Autograph has been on the market since the beginning of 2023 and is distributed by the plug-in manufacturer Re:Vision Effects, but is produced by the French company Left Angle. The software is available on Windows, Mac and Linux and can be purchased either as a subscription model (monthly and annually) or as a perpetual licence. In addition, there are three versions of Autograph: Creator, Studio and Render. While Render is really intended as a pure command line rendering tool, Creator is a licence for freelancers and smaller companies with less than 1 million US dollars in capital or income. For those who land above that (congratulations!) there is the Studio licence, which also includes the Python API. A monthly subscription for Autograph Creator starts at 35 US dollars, for Studio it is then 60 US dollars. Those who prefer to buy their licence will then pay a price of 945 US dollars for Creator and 1,795 US dollars for Studio. Before that, however, you can easily get a first impression in a ninety-day trial – to be found at left-angle.com.

The first impression

When you first look at the interface, you quickly think that you are simply sitting in front of a darker After Effects. And indeed, anyone who is familiar with Adobe’s motion design software will quickly find their way around Autograph. The viewer in the centre, properties on the right, the project window on the left and the timeline at the bottom. Similar to Nuke, you can create your own workspaces for different tasks so that you can customise the layout to suit your needs. However, there are only very few types of panels, which is why the design options are quite limited. Footage can be imported into our project via the project panel and compositions can be created. However, as with After Effects, the real centre of attention is the timeline. Individual layers can be dragged in here and the parameters can be expanded and edited accordingly. The filter function, which not only helps to isolate individual layers but also specific operations, is very convenient. These filters are also assigned to hotkeys and can be combined with text input. As an artist, you can then isolate the position value of a specific layer quite quickly, for example, in order to animate it. All parameters that are displayed in the timeline are also available in the Properties panel. This may seem redundant, but users who don’t want to scroll through thirty layers will find an alternative workflow here. And if you only want to work in the timeline, you can modify your workspace accordingly. Speaking of parameters: Those who prefer to enter values gesturally rather than numerically will be pleased with the hotkeys that adjust the intensity of the value input. For example, Shift lets you “drag” larger values, while Ctrl ensures finer input. Autograph does one thing fundamentally differently to After Effects: the Curve Editor is not integrated into the already fully loaded timeline, but is available as a separate panel. However, the viewer will undoubtedly remind many experienced artists of Nuke. No wonder, under the bonnet Autograph processes all files in 32-bit float – and so the viewer has silders to not only apply gamma and gain, but also saturation as a post process to the viewer, so that you can visualise even the last highlight in the EXR file. The viewer has two inputs, where a HUD and the blend modes known from Nuke are also available, for example to compare a reference with the current comp.

On to something new…

Up to this point, the interface looks familiar. But let’s take a look at the things that Autograph does differently from the two software packages between which it is positioned. The first thing that stands out is the use of the HUD. While Nuke’s on-screen controls are often very small, Autograph’s are not hidden at all. They are clearly visible and an important part of Autograph’s fast and fluid operation. The fact that right-clicking and the associated context menus are largely dispensed with also contributes to this fast interface. Pen and tablet users in particular will really appreciate this. But it is also the absence of clicks that makes the interface incredibly fast. For example, there is an option (which can of course be switched off) to briefly highlight layers that can be clicked on in the viewer so that you can see whether you have made the right selection. And how nicely the alpha channel is respected here – others can take a leaf out of this book. Hovering over drop-down menus works in a similar way. The selected parameter is displayed as a preview, so to speak, before you have even clicked on it. Not as a thumbnail, but actually applied to the current comp in the viewer. This works for the layer blend modes, for example, but also for the more than fifty interpolation presets. So you can run your animation in a loop and simply scroll through the presets and see what these different interpolations would look like.

Let’s stay on the subject of animation for a moment: Autograph is also trying to rethink things here. All image content always has its pivot in the centre of the image and the zero point of the coordinate system is also in the centre of the image. This may sound banal, but it is enormously helpful when it comes to adapting an animation for different output formats. It also helps tremendously when creating symmetrical animations. Autograph has a sophisticated system for dynamically linking values to each other without the artist needing a maths degree to write the corresponding expressions. Autograph uses so-called modifiers, which can simply be dragged onto the corresponding layer like an effect in After Effects. Somewhat confusingly, Autograph also refers to classic effects such as keyers or colour correctors as modifiers, but this shows how important they are for the workflow in Autograph. In addition to the modifiers, there are generators that can produce everything from images, i.e. classic colour areas or gradients, to text and numerical data and even 3D primitives. This now brings us to the 3D environment. This is based entirely on USD and as the USD files are only linked and not imported into the Autograph project, common formats such as Alembic or the rather old FBX are not supported. Autograph uses the filament developed by Google as its render engine. As a result, the 3D environment is quite fast and is likely to bring tears to the eyes of some Nuke artists. Camera and lights can simply be dragged into the timeline and if 3D layers are stacked directly under each other, they are treated and rendered as a single 3D scene. This makes it possible to mix 2D and 3D elements in the same timeline without having to create precomps. And by using filament, post effects such as ambient occlusion, fog and depth of field are also available.

What is missing?

The developers of Left Angle do not seem to lack enthusiasm, as can be seen in the interview with Francois Grassard on page 30. For a first version, Autograph makes a well-rounded impression. OpenFX plug-ins are supported, even if at the moment only those from its own distributor, Re:Vision FX, are available. The extent to which plug-in manufacturers support the newcomer will be seen in the coming months. However, both Nuke and After Effects, the two programmes that Autograph is competing with, both have an ecosystem of plugins and gizmos that has been growing for several years, not to mention a very large user base, which naturally does not exist right from the start. Nuke artists will miss their Node Graph the most. Even if the toolset is impressive, the speed of the interface and software is blazingly fast and the rendering is super-precise, the logic in a timeline is of course different from that in a node graph. Motion designers coming from After Effects will certainly find it easier to familiarise themselves with Autograph.

Conclusion

First impressions can often be deceiving. Autograph is far more than the After Effects clone that some people thought it was at launch. The user interface in particular has some innovative surprises and concepts in store that the developers of the long-established software either didn’t dare to tackle due to their user base or simply didn’t think of. This breath of fresh air is something that has definitely been missing in recent years. It’s reminiscent of the days of 5D Cyborg and Socratto, where innovative software forced established developers to rethink things and push their software forward. Competition stimulates business. That alone is a reason to be happy. Autograph is already interesting for a certain group of artists. Anyone who moves between motion design and compositing or wants to make the leap from motion design to compositing will find Autograph a possibility without having to learn a completely new paradigm such as working in Node Graph. Finishing artists can find a complement to Flame or Resolve without having to commit to a monthly cloud subscription. And if the developers of Left Angle continue with the same verve, this circle will certainly grow even larger.

The post From the left.. first appeared on DIGITAL PRODUCTION and was written by Christoph Zapletal.

Here is a short review of the slider in german. English subtitles are available.

This is also the case with the new camera slider model that YC Onion gave me for this test. It is the 52 cm Chocolate Pro Cheese Motorised Slider with Supporting Stand, the second longest model in the Chocolate series, which also includes 20, 30, 40 and 70 cm models. The “cheese” is a removable motor with a touch display for programming directly on the appliance. there is also control software for Android and iOS that can be used to control many other YC Onion appliances.

The motor can be inserted into the slider from above and is then locked with a small lever. For the power supply, Sony NP rechargeable batteries can be inserted at the side, which supply the slider with power for between 4 and 10 hours, depending on the size and mode. This is also sufficient for longer time-lapse recordings. If it takes even longer, the motor can also be powered via the USB-C socket with a power bank or mains adapter. Programming with the touch display on the motor is intuitive and self-explanatory. First select one of the 3 modes. Video, timelapse or stopmotion. In video mode, you then use the arrow buttons to move to the start point and confirm it. Then select the end point and confirm this too. On the next menu page, you can then set the speed and activate the loop mode. The other two modes work in a similar way.

If you address the motor via the app, it can be completely remote-controlled. Six waypoints are then also available. For timelapse mode, there is also a mini jack socket for camera trigger control. And it is possible to couple the controller with various gimbal models. This allows complex camera movements with pan and tilt functions to be realised. The slider itself can also be “pushed” manually and is made entirely of metal. It has four fold-out feet so that it can be placed stably on the table or floor. Of course, there is a thread on the underside for tripod mounting. With a sturdy stand, the slider can also be moved at an angle or vertically. As the base and tripod plate travel with the slider, the travelling distance almost doubles in relation to the length of the slider. The maximum weight is reduced from 8 kg to 2.5 kg in vertical mode, but for a DSLR camera, such as the GH5 in my case, the force is easily sufficient. The operating noise of the
Motor is very quiet and hardly disturbs interview recordings.

The compact slider is a real asset when shooting with a DSLR camera, but it can also provide a cinematic look when shooting with a mobile phone or Osmo Pocket. The length of 52 cm and the distance of 88 cm due to the travelling base and on a tripod are a good compromise between a mini slider and a long rail slider. Supplied in a compact bag, the device can also be carried in small luggage. And it’s fun to watch the cheese slowly glide over the chocolate. Incidentally, the menu combination presented here costs just under 1000 euros. On my postproduction-tutorials.net page there are some videos on the subject, including the YC Onion Stand.

The post YC Onion Chocolate Cameraslider first appeared on DIGITAL PRODUCTION and was written by Nils Calles.

There are commercial solutions for this. But not everyone wants to invest money straight away when they have just bought a computer and editing programme and are now faced with another problem. Or you work with the free version of DaVinci Resolve (DR for short) under Windows or Linux and don’t want to buy the studio version for the time being. This is less relevant for the Mac because you have already acquired support for many codecs with the purchase of the machine, but not everything works there either.
A very helpful programme comes into question here – Shutter Encoder(shutterencoder.com), which I expressly do not want to call “free” for good reason, but simply “donationware”. The developer, Paul Pacifico, has clearly earned our donations. It is based on ffmpeg, which means that anyone who is not afraid of command lines could do much the same thing. For everyone else, it is the Shutter Encoder (SEnc for short) that simplifies all its possibilities for users. It runs under Windows, Linux (Ubuntu) and on Macs with Intel or Apple Silicon. At first glance, it is only available for 64-bit systems, but under “older versions” you can even find 32-bit versions.

Thanks to a number of helpful translators, the programme is available in many languages, including German. However, I would like to mention one of the few points of criticism here: It is irritating when “Conversion” is listed under “Without conversion”. This is an unfortunate translation, as the English term would be “rewrap”. Although this is more correct, it is certainly not familiar to everyone. The translation is also not ideal in a few other respects. I do not want to replace the comprehensive documentation (English) in the following, but only explain a few basic applications.

Codecs and containers

Of course, the terms are familiar to experienced professionals, but they are probably also familiar with the programme or can easily hack their way through ffmpeg. For the less experienced: Containers are like digital boxes in which the actual video and audio data is packed. These include MP4, Quicktime, MXF, MKV and AVI, to name just a few of the most common. The processes used to digitally store video and make it visible again at the end are called codecs (coders and decoders). Today, this almost always involves a compression process, otherwise the video files would be gigantic. The sound is also often compressed, with processes such as MP3, AC3 or AAC, although compression of audio data is rather mistrusted by professionals
Compression of audio data tends to be avoided by professionals.

There are now dozens, if not hundreds, of such codecs, and the same container can contain completely different ones. So it is of little use if someone delivers you a file and says, for example, “This is a QuickTime, have a look at it.” If you can’t watch it after all, he or she had better tell you which codec is in it. If you don’t know this, SEnc can also provide detailed information. It uses the freeware MediaInfo to do this without you having to switch to that programme. While SEnc can handle an almost infinite number of container and codec versions, this is not a matter of course for an editing system.
There may be cost reasons for this, for example DR accepts fewer codecs in the free version on PC than the studio version. Or the codec is simply too exotic or outdated, so that the developers have dispensed with it. Some formats are not even displayed by the standard player of the respective operating system. A second reason for problems can be that some codecs are recognised, but are so demanding that they cannot be played back smoothly. This applies, for example, to H.265 (aka HEVC), especially for UHD. A number of RAW formats can also require a lot of computer power, but these are pretty much the only ones that SEnc does not read. However, it can usually extract the audio from such a file.

Repackaging

Sometimes it helps if a codec is simply moved to another container (rewrap). Some editing programmes, for example, do not take everything in MXF, but then as QuickTime. It is definitely worth a try, as this process is not only very fast, but also completely lossless. To do this, select “Conversion” under “Without conversion” in the German version and then the desired target container. If you wish, you can also shorten the clip first, whereby GOP codecs can only be cut on an I-frame (not quite correctly referred to here as a keyframe). These codecs include, for example, the common variants of MPEG, H.264 and H.265.
Repackaging an .mpg file (MPEG-PS) to MOV, for example, already helps DR to process the file. However, a correct timecode is essential for reliable processing in the editing programme and even more so for transfer to other editors. However, this is not always read correctly (e.g. with MP4) or the camera has not written any at all – it is suspicious if it always starts at zero. SEnc can then add a timecode during the rewrap, the start value of which you specify yourself. When processing a batch list, this can also be incremented if required.

Conversion

Converting the codec into another codec (also known as transcoding) is necessary if a codec cannot be read in a video programme regardless of the container. This also makes sense if a codec places too high a demand on the hardware in order to run smoothly. This typically applies to H.265, where some versions are not even recognised by the free DR on PCs. Even if they are accepted, the hardware performance of older PCs (especially laptops) and many Intel Macs is not sufficient for smooth playback. This is where converting to a less demanding codec helps, whereby the metadata can be retained if desired.

Now you will probably already know that you should not convert a highly compressed codec to another codec of the same type if you value picture quality. So please don’t convert H.265 from a camera to H.264 just because your computer can handle it better. The so-called editing codecs were developed for this purpose and can be found in SEnc under “For further processing”. The most common are ProRes, DNxHR and CineForm. They have different variants, but with the same amount of data they are also comparable in terms of image quality. From ProRes 422 HQ, DNxHR HQX or Cineform high, these codecs are visually lossless, especially for originals from semi-professional cameras.
Such codecs put much less strain on the computer, and current Macs even have their own hardware built in for ProRes. In practice, this means that you can also edit high video resolutions smoothly or use more tracks for multicam. Of course, nothing comes from nothing: the converted files are considerably larger, but they are usually only needed temporarily.

What’s more, storage space is much cheaper than a new computer these days. Incidentally, CineForm is somewhat more compact with the same visual quality. QT Animation, on the other hand, is hopelessly outdated and inefficient, while Uncompressed simply produces gigantic amounts of data.

This is not a problem with sound, where you should use an uncompressed format such as AIFF or WAV for further processing. Superfluous audio tracks can be deactivated before editing. After editing, you can also use SEnc to compress your final product without hesitation if your computer is unable to cope with simultaneous rendering and compression. All this is very luxuriously supported with a batch list, several output formats and automatic start when the computer is idle.

Clips from smartphones are converted to a constant frame rate using “conform” with various algorithms. The programme is even capable of working with subtitles, text overlays or watermarks. Finally, de-interlacing and slow motion should also be mentioned. Specialised programmes such as TVAI (see page 48) can do this even better, but SEnc is really fast instead. We could fill several more pages with the complete list of functions, but if you understand English, you can find everything here: shutterencoder.com/documentation.html

Comment

A programme with such a wide range of functions for which the author only expects a donation? Donate plenty! We have noticed one more little thing: When right-clicking on a video file, the programme is not automatically offered as a target. This is a tolerable flaw, as there is drag’n’drop and even a watch folder
even a watch folder – of course, “watch folder”.

The post Shutter Encoder – Multitool for video and audio first appeared on DIGITAL PRODUCTION and was written by Uli Plank.

So far, we’ve only scratched the surface of the question: “Yes, but what am I listening to anyway?” So now it’s all about content. It’s not so easy to say in general terms where 3D audio actually enables good content. Depending on the context, there can be a completely different technology behind it in the form of formats or game engines.

That’s why I’ve come up with a structure that I’ll simply call the “3D audio matrix” – or pyramid? Be that as it may, the whole thing is intended to provide a reference for applications and their prime examples, the advantages and disadvantages of 3D sound, and of course an overview of formats and tools with their respective peculiarities. You have to take a few steps to understand this: 3D audio is not just 3D audio. So let’s first go back several dimensions to the origin, i.e. from 3D audio to 2D, 1D, 0D..

The 3D audio matrix overview

How, 0D? Admittedly, that’s a bit abstract. How can you visualise it? I’m taking a mathematical approach here. Don’t worry, it won’t be any more difficult than in your first geometry lesson. Let’s imagine a coordinate system in which our head is at the origin.

Now let’s combine this with the question: what kind of geometric object do I have?
0D: A point in the coordinate system without spatial information, the origin (0|0|0)
1D: A line, here I can move from left to right (x|0|0)
2D: A plane, now I can also move forwards and backwards (x|y|0)
3D: A cube/sphere that adds height information (x|y|z)

Audio formats that you already know

So far so good. Now imagine you want to place an audio object in a room. There are already audio formats that we know from our everyday lives.

0D: Mono. No room information can be added.
1D: Stereo. You can at least move your sound to the left and right
2D: Surround. With 5.1, for example, you can also place sound at the back
3D Audio: Here you can also move sound up or down.

But wait, there’s more: degrees of freedom

All overviews in this direction that I know of stop at Dolby Atmos, but beyond that it’s just getting started with 3 or 6 degrees of freedom. OK – what are degrees of freedom? Also known as DoF (Degrees of Freedom), the degree describes the following.

• 0DoF: It is not defined where you actually look while consuming the content, except that you look forwards, like in films, you are not supposed to turn around.
• 3DoF: Here you can also rotate your gaze, as we know it from 360° videos. Also known as head tracking (rotation).
• 6DoF: This concept is already familiar from 3D games, where the player also moves through a 3D space (translation).

Headphones simplify understanding

It’s as simple as that – or not. Because it’s always a question of which direction I’m looking at my concept from. Here I am referring exclusively to headphone playback. Do some of you remember the localisation and externalisation in the head from the last article? This is a good example of how mono and stereo are always perceived in the head. Even if I add reverb to create a depth gradation, I can still only move an object to the left and right and only create a difference in volume and time (ILD, ITD). But the third factor (HRTF) is missing, with which I can really differentiate between front and back. For me, stereo is therefore one-dimensional (lins right), which is often confused with the two channels.
With surround sound via headphones, a binaural attempt is made to generate an impression of the front and rear – with 3D audio, this also includes height information. This is where the aforementioned HRTF comes into play in order to really “let the sound travel out of our head” during the calculation. So we go from an in-head localisation to an externalisation. Although at the end of the day, a two-channel stereo signal is played back. But watch out! It’s not “normal” stereo, but binaural, with HRTF-filtered extension.

The walls of the concept are shaking for loudspeakers

It’s not so easy for loudspeakers, because even if I only have one loudspeaker in the room, it’s still “somehow three-dimensional”. Besides, the term stereophony actually says: more than mono, so it would even include 3D audio, but I think that in everyday production everyone thinks of stereo as a “two-channel audio file”. With stereo
Playback, you place two speakers with yourself as the third point in an equilateral triangle. And as we know, triangles are actually two-dimensional. Nevertheless, I can only move my sound between the two loudspeakers, not beyond them. Even if I add reverb, you get the feeling of depth
Feeling of depth, but you don’t know whether the room is actually in front or behind you. Nevertheless, two-channel stereo is still a stable reproduction method and, in my opinion, not broken, regardless of what the various hectic marketing newsletters claim.

0-2D aka “normal media”

OK, of course I could go on and on about where to find mono, stereo and surround content. But that’s everyday life. We use mono every day for voice messages, we stream music in stereo and if you have the right TV, you can watch films in surround. The more I think about it, the more I realise that surround is not that far removed from 3D audio in this respect. You still have height information – applause. So the quantum leap from stereo to surround actually seems greater than that from surround to 3D. I would also subscribe to this for film, for example, because when streaming, the great 3D sound from the cinema has to be compressed and is then usually a 5.1 that tries to retain a few treble elements.

Audio should be immersive

But we remember that immersive audio should ideally be so natural that we don’t even think much about the technology. And 3D audio brings us a lot closer to this impression than surround sound alone.
However, there is another factor why 3D audio can bring even more benefits under the bonnet than height information. The surround formats meant here are either quadrophones 4.0, 5.1 or 7.1. The numbers are channel information, e.g. for 5.1, channels 1 to 6 are: Front Left, Front Right, Centre, LFE (low frequency effect), Left Surround, Right Surround.
So if you want height information, you need even more channels, such as 5.1.4. You then have four more speakers on the ceiling. But as you can guess, that’s kind of impractical. And what if I have a 7.1.2 system, how are the channels converted? That’s why audio technology is moving away from channel-based formats and towards so-called NGA, next-generation audio formats.

Object-based audio

To understand why 3D audio can be even better than surround sound, let’s take a brief look at object-based audio. A major advantage of object-based audio is the independence of the channels, as the rendering only takes place at the end user. Systems that want to use Next Generation Audio must therefore have a corresponding decoder integrated. This ensures optimised audio playback at all times.
Another exciting possibility in addition to the movement of audio content in 3D space is the personalisation of these audio objects. My favourite example is watching a football match, where I can simply mute the “Commentator:in” audio object. Podcasts would be an equally exciting application. Let’s say we want to listen to a news podcast that is an hour long, but we only have 10 minutes. We tell our smartphone this and the podcast is automatically shortened to the most important 10 minutes using metadata.
MPEG-H is able to do this and is already standard in broadcasting in Korea and Brazil. The biggest competitor is AC-4 aka Dolby Atoms, which only allows such personalisation and interaction according to its own specifications.

Spatial Audio 0DOF with/without picture?

Let’s take a look at what 3D audio content is really available now. Most of them should be familiar, because thanks to Dolby Atmos, consumer favourites such as films and music streaming services are currently being supplied. Podcasts too, but in the vast majority of cases they make more sense in stereo – or even mono.

3D audio is better than 3D video?

When I tell people that I do “something with 3D audio”, they immediately ask if I work with “Dolby Atmos”. In fact, Dolby is not that relevant in my immersive audio bubble because it doesn’t enable many things that I need in my daily work. But more on that when it comes to degrees of freedom.
Nevertheless, Dolby Atmos (AC-4 is actually the audio format behind the marketing term) is particularly relevant in the film industry. Thousands of Hollywood blockbusters have already been mixed in this format and people are happy to spend a few euros more to enjoy a surround system in the cinema. Sure, the sound experience is more fun when a helicopter suddenly sounds like it’s flying over your head. But at the end of the day, I would argue that all films also work with stereo sound – or mono. You don’t look in all directions, you only look forwards.
Even though I like to shoot in the direction of Dolby Atmos, they do a good job of bringing this surround sound into the living room. Many soundbar models now support playback via the TV using streaming apps. And playback via Apple headphones is also really fun, even on an iPad. Although the “Airpods Pro” are in-ear headphones, you have the feeling of being enveloped by the sound and can almost save yourself an expensive home cinema. Now let’s take a look at pure audio enjoyment without visual content: Music streaming has long been part of our everyday lives and is becoming increasingly popular. 3D music streaming is the latest innovation in the industry, adding a third dimension to the listening experience.

3D music

But now Dolby came up with the idea of converting their 3D audio format for music production. Admittedly, I’m a little sceptical here too, because I felt that most songs sounded better in stereo than with the 3D audio formats. In addition to Dolby Atmos Music, Sony is trying to add 360 Reality Audio to the list of 3D formats.
However, one advantage is definitely that you have to make fewer compromises when mixing music and have more options for placing the individual audio tracks. This gives some tracks more depth and you can hear the individual instruments better. You have the feeling that the musicians are sitting around you in the studio.
We are currently in a major learning phase here, similar to the move from stereo to mono. The first Beatles songs sound interesting by today’s standards. Today we know better how to mix in stereo. The same applies to these 3D music productions. So just see for yourself which streaming platform is already there and listen to it. I think the quality is getting better and better and since Apple Music got involved, there’s been a bit of a gold-rush atmosphere in the audio community.
And what is 8D Audio now? Admittedly: maximum confusion to categorise this again, but 8D doesn’t even stand for dimensions, but directions. Let’s just accept the phenomenon and define it for what it is: music circles around our heads in mono and works quite well through headphones. But in the long run it is a bit tiring and monotonous, so 3D music tries better. Here you work with individual tracks of the various instruments and place or move them through the room where it suits the composition. It rarely sounds as clearly 3D as 8D audio. So just listen to the examples that sound better or worse on my blog and form your own opinion.

Dolby Atmos Podcast

The last remaining audio-only format is podcasts, which are now also being tackled by Dolby Atmos. Here, however, my toenails roll up a little, for reasons that go beyond the scope of this article. The short version is that Audible has now jumped on the bandwagon, but AC-4 as a 3D format does not allow certain sound sources to be made non-3D. With audio dramas in particular, this means that the narrator’s voice is suddenly in the 3D scene with the protagonists, which means that the listener can no longer really distinguish who is actually part of the action.
But I’m already in dialogue with Dolby about this too, because what’s the point of always complaining :-) But there are already other courageous productions that have mixed with or without Dolby in order to get answers to the questions of how well radio plays work as 3D audio productions. Just listen to it yourself and form your own opinion. Head tracking comes later.
Most productions have “the problem” of producing too classically. In other words, recording speakers in the studio, then letting them move around virtually with 3D spatialisers and adding an atmo. But that doesn’t really sound convincing or immersive. Can you cite your own productions as a positive example? Then I’ll throw my radio play for BKW Engineering into the ring. A more classically produced one is the “Erdsee-Hörspiel” from WDR. I’m curious about the opinions.

Advantages and disadvantages of 3D sound

Said productions with 3D audio can be fun if they fit the content well. It is also often used as a marketing gimmick to simply offer the listener something new, to have a unique selling point. This is also a disadvantage, as 3D audio is not a seal of quality. However, you can usually rely on your ears to determine whether the production works for you or not, apart from your own taste.
That’s why productions are often made in 3D that would probably have worked just as well in stereo. Especially in the music sector, there are genres that have spatialisation but, in contrast to the stereo version, have less pressure. In addition, the conversion to headphones is not yet perfect. The soundtrack often sounds somehow duller than you are used to with stereo directly to the ears.
Here, too, we are in a transitional phase. Users first have to get used to this spatial sound again. It usually works better via loudspeaker systems and I have to admit that I had a lot of fun with Dolby Atmos Music in a demo car.
However, as you can imagine, with 3D audio mixes there are even more parameters that I can set for a sound. That’s why such mixes are usually more complex, more time-consuming and more expensive than a stereo mix. And, as I said, you usually need a selection of special devices to really get the full benefit.

Formats and object-based audio

As mentioned, all formats are somehow based on the assumption that the
the listener is looking towards the front, where there is usually a centre speaker or screen. You don’t actually want people to look across the room because there is usually a TV picture in front of them while they are listening to 3D audio content.
As I said, Dolby Atmos has established itself in the form of AC-4 with a large marketing budget and is spreading from films to music, podcasts and gaming. The alternative is MPEG-H made in Germany, which is particularly suitable for live streaming in the broadcast sector. The competition from Dolby Atmos Music is an adaptation of MPEG-H, called Sony 360 Reality Audio, which should provide a boost for the Sony Music label in particular. Both formats can even be found on Amazon Music, although Dolby already has around four times as many tracks, so the battle seems to have been decided.
One format that has not established itself for over 30 years, but is currently experiencing a renaissance, is Ambisonics. This sound field-based format has audio channels, but instead of loudspeakers it maps spatial axes. It all sounds a little unusual and only has a very small sweet spot when played back via loudspeakers. However, this disadvantage does not exist with headphones because you have the perfect playback position directly on your ears. The format can also be easily rotated around the X, Y and Z axis. This is why it has established itself more for the use of 360° videos and thus into the world of three degrees of freedom.

New audio freedoms in three degrees

With advances in technology, new 3D audio techniques are opening up unprecedented possibilities for media production. 3D audio with three degrees of freedom (3DOF) is one such concept that enables unprecedented immersion and dynamic sound experiences that benefit from the fact that you’re not just staring straight ahead.
Below we look at the pros and cons for specialised headphones, applications where it can be used effectively and different formats available when implementing this feature into a production workflow.

360° videos from a sound perspective

Probably the best-known representative of this genre are 360° videos. These spherical moving images triggered a real hype half a decade ago. Suddenly you could watch such videos
on the largest video platforms Video YouTube and Facebook. Strictly speaking, there are many types of user experience:

• On desktop devices, using the mouse to turn your gaze while looking further forward at the screen.

• On smartphones, where you hold your device in front of your nose, not turning your head, but rotating your body on its own axis.

• Head-mounted displays (HMDs) are in the top class because the image really does adapt to the movement of your head in real time. Such 360° videos are also available as stereoscopic videos, as 3D videos, if you like, where each eye gets to see its own 360° panorama, creating a more vivid image in the brain. But of course there are also 360° videos with 3D sound. In this context, I also like to call it 360° sound, because you immediately understand that the sound is also spherical like the image.

Audio head tracking

If you now remove the image component, but still want the sound to react to head movements, then we find ourselves in the world of audio head tracking. Apple is already building this technology into ALL Airpods, from the entry-level Airpods to the Airpods Pros and, of course, Airpods Max. However, the use cases are currently limited.
Technically, it is now possible to listen to Dolby Atmos Music tracks via Apple Music, but, as I said, these tracks were never mixed with the intention of “listening around” in the music. In addition, Dolby metadata is even bypassed so that this feature can be activated at all. As a result, Dolby Atmos Music tracks sound different on Amazon than on Apple Music – not exactly what you want to hear as an audio engineer. But Apple needs content to be able to use its technology as a selling point.

3D sound is more than entertainment

But there are applications that actually solve problems with 3D audio and head tracking and are not just a fun factor. The medium is also becoming increasingly relevant for communication. MS Teams has added the “spatial audio” feature to its video calls. The good thing is that you don’t need any additional hardware. Simple headphones are all you need and the microphone signal is automatically spatialised in the cloud for the other users – even without head tracking.
In a video conference with several people, things can quickly become chaotic as the current mono system has problems keeping the different voices apart. Our brain has difficulty differentiating between the voices as they all come from the same direction. 3D sound makes the conversation situation natural and actually makes it measurably easier to listen, because it takes the strain off our brain – similar to the cocktail party effect. The 3D audio spatialisation of voices makes it much easier to differentiate between them and noise is less noticeable.

Head localisation is being fought – wrongly

Films can also be watched with head tracking, but there is a fundamental problem here: the approach is to make the entire sound 3D. However, this means that even elements such as narrative voices or background music are part of the scene where they shouldn’t be. Here is a very brief explanation on the subject of diegesis. 360° videos are the most vivid. Everything you can see should also be three-dimensional in terms of sound, as you are in the scene (diegetically). But it shouldn’t be a narrative voice that can’t be seen (non-diegetic). Otherwise you will hear a ghost coming from somewhere and wonder who is talking to you. But if the voice is played in mono, it doesn’t change and you immediately understand that a person is talking to you who isn’t even part of the scene.
In other words, in the world of 3 degrees of freedom, not everything is just 3D audio so that you have the feeling that it is coming “from outside”. Rather, the ability to combine the soundtrack with mono or stereo signals via headphones is important so that listeners understand which sound is coming from which narrative level. This is not possible with loudspeakers because they are always perceived from the outside, whereas with headphones you can – and should – make use of the localisation in the head.

Confused? Then let’s go round in circles again

Apple does a good job of being able to understand whether I’m listening to stereo or multi-channel content. This is all the more interesting when you realise that Bose burnt its fingers on this very subject years ago. In Cupertino, however, they believe in the technology and have already built it into every pair of in-house Airpod headphones. This also helps with the market launch of the Apple Vision Pro, but more on that later. When it comes to audio playback alone, a distinction is made not only between the two options, but a total of five.

Input: Stereo.

You can hear the sound as normal stereo, nothing special at first. However, if you don’t like this upside-down localisation, you can activate “Spatialize Stereo/Stereo to 3D Audio”, which adds a reverb algorithm to the signal to make it sound more “natural”.
In addition to this spatialisation, you can also activate head tracking, which makes the signal sound through headphones as if you were listening to it through two speakers in the room (3DoF).

Input: Multi-channel

Mostly Dolby Atmos via film platforms such as Disney or music streaming such as Apple Music. The sound is automatically converted from multi-channel audio to binaural stereo so that the sound sounds as if it is happening around you. This setting makes the most sense if you move around and don’t want the sound to change all the time as you move your head.
You can also activate head tracking, which is particularly useful for films if you have a visual reference point or if you want to distinguish objects from the front from those from behind (which is always difficult with binaural sound). I would call this level 3DoF.

Freedom comes with pitfalls – advantages of head tracking

A big advantage of this technology for 360-degree videos is the fact that you can now hear better when something is happening behind you, for example. Something you wouldn’t be able to see because we only have a limited field of view, but our hearing is always mapped in 360°. I have often seen VR experiences that slap their beautifully designed visual scene with arrows so that people look in the right direction at the right moment. Cleverly placed 3D sound can solve this problem intuitively.
It also solves one of the biggest obstacles of binaural audio. You often have the feeling of spatiality and that the sound is happening around you. However, you can rarely distinguish the front from the back. But if you are able to turn your head even slightly, you can immediately understand which sound is where. Head tracking does not mean that you have to move 360° – but you can. This solves the aforementioned problem of narrative levels. With films, you don’t have to ask yourself whether an element is dynamic or not. When we listen to epic music in the cinema, we don’t ask ourselves “where is Hans Zimmer now”. But because you now have this clear separation, you have to question how you actually use voice-overs and music. In most cases, a scene with well-designed sound effects is better than desperately trying to keep people entertained with music and speech. The brain is usually already well supplied with 360° images anyway, so three levels of sound (speech, sound effects, music) are more likely to cause confusion.

Disadvantages of head tracking

As already mentioned, the whole thing is not so easy to implement via loudspeakers. Theoretically, it is also possible to display 360° videos as a projection in planetariums, for example. For surround sound, you are surrounded by loudspeakers. But the speaker’s voice still somehow comes from one direction. What still works well in the cinema suddenly becomes a problem with spherical videos and a series of workarounds and compromises are necessary.
Unfortunately, you rarely know when listening whether the mix should be heard with head tracking or not. There are mixes that really fall apart when you have the opportunity to turn your head. While other productions don’t even make sense if you’ve deactivated head tracking.
Admittedly, I always talk so cleverly about what you should and shouldn’t do. But the reality is simply that there is usually neither the time, money nor knowledge to make your immersive media production really good. If anything, an Ambisonics microphone is put up and labelled as immersive audio. Only to be mixed in stereo in the end for budget reasons “because nobody can hear it anyway”. It may be that listeners who come into contact with immersive media for the first time don’t necessarily scrutinise the sound. But the more points of contact you have, the greater the desire for sound. All the top podcasts are now produced in a studio, even if they started out as a hobby – there must be some kind of quality reason for this ;-)

Formats for audio head tracking

I have already mentioned the most famous representatives with Dolby Atmos, 360 Reality Audio (based on MPEG-H) and Ambisonics. However, these are all 3D formats that were not primarily developed for audio head tracking.
That’s why I don’t want to go into the technical details here, but rather briefly explain why Apple is once again showing a very good approach here. Even if Dolby likes to describe its format as future-proof, at some point it will reach its limits.
As mentioned, in the world of degrees of freedom, it’s not just 3D sounds that are relevant. But also precisely those non-diegetic sounds that are not part of the scene, but make sense for music and voice-overs in 0D. But of course there is more than just black and white thinking, i.e. 0D and 3D. Because there has to be something in between. This is often referred to as a bed. Apple refers to the three factors as:

• 3D Audio Objects
• Ambience Bed
• Head-locked audio

The 3 layers for immersive soundtracks

we have already looked at 3D Audio Objects, which are the objects that I can place in the room. You usually have the option of setting distance parameters or the size of an object so that it doesn’t stand out from the scene. Let’s take a cheering 3D audio fan in a stadium as an example. Then I would like to have a reverb that gives you the feeling of being in the same place. But if I simply add a reverb to the object, the reverb will only come from this corner. However, sound spreads in all directions, so it would also be heard all around us. Theoretically, I could send the reverb to our head-locked audio track. But then the reverb would not be 3D.
This is where the aforementioned bed comes into play. All signals that should be spatial but can be diffuse can be sent here. So if you have not just one fan, but hundreds, you would otherwise have to fill 100 audio tracks with objects. This way, you can simply send the group to the bed and only need a fraction of the audio tracks.

What do the other formats do differently?

Dolby Atmos, for example, works with a channel-based 7.1.2 bed and you can add up to 128 mono objects. However, it doesn’t actually have a head-locked stereo track because the format is based on loudspeakers. So for me it is not suitable for podcasts. In principle, the Dolby Atmos renderer offers the option of marking an audio
Object as “disable binauralisation”. This means that it is not played spatially. However, if you activate head tracking, the Apple renderer bypasses this meta data and only reads out where the object is located in the scene. This means that all Dolby Atmos mixes were never mixed with the intention of head tracking and therefore rarely utilise the advantages of the technology.
Ambisonics, on the other hand, has 4, 9, 16 or more channels, depending on the order. So it has a bed, I can even work with head-locked audio, but again it has no objects. Which is why the sound is always a bit diffuse, or I would have to spend a lot of audio channels to get close to the resolution of object-based formats. It therefore supports head-locked audio in mono, but not stereo. However, this optional stereo track is standard with Facebook360 and YouTubeVR, for example. An Ambisonics file is supplied, which rotates depending on the viewing direction. If required, an additional stereo file that is always played in the same way, no matter where you look in the 360° video. This gives you the best of both worlds and a good compromise between resolution and quality.

6DoF – anything but dumb

Let’s move on to the premier class of 6 degrees of freedom. Here you not only have three possible rotations around the X, Y or Z axis – but also three translations to these axes. In less unnecessarily clever terms, this means that you can also move towards or away from the sound, making it louder or quieter, for example.
All the applications discussed so far have been based on the fact that the listener is at the centre of the action, in the so-called sweet spot. This is where the optimum listening position is. But now we can move away from this point all at once and you can already guess that this makes sound design even more complex. So that there are no acoustic holes in the 3D scene or you are distracted by too many sound sources.

Games (more than a gimmick)

Here you will inevitably find yourself in game engines. This is why games are the best-known representative of this category. But not every game uses 3D audio. Once again, the genre question is a legitimate one.
A 2D game needs sounds from behind/above/below just as little as a strategy game in which I look down on my people from above like a god. Left/right is perfectly adequate here. 3D audio could be used here at most in the ambience, similar to films, so that you have more of a feeling of being part of the scene. Enveloped by sound is the keyword here again.
All games that take place in 3D worlds, especially first-person games, benefit from spatial sound. Fans of shooters have long known that being able to hear the enemies behind you before you get them in front of your virtual “weapon” can make all the difference in the game. That’s why gaming headsets are popular in this respect, so that your ears can tell your eyes where to look as well as possible.
However, such surround headsets are usually not even necessary to be able to hear spatially. We remember that our brain can do this with just two cleverly rendered audio channels. In most cases, the game automatically recognises whether you are using speakers or headphones and renders the sound accordingly. Nevertheless, it may well be that such surround headsets are even more customised to the software and, above all, enable communication. The Playstation 5, for example, advertises with the Pulse 3D Audio Engine and its own headset, which are very well matched to each other. Recently also in combination with Dolby Atmos in order to be able to control multi-channel soundbars.

AR (augmented reality audio)

Pokemon AR is often used as best practice for augmented reality applications. Even though I like to say that the game didn’t go viral because it was AR, but because it was Pokemon and had a great multiplayer character. The built-in cameras of smartphones are usually used here. The Lidar scanner, which enables even more precise tracking, is also increasingly being used.
AR glasses have not really become socially acceptable yet. Magic Leap or Hololens cost several thousand euros and are even slowly establishing themselves in the industry. Google Glass was way ahead of its time, but the features you would want from such a device are correspondingly limited. That’s why augmented reality is currently even more exciting from an auditory perspective. The technology here is already very advanced and, as already mentioned, headphones usually have at least head tracking built in. Combined with the smartphone, experiences with six degrees of freedom are also possible. Applications could include audio guides in museums, where the paintings or statues are brought to life by sound and tell their life stories, for example.

VR (sound for virtual reality)

When it comes to VR applications, most people also think of gaming. This is supported by the fact that such projects are almost exclusively developed with game engines such as Unity or Unreal. However, the VR bubble is much more diverse than you might think. Training and simulations in particular are currently in the B2B industry without consumers realising it. The possibilities are virtually unlimited and, in addition to the aforementioned rollercoaster games for which the medium is mostly known, real use cases are establishing themselves that offer added value – such as saving time and money when training employees. Nevertheless, it is not easy to transfer games, apps and the like from 2D screens to VR. The user experience with HMDs and controllers is simply fundamentally different. In most cases, VR attempts to replicate reality and is just a poor digital copy. However, the immersive medium really comes into its own when you do things that you can’t do in real life.

A prime example is “Notes on Blindness” (is.gd/notes_on_blindness). A VR experience in which you slip into the character of someone who is slowly going blind. Automatically, even non-sound people pay much more attention to the subtle nuances in the sound. But in the vast majority of cases, sound is usually neglected by developers due to a lack of knowledge and time. That’s why most apps sold as immersive experiences sound quite sterile. We are a long way from AAA budgets and need to give this young medium some time.

Spatial computing meets spatial audio

Let’s not get confused by Apple’s new term. For me, spatial computing is the same as XR (eXtended Reality). You are in virtual reality, extending your reality or somehow in between. The boundaries are no longer so easy to separate when even VR glasses have cameras that look into reality. So if I see reality through an HMD, is that VR or AR?
Apple doesn’t make the confusion any easier because they didn’t want to use terms that are already used by other companies. Virtual reality or the metaverse from Meta. Or mixed reality from Microsoft. In the long term, the result will be that we don’t have a VR device and an AR device, but a device that can depict all realities. I won’t comment on how AI and other buzzwords such as blockchain will play into this ;-)
For me as a sound engineer, however, it is important to separate which sound is part of which reality. In VR, I want to isolate myself, so I use a surround sound that matches what the display tells me. Whereas in AR, I want the sound to sound as if something is happening in my living room, where I am right now. Apple is also closer to a solution here than other companies because Apple Vision Pro, for example, also introduced ray tracing, which recognises the geometry of our surroundings and renders the sound accordingly.
In addition, Apple has already built a good infrastructure for 3D audio with the Airpods. If you want to further optimise the sound, you take pictures of your ears, which generates a personalised HRTF. Our hearing as a 3D model, so to speak. This allows the iPhone, for example, to tune the sound for us even more precisely and the distinction as to whether the sound is with us in VR or AR becomes even clearer.

Game audio – playful or gambled away?

Gamers know how important it is not only to see your opponents in time, but to hear them beforehand. For this reason, many people like to spend good money on expensive headsets that supposedly give them an advantage in the game. However, the sound design in AAA games is also very well budgeted and therefore correspondingly complex. A short beep sound is enough for the player to know immediately what is happening in the scene.

Communication and collaboration in 3D audio and with 6 degrees of freedom are crucial aspects for the feeling of presence in social VR. I recently had the opportunity to be in Social VR myself and was surprised at how long I was in there at a stretch. Even though the room was virtual, afterwards I had the feeling that the other person was actually in the same room as me. The cognitive load on our brain is lower because the sound is not coming from just one direction, as is the case with video calls, but a natural conversation situation is depicted.

But augmented audio can also make our everyday lives easier when we are on the move. Everyone is probably familiar with the problem of travelling on the underground using a map service for smartphones. You arrive somewhere on the
somewhere on the surface, but have no idea where to go because the sat nav is confused as to which direction we are travelling in. GPS is simply too imprecise. But if you have a second reference system – in the form of surround headphones that know the direction you are facing – you could simply hear a voice from the direction you need to move in.

Game over with these problems?

We humans have been used to hearing in three dimensions since birth. Now we can finally approximate this impression naturally. It all sounds very simple, but to get back from stereo to the original, an extremely large number of parameters are required. Unfortunately, it is not enough to load an audio object into a game engine and tick the “3D Audio” box. Here is a brief overview of what is required for 3D audio:
Before you insert an audio clip into a game engine, you should ask yourself a few questions. Where did you get the sound from and does it fit in with the other sounds in your library or recordings that you may have processed with EQ? What is the purpose of the sound? Will it run in the background or will it serve as a trigger for special actions in the game? The secrets of game audio have evolved in recent years. Usually the result is to have loud sounds in a 3D world, but they never really work together.
Since you not only give the sounds parameters as to where they are in the environment, but you also move through the world as a character, there are a variety of parameters that you can give your audio object. As already mentioned, a distinction is made between mono and 3D sound and the size of the sound. Another important parameter is the attenuation curve, which regulates how quickly and how much the volume of the sound decreases in the room. By setting the focus parameters and the air absorption and occlusion factors, you can further determine how the sound spreads in the room.
So far so good, but so far the sound still subjectively sticks very close to your face. It has a certain distance, but our brain does not yet know what kind of room we are in. At the moment it is an abstract sound source in an empty room. So it’s time for reverb. An important consideration when creating realistic sound effects using 3D reverb is the size of the room and the material of the walls. Here, too, the calculation is usually only approximate. You would actually need real-time ray tracing to be able to realistically simulate initial reflections and reverberation. However, with the right combination of the parameters mentioned above, you can get quite close and don’t need a render farm.

If the software wasn’t so hardware..

Hard… you know? Anyway, in this context you usually come across the term “game audio”, which deals with the design of interactive audio content. The three new degrees of freedom create two new problems: You don’t know exactly when this player is actually where.
That’s why audio production doesn’t end up with one long audio file with several channels. Instead, many small assets are delivered. These can be loops such as a forest atmosphere, for example, which is repeated in the background until we are no longer in the forest. The second category is trigger sounds, for example if I want to hit a tree with an axe, a “tin” should also come at the right moment.
The only audio format that is still under development and that could depict everything is Fraunhofer’s MPEG-I. But that will take a few more years. However, this will take a few more years, which is why it is mostly found in game engines such as Unity or Unreal Engine. The former is only equipped with very rudimentary 3D audio features. With Epic Games, you can go further. Nevertheless, both platforms quickly reach their limits, which is why it is common to implement middleware for your project. Audiokinetic Wwise and FMOD are popular programmes for this and usually provide everything you need. And if not, you can always write your own scripts. Easier said than done, because here the sound designer has to become more of a developer. With endless possibilities but also complexity.

Conclusion on the large 3D Audio Matrix

To summarise, 6DoF makes it possible to move freely in space and changes the way we experience sound, initially in a playful way. Even if games have somehow been using this for decades, there is a much greater added value than entertainment. That’s why it doesn’t make sense for me to call 3D audio the stereo killer now, as Dolby likes to propagate. I’m starting with what wouldn’t have worked with stereo, true to the motto “Sound First”. In the sound community VDT (Association of German Sound Engineers) and AES (Audio Engineering Society) there is a lot of talk about “immersive audio”, but it’s mostly just about 3D music. And it feels like nerdy details that users don’t understand anyway.
Audio professionals have to ensure that the scene is set to music in such a way that it doesn’t sound empty but also not overloaded. Developers suddenly have to deal with very complex audio parameters that they have to operate on their own more often than I would like. However, the bigger the project, the more budget there is for the respective specialists and games can certainly be taken as a prime example in terms of creativity and technical realisation. It’s an exciting time for the audio world because 3D audio is really celebrating one breakthrough after another in a wide variety of areas. So if you have a project that you need help with, or would like to learn more about this area with my video course, just get in touch!

The post 3D Audio – Into the acoustic matrix first appeared on DIGITAL PRODUCTION and was written by Martin Rieger.

Disclaimer: A Hollow Flame is a non-commercial fan cinematic. Fallout, Brotherhood of Steel, Vault-Tec and related logos are trademarks or registered trademarks of Bethesda Softworks LLC in the U.S. and/or other countries.

Nikita’s family bought their first computer when he was just 10 years old. Like most people living in the 1980s and 1990s, the purchase of a home PC opened up a world of gaming. It was 1999, and the game was Fallout. His infatuation continued, and those who know it are not surprised that it garnered so much praise and has been consistently ranked among the best video games of all time for the past 25 years. After playing the game, Nikita was a completely different boy than before and decided to work as a 3D artist and later as a game designer and game producer. His parents had no idea at the time that their child’s contact with this new technology would have such an impact on his life. When he was thinking about a personal project, he decided to design a tribute to the game.

Why BMD and Blender?

Before Nikita decided on his editing, colour grading and compositing software, he first had to decide how he wanted to create the footage himself. He wanted to create everything in 3D without the luxury of shooting his film in real time. As a hobby filmmaker, he also needed to keep costs as low as possible to make the project financially viable. Fortunately, Blender exists and it was Nikita’s choice for this project.
He only used a small package of branches and some video assets. Apart from that, all cinematic assets (models, textures, animations, mocap, effects) were created just for Hollow Flame – all assets used can be viewed here: bit.ly/hollow_assets

Those familiar with Blender will know that it has a range of compositing features and functionalities. Like Fusion, it is node-based and has a number of features that allow modelling, rendering, comp and post to be performed in the same application. With version 3.5 of Blender, a new feature has also been added to the compositing toolset: a real-time compositor. The fact that compositing can now be done interactively and on the GPU means that the results are almost instantaneous and the efficiency of the creative workflow is greatly accelerated.
However, Nikita decided against using Blender for this part of the workflow and instead opted for DaVinci Resolve with Fusion. DaVinci Resolve and Fusion have a more powerful and comprehensive set of tools that are essential for complex projects. They are also faster and more responsive, which proved to be perfect when editing, colour correcting and compositing shots. Nikita knew that with DaVinci Resolve Studio and Fusion he would have a one-stop shop, and although he had never used Fusion before, he was willing to give it a try, mainly because he had previously found DaVinci Resolve to be very intuitive.
As well as all the editing, colour grading and compositing features, he found the audio editing features of DaVinci Resolve Studio more than adequate. He could have opted for a programme like Adobe Audition, but he found this alternative less intuitive. He also didn’t like the subscription pricing structure and the unnecessarily slow installation process. The ability to consolidate everything into a single application was a big win for Nikita.

Floating Pipeline

Nikita wanted a strong cinematic feel, especially in the area of colour grading. It was important that his files had the high dynamic range required to handle this – for a 32-bit workflow.

For live action filming, videographers can use cameras such as the Alexa, RED or Blackmagic Ursa to record in file formats that support this amount of data. Traditional DSLR cameras like Canon and Nikon can record in a RAW format, which can then be converted to EXR files that can be used in Resolve. In 3D, it works a little differently, but still pretty simple: export directly from Blender to 32-bit EXRs and bring all that beautiful data directly into DaVinci Resolve.

One of the advantages of working in a 3D pipeline is that you can export a lot more data from a 3D application like Blender than just the final rendered image. Nikita took advantage of this by exporting depth and motion blur data. These additional layers to each image were included in each exported EXR file. Once the files were rendered, Nikita was able to bring all of these layers into Resolve Studio to target his grading and post-production effects. Unfortunately, access to these different layers of information is not always possible in traditional live action film projects.

Colour grading

One of the most iconic types of colour grading in the film industry is the contrast of teal and orange. These two colours are particularly popular because they look beautiful in contrast to each other, and Nikita has taken advantage of this. One of the best implementations of this technique in the film is the shot with the industrial fans and the metal bar. The fans on the left and right side of the shots are fiery orange in colour, while the walkway is turquoise. This effect is achieved by pushing the blues into the shadows and the oranges into the highlights. This is something that is very easy to achieve in DaVinci Resolve, and Nikita has used it to great effect.
The debate over whether colour correction is an art or a science continues – some colourists are very much guided by their eye and what feels good for the scene, while others rely on histograms and charts to determine what changes need to be made. Nikita, as an amateur and inexperienced colour grader, did everything by eye.

Using reference images from the original game, he was able to play with curves, colour wheels and other colour adjustments to get something that matched the original game. This is probably most evident towards the end of the film when the monster is bathed in aggressive red light, which is compared to the sickly greenish light that fills the scene outside. This kind of cinematic effect is easy to achieve in Resolve – especially if the artist has spent the necessary render layers to isolate parts of the image.

Post Effects

Fusion in DaVinci Resolve Studio and Fusion Studio are true compositing powerhouses. Nikita used only a small portion of Fusion’s features, but used them effectively. He also utilised a range of post-production effects that took his work to a new level and added layers of depth to his already impressive work. The post-production effects he used were motion blur, film grain, lens distortion, chromatic aberration and a vignette. These effects are easy to add if they are not present in the original material. Rendering these effects in 3D is often very time consuming and when using film cameras it is also difficult to get them perfect.
There are no such problems when adding effects in post-production. Effects can be added or removed and customised for a perfect look and feel. Fusion’s node-based workflow allows for a completely non-destructive choice for any digital production pipeline.

Lessons learnt

The advantage of not having a client was that Nikita could do exactly what he wanted. For hobbyists, these projects offer a high level of experimentation and the flexibility to learn new skills while still in the middle of the project. When a project comes to an end, there are still lessons to be learnt and implemented next time.

Nikita would have chosen to implement a lot more of the post-production effects in DaVinci Resolve Studio with Fusion instead of shooting them with the 3D camera. Effects such as particle effects, volumetrics, steam and smoke can be time consuming and difficult to set up in 3D or on the film set, whereas this is much easier and cheaper in post production.

Conclusion

A Hollow Flame is a rarity: an enthusiastic fan has created a beautiful homage to a game he loves. He skilfully uses Blender and DaVinci Resolve Studio with Fusion to create something truly cinematic. By combining 3D footage, audio, colour grading, compositing and post-production effects, Nikita has created something amazing. His heart and dedication will hopefully inspire other creatives to get their hands on some excellent free tools and try something similar themselves. And now let’s dive into the actual pipeline and creation of “Hollow Flame”!

Resolve/Fusion for Hollow Flame!

Now in the tutorial I want to show you how I used DaVinci Resolve and Fusion to give my short CGI film “Fallout” a cinematic look. I used Blender to create the footage, but this workflow will work with any footage you have created, whether it’s 3D or live action. I rendered a footage in 32-bit, which is ideal for extensive colour grading and post-processing. It provides the necessary data to make significant changes to the colours without unsightly results. Only the best cameras allow output with such a wide dynamic range, but if possible, always record at the highest possible bit rate, then at least you have the best chance of getting good results. In terms of time, I was able to use the Colour tab immediately and make adjustments straight after importing capture sequences. With Fusion, it took a little longer because I’m not familiar with the node system (I mainly used Blender nodes), but I quickly got used to it. After that, colours and post-processing were done in a few minutes. In this tutorial, I assume that the footage is already imported into DaVinci Resolve and ready for colour grading. It’s important to realise that the workflow in DaVinci Resolve is perfect for experimenting with different ideas. With good hardware, the results can be seen quickly and you can make comparisons. I encourage you to play around with the parameters and controls to get a good feel for what’s possible.

Step 1: First steps in colour correction

Now that the files are imported, we can start colour grading. I’m not a professional colourist, but it’s pretty easy to make basic adjustments. You can use colour wheels to do this. First try changing “Contrast”, “Pivot” and “Shadow/Highlight” to adjust the contrast, highlights and shadow intensity. For basic colour correction, use “Log Wheels” to change the colours of the shadows, midtones and highlights separately. Then you can use “Temp” to apply a colour filter to the entire image
Image. Later you can use more sophisticated methods such as colour curves, qualifiers and colour distortion for detailed corrections.

Step 2: Save colour edits

You can save your edits for later use by using the “Still image” function. Simply right-click on the sequence preview and then click on “Still image”. The preset will be saved in the gallery. You can simply drag this preset onto the preview to apply it to the sequence. It’s worth noting that you can experiment with colours on the “Colour” tab without worrying about messing up the image. Simply create a new “Serial Node” and make any changes here. If you delete this node, the standard sequence will still be available to you.

I’m a hobbyist and a novice when it comes to colour correction/grading, so I used my knowledge of colour correction in Photoshop and optimised the colours by eye. To do this, I used “stills” to test colour correction “samples” I had created, but ended up using a LUT as a base, correcting them for brightness/contrast and adding some masks to highlight some important parts of the shots. I worked with a very specific colour management profile, “Blender Filmic”, because I rendered all my shots in Blender with this profile. So before colour grading, I tweaked DaVinci a bit and installed the Blender Filmic
LUT to make sure that the colours in Blender and DaVinci matched.

Step 3: LUTs

LUTs (or Lookup Tables) are sets of predefined colour values. They are easy to use and can give the sequence a specific look in seconds
in seconds. However, it can happen that you spend too much time searching for the “perfect” LUT and end up using some favourite preset that makes the video look very “standard”. I used the LUT “Matilde – Rec709” as a base preset and optimised each clip. You can easily install and use all these LUTs by placing them in the “LUTs” folder on the “Colour” tab.

Step 4: Compare LUTs

You can add different LUTs to different serial nodes and compare them by using the key combination “Ctrl D” to deactivate the node and
to see a different image and adding new serial nodes to optimise each LUT the way you want. You can use colour wheels or curves to edit the LUT, or simply mix two nodes with different LUTs applied – it’s up to you,
the software gives you many options to work with colours.

Step 5: Shot Alignment

We’ve sorted the shots and we like them, but we still need help with the overall consistency. Some shots look too light or too dark. That’s why we need to
align all the shots together. I did this “by eye” with the help of clips in the “Colour” tab. The clips show all the shots of the video in one place, which is
useful – by taking a quick look at them, you can already spot inconsistencies and optimise individual shots. In my case, I needed to emphasise some bright areas. To do this, I used a simple mask, added a “Parallel” node and created a simple mask with the “Curve Mask” tool. Then I edited the node to increase the intensity of the light from the arc.

Step 6: First post effects

Now that we’ve edited and graded our video, what’s next? To make the look of the video even more cinematic, I added some effects that simulate an old, analogue film look. To do this, we need to go to the “Fusion” tab and use special “Fusion” nodes. I used the “Film grain” node to add some grain to my video. Then I used “Lens Distortion” to simulate the imperfections of a real camera lens. In addition, I used “Chromatic Aberration” to simulate a slight colour channel deviation.

Step 7: Even more pose effects!

Following on from the previous step, I scaled the image a little using the “Transform” node. Finally, I used “Vignette” to slightly darken the edges. All these nodes are available in the “Effects” list on the “Fusion” tab. Each node has its properties that can be easily customised, so the result depends entirely on your taste and vision.
And – this also applies to chromatic aberration – I believe that subtle effects create a more believable, natural image. It’s easy to overdo the effects, so I prefer to add just a touch.

DaVinci Resolve is nimble and perfect for quick editing. The nodes of the Fusion and Colour tabs are different entities and work independently, so it’s better to split the colour correction and post effects into different stages.

Step 7.5: Motion Blur

Another post effect I added was motion blur. This one was the most complicated to set up, but a quick note: Normally, motion blur takes a lot of processing power to render directly from the 3D software. I was using the EEVEE renderer in Blender, and although EEVEE is a very fast renderer (almost real time), rendering motion blur takes quite a long time. I therefore decided to add the motion blur only in the comp.
I used Cycles for the motion blur, rendered the V-Pass in Cycles and used it in Fusion to get the motion blur with the “Vector Motion Blur” node, but then used the “Optical Flow” node for most scenes instead because the result was better. I tried to render the motion blur in Eevee, but each frame took ~40 minutes, which was too long (considering I have more than 3500 frames). Basically, I used Eevee instead of Cycles because I wanted to create a good looking cinematic with a real-time renderer. And of course, because Eevee is much faster than Cycles (well, was, until the current time).
I ended up rendering a separate vector layer in Cycles and saving it as a sequence in MultiLayer EXR format. Then I imported the sequence into DaVinci Resolve and added it as a parallel input to Fusion. To use the layers in Resolve (they are called channels here), you need to select the sequence node in Fusion and click on “Inspector” to open the node settings, go to “Channels” and assign the vector layer to the speed channel (both X and Y). Then both inputs need to be routed through the “Vector Motion Blur” node (also in the effects list) to add motion blur to the shot.
This is one option, but there is another. With the built-in functions, you can add motion blur much more easily. Instead of rendering a vector layer and blending it with the sequence, you can simply add the “Optical Flow” node and connect it to the “Vector Motion Blur” node, and voilà – you have motion blur in your shot
blur in your shot! I used the optical flow in almost all of my shots and it saved me a lot of time.

Step 8: Final check

After all these steps, it’s time to render the video. All output settings can be found on the “Deliver” tab. Here you can set the file format, output quality and file path. Resolve offers several presets I recommend rendering your video in the lowest output quality first – just to check that everything fits. I rendered my short film several times in low quality, and each time I found minor errors, fixed them and rendered it again. To speed up this process, you can render individual clips or a custom area to check specific parts of the video.

Step 9: Output

Okay, we’ve checked our project and found no problems. Let’s set it to high quality and render the final result! Set the settings, click “Add to Render Queue” and then click “Render” on the right side of the screen. That’s it! With DaVinci Resolve and Fusion we can cut, colour correct, apply post effects and edit audio in a single package. The fact that it fully supports a 32-bit workflow makes it an unbeatable tool.

The post Fallout becomes cinematic. first appeared on DIGITAL PRODUCTION and was written by Bela Beier.

by Jonathan Braun (Head of Data), David Schaefer (CEO), Dmitry Glazyrin (CEO) and Zhanna Travkina (Creative Producer)

Creative CGI studio Mondlicht Studios invites you to take a closer look at how to get things from the drawing table to your framebuffer without breaking your pipeline.

Step 1: Getting engineering data

The work starts right after the client contacts you about the project, because the first step is to ask questions or otherwise you might end up burning the midnight oil in misery. It is pretty obvious that you need to ask about the scope and that creating one nice still image is easier than setting up a pixel streaming configurator, but besides that, you really need to know what kind of data the client is offering as an input – and what quality the data is providing. Let’s talk about CAD-files first.

If you are dealing with a small company, you probably can get in touch directly with the person who creates the CAD-files, which is always a good idea, because they can make your life a lot easier. The larger the company, the more departments will be involved, up to the point where every-thing might be stored in a company-wide product-data-management (PDM) system while the creator and the user of engineering data don’t even know each other. In that case you usually send your contact searching for a product manager, which himself should know who to talk to in various departments cascading from top to bottom. Keep these lines of communication in mind, when planning deadlines and writing requests, because inquiries passing through large companies can eat away all your buffer time and the person pushing the “export”-button might never have seen the briefing.

Having found someone willing to help, what do we even want? Similar to the CGI software landscape, there’s a few big software companies dominating the engineering software market and some less known solutions, but luckily there are a few exchange formats that almost any of those software suites can export and that you can handle. For example, STEP and IGES (*.stp and *.igs) are go-to file formats you can always ask for when the client will share 3D-engineering data. You will probably not have CATIA at hand, so ask for a proper export.

Another valuable tip: a commonly used software for data preparation inside companies is Autodesk VRED. If the client wants to send you *.vpb files (VRED Binary), you will be able to open these files only if you have ‘’the big VRED Professional’’, which is as of now only cheap at 1.805 USD monthly (or 14.440 USD annually). There is one “small” version of VRED (VRED Design), but it doesn’t open *vpb. The easy solution here is to ask them to export a *.fbx file from their VRED and share that with you.

Back to the data itself and the first pitfall: Do you want parts or an assembly? That is one question your contact might ask you, so it’s important to understand what he means. Basically, he is talking about what you might refer to as “references”. If a draftsman creates a screw, he will most likely position it in 0/0/0 of his 3D-space for general use and everyone who wants to utilize this screw will position it himself at the needed position of his contraption. You could just copy-paste the screw and the other parts creating one heavy file with your contraption, but an engineer most likely wouldn’t approve of this method. Engineering is an iterative process and handling changes is one of the most important aspects of the job.

What if you find out that a new legislation forbids you from using that exact screw? You would have to manually find, replace and reposition all screws in your file, resulting in tedious labor. That’s why engineers usually create so-called “assemblies”, linking part-files into a meta-file, only containing transformations and other information, but no geometry data itself. Changing the screw will then automatically update all its uses in the assembly. Why does that matter? If your client just dumps you a pile of parts, you might lose all positioning and you might end up re-assembling the data part by part, which is not something you want to do. And you might overlook that he also delivered a tiny file containing all the “missing” information.

If you want to be safe, you should ask for a parts-export from the assembly. That means taking all the linked geometries and information from the assembly and putting it back into one single file with all transformations applied. If the product is too large or too complex for that, your contact will probably suggest to split this into several files, which is alright, because you don’t want to watch your computer taking an eternity, trying to tessellate a 10 GB part file containing a complete combustion engine.

Step 2: Tess-uh-what?

CAD-files are based on NURBS surfaces (unless you received a *.fbx export, which is not NURBS, but triangulated meshes). NURBS are mathematically accurate sur­faces which do not have a set mesh resolution. There are different kinds of NURBS data, but that should suffice for now. You could try to import this CAD data directly to the 3D application of your choice, so to say importing the NURBS surfaces as NURBS, not converting them into triangulated meshes. Many will succeed in doing so, but you will notice pretty fast that most of your workflows and pipelines are made for polygonal data and will fail on these NURBS surfaces.

Converting CAD data from NURBS to polygons is called “tessellation” and is a common step for engineering to CGI pipelines. In theory, many of your 3D applications are able to do it, but most of them deliver bad quality, confusing settings or will slow you down with poor computation or annoying dialogues. The best way to go is to decide for a dedicated tessellation software of your choice like “PiXYZ” to batch tessellate your client’s input and export a fbx-file. There are many more, but we find this provides good quality, speed and is affordable. If you are not planning to invest in such software, you can use Unreal Engine for the job, which delivers decent results and the possibility of *.fbx export. Cinema 4D also has a decent tessellation algorithm. There is not much you need to know about tessellation apart from a few parameters that control the resolution of the output, the most important ones being maximum chord (length) and angle deviation. For both, lower numbers mean a more detailed result.

Step 3: Tailoring the data according to your needs

So just set both to near-zero, right? Not quite. Adjusting the tessellation settings can make a huge difference regarding the poly-count and file size of your project, so these values should be chosen wisely. If you are planning to deliver 14K still images showing product close ups, you of course need pro­per mesh resolution and don’t want your shapes to be cornered and chiselled. But even then, it might be wise to use a mid-resolution setting for most parts and just use higher values on some highlight shapes. Investing some time to find values your artists feel comfortable with is worth it, as excess poly­gons will slow you down heavily later in the process. Try to keep everything as lean as possible.

Talking about excess polygons: When processing engineering data, keep in mind that it’s not specifically designed for visualization purposes. Engineers store a lot of information in there, that might be relevant for manufacturing or crash calculations, but not for you, resulting in additional points, curves or geometries that you do not need. Deleting 400 spheres representing welding points or an additional pre-processed engine block without drilling holes can save you millions of polygons. On top of that it will also make your product depiction correct.

If your project aims for real-time or gaming applications and you need to reduce even more, you have two options, both valid: Go for high resolution and try poly-crunching, retopology or remodelling to create light meshes based on detailed input or you could try to tessellate the data with coarse settings to achieve low-poly meshes directly from tessellation. This can be especially helpful for complex shapes you don’t want to remodel but need to be present in the background or out of focus.

Depending on your input and the tessellation application you are using, you might want to take a look at some things before moving on. It is common in engineering to create simplified, airtight bodies from assemblies for further processing in digital packaging. You might want to call these things placeholders. Engineers need them for accurately crafting the models. They need to know where things are, even if they just get an exact shape, not the detailed
part itself. This is common with OEMs who receive data from various different suppliers. But this means multiple objects are baked into one NURBS object. If you are lucky, the engineers kept the materials from the original objects, enabling you to separate them by material after tessellation either directly in the tessellation software or later in your 3D application. If you are not that lucky, you have to select the objects manually. The worst case is that everything is merged together.

Then the only way is to select and extract surface patches (connected meshes inside the big mesh) from the tessellated mesh. Don’t worry, if you are not familiar with the term “patch”. Tessellation algorithms usually keep the object layout from the CAD input. This means for example to define a cube in CAD, 6 nurbs-surfaces will be stored in one object. After tessellation, you will have a mesh containing 6 surface patches, no matter how many polygons within the patches are contributed by the tessellation settings. But what’s the difference to a cube you can create with your 3D-software? The difference is, that the outer vertices of this “CAD cube” are not welded together, enabling you to easily select and extract the original surface-patch in your production software. You can select them in almost any 3D software by double clicking a triangle. It will expand the selection to the most outer boundary of the mesh, thus selecting the complete surface patch. Maya pro tip: Apply a planar UV projection and then switch to component mode (= select faces/polygons) and then use the SelectMeshUVShell function. Super fast, makes a very unpleasant task almost fun…

If your pipeline requires you to merge vertices at some point, it is recommended to do this after you have finished shading your assets or have product correctness approval from your client, because otherwise you will go back, split and re-import a lot. Merging vertices only improves performance by a very tiny bit. So, it’s better to just keep unmerged vertices. Unless you want to create nice UV layouts. But even then, it’s possible to keep them. Duplicate the mesh, make your UV magic and transfer the result back to the original mesh. The vertices are the same, so that’s possible without any issues.

Let’s talk about shading, because we just touched UVs. By default, CAD-data doesn‘t have any proper UVs and tessellation software usually applies UVs per patch and sometimes even random multiple UV sets for one part. So, in short: The UVs of enginee­ring data are a mess, unless you actively change that.

Working in CGI, you probably know your way around UVs to some degree, but the unique properties of engineering data can make this task quite challenging. There are no nice edge flows you can select for cutting and sewing on triangulated meshes and you probably need to sew a lot of shells first, before you can think about actual unwrapping or fine-tuning. Take your time to devise a method that works for you and your pipeline. Time invested into creating a UV-clean-up-script for your production probably pays off pretty fast. You can, of course, just go with triplanar-mapping for most objects, if your project allows it, but cleaning out unneeded UV-sets helps to optimize your scenes. It won’t speed up or dramatically decrease
the amount of data, but it makes handling the meshes much more comfortable.

Before spending any time on rigging and animations, you should check the hierarchy and object naming of your engineering data. Especially real-time engines are sensitive to scene-tree depth and are likely to crash or slow down when importing engineering data without adjustment. Tessellation algorithms have a tendency of creating long mesh names and numerous layers of hierarchy derived from the CAD file and internal translation which can result in complex parent-child relations that you usually don’t need. Those who have encountered this know how crazy this can get. 100 me­shes and thousands of groups. Unparenting, regrouping and restructuring is advisable, also clustering objects in groups that are suitable for your animation and rendering requirements. Also merging objects toge­ther in a reasonable way helps a lot. But always be careful if you “destroy” the hierarchy. If changes come later, you have to update things manually.

Another thing that can slow you down is the sheer number of objects in your scene. Engineers and designers are not concerned with frames-per-second or amounts of objects. “Somehow feasible” is good enough for them. They even need it, it’s their sophisticated workflow. It has a reason. But this can result in data containing a vast number of small objects and fragments like ball bearings in a gearbox or even single black print dots on a windshield, sometimes CAD-modelled seams and stitching (it’s rare, but it happens). It is advisable to combine as many of those like-shaded objects into one as possible, keeping certain necessary separations for animations and Z-buffering in real-time in mind.

Also, take a look if you imported data types that you don’t need. Engineering data can contain lots of curves and visibility-layers that often survive tessellation and that you in most cases have no use for. Deleting them helps to maintain the readability of your scene and avoid cluttering your GUI with cryptic stuff.

If you still need to reduce file size or require more performance, you can think about deleting unneeded triangles from meshes, trimming them to the visible sur­faces so to say. This is a very time-consuming process when done manually and heavi­ly relies on smart selection methods, like patch selection and sound understanding of angle-dependent visibility. If you do not have that kind of manpower, you could also consider using ray tracing based optimizers that shoot visibility rays from different viewpoints, deleting all triangles that are never touched by a ray. Bottom line: If you have clean data to work with, it will be a joy. If you don’t, it won’t.

Step 4: Candy-crush screws and bolts

Having removed all the junk from tessellation and conversion you now probably ended up with a 3D scene containing bits and pieces coloured in all shades of the rainbow, lacking a certain degree of realistic look and feel. Although some things regarding that topic have changed in the last few years, engineers usually don’t care about realistic shading when working with CAD-files. They focus on good recognizability in assemblies applying as many different colours as they can. But are they completely random?

Usually not. Often you can recognize identical physical materials by their common colour, for example all steel parts being turquoise or cast iron parts being purple. Plain red and yellow however are to be approached with a certain caution, as they are often used not to indicate material, but recent changes on a geometry. Asking for reference images from the customer is probably unavoidable, but taking engineering shading into account can save you a lot of time applying your viz-materials. If you are the curious type or your client fails to obtain any references, you could try taking a look at the CAD files yourself using a free CAD software like FreeCAD investigating the CAD materials. Often the CAD files or accompanying drawings contain detailed descriptions of materials like “St.8.8 Zn-Ni BLACK” that can be easily googled for reference.

If you are planning to create real-time applications, you probably want to merge as many materials as you can without losing too much product correctness. Again, try to combine as many geometries of the same shading as your rigging and animation
allows, as it will greatly improve computation cost.

Step 5: Your actual work

Congratulations, you finally get to do the actual creative work. How hard your journey was until this point depends very much on your client and his data hygiene as well as his ability and willingness to get in touch with the engineering people. The good news is, many of the steps aforementioned can be automated or scripted to some degree, usually doing some python scripting. But it is important to know what’s happening and why it is happening. For some jobs, processing the data won’t be enough and you will have to get involved in soft part modeling. For engineering, a rough representation of a motorbike bench or steering wheel is good enough for their purposes. Sometimes you might only get the inner part on which the upholstery is applied. To CAD realistic fabric stretching, wrinkles and stitching are details which do not matter and that you will have to add manually based on references, 3D curves, drawings or any other information. Ask for every bit of information the client can get on these parts and if possible
request for a reference photo shoot.

If your workflow allows it, try to keep one “master file” representing your single point of truth for the project to avoid local deviating versions. As mentioned before, engineering is also about handling change, and you might have to replace assets in the middle of production due to an update. This helps also to manage the lifecycle of products which might change over time, blessing you with image update requests. Thinking about future projects or pitching, master files are also very helpful as a starting point for cross-platform derivatives. If you have a clean Cinema4D scene set up, data smith import to Unreal is pretty straightforward and reproducible, even after you changed your cinema scene.

Bonus step: You can get visualization data from your client

Sometimes, clients offer to deliver their own visualization data as an input, so that’s the data prep jackpot, right? Again, that strongly depends on the project you have in mind. If the job is to create a couple of fancy still images you will probably be fine and the worst thing that can happen is that you have to recreate shaders for your own render pipeline. Sometimes you get horrible input which you have to touch a lot anyway, but sometimes you get a readily shaded, fully textured master scene in your choice of 3D application. It did happen to us sometimes.

However, if the client wants you to create a real-time-performing dataset, caution is advised and you should probably ask for some sample data before taking the job. But what’s the risk? Especially large companies tend to optimize processes with certain automations and go for approaches that somehow work for most things but don’t always produce clean data. This can mean that they tessellate everything with slightly raised values leading to insanely large polycounts or this can result in some LOD-pipeline delivering nice polycounts but weird topology and hierarchy that can be really nasty to handle. If performance is key, don’t hesitate to ask for the original CAD files, as it might be much easier to re-process the data than trying to get someone else’s highpoly assets to match your purposes.

Summary: If you know why, you will find out how

Everything completely depends on the exact job. Real-time or high resolution still images, animation, or everything in one project. The best practice is to analyze what the deliverables are and how to get there the best way. Not blindly running and preparing data,
noticing later, you’d need a totally different approach. Even if you run into problems and don’t know how to repair a mesh or even get it tessellated in the first place (some broken parts just don’t work), knowing what’s behind all this will help you to overcome these challenges. We hope this was enlightening to you and will help you to work with engineering data of your clients. Godspeed!

About Mondlicht Studios

Mondlicht Studios is a CGI Studio located in the heart of Germany. Boasting a significant background in the automotive sector, their ex­pertise is a testament to their deep roots and understanding of the nuances in this industry. The foundation of the company is built on the rich experience of their co-founders, each bringing close to 20 years of professional expertise to the table.

Surrounded by a small team of veterans and professionals the team is delivering award winning content, providing aid and helping clients to step up. Among these clients are Lucid Motors, LVMH, Nissan, Panerai, BMW, Netflix, Amazon, Netflix and many other renowned brands.

Over the years, they‘ve been honored with numerous awards including Promax Global Excellence and Clio Entertainment Award. The team‘s dedication to excellence has also led them to be named among the top 200 digital artists in 2021, 2022, and 2023 in the Lürzer’s Archive.
Mondlicht Studios has also been celebrated multiple times at the Motion Design Awards
and in 2023 became certified service partners of Epic Games. Beyond their achievements, they believe in giving back to the community. Members of the team have had the honor of serving as judges at renowned platforms such as the NY Advertising Awards and Promax North America.

mondlicht-studios.de

The post Engineering data to CGI –what could possibly go wrong? first appeared on DIGITAL PRODUCTION and was written by Mondlicht Studios.

Before we start moaning about budgets, the main reason is that I just really like grading work and like to put a lot of attention to detail into it that could only be replicated in grading sessions for high-priced commercials. There are several reasons why colour grading has emerged from the black box in recent years: On the one hand, the greater prevalence of cameras that record with a logarithmic gradation curve means that you are forced to edit your material. Some are satisfied with the manufacturer’s conversion LUT or another LUT from one of the many providers, usually applied directly in the editing programme.

This is not necessarily a bad thing. Others, however, whose aspirations and creative vision go beyond this, reach for the (temptingly free) version of DaVinci Resolve from Blackmagic (DR for short).

There are thousands of tutorials to get you started and beyond, many freely available on YouTube, others hidden behind a paywall (such as LowePost, Mixing Light and others). I took the more traditional research route and interviewed four senior colourists (see box). They were very open and generous in sharing their knowledge and wealth of experience from many years in this segment with me. Many thanks again at this point! Behind all these efforts and weeks of research was the idea of bypassing the noise and trends of the internet and talking to people who actually grade every day and full-time.

From Dietramszell to Mumbai

Our panel of colourists is spread across the world, working on different projects and formats. Some of them use Baselight, but most of them use Resolve – so they’re cooking with the same water as we are. That’s also part of the objective: to see what goes on behind the doors that lead to the dark grading rooms. Although they use the same software, importing, grading and exporting as we do, there are crucial differences.

The first big difference between the four experts and us “normalos” is that they mostly work with good to excellent material. Not everyone would agree with this (there is also faulty material or archive material), but a production that can afford high-quality external grading usually also has extremely good source material available.

The second distinguishing feature is that everyone has such a practised eye that a lot of things happen very intuitively. As intuition and routine are difficult to explain, it was always a challenge during my research to penetrate this in order to be able to convey it in an understandable way.

Sebastian Göhs
Sebastian Göhs is a freelance senior colourist from Berlin and works mainly on feature films, TV series and commercials. Most recently he worked on the Amazon show “Der Greif” and Ilker Çatak’s feature film “Das Lehrerzimmer”. Sebastian began training as a film and video editor at Geyer Berlin in 2002. After graduating, he moved to Digilab and advanced to Senior Colourist. This was followed by the development and management of the image department at Rotor Film and finally self-employment. www.sebastian-goehs.com

Stefan Andermann
After training as a film and video editor at ARRI Film and TV Services, which lasted three years, Stefan Andermann worked in the dailies colour grading department for around two years, but was quickly given the opportunity to work in fine correction
but was quickly given the opportunity to take on fine corrections. In the following 12 years, he worked in commercial grading at ARRI. He has specialised in feature films for around seven years and works at Pharos on cinema productions, for streaming services and TV productions. www.pharos.de

Andreas Brückl
Andreas Brückl started as a colourist at Bavariafilm in 2005 and worked as a freelancer from 2010. After four years in the advertising industry in Istanbul and a stopover in Malaysia, he moved to India in 2017. After four years of chaos and colour saturation in Bollywood blockbusters, he moved to Dubai for two years. He has been back in India with DNEG as a supervising colourist since 2023. www.baselight-colorist.com and www.vimeo.com/colorgrading

Florian Wolf
In 2003, Florian Wolf began his career as a career changer in post-production and telecine at Pictorion das werk. He worked there as a junior colourist until 2009. From 2009 to 2011, he moved to ARRI Commercial Munich and was promoted to Senior Colourist. After a brief intermezzo at das werk Munich, he took the plunge into self-employment and has been working as a freelance senior colourist since 2011. In addition to colour grading, he offers the production of commercials, image films and music videos from concept to shooting including the entire post-production. www.flowolf.info

Both great material and routine naturally make grading easier. However, the resulting higher speed is also necessary, as more and more is demanded of this trade for technical reasons. For the US American lawyer series Suits, with around 770 shots (to be seen in the edit page in the Index section) per episode, one and a half to two working days are scheduled (as reported on LowePost). And even established grading studios in our country often require 500 settings per day. This also cancels out the advantage of high-quality shot images, as such an incredible amount of images have to be processed.

Target group

The aim of this two-part series of articles is therefore not to breed more professional colourists. Their routine and experience can only be achieved with practice. However, we can utilise their approach (best practices) for ourselves. This article and the following one on the subject of beauty retouching are intended to appeal to those of you who do not have an external grading service in your budget, but who nevertheless have high expectations of the end product and already have a certain amount of advanced knowledge in the use of Resolve.

As the recommendations from the various sources sometimes differ widely, I am acting as a curator. I have summarised all the information and my knowledge from expensive training courses (2011 Resolve Colorist Advanced), free tutorials, paid online courses and detailed discussions with the experts with a view to this objective to create a guide that is as concise as possible.

Colour management

Here the opinions of the professionals differ: from completely unmanaged to ACES to the use of secret company-internal LUTs, the spectrum is wide. I would recommend a colour managed workflow in Resolve, i.e. select “DaVinci YRGB Color Managed” under Color Management in the Project Settings. The default selection “SDR” and “SDR Rec.709” in the dropdowns below is certainly also suitable for many of you. Some tools (such as the HDR Wheels) are now “Colour Space Aware”, i.e. they react in an intuitive way. In rare cases, an unmanaged workflow might be easier for certain looks, some say, because you don’t have to work against the steep gradation curve. In my opinion, however, this is rather rare and is more than compensated for by the speed advantages. Colour Space Transform Nodes lead to the same results with the correct settings for each camera, but do not make tools colour space aware. In addition to ACES workflows, which require a relatively high level of prior knowledge, I would also advise against using LUTs for colour management – apart from look-specific colour mappings, they offer few advantages.

Balancing

Primaries or balancing, i.e. the equalisation of colours and brightness to achieve a natural and balanced image impression without colour casts, is an essential step in colour grading. All the experts agreed that the very first thing to do is to balance images within and across a scene. All further steps are based on the consistency this creates. If the balancing is not right, you get yourself into hot water later with looks and secondary adjustments (qualifiers, masks, etc.).

But balancing and also shot matching (balancing across several settings) is actually a relatively difficult matter and requires a trained eye and a lot of experience. Especially when you have a high workload. What I have realised is that professionals approach this step almost instinctively and often have difficulty explaining their approach.

However, it has become clear that the use of “simple” tools that tackle the whole picture is an important point. There are several candidates for this. The professionals almost always use lift, gamma and gain on their panels. That’s how I learnt to do it and this is a valid and particularly “gentle” method. For the inexperienced, however, there is a risk of getting colour casts in different brightness ranges. This is because the tonal ranges of Lift, Gamma and Gain overlap so much compared to the LOG wheels. Adjustments in dark areas always influence the medium and light areas to varying degrees.

As we work in a “YRGB Colour Managed” workflow, my first candidate for a recommendation would be Temperature and Tint in the Global Wheel of the HDR Tools. These are “Color Space Aware” due to our chosen colour management and (according to the Resolve manual, p. 2901) “react photometrically correct and produce results that are similar to the human perception of colour temperature adjustments on set”. If you combine these with pure brightness adjustments in normal lift, gamma and gain, you are already doing quite well.

Offset is an alternative. Many panels have an extra ball for this and even MidiGrade (see DP 22:06) supports it as standard. Offset changes the entire signal, so you don’t get any stitches in individual brightness ranges. These adjustments, formerly known as “Printer Lights”, can even be mapped to the keyboard or, even better, a (separate) numpad. And in whole, half and even quarter printer points.

Once you have a well-balanced image and, above all, no clipping, you can move on to shot matching. Here it can help to sort the clips in the order in which they were recorded instead of how they appear in the final cut. To do this, select the “Source (C Mode)” mode under View -> Timeline Thumbnail Mode. Of course, this only changes the view and not the actual order of the clips.

Prefabricated node trees

Fixed node trees are a topic that I only came into contact with during this research. I was used to creating more or fewer nodes depending on the requirements and the shot, and deciding anew for each clip. Of course, that took a lot of time. If I knew beforehand that certain work steps would be repeated, I packed shots into groups or simply copied nodes, but that was it.

Fixed node trees are supposed to help you save time by having the same prefabricated node structure on every clip. If you follow this concept, there are three advantages: The first is, of course, the overview. Even after many weeks, months or years, everything is still traceable and collaborations also have the potential to run more smoothly.
The second advantage: As certain editing steps are always on the same node, it is possible to transfer them to different clips via node rippling (under Colour -> Ripple Node Changes to Selected Clips). It should be noted that this cannot be undone, as Resolve’s Undo History follows its own rules (for more information, see the official manual p. 489). If you want to be on the safe side, simply save before each ripping and load the last save if necessary. It can help to deactivate the “Live save” option in the programme settings under “Project Save and Load”
“Live save” option in the programme settings to avoid automatic saving after rippling. Rippling between active nodes is consistently possible, but sometimes nothing happens when rippling between nodes that are enabled/disabled.

The third advantage of fixed node trees is that you can build prefabricated modules. A simple example would be a vignette node that already has a mask in the right size and a slight darkening in the mid tones. I need something like this more often and I don’t particularly enjoy creating it every time. It’s the same with many nodes. In addition, you might just want to try out certain adjustments without having to rebuild them every time.

I had an insight into the fixed node trees from Walter Volpatto, Mark Todd Osborne (both via LowePost) and Sebastian Göhs and created my own for my requirements, which I would like to go through with you below. It has become relatively comprehensive, which is not necessarily the case. Of course, this is only a suggestion; you will certainly have different requirements of your own for your projects. However, the explanations of the concepts and considerations that have gone into my node tree may help you to create your own.

Let’s get started: The first node only has an aesthetic function, namely to bundle the inputs of the following three parallel nodes. If Resolve had something like pipe routers in Fusion, which could be used to organise the connections between the nodes, this node would not be needed. If we also had the underlays (organisational aid for visually grouping several nodes that belong together in terms of content) from Fusion at our disposal, the next three parallel nodes could be grouped together thematically under the term “Highlight Recovery” – as can be seen in the title image of the article.

This is about darkening the highlights before the general balancing. Some people put this in a pre-group, but most professionals agree that this step in the node tree should always take place before balancing. This does not refer to the order of work, as it can make sense to carry out balancing first and deliberately clip the highlights slightly in order to achieve a nice contrast, and then use this recovery to push them back into the legal range in the next step.

Two of the three highlight recovery nodes use a very soft qualifier as a method, which is already combined with reduced gain. As the parameters in the two nodes are set differently, I can quickly try out two variants by briefly activating the nodes. The method of the third node is the most aggressive and uses a sat vs. lum curve that slopes from left to right. Another conceivable variant could be to lower the highlights in the log wheels. This is usually too far-reaching for my standards, as other tonal values are also included in addition to the actual brightest highlights.

The next block of three nodes is intended for balancing. If, as some people suggest, I want to adjust the contrast independently of the colours, two labelled nodes are already available for this purpose. In order not to confuse the numbering of the nodes during grading by subsequently inserting nodes, empty nodes are repeatedly inserted into the node tree at different points, which can be used as desired. The third balancing node is also intended for this purpose.

This is followed by a group of parallel nodes with frequently used mask shapes, which could be summarised under the term “light shaping”. In my case, this would be a narrow, soft mask to emphasise an important image element, a vignette with a corresponding outer vignette, a spectacle-shaped mask to sharpen the eye area, a gradient from top to bottom to adjust the sky (or if rotated to darken the left, right or bottom) and two empty nodes if a shot calls for something more. In all these nodes, I only adjust the brightness, not the colours, because if I need to balance the colours in the balancing nodes immediately afterwards, I don’t want to mess everything up here. It’s a matter of taste whether you pre-set the reduction (or increase) of brightness in the individual nodes.

Windows with blurring would of course also be conceivable at this point in the node tree for focus control. This is followed by two nodes for the look. It is often recommended to only use the colour wheels here to make it easier to ensure consistency across multiple clips. If both the balancing (of the individual clips) and, above all, the shot matching (across the scene) have been carried out well, it should now be reasonably easy to transfer the look to other clips via node rippling and then only have to make minor adjustments.

One more tip at this point: If you need ideas for looks, you can simply put together a collection of (free) LUTs from the Internet, throw them alternately onto a node and reduce their effect with the gain control in the key tab if necessary. This allows you to try out many looks in a short space of time.

The second look node is also optionally available should you need it.
In the second row, the first node tidies up the inputs again for the sake of aesthetics. This is followed by three nodes that are specifically related to my image film productions. In the first node, a reduction of the low-saturated colours in Sat vs. Sat is prepared, which I have called Clean White. As I often produce films for the healthcare sector (many white image elements: white rooms, white clothes of the medical staff etc.) and also have to deal with mixed light again and again, I don’t have to build this very useful tool again and again.

The second node contains a glow, simply to be able to try it out quickly. I called the third node in the group “Pop” and it contains a simple S-curve. As I am sometimes a little too cautious with the contrast, I can quickly try out whether a little more oomph is possible here.
More generally, it makes sense to create nodes in this area of the tree that specifically match your own workflow. For example, halation, flares, skintone keys, preset third-party plugins and much more could be useful here.

Next comes a block whose function is still open. After all, colour grading is also a learning process and so we may not know what nodes will be added in general or specifically for a project, but at least we have provided a place for them. So there is room here for everything that has not yet found a place elsewhere. For many professionals, these are nodes that take into account the aesthetic wishes of other trades (such as DoP, director, etc.) or are project-specific, such as keys to certain parts of the set, film simulations or others. How many nodes are required here is of course completely individual.

Regarding the last block: Spacial and/or Temporary Noise Reduction is placed at the end instead of the beginning of the node tree. If individual qualifiers have inadvertently caused noise, this may also be picked up by the noise reduction. Of course, if you do everything right, this won’t happen, which is why the pros always have the noise reduction as the first node – of course, you can also do it this way.
This is followed by Sharpening on a node, set to a value of 0.47. There would be no objection to placing both in the post group or the entire timeline, but this way the individual parameters can be adjusted on a clip basis. At the very end of the tree, two empty nodes are again prepared in case, for example, general requests from the customer or a team member need to be included at the last minute.
Finally, I would like to point out that not all professionals use ready-made node trees. Some create all nodes as required and for each new project, while others only have a long series of serialised nodes as a template. However, I find the advantages of a fixed node tree in terms of speed and structure to be enormous.

Giving boring images more punch

We often come across shots that look dull, flat or simply boring. But I have also been able to put together a few tricks for this case. Many colourists tackle this issue by directing the viewer’s attention to a specific image element. Very soft masks with a slight darkening or blurring are often used for this. Sometimes the direction of light is taken into account, but often physical logic is completely ignored, which is usually not noticed by anyone due to the fast editing rhythm. Only inaccurate tracking of the masks makes the effect unpleasantly noticeable.

If the image still looks too one-dimensional (often in drone shots), simply use individual small, soft circular masks and place “blobs of light” in the image. This creates exciting contrasts and, in combination with cleverly used tints, additional colour contrast. Flares also often add depth and excitement to the image. You can either use plugins such as LensFlareAutoTrack from Boris FX or (as I often do) soft masks with strongly raised blacks. The latter will give you softer glares, but that’s usually enough.
If images suffer from a certain monochromaticity due to the subject (all the colours in the image are in the same part of the spectrum), for example when shooting at dusk, it is worth giving image areas outside the image focus a different colour and thus increasing the overall colour contrast. It can also be useful to slightly increase the brightness of the half of the face on which the key light lands in order to further emphasise the focus.

General best practices

Finally, a few “general best practices” – i.e. processes and empirical values that have proven themselves in practice with professional graders. What was frequently mentioned is the focus on the basic tools, above all the wheels. In order to work quickly and efficiently, to be able to transfer work steps from one clip to the next and to avoid getting into technical and aesthetic difficulties in the medium term, almost all professionals advised using the colour wheels for everything. With their softly overlapping editing zones and enormous accuracy, there is often little to go wrong in the workflow (except perhaps with balancing, see above). As mentioned above, I would look at tools like Temp and Tint in a similar way. That’s why it’s worth going through the entire timeline with basic balancing first and getting good shot matching (which is probably the most difficult task anyway). This will give you a film that is presentable if necessary, with a view to time management. Only in the next step should you venture into the look and secondaries, i.e. from the rough to the fine. As a beginner (as was the case for me), you quickly tend to edit individual shots in great detail.

Keeping an eye on your time is another important point. As mentioned above, it helps to divide the time available by the number of shots and thus get a better feel for how many images you need to create per hour, for example. To remain consistent, it also helps to always compare with the same setting when matching, i.e. Shot 2 with Shot 1, Shot 3 with Shot 1 and Shot 4 logically with Shot 1 and so on.

Word has certainly got around that highlights should not be clipped, but during my research I was repeatedly advised to increase the blacks slightly.
If you want a soft roll-off in the highlights, you can simply use the custom curves by setting a point at 40 per cent of the Y-curve (Alt key leaves the point on the reference diagonal) and then pressing the
Reference diagonal) and then dragging the white point downwards. If you place this adjustment in a node before the balancing, you should achieve a smoother fading of the highlights.
With regard to the order in which you edit your clips, it can also be worth starting with the “worst” image in the scene. Adjustments of (technically) very successful shots to this shot are usually easier than the other way round.
Incidentally, if you find it difficult to find a balanced contrast, you can simply remove all the saturation from the clip temporarily. I can say from my own experience that with this black and white image the contrast is very intuitive to find.

To counteract eye fatigue, it can help to look at a reference. Bela recommends simply placing a standard grey card and a piece of white paper/cardboard next to the screen and looking at it from time to time. As a rule of thumb: look “away from the screen” for 20 seconds every 20 minutes. This also fits in perfectly with the Pomodoro efficiency logic. And last but not least, you should always double and triple check everything you render – the professionals do this too, and above all. A mask may not be tracking, you may have extended a clip, the qualifier may be noisy, you may have rendered incorrectly: There are many problems that can be overlooked in the process.

Final words

Colour grading is also a personal process to a certain extent and involves
many subjective elements and creative approaches with the different tools. In the end, it’s the final product that counts, regardless of how you got there. However, the methods and techniques mentioned above should make it easier to achieve a good basis consistently and perhaps even more quickly. This frees up time and perhaps also mental resources to concentrate on working with colour as a tool for image design and storytelling.
Finally, a technique to push yourself a little and experiment a little: “Go overboard to find your sweetspot” recommends Mark Todd Osborne on LowePost. This means going overboard with individual parameters in Resolve to see how far you can go.

Macintosh Gamma Shift

Before we come to the general best practices, here is a short insert for all readers with Mac computers that do not grade with an external monitor connected via SDI. For many years, the problem has been that footage looks different in Resolve than it does after export or roundtrip in Premiere, for example.

As soon as clips leave Resolve, they look less contrasty and therefore a little desaturated – a so-called gamma shift has taken place. Put simply, this has to do with the colour management of MacOS (ColorSync), which is not applied to Resolve. In a future issue, this will be discussed in more detail – Michael Radeck is already working on it. Many methods (such as exporting with a special LUT or adjusting with Lumetri in Premiere) of trying to correct this shift after grading are more a matter of pi times thumbs. If you want to be sure that you always see the same image on your Mac both during and after grading and when publishing online, you have to change a few settings within Resolve.

First, go to the Resolve application settings (⌘ ,) and tick the “Use Mac display colour profiles for viewers” and “Automatically
tag Rec.709 clips as Rec709-A”. Then go to the project settings (File > Project Settings or  9) and as we recommend a YRGB colour managed workflow, you usually have to deactivate the tick under “Automatic colour management” to retain all options.

For “Colour Processing mode” we select “SDR Rec.709” and for “Output Color Space” we select “Rec.709-A”. The A probably stands for Apple – just as a mnemonic. Now we just have to assign the tag “Rec.709-A” under “Gamma Tag” in the “Advanced Settings” when exporting. This should (finally) display everything consistently – in all programmes and also after uploading to streaming services such as YouTube.

The post Grading tricks from a professional first appeared on DIGITAL PRODUCTION and was written by David Feuerstein.

In its own words, the biggest upgrade to date advertises the best 2D rigging system, the combination of unique traditional and non-traditional animation tools, a completely new graphics engine and many new tools, such as Liquid Shapes, Multi Strokes, Smart Line
Boil, Delayed Constraints, Vitruvian Bones, Physics, Particles, Dynamics… Despite obvious adversities in the past and great competition, Moho now wants to find and maintain its place among professional 2D animation programmes.

Moving story – from Moho to Moho 14 (2023)

According to Wikipedia, Michael Clifton, founder of Lost Marble LCC, published his animation software before the year 2000. Simple vector animation and a bone system that was already advanced at the time immediately attracted the attention of users interested in animation, who hoped for further rapid developments. As Clifton originally developed Moho for BeOS, which came to an end in 2001, he had to port his programme. He subsequently adapted his software for Windows, Mac OS and also Linux. in 2006, the programme – and the developer – moved to the company e-Frontier, which wanted to continue marketing it. A new change came in 2007, this time to Smith Micro Software. From then on, the software was called Anime Studio, but was renamed Moho again in 2016.

in 2020, many users in the Moho forum were delighted to learn that Moho was back in the hands of Lost Marble. Lost Marble acquired the software with the support of Cartoon Saloon. The Irish animation studio, Cartoon Saloon, is probably known to connoisseurs for animated films such as The Secret of Kells, Song of the Sea and Wolfwalkers. It has received five Oscar nominations, among others.

“We always called Moho ‘our secret weapon'”, says Tomm Moore, co-director of Wolfwalkers
Director of “Wolfwalkers” and “Secret of Kells”, is said to have said at the time. This autumn, Lost Marble announced the release of Moho, Release 14, as a pro and debut version. While these lines were still being written, the first update, 14.1, was released, which Moho’s search for updates on a Windows 10 system did not (yet) recognise. The list of new features made us curious..

A lot of talk about a little 2D animation?

Despite the huge boom in the 3D sector, 2D animation will retain its justification in various sectors and in different media. This is not really the place to discuss design or media trends or the pros and cons of 2D or 3D. A look at 2D software, such as Moho, reveals a rather complex situation that is constantly being determined by new technologies and may soon change fundamentally through the use of AI. 2D animation has long since ceased to be limited to pure frame-by-frame animation, i.e. the classic animated cartoon that older people may still remember from early animated films, but now refers to various 2D animation techniques. The market now offers a whole range of programmes for animating with different approaches, including programmes with very different prices, such as Animate CC (adobe.com), Toon Boom Harmony (toonboom.com), TVPaint (tvpaint.com), CelAction2D (celaction.com), Pencil2D Animation (pencil2d.org), Cartoon Animator (reallusion.com), Synfig (www.synfig.org), OpenToonz (opentoonz.github.io/e)… This list is arbitrary and absolutely incomplete, but should give an impression of the fact that “simple” 2D – and the software for it – is still in demand. A look at current media also shows this: animated films still exist, advertising and explanatory videos can be found on almost every website and some stories or product information are more convincing through the graphical means of 2D design than through more elaborate, hyper-realistic 3D animations.
Meanwhile, the many programs and online platforms promise to simplify and revolutionise 2D animation and achieve the best results with the least amount of effort. Moho also aims to be such a tool. Moho is a versatile software with lots of tools that attempts to cover every area of (2D) animation and make everything animatable.

What can Moho do?

Moho Animation Software promises professional, powerful and user-friendly 2D rigging and animation tools as well as new drawing, deformation and effect tools. The list of new features is long and fills several pages in the manual, but only supplements the many tools already available. Moho is a vector drawing programme with a range of special drawing tools that now also enable a natural look similar to that of raster graphics. Thanks to the newly developed graphics engine, drawings, styles and effects should be displayed much better and more smoothly. The current version offers various drawing styles such as wet ink, charcoal, gouache, rough ink, watercolour and others, as well as several dozen preset brushes with complex adjustment options. By using styles, you can control the exact look of your drawings and edit multiple drawings at the same time, changing the shape, colour, line width, brush settings and effects.

Animations can be achieved by simply manipulating individual selected control points of the drawn (vector) curves – optionally also controllable with Bezier handlers or, for example, with a variable magnet tool or various deformation tools in the timeline, whereby Moho automatically sets keyframes. Fills and strokes can be varied as desired and thus also animated. There are a number of different tools and methods for deforming drawings (e.g. Liquid Shapes are new). Keyframes can also be set for effect settings.
In fact, everything can be animated in Moho. Moho supports the animation of vector points, strokes, shapes and layers as well as various effects in different ways. Moho allows better frame by frame animation in the current version, especially through improvements in the timeline. Although we are talking about frame-by-frame animation, with Moho this means that all of the programme’s manipulation options for vectors are available in every frame. Rotoscoping should also not be a problem.

Moho aims to support all conceivable animation options and technologies, which are almost impossible to list here: Already mentioned are animations of vector shapes (points, fills and lines), object and path animations, deformations through warping (also for imported images), bones that can be controlled with constraints (targets, IK stretching, IK solvier), or the use of physical rules, such as reactions to gravity or wind and adjustable collision detection.

New in Moho 14 are Delayed Constraints. A bone (Control Bone) to control everything! In the Bone Constraints, where Bone Dynamics and Control Bones are defined for controlling angles, position and scaling of individual bones, there is now also the option of specifying a delay for subsequent movements of bones. Controlled bones can be “told” when they should follow the control bone and by what percentage (before or after their movement, with smaller or larger amplitudes). There is an example in the Moho tutorials that shows how, for example, an (octopus) tentacle can move very lively and elastically after such settings.

Also new are the aforementioned Liquid Shapes: For shapes within a layer, it is possible to apply Boolean operations to each other. Each shape can be assigned either “Normal”, “Add”, “Subtract” or “Clip”. “Normal” shapes that are overlapped by others can then be “manipulated” by the shapes above them, as with Boolean functions in other programmes (the order of the shapes can be moved up and down within a layer using the Shift and arrow keys). The function is made particularly interesting by the “Blend” option, which, depending on the value, softly rounds off the resulting shapes. The result can be fixed with “Merge”.
As long as “everything is still in flux”, everything remains animatable and it is amazing what a variety of shapes and smooth animations are possible with this new option. The shapes behave in real time, like fluid forms that displace each other or seem to flow into each other. This technique, combined with various layer effects, is demonstrated in the trailer for Moho’s tutorial videos.
Although Moho is a 2D software, movements are also possible in 3D space. Moho can import 3D objects and also generate simple shapes from 2D shapes (rotation and extrusion bodies are defined in the layer properties, or layers are rotated and positioned around the X or Y axis).
Moho offers a rudimentary 3D view for working in space. When the 3D button is clicked, Moho still jumps to an obviously preset view, which has to be corrected every time you want to use the 3D space. A “smoother” camera pan or even the selection of possible camera settings would be more pleasant. Most users probably use the 3D options mainly to arrange objects in space and to create camera movements with parallax and depth of field effects.

Many tools

Many tools offer many possibilities, but also require a lot of work with the programme in order to be able to exhaust all the possibilities and use the respective settings and options effectively. Everything is easy if you know how to do it. – Or as the old maths teacher used to say: “The solver can see immediately what the result is”. This also applies to Moho.

A beginner can either be delighted by the ease with which lines and shapes can be animated in Moho, or frustrated by the sheer endless possibilities – and the corresponding settings and options that need to be “figured out” first. Accordingly (and irrespective of the fact that in many computer forums “your own software is always the best”), opinions also differ there, from rejection because it is too complicated, to euphoric statements about the “best 2D animation programme”. The tenor is that Moho offers a professional approach with a corresponding learning curve.
The community of Moho users appears to be quite large and there are numerous tutorials online (also on commercial learning platforms such as http://www.udemy.com or http://www.toonfiles.com) and also help, e.g. in the Moho forum of Lost Marble (see the box with links). Lost Marble provides dozens of tutorials and webinars on the software and also offers learning courses on its website in addition to various paid content and some brush packs (including character, cartoon and Halloween content packs…).
A first look at the software shows us that there can’t be too many of these courses.
The current version of Moho can be used in several languages, including German. We find the German version more convenient to use, but most of the help can be found in English-language sources, which is why we use modern, occasional German or English expressions in the programme. In addition to context-sensitive tool tips for all functions and information on the currently selected tools in the programme’s status bar, Moho offers help in the form of a wiki. Unfortunately, this is only available online. In some forums there have been requests for a downloadable PDF file or similar, which would be easier to access and perhaps more searchable.

We are sure that not all aspects of this software can be discussed here. We have limited ourselves to a general overview and some of the “specialities” and new features of Moho in order to convey at least some of the “Moho feeling”.

An easy start?

At first glance, Moho seems as simple as software is usually advertised by the manufacturers. And indeed, even beginners seem to find it easy to take their first steps in the programme. By default, a randomly selected sample animation opens after the splash screen – a nice way to see Moho in action. You can switch this off in the preferences, along with other settings, but you can also simply open several files in parallel.

Of course, everyone has already seen a graphics application, perhaps even an animation programme. Then the default UI of Moho will already appear somewhat familiar.

If you want, you can change the brightness and the display of some editor colours, e.g. for objects, selections or the canvas background, in the default settings. Menus can be docked or undocked in the Window menu and, of course, customised shortcuts are possible.
There is the usual menu bar at the top of the screen. In the centre of the screen is the canvas, Moho’s drawing area (the view can be split, display settings can hide certain content), to the left of it is the toolbar, which is equipped with different tools depending on the selected layer type. A new file opens with a vector layer by default. Accordingly, some drawing and layer tools as well as buttons for controlling the camera and screen navigation are offered (if a bone layer is active, the corresponding bone tools are activated). Depending on the active tool, a tool options bar above the canvas shows further options. To the right of the canvas, either the Style or Actions menu can be displayed. Below this is the layer menu (similar to the layer palette in other applications).

The centrepiece for animations is of course the timeline with the playback controls at the bottom of the screen. The timeline shows the current position of the playhead for the animation (a very narrow vertical line), animation channels for objects and, of course, keyframes, which can be set, moved, copied or deleted here.

There are various display modes (Channels, Sequencer, Motion Graph). The interpolation methods between keyframes can also be changed here. Onion skins can be switched on or off and the length of the animation can be set. The Relative Keyframing option allows animated objects to be repositioned and existing keyframes to be moved “relatively” with them. Perhaps a small pitfall for newcomers to the programme: objects and drawings, as well as receipt assignments… are created in frame 0 by default in Moho! No animation can be tried out here, while no drawing and no objects can be created within the timeline (except in FBF mode).

A look at different layers

Moho uses different types of layers for different purposes. Vector layers are the standard layers for all vector drawings, similar to
in common graphic programmes such as Adobe Illustrator.
There are a variety of layer options for different purposes. Among other things, there are settings for effects such as shading, the vector display (e.g. whether an animated “noise” of outlines should simulate the style of hand-drawn animations), there are also options for display, opacity, motion blur… In the layer settings, you can also specify whether elements of the layer are captured by Moho’s camera at all, whether DOF affects them – or how a layer should interact with other objects, e.g. follow a path or react to a warp layer..
Whether depth of field (as Moho calls the depth of field in the German translation) affects the layers at all, which render style, image resolution is used, or whether, for example, layers are displayed sorted by depth, is specified in the project settings. There are layers for imported images and image sequences. There are currently no more bitmap tools in Moho. Technical difficulties in editing and displaying bitmaps in earlier versions are cited as the reason. But apparently the developers are planning to fix this problem later. For the time being, the main focus was on the vector tools and the new graphics engine.
However, bitmaps can be imported and deformed, e.g. with Bones. Meshes can be
Meshes can be defined as warp layers or an automatic function for mesh creation can be used (Draw/Create Mesh Layer). Images can be masked in the same way as vector layers. Various options can be set under “Masking” in the group settings for layers. The lowest vector layer serves as a mask for the layers above it. Masks can be used to simulate light and shadow effects by applying layers with different colours, intensities, blur and layer modes to objects – but only up to the edge of the mask. In the test with a Windows 10 system, an assigned shadow effect was often only displayed in the render preview.

Patch layers offer a special type of masking, e.g. to conceal overlapping contours at joints. Patch layers (round vector shapes) “hide” the overlapping layer, e.g. exactly at the joint.
The new Curver layers are used for simple but effective deformation of vector or image layers. If a layer is selected, a new type of warp layer can be created under “Draw/Create Curver Layer”. The layers to be deformed are specified in the layer settings, under “Vectors”, to which warp layer they should react. Curver layers, originally consisting of just two points, can be customised in terms of size, thickness (area of influence) and number and position of points. Smooth distortions are then possible in the timeline.

Curver layers can be combined with Bones. The animations can also be controlled with Smart Bones. Compressible Curver layers do not distort their objects “linearly”, but according to exactly which points are moved. This means that part of a figure can remain “untouched” while the rest is wildly distorted. This is a very simple way of creating organic-looking deformations (which can of course be animated).
Special layers also contain particles and bones. Switch layers contain elements that are only displayed “switched”. Switch layers group different elements, but only display one at a time. This means that predefined or drawn hand positions (as vector or bitmap graphics) can be exchanged in animations using keyframes, for example. When importing Photoshop files, layer groups in which only one element is visible are automatically converted to switch layers. Moho also uses the same principle for FBF animations
principle is also used for FBF animations, as well as for lip synchronisation. Here, as with
Switch Layers, the most suitable mouth shapes are made visible according to the imported sound curve of a speaker. Moho attempts to do this fully automatically, but also works together with Papagayo, for example, a tool that simplifies lip synchronisation. Of course, the images for the respective phonemes can also be set by hand or Smartbone, as in “normal” switch layers. If vector drawings that have been “morphed” are used for the mouth shapes, Moho can even calculate smooth transitions here. However, the smooth transitions should not be visible when using Smartbones.
Layers can be sorted and summarised in groups in the layer menu. Different blending modes (as in other graphics applications) can also be selected there, for example.
If bones are used, all elements to be animated must be subordinate to a bone group. To do this, either create a bone layer in the layer menu and then move the layers to be manipulated underneath it, or create a layer group by right-clicking on selected layers, which can then be converted into a bone layer. Switch layers can also be created in exactly the same way. And Moho also has a text layer.

Bony

Lost Marble advertises a sophisticated bone system. Not all bones are the same in Moho. There are bones that are “simply” drawn hierarchically.
Every bone that is drawn on an active bone is automatically bound to it. With the “Reparent Bone” icon, the hierarchy, represented by green and, for selected bones, red arrows, can be viewed and changed. Sketch bones are drawn even more simply, in a single line. This allows long bone chains to be created. The length of the individual bones can be preset. Smart Bones can probably control every animation that has been summarised in an action via its respective angle – including the order of the objects in switch layers. Actions can be reused as required. Under “suitable” conditions, e.g. if bone structures with the same structure are used, they can even be used in different projects.
The Vitruvian Man, or rather the famous depiction by Leonardo da Vinci, was the model for the name Vitruvian Bones. When using these bones, similar to switch layers, elements (e.g. body parts, but with the bones included) are exchanged in the animation. For example, a character has four arms, like Shiba. Each arm has a different posture and position. By switching the individual poses of the arms, animation time is to be saved considerably.
Pin bones are created by simply placing a point when drawing a bone. They can be used like “normal” bones but, depending on the specified bone strength, they enable special distortion effects to be created both for vector drawings and imported bitmaps by moving (T), rotating (T Alt) or scaling (T CTRL). As mentioned, bones always affect the objects that are subordinate to them.
Binding in Moho, as in 3D software, is the linking of bones and “content”. Flexi-Bind Points is the standard method; it binds points to bones in such a way that they are influenced by all the bones in the parent bone layer, depending on the Bone Strength, the area of influence of each bone. More control is achieved with the direct binding of selected points to selected bones (bind points). When recreating classic cutout animations, it makes sense to have all the body parts of a figure to be animated in individual layers. Bones that are connected to these layers (bind layers) then only affect these elements.

Moho still offers some options to make rigging more efficient, e.g. the option to “Create Smooth Joint for Bone Pair” (tries to create smooth joints between two bones). This is probably best suited for bending bitmaps, e.g. when bending a leg.

Conclusion

Moho clearly claims to do 2D animation at a professional level, and not just because of the $400 price tag for the Pro version. Moho offers a wealth of drawing and animation options that cover almost every possible 2D task. However, the software also requires a corresponding amount of effort before it can be used effectively. We will save ourselves a comparison with other applications that are either easier to use but offer fewer possibilities – or can do even more, but are then also more complex.
If you are serious about 2D animation and want to try out the advantages of vector graphics and, above all, Moho’s Bone system, you should take a look at the demo versions. We think that the latest upgrade, especially because of the new graphics engine, has really taken Moho to a new level. We wish Moho a good way into the future and hope for good news from Lost Marble.

A collection of Moho scripts

There are various instructions for installing the scripts in Moho, including here:
is.gd/moho_scripts_install. At least for most of the scripts in this list there is the option “Download for Install Script command”. In this case, the installation was successful with just a few clicks using the installation script integrated in Moho. Please make sure that the scripts are intended for the current Moho version!

Some suggestions for testing
MR Curve Tool: This script simplifies the drawing of curves. One option is
z. Drawing outside frame 0. is.gd/mr_curve_tool

MR Pose Tool: Promises to speed up posing by combining the Transform Bone and Manipulate tools. is.gd/mr_pose_tool

MR Key Motion: Manipulates curves in the timeline by moving selected keyframes (Smooth, Pull
(Smooth, Pull, Scale, Flip). is.gd/mr_key_motion

MR Path: Should help to analyse animations by drawing “bone” and “point paths” across the animation layers for visualisation. is.gd/mr_path

MR Tween Machine: Anticipation and exaggeration – anticipating movements and “overshooting” are among the well-known principles of animation. This script generates additional keyframes before and after the end of a movement in order to achieve the desired behaviour. is.gd/mr_tween_machine

MR Transform Rig Tool: Parts of the rig can be deformed without affecting its functions. is.gd/mr_transform_rig

MR Guides: Create and edit guides without having to leave the current layer and action. is.gd/mr_guides

MR Overlay: Create an “overlay” to obtain a reference layer. is.gd/mr_overlay

MR Continue Animation: Extend existing animations based on two neighbouring keyframes up to the current frame. is.gd/mr_continue_animation

MR Track Bone: Transfer the transformation of a bone from an animation to any skeleton. is.gd/mr_track_bone

Script for importing Affinity Layers or Krita, Spine, Photoshop, AfterEffects, Gimp… into Moho is.gd/import_layers

LipSync with papagayo: An app to easily automate lip synchronisation lostmarble.com/papagayo

Specsheet

Moho Pro 14: 399.99 US dollars
Moho Debut 14: 59.99 US dollars
For the differences between the two versions, see here: is.gd/moho_debut_14
or is.gd/moho_pro_14

System requirements

Windows 10 or higher, 2.0 GHz Intel Core i3 or higher, 4 GB RAM or higher, 16 GB free hard drive space or more, OpenGL 4.1-supported graphics card required (1920 × 1080 recommended)

macOS 10.15, or higher, 2.0 GHz Intel Core i3 or higher, 4 GB RAM or higher,
1.6 GB free hard drive space or more, OpenGL 4.1-supported graphics card required (1920 × 1080 recommended) – Moho 14 is fully compatible with Apple silicon chips (M1) (M2)

For displays with high pixel density, the minimum resolution varies depending on the scaling level recommended by the operating system. For example, if the operating system recommends a scaling level of 200 per cent, the minimum requirement may be a resolution of 2736 × 1824. Or if the operating system recommends a scaling level of 150 per cent, the minimum requirement can be up to a resolution of 2160 × 1440.

A demo version is available. This is fully functional, allows opening and saving of Moho files, and is valid for 30 days. Restrictions are no import of external file formats (images, films) and no export of animations to other formats (MP4, MOV, PNG…)

The post Moho 14 – the most advanced 2D animation first appeared on DIGITAL PRODUCTION and was written by Ralf Gliffe.

by Pia Röder

Okay, I admit it. I hate (h.a.s.s.e.) installing software and getting it to work. I’m a copywriter and use Word, PowerPoint and, if things get stupid, Excel. Unfortunately, there are two laptops here that I have permanently in the office and slowly (after 5 years) I’m getting really fed up with having to move the USB mouse from laptop A to laptop B. Apart from that, laptop keyboards are somehow semi-optimal anyway and not ergonomic at all. I need something simpler, something better. Something that allows me to operate both luggies with one mouse and one keyboard. That’s when I found the MX Keys S Combo from Logitech.

The unboxing experience

I’m not a big fan of the bitten apple, which is why I don’t like all this unboxing hype. The hardware I order comes in black cardboard boxes – cuboid, practical, good. That’s all the unboxing experience I can get as a PC user. So well, I’ll take it. Everything in the box is wrapped in greaseproof paper – keyboard, mouse, silicone wrist-rest cushion (a big hello to all those learning German as a foreign language). But at least it avoids plastic. One UN sustainability target has already been met.

Connection via Bluetooth

The instructions on how to connect the devices to the laptops are printed on the top of the lid. One option: via Bluetooth in just 3 steps. It can’t be any more than that, otherwise I’ll jump off halfway through the user journey. I won’t even try the connection with the logi bolt USB receiver (4 steps). If anyone wants to test it, let me know if it works.

So, Bluetooth switched on in the PC settings. Keyboard MX Keys S switched on at the back, mouse on the underside too. It works. The numbers 1, 2 or 3 flash on the Easy Switch buttons, like children’s television in the nineties. I decide in favour of 3 and press firmly. Last chance… over! MX Master 3 S is connected, says my laptop. Hooray! (I’m annoyed that I didn’t press 1, but never mind now.) The keyboard has secretly connected itself via key 2. Guys… really now.

I ignore the problem and concentrate on connecting to the other laptop. Same game. This time it’s mouse button 2 and it works. Only the keyboard doesn’t work. The setup wizard on mxsetup-logi.com promises help. Clear design, but they have separated “setup instructions” incorrectly. I get sad. At least it’s quick and easy. Switch the keyboard off and on again, press and hold Easy-Switch button 2 until it flashes, enter the PIN in the pop-up on the desktop using the keyboard, enter and you’re done. It wasn’t that difficult for laptop 1 either. Now I can use the mouse and keyboard with both laptops.

Hardware with Fancy Keys

The keyboard has already been extensively discussed in a previous issue: solid, comfortable to type on and easy to charge via USB. I can confirm this. The really cool thing is the function keys at the top. In particular, “Microphone off” when the cat in the background throws up on the carpet during a meeting (or, alternatively, the baby cries in the case of a low A on the Crazy Cat Lady scale). The dictation function at the touch of a button works surprisingly precisely and the one-click calculator is indispensable as a copywriter with three points in the (Hessian) maths A-levels.

While the Logitech software Logi Options is installing, I stroke the ergonomic mouse, which caresses my palm like a Motorola Pebble from 2005. It sits really well in my palm, has a left and right button and a scroll wheel. So far, so familiar. The highlight: I can assign individual functions to the various buttons.

Logi Options software

It has to be said, Logitech has it all when it comes to UX. The software for the keyboard/mouse combination is slim, clean and doesn’t bother me with complicated bells and whistles. I can use it to check the battery charge level on the devices, check for updates and set all the functions. One click in the top navigator takes me to the Smart Actions – the “If this than that” feature for the hardware. I was sceptical at first, but now I use the middle mouse button to open my fee Excel or send the editor-in-chief of this magazine “Entententententente” via WhatsApp. Just the important things. This also works with the keyboard, but unfortunately only with the F keys in the top row. Would like to have 30 pizzas delivered to my ex-boyfriend automatically using the “A” key. Doesn’t work.

There is room for improvement. I will soon see what is possible with Autohotkey (alternatives here: is.gd/remapping). Using Logi Flow, you can theoretically simply switch to the other computer using the cursor and transfer files via copy-paste. Theoretically, because that doesn’t work for me. This is probably because the (customer) laptop does not tolerate such invasive behaviour. This function would actually have been extremely practical, because I wouldn’t have to send files to my own office laptop for printing by e-mail, for example. Installing a third-party printer driver is not an option on the customer’s device either.

Personal conclusion

Even tech n00bs have fun with the MX Keys S Combo. After a bumpy start (the error is usually in front of the computer), the connection for both laptops was quickly set up. Minus point: You have to turn the mouse round each time and press the Easy-Switch on the underside to activate it for the other laptop. However, this is also possible using Windows Powertoys(bit.ly/powertoys_mouse). The Easy-Switches on the keyboard are in the path of my fingers and I deactivate them from time to time. It’s a matter of getting used to it.

But the most important thing: I save a huge amount of time because I no longer have to fiddle around with the USB mouse, I type like a young goddess and everything looks smart (girls love that). Hallelujah.

The post Shortcuts for more connection first appeared on DIGITAL PRODUCTION and was written by Bela Beier.

January saw France-based Left Angle (left-angle.com) introducing Autograph and while at first glance it looks like a “Darker Version” of After Effects, there is actually a lot more to it (See our test).

We were able to sit down with Francois Grassard, lead developer at Left Angle during IBC and talk about where Autograph is right now and what to expect in the future.
With over 25 years of expe­rience in 2D & 3D motion design and visual effects, François knows a thing or two when it comes to industry standards. Over the years, François has worked in va­rious development, graphic design, and managerial roles across the broadcasting, film, and advertising industries. His client roster includes Walt Disney, Universal, Fox, Warner, Renault, Dior, CBS, BBC, NBC, Yves-Saint-Laurent, and Unilever. François is also an author and has taught university students, sharing his insights and expertise.

DP: Autograph is a new contender in a market dominated by After Effects and Nuke. What sets it apart from the competition?
Francois Grassard: Autograph, created by Left Angle, is a software intended to be between After Effects and Nuke. After Effects, because we are a layer based compositing system. So obviously it looks like After Effects, but we are closer to nuke in the technical aspect because we are working on 32 bit per channel and we manage all the colour spaces. So we try to be more precise. So we try to have a very fast workflow for motion designers, but also VFX artists. We are working on a new way to switch from layer based approach to a nodal approach. So this will be a bi-directional way without committing to one workflow or the other. But we decided to create Autograph for different things. We think that the progression of software such as Nuke or After Effects was pretty slow because there has been no real evolution in this software since maybe 20 or 25 years.

I actually started to use After Effects in 1995. So it was version one or version two, I don‘t remember exactly. And since this day the process is almost the same and a lot of things have changed in the media industry. Regarding 3D and the way we composite together different elements. So we decided to create a new product with a focus on specific topics such as the possibility to work on different formats at the same time. For example, you can work in a responsive design composition, it can be turned into 16 by 9 or 1 by 1 or square or 9 by 16, or any kind of format in parallel, just by defining rules of creative positioning in the composition. So if you want to place a layer on the left of your composition, if you change the size of your compositions, usually it would be replaced, according to this new format. So for this kind of specific issue, there‘s actually not really a solution in After Effects for you, except you can script your own solution.

And the scripting is obviously a good answer to this, but it‘s also a really technical one. So we are really aware that all these really common tasks can be not really easily achieved in other applications. We also want to break the bridge between 2D and 3D because when you work with a 3D scene and want to work with the image produced by this scene, you have to render these images onto disk and you have to integrate these images in your compositing package. That way you have a huge footprint on your drive. You need to have a lot of storage to keep this render on the hard drive. Also it shouldn’t be such a linear process between the two tasks. So with Autograph, we try to be able to render a 3D scene on the fly and grab images from this 3D rendering and do the compositing of the side within the same application.

So that was another part of the project initially to break the bridge, to make a bridge between 3D and Compositing. I think it‘s nonsense to separate these two things. And we can see this nowadays with virtual production. And most of the time when you see some virtual production based on a game engine or any or any other system, you can see that the missing part in this process is compositing. Of course, Game Engines offer things such as Glows or ambient occlusions, but real compositing has always been missing. So the goal was to propose a proper solution. At the same time, in the same software. We are not saying Autograph is a real-time engine because we know that every real-time engine is real-time until it is not. So we prefer to define Autograph as interactive rendering.

DP: So, speed is not only important to rendering, but also in using the software. What is Autographs philosophy in regards to speed and user interface design?
Francois Grassard: I am personally very frustrated with the interfaces in other applications. The way you can interact with for instance dropdown menus: You have to click on any kind of style, any entry to test it, but in Autograph you just have to hover your mouse over the style you want to test. The interactivity of the feedback, that‘s one of the things we want to focus on, because so much time is lost just by navigating the interface. In the current release, we changed a few things about the way you interact with shapes.

Another way we achieve speed is by utilizing all the cores available with the CPU and the GPU as much as possible. But, there‘s always a bottleneck. The question is what is the bottleneck? Is it the GPU, is it the CPU Cores, is it RAM or is it that may be the bandwidth of your hard drive. So the question is to find the right balance between both to achieve almost 100 percent for each kind of device. So we try to find a good relationship between the interface and the reactivity of the software because when you have to wait for the software to have a response, you are losing time.

DP: VFX applications typically use a node based approach. Autograph, however, uses a timeline. Could you elaborate on the choice to take the timeline as a starting point?
Francois Grassard: Yeah. Autograph has been designed for motion design artists and VFX artists and when you work on VFX you mainly work on one shot. So the concept of timing is not the most important aspect of your creative process. But when you work on motion design, you have to put a lot of different sequences together, one after the other, and it‘s really important to have the concept of timing in mind.

In Autograph, a lot of things are related to timing. If you work with sound, for instance. In the next release, we are going to have the full audio pipeline and you can source this audio and extract frequencies for this audio drive any parameters also display curves based on this audio and all of this thing is related to a ribbon. So timing is pretty important.

So there‘s a reason we try initially to focus on the timeline approach. On the other hand, if you are a VFX artist working on a single shot, you want to have a really precise control over the process.

DP: And Autograph handles this how?
Francois Grassard: In Autograph, even if we have a layer-based approach, we actually have underneath a node-graph created, but with thousands of nodes. So if we show the graph created by maybe stacking ten or 20 layers, we have a bunch of modifiers – that‘s the name of your effects – you are going to have thousands of nodes. So if we have in a few months or in a few years a node-based approach, when you can convert your layer-based approach to a node-based approach, we have to think about maybe a bigger node with more parameters.

In Nuke, for instance, you have to create a read node and then a transform – two separate nodes. In most cases you want to use a transform just after the read node. So why not just merge a transform into the read node? It‘s something that you have to think about to have a ma­nageable node graph. So the first question was to have good control over the rhythm, the music and so on. So that‘s the reason we decided to introduce the layer-based approach first.

DP: USD is quickly rolling out everywhere and is already a de facto industry standard. How does autograph handle USD files?
Francois Grassard: Yeah, we decided to integrate USD three years ago when we started the development of the 3D part of autograph. We started to work on Autograph about seven years ago, but the main development started about three years ago. So we decided to use USD as a basis. Each time you add a new primitive or you inject a new geometry, everything is based on USD. We create the USD scene under the hood. At the time we took a risk, because it was just emerging and Pixar was saying: ”This is the future.” But at that time USD was like an empty box. You know, you open the box, you can see a few things about the geometry, but nothing about the materials. And it‘s still the case today.

There‘s some evolution there, a lot of things that are on MaterialX. That’s a good way to interchange material between applications and something you appear about two months ago with the creation of aerials for Open USD with Adobe and also Apple and a lot ofactors around USD to work on something called Open PBR as a physical based rendering, and it‘s a way to standardize the ma­terial between applications. So in a perfect world you should be able to exchange material between the render engine and the compositing application and physically get the same result. So it was risky at the time, but actually it was a good choice because three years later you see it was the way to go. So basically we have a head start in integrating USD, which is not an easy task.

DP: Autograph was released in January 2023, so it has got a very fresh user-base. Who are the users?
Francois Grassard: We have two kinds of clients. Freelancers, because the software has been designed for small computers, initially. Actually here at IBC, I am at the Dell booth has I have a huge computer to make my demos, but every day I use Autograph on my eight year old laptop. That allows me to optimize the code and detect bottlenecks. And we need to detect those bottlenecks.

If you have a big computer, it‘s going to be difficult to see if there is a problem in this part. And it is also used by big studios to do heavy compositing, even if we don‘t have nodes yet. But for some studios, it‘s not really a problem because we have a 32 bit channel processing linear workflow. So they love the precision of the software and also the speed of the software. That‘s something we hear pretty quickly when someone tests the software “Oh,it‘s fast!”

DP: And what has been the feedback so far?
Francois Grassard: So obviously it needs time to get adopted because it‘s a new concept, even if you come from after Effects because while the interface is pretty close, we have some very different concepts like generators and modifiers and the way you can link parameters together is pretty complex. So yes, we have two different kind of user profiles: motion designers and big studios. Obviously some people also work in VFX and these are freelancers as well, so we are really talking about three user profiles.

But right now it is mostly motion designers because we have a lot of functionality for animating elements, we have a lot of interpolation types built in. In After Effects, If you want to have a bounce movement or an elastic movement, you have to write an expression for this. In Auto­graph we have 49 interpolation types built in, so you just have to create two keyframes and select your interpolation type. So motion designers love this.

There‘s so many small things that actually make a huge difference, such as the fact we have the origins of zero. The origin of the coordinate system is at the centre of the composition. It‘s a good way to quickly create symmetrical ani­mation. We also have really technical users who are really happy to see that they can go really high into controlling the precision of the picture.

DP: Is there a trial version available?
Francois Grassard: Of course! Our free trial comes with the latest version of Autograph Creator and access to free templates and demo projects, so you can jump right in. Sign up for it at is.gd/autograph_trial.

DP: And what are the upcoming features?
Francois Grassard: Yeah, we are working on the new audio system that will be available as a next release at the end of September. We are also working on a new text engine that allows you to have a lot of more control over the animation. And we are working also on the new shape layer feature. So previously in Autograph you had to create a generator to generate, for instance, a stroke or fill. And in the generator you have a path group, a group containing a bunch of paths. And we are actually reversing this process to have the path group first. So you will be able to define your path and then you can apply a style to this path. Right now we have really classic styles such as stroke, solid and fill.

But in the future, because we have two people in the company who were previously in the field of non-photo-realistic-rendering, just like pencil and watercolour. And we are also working on converting a 3D scene to paths. So the goal is to create a non-photorealistic drawing from 3D, but not by drawing directly the pixel, but by defining paths and using these paths, you are going to add style. So it‘s a good feature for people who work in 2D animation, for example the Japanese animation industry. We worked a lot with 2D and 3D at the same time. And for this kind of industry it‘s going to be really useful.

The post The new kid on the block! first appeared on DIGITAL PRODUCTION and was written by Christoph Zapletal.

At this year’s Adobe MAX sneak session, it was clear to see where Adobe’s headquarters are located in Germany. The gigantic background projection presented itself in the white and blue Oktoberfest look and the co-host Co-host, actor and comedian Adam DeVine, was presented with a beer mug and pretzel in true Oktoberfest style. As always, numerous innovative, experimental technologies were presented by Adobe researchers and engineers. Surprisingly, the focus was on artificial intelligence. You can watch the complete show on Youtube(is.gd/youtube_adobe_sneaks). Here is a short summary of the most interesting innovations!

Project Fast Fill

What is called Generative Fill in Photoshop is now also finding its way into the world of video. The removal and addition of objects in videos. In the examples shown, this worked amazingly well and without any noticeable artefacts. We are curious to see when
the tool will be available in After Effects and Premiere.

Project Draw & Delight

There are now many text-to-image generators. Whether DALL-E, Stable Diffusion, Midjourney or Adobe Firefly, they all have the same problem: the results are unpredictable and the image structure can only be roughly influenced. With Draw & Delight, you can influence this not only with prompts, but also with rough doodles. In the example shown, a comparatively high-quality vector graphic illustration was generated from a very amateurish sketch in just a few steps. It starts with an outline drawing, which can then be filled with any colours. Similar to a colouring book. Backgrounds can also be created. And the whole thing can then be edited as a vector graphic in Illustrator.

Project Neo

This online tool is used to generate 3D shapes from simple basic shapes. The workflow is somewhat reminiscent of Substance 3D Modeler. The focus here is on ease of use when creating 3D elements using Boolean operations and object blending. The finished 3D scene can be virtually illuminated, with the “shadow vectors” adapting to the direction of light. At the end, it can be exported as an Illustrator file. The two-dimensional vector elements after import can be further modified in the graphics software.

Project Glyph Ease

Developing your own fonts for special designs used to be a lot of work. One of the difficulties was to ensure a uniform style. In Project Glyph Ease, only three reference letters need to be designed. The AI then uses these to generate the complete font including textures. This even works with hand-drawn designs that have been imported into Illustrator.

Project Scene Change

Changing the background of shots taken with a green screen is old hat. If you include enough tracking marks, this is also possible with a moving camera. But cutting out actors from normal shots without a green screen also works quite well now. Be it with the new version of the now also AI-supported Rotobrush in After Effects or the AI clipping tool from Runway.ml. Scene Change now combines and automates both techniques by converting the background scene into a 3D environment using photogrammetry. The camera data tracked from the foreground is then transferred to the background scene so that the actor can be credibly integrated into the scene. The positions of objects in the Z-axis and the actor are also analysed so that he disappears behind the foreground objects and casts shadows.

Project Poseable

This software facilitates storyboarding and illustration by combining Mixamo with a text-to-image generator. A 3D puppet is brought into the desired pose using the joints of its skeleton
into the desired pose or placed in an armchair using automatic adjustment. The IK system ensures that only natural positions are possible. The camera perspective is then aligned and the prompt for the desired result is entered. However, the pose does not have to be set up manually, but can also be taken from a loaded image. The 3D mannequin models can also be imported from Mixamo into Poseable.

Project Res Up

High-quality upscaling of videos is already familiar from Topaz Video AI (see this issue, page 52) or the AI upscaling of DaVinci Resolve. While DaVinci scores particularly well when upscaling good low-res material, the Topaz tool is also able to correct image errors such as colour noise, blurring and interlace problems to a certain extent. Adobe is now developing its own tool. In the examples shown, Res Up also appears to use comparison images during upscaling and recreate the structures from them. Whether and with which material it can really do this better and/or faster than the other tools remains to be seen. However, the examples shown were impressive.

Project Dub Dub

In this case too, various AI tools such as Descript already exist. It is about lip-synchronised text translation. Dub Dub Dub uses generative AI to automatically synchronise videos or audio clips in more than 70 languages and over 140 dialects. The tone of voice, intonation and acoustics of the original clip are also taken into account. And with the voices from the original video. The intonations in the last example shown were somewhat strange and difficult to understand in the German version of a very fast-paced dialogue with Cary Grant. But impressive for a 1-click automatic solution.

Project See Through

How often have photographers been annoyed by the fact that a pane of glass between them and the object of their desire has unfortunately obscured their view and they have mostly seen their reflection in the photo. Until now, retouching such photos has been difficult or even impossible. With this tool, glass reflections can now be removed and even extracted. In this way, the reflection can even be blended into the image. What the AI achieves in the examples shown borders on witchcraft. To dispel my doubts, I’ll probably have to try it out for myself when the beta phase arrives.

After Effects

Victoria Nece showed us Roto Brush 3 at the IBC, which now works much faster and more effectively with AI support. However, to ensure compatibility with old projects, versions 1 and 2 are still available. The new 3D workspace now also has physically based rendering and environment lights. The workflow with the Advanced 3D Renderer is much smoother than with the Cinema 4D Renderer. However, it is still somewhat difficult to set the lighting conditions so that the render results look really good. This is still easier with the Element 3D plugin. The Glb format works best in the new 3D workflow. Here the textures are imported 1 to 1.

Firefly / Adobe Express

Version 2 of Firefly was also released for Max. In addition to the improved image generation and the higher resolution, it is now also possible to enter negative prompts. You can also influence the style using a library of reference images or even upload your own reference images. And in the photo settings, you can now set the aperture, shutter speed and field of view.

With the Edit menu you can now also use the generative fill and the removal of unwanted elements as in Photoshop, or crop the foreground object. Although the file can then be downloaded as a PNG, the mask that has just been created is not taken into account, but the complete image with background is saved. You can transfer the image directly to the browser version of Adobe Express.
There, however, you must first set the background colour to transparent and then download the cut-out object, this time with the alpha channel.

This is unnecessarily cumbersome and I hope that Adobe will simplify this workflow by the time this article is published. It’s still in the beta stage. Firefly and Express continue to grow together. The web version of Express also features AI image generation, generative fill and background removal. With simple animations in several individually animatable layers and the integration of videos and music, short motion graphics trailers can be created without having to open After Effects. The “Text to template” module is also new. This allows editable templates to be created using text prompts, which can then be easily customised to your own requirements. Design projects can now also be synchronised with Photoshop and Illustrator. For example, Photoshop and Illustrator files can be thrown directly into the browser window. They are then converted by Express into editable templates with layers. With Illustrator files, all vector layers are retained – and the text also remains editable. I have just received the beta update of the mobile version of Express and it is hardly any different from the web version. You can now open files from the mobile version in the web version again and even work in parallel. As you can see, the development of AI integration at Adobe is making huge strides – this helps with brainstorming and makes some time-consuming edits easier.

Illustrator

In the current beta, text to vector graphics is now also possible. This means that even “vector legastics” can generate interesting templates. There is a choice of motif, scene, symbol and pattern in the settings. The generated graphics can also be coloured using the AI-controlled colour selection. the new modelling function also makes it easier to create interactive mock-ups. An image is analysed for its geometric properties and the vector object can then be placed correctly on the surface in terms of perspective.

The image generated with the prompt “SciFi Spaceship Interior with large Monitor” was analysed via the Object>Model>Create menu. The star logo could then be placed on any surface. The preview on the right shows the logo on various mock-ups from the Adobe Stock Library. Live Text Retype Beta can make texts that have already been converted into vectors editable again. The appropriate font is searched for and a suitable one can be selected from a list. In principle, this even works with texts on pixel images. An attempt is also made to reconstruct the background using Generative Fill. Depending on the material, this works more or less well. But it’s still beta. Another new feature is the Smooth tool, which smoothes vector paths automatically or with a slider. In contrast to the simplify tool, peaks are also rounded here.

The post Update to the MAX first appeared on DIGITAL PRODUCTION and was written by Nils Calles.

Nowadays, you don’t have to be an engineer or electrician to make your own tools. Thanks to the technology available today and the wealth of information available, many problems can be solved. Building your own equipment is particularly appealing. Not only does it allow you to customise it to your own needs, but it also creates a deeper connection and appreciation for what you create. In this article, I want to show that everyone has the opportunity to make their own tools. To do this, I will present a project that is very close to my heart. I will explain a problem and the steps to solve it, with the confidence that many people can find approaches to overcome their own creative challenges.

by Robert Krupa

The construction shown here is a DIY film scanner for Super 8 films. This device makes it possible to copy analogue Super 8 films into a digital file format in very good quality. Each film frame is scanned individually to avoid image interference such as ghosting or flickering. A conventional camera is used for digitisation, giving the user a high degree of control over the image quality.

• Links!
  • My website: RO-KR.net
  • LinkedIn: linkedin.com/in/robert-krupa-3d
  • Website of the Hamm-Lippstadt University of Applied Sciences: www.hshl.de
    (Computer Visualistics and Design programme, contact person Prof. Rainer Baum)
  • Arduino: www.arduino.cc
  • Electronic camera control: www.doc-diy.net/photo/remote_pinout
  • Adapter for microscope lenses: bit.ly/youtube_makrolinse
    

Where did the idea come from?

It all started when my aunt gave me a large collection of old Super 8 films. First I got myself a film projector and learnt how to handle the films properly. After a short time, I had successfully played the films. Of course, I wanted to show them to my aunt, so I filmed the projected image from the screen. But I soon realised that there was a problem. The Super 8 film runs at 18 frames per second, while my mobile phone films at 30 frames per second. Due to these different frame rates, the image flickered a lot, double exposures and ghosting artefacts appeared. An initial solution was to equalise the two frame rates. The film projector had the option to increase the film speed to 24 frames per second, which was also the lowest frame rate my DSLR could record. I therefore filmed the projected image again at this adjusted speed and later adjusted the speed back to 18 FPS on the PC. This led to a considerable increase in quality, as the flickering was now greatly reduced. However, there were still ghosting effects and slight fluctuations in brightness. The reason for this was obvious: the film projector was an old device that was relatively simple in design and, above all, mechanically operated. There was no digital computer to control the timing down to the nanosecond. This did not play a major role in the viewing of the films
Role. As long as the image reproduction was around 18 frames per second, minor timing differences were not particularly noticeable. For the digitisation of the films, however, this was a decisive factor.

However, there were other problems such as vignetting and blurring. So I looked for other options to digitise the films. One option was to send the films to a digitisation service, although there is always the risk that they could be lost. Also, film digitisation is expensive. With the amount of films I have, digitisation by a service was not financially viable for me. Another option was to buy or rent a film scanner. There are film scanners for home use that allow users to digitise their films themselves. However, there are problems here too. There are examples of the quality of such scanners on YouTube. They have a low dynamic range, fluctuations in brightness and oversharpening artefacts. In addition, the colour reproduction is very poor and colour fringing occurs. I therefore wanted to find a cost-effective solution that would give the user full control over the digitisation process and deliver good quality digitised films.

Structure of the device

The design of the device is very similar to a portable analogue tape recorder. A stepper motor rotates a reel that moves the film one frame at a time. The film is fed through a device that illuminates it from behind with an LED so that the respective film image can be photographed from the front with a camera. Between each rotation of the reel, the film stops for a moment and an electrical signal is sent to the camera to trigger it. An Arduino microcontroller is used to control the electronic components.

For automatic film unwinding, the machine has two turntables on which the film spool and an empty spool are placed. These are each driven by a belt with a motor. The films lie on a flat disc that rests loosely on the turntable. This keeps the film taut and reduces the risk of tearing due to excessive tension.

The film gate

The film is guided through a 1 mm wide gap on a 3D-printed device, the “film gate”. There is an area where the film slides freely through the gap, i.e. without touching the device. At this point, an LED with a satinised plastic is mounted behind the film. This illuminates the film with even light so that the film image can be optimally photographed. Further along the film guide, a light barrier is mounted exactly at the height of the film’s perforation hole. This ensures that the film remains in position. The camera with the macro lens is placed in front of the film gate so that the film image is in focus.

Operating the device

The device is equipped with a control panel that is connected via a cable connection. The control panel consists of 9 push buttons that can be used to perform various operations such as forward and rewind, manual release of the camera or precise positioning of the film image. In addition, user inputs are shown on a small LCD display to give the user full control over the scanner.
To connect the control unit to the Arduino, it was important to reduce the number of tracks. As the control unit has 9 pushbuttons, at least 10 tracks would otherwise be necessary. Instead, the buttons are connected in series with electrical resistors. Depending on which button is pressed, a certain voltage drop occurs, which can be measured via the analogue inputs of the Arduino. In the scanner software, an if query can be used to define a specific voltage range in which the respective commands are executed. See the pictures on the next page.

Controlling the camera

In order to capture the individual images, it was important that the camera was triggered automatically at the exact moment when the film image was in the correct position and the film reels were stationary. I bought a remote shutter release for this purpose. My first idea was to bridge the mechanical push button with a transistor to send the release signal to the camera. However, when I unscrewed the remote shutter release, I realised that only 3 contacts are short-circuited inside, which means that the remote shutter release is ultimately not necessary.

From this I was able to derive the circuit diagram above: Depending on which switch is pressed here, either the autofocus or the shutter is triggered. The majority of all camera models work according to this principle, which makes it easy to use different cameras. As the lens I am using does not have autofocus, only the switch for the shutter is important here. Instead of a mechanical pressure switch to close the circuit, a switch that can be controlled via an electrical signal had to be used at this point. In my opinion, the best solution for this is to use an optocoupler. This works in a similar way to a transistor, with the advantage that, unlike a transistor, the respective circuits are separated from each other. This means that there is no risk of damaging the camera by applying voltage.

In my case, I constantly connected the focus connection to GND. In hindsight, it would have been better to control the connection with an optocoupler as well. Otherwise, the camera can no longer be operated, as the permanent connection of the focus contact to GND simulates a permanently half-pressed shutter release and thus blocks the operation of the camera.

Correct colour reproduction

To ensure that the colours of the film are reproduced as faithfully as possible, a neutral white LED with a high colour rendering index (CRI) of over 90 was used. This means that the colour information is reproduced with a very broad colour spectrum during digitisation. In addition, the white balance of the camera is set to the Kelvin value of the LED in order to avoid unwanted colour casts.

Using the scanner

After the film has been fed through the film gate and attached to the take-up reel, a camera with a macro lens is positioned on a tripod so that the individual film image is focussed and can be seen with an overscan. The scanner has a 2.5 mm jack connection at the front, via which the camera receives the trigger signal. The camera is then connected to the remote trigger input and the jack socket via a cable.

At this point, the control unit is rewound to the position at which the first image in front of the
Image appears in front of the LED and on the camera image. The camera must be set to manual mode so that every picture is taken with the same settings. The automated digitisation process is started by pressing the red button on the control panel. Once the film has run through completely, the film can be rewound.
Practically any camera can be used to take the pictures as long as it has an electrical input for a remote shutter release. In my case, I opted for the Sony Alpha 6000 as it is relatively inexpensive and can also take pictures in RAW mode. This allows more freedom in terms of colour and brightness adjustment in post-processing.

I also used the 25mm ultra-macro lens from Laowa, as I think it is perfect for this application. A cheaper alternative, which I have not personally tested, is to use a microscope lens with a 3D printed camera mount. There are instructions for constructing this on YouTube.

Post processing

After the film has been completely scanned by the scanner, the images are now available on one or more SD memory cards. These must now be copied to a computer and combined into a video. To do this, I made a rough histogram adjustment to the RAW images. Essentially, it’s all about ensuring that no clipping occurs and that the image matches the original as closely as possible. The individual images were then exported in lossless TIFF format. The advantage of the images is that the files are already numbered sequentially. This makes it relatively easy to convert them into a video as an image sequence. To simplify further processing, a proxy file was created from the TIFF sequence. This is a file of lower quality and size, which requires less computing power
and is therefore better suited for further processing. As the film image is photographed directly from the film, it is initially inverted, which is why it has to be mirrored back on the computer.

Position errors must also be corrected, which can occur if the film images are not always in exactly the same position during the digitisation process. To correct this, the film’s perforation hole is best suited as a tracking point. Once the image stabilisation has been carried out, the image section can be defined. My personal preference is to fit the film image, including the perforation hole, into a 16:9 image. This means that no image information is lost and it is immediately recognisable that it is a Super 8 film, which gives the film its own charm. Alternatively, part of the edge of the film could also be cropped to create a clean image edge. Finally, the proxy file is exchanged with the TIFF sequence again before a new TIFF sequence or your video file can be rendered.

Digitisation of the soundtrack

The sound is not digitised directly from the scanner, but in a separate and simpler process. The advantage of the soundtrack is that the audio signal can be output in electronic form for speakers or headphones on the projector. On my projector there was an electronic socket for a “DIN” connection. It was easy to find an adapter to 3.5 mm jack. The output audio signal only needs to be recorded with a digital device during film playback. I used a field recorder for this, but it should also be possible to record the signal directly via a line-in input on a computer. The resulting digital file can then be synchronised with the digital image on the computer. The timing often needs to be adjusted. It is important to note that the pitch also changes when the timing is adjusted, i.e. the sound becomes lower when the audio track is extended. This has the same effect as a slow running tape, where the voice sounds lower.
The quality of the sound is relatively good, but there are occasional noises such as crackling and hissing. This is not surprising with old analogue tapes. Such problems can be eliminated very easily with the help of digital processing.

Replica of the scanner

The implementation of the project required knowledge in various fields, including electrical engineering, programming, CAD modelling and mechanical engineering. Nevertheless, everything was kept at a simple level, which meant that a replica should not be too difficult. Through my degree course in Computer Visualistics and Design, I was able to successfully apply some of the skills I had acquired, such as CAD modelling and programming. However, I also had to acquire new knowledge, especially in the field of electrical engineering. In addition, I was able to significantly improve my manual skills, such as soldering, through the project. I received valuable support from the competent assistants at our university. The university has a project workshop that I was able to use. I had access to a stationary drilling machine there and also received valuable tips.

My plan is to provide comprehensive design instructions as well as all the necessary parts and software to enable easy replication. When this is ready, we will put a link on Digitalproduction.com!

One of my aims with this project was to use only components that are open and readily available, rather than, for example, converting a film projector. As this is old technology, not all parts are available for purchase. For this reason, I also made my own components using 3D printing. For example, I made spools with teeth that fit exactly into the perforation of the film to be able to move it. To develop the components, I took measurements from the existing film spools with a caliper gauge and created the 3D files using CAD. The components were also matched to different motors. I used two small N20 DC motors (12V, 300 rpm) and a NEMA 17 stepper motor. In addition, I used a laser diode with a corresponding sensor to detect the perforation hole of the film. However, this function is optional and does not work for all types of film.

The electrical components are controlled using an Arduino microcontroller and a motor control board from Adafruit. In addition, a small LCD display shows how many pictures have already been taken. First of all, holes and cut-outs have to be drilled and filed in a tool case. The case serves as a housing for the device and makes it easy to transport or store the apparatus. The cut-outs are necessary to mount the LCD display, the power connection and the connection sockets for the camera, control unit and interface for the microcontroller. Brackets are also attached to the inside of the case. A Plexiglas disc (or alternatively a metal sheet) in the same format as the case is then produced with holes and cut-outs for the various components of the scanner and mounted on the case with hinges.

The 3D-printed components also have to be assembled, for example mounting devices are attached to the motors and pulleys are attached to the motor axles. The components are then attached to the mounting surface. This is followed by the wiring of the electrical components. For this purpose, breadboards were used to which the components were soldered.
A major advantage of the self-build is that the project can be easily adapted to other requirements.

Here are some ideas or possible improvements

• Remote triggering of the camera via wireless methods: Triggering by cable is not possible with every camera and is also somewhat cumbersome, as the cable connection makes it difficult to position the camera. One solution could be to control the camera wirelessly via Bluetooth, WLAN or infrared signals, provided the camera is suitable for this.
• Wireless control: In the same way, the control panel could also be converted into a remote control. For this purpose, the control panel could be equipped with its own microcontroller, which processes the signals and outputs a signal via an infrared LED. The signal could then be received and executed using an IR sensor on the scanner.
• Recording the images with a smartphone or a WLAN-capable camera: One disadvantage is the large amount of memory required. In my case, a 128 gigabyte memory card was enough for around 5000 images, which requires several memory cards and copying processes for longer films. It would therefore be conceivable to transfer the individual images directly to a cloud or private server during the digitisation process. This can save time that would otherwise have to be spent copying the images and replacing memory cards.
• Adaptation of the scanner to other film formats: Depending on your needs, the scanner could easily be adapted to different film formats. For example, if standard 8mm films are to be digitised, spools could be produced that are adapted to the perforation of the standard 8mm film.
• Development of security mechanisms: At present, the digitisation process requires constant monitoring. There are no mechanisms that automatically switch off the scanner if, for example, a belt breaks and the film is therefore no longer wound up. This could cause the film to get caught elsewhere and be damaged. Sensors could therefore be attached to the winding spools to interrupt the digitisation process if the film is no longer being wound or unwound correctly or if the film has been completely scanned. It should be noted that no damage to the film has occurred in my tests to date.

Advantages and disadvantages

The biggest advantage is the cost saving when using large quantities of film and the high quality results. This also makes a replica of the scanner interesting for smaller video productions. For example, it is quite conceivable that the scanner could be used for the production of wedding videos,
as Super8 film is popular due to its characteristically unmistakable image. In addition, in contrast to commercial film scanners and thanks to the simple design, it is relatively easy to carry out repairs or modifications. Disadvantages, however, are factors such as the slow scanning time and the time required for post-processing. In addition, the scanner has no safety mechanisms, which is why the digitisation process should not be left unattended.

Possible improvements

The scanner still has room for improvement in many areas. For example, at the film gate, where the film is currently only fed through a gap about 1 mm wide. This prevents abrasion of the film, but sometimes the film is not exactly in focus. Therefore, for the next iteration, I would like to develop a film gate that guides the film over more spools to keep it a little tighter without the film being subjected to too much friction. The backlight should also be better shielded. The current film gate lets too much light past the sides of the film, which leads to reflections in the lens. I solved this problem by attaching some insulating tape to the film gate to block out the light. However, the light is also dazzling during use. In addition, the wiring is still a bit confusing at the moment. This would be remedied by a separate circuit board that makes the electrical connections and provides connections for the camera and control unit. This board should be made in the form factor of an Arduino shield so that it can simply be plugged onto the Arduino.

Looking back, the most complicated part was the conceptualisation. The actual manual assembly is very simple compared to this. This mainly involves drilling holes, filing some of them out and then fastening the components with screws. On the electrotechnical side, parts have to be soldered or plugged together. Therefore, a replica should not be very complicated. For me personally, this project has above all taken away the fear of facing new, unknown challenges. Subject areas such as electrical engineering were previously difficult for me to fathom. By building the scanner, I was able to gain a basic understanding of it.

I can advise anyone who wants to tackle this or a similar project: Just get started, don’t be afraid of making mistakes. There will certainly be points where you get stuck. At such times, it makes sense to continue elsewhere until you find a solution. There are enough people who are willing to help, and it doesn’t cost anything to ask. My plan is to publish the construction plan, the circuit diagram, the software and all the 3D data. I hope that this will allow the project to grow and improve. I have already worked on a first version of the construction manual as part of my bachelor’s thesis. This will be available online and will be easy to understand, even for non-experts, as each work step is shown in an interactive 3D visualisation.

The post Building your own Super 8 film scanner first appeared on DIGITAL PRODUCTION and was written by Bela Beier.

Let’s start with transcription: The value of a new feature is, of course, extremely dependent on your workflow and specific needs. But for documentaries, especially with interviews, AI-based text recognition (only in the studio version) is perhaps the most useful new feature in 18.5. It can even be helpful in feature films if the director likes to let the actors improvise.

We tested speech recognition on an entire feature film in order to test it under critical conditions, such as music and strong background noises. Even on a modest MacBook M1 Pro, it ran surprisingly quickly at 4 minutes and 15 seconds for an hour and 42 minutes of film. But then the typical behaviour of today’s AI became apparent – the result was somewhere between Wow! and What? Sometimes we were extremely impressed when it still recognised the text correctly despite the film music, where we had difficulties ourselves and had to listen several times.

But there are also a lot of results that don’t (yet?) replace humans. In passages with long periods of silence or only quiet noises, it started to hallucinate. And not just in German, although we had switched from automatic to our language. Chinese or Korean text could still appear from time to time. Or absurd poetry such as “If you think the body Bugün? Cve is that giant?”

And who says AI isn’t creative? Word creations that probably don’t appear in the internal dictionary were, for example, “Kapitaneverbräucher” as an interesting alternative to “Kapitalverbrecher” or a nightmare that mutates into “Eiltraum”. Sometimes, however, the context is ignored in such a way that “photo album” can become “fodder album” when spoken quickly.

All of this may be rather amusing, but word turns that change the meaning, such as “What happened?” instead of “Did what happen?” can easily be overlooked. Strangely, sometimes even longer, clearly understandable passages are completely ignored. Conversely, the AI occasionally inserts a short text passage several times, which actually only occurs later in the film. In short: you can’t get by without careful checking and manual correction, although the function is better than in Premiere.

Editing assistant

Nevertheless, the AI can save a lot of time, especially when editing, because the recognised and, if necessary, corrected text remains linked to the image. If you have selected a text passage, of course also via text search, the corresponding point in the film is shown at the same time. A moving cursor appears in the text while you play the video. As corresponding cut points are also set temporarily, a section can be cut into the timeline immediately. This works in both the Cut and Edit pages. The corresponding functions can be found directly in the transcription text window, from playback buttons to insert or append to setting markers and creating subclips.

There are also text functions such as search and replace, changing the font size and optional display in black on white. Only major changes to the text length in this window can have unintended side effects when editing. Therefore, there is a copy function for the external use of text, but no paste function. The backspace key crosses out selected passages, but does not delete them. This results in the passage being omitted during editing. Of course, you can also export the entire text here if required. The text window can be freely configured and moved to a separate monitor.

Subtitles

As the transcription is based on text and timecode, it makes sense to generate subtitles automatically in a similar way. The “Create Subtitles from Audio” function is available in the timeline menu for this purpose. This takes a little longer than the transcription, around three and a half minutes for half an hour of film. After that, the subtitle track contains clear texts if the speaker, in this case Cullen Kelly, was clearly audible. This time we tried a tutorial from Blackmagic (BM for short), in which the voiceover was completely clear and could even be heard without background music (it works!).

Now we wanted to create German subtitles for this video using colour management, as no German version exists in this case. In principle, this also works if you simply enter German instead of English in the dialogue box for the title generation, but unfortunately such an internal translation is not really convincing. We have therefore renamed the exported *.SRT file to *.TXT and had it translated by the respectable DeepL. Unfortunately, this is only possible with the full version, as otherwise the time codes and line breaks in the file are not retained (but there is a test period).

We only had to rename the result to *.SRT again and were then able to import it as a subtitle file. Thanks to better technical conditions, the text recognition in English was already more accurate, as was the translation by DeepL. Nevertheless, it is essential to review the foreign-language version and correct it if necessary. This applies all the more to the German translation, as even DeepL does not yet have a complete grasp of technical terms such as those used in this complex topic. What, please, is a flat log state? The appearance of a log file..

So here too: Trust (in AI) is good, control is better. Nevertheless, it can save a lot of time in both use cases, depending on the sound quality and level of sophistication of the language. You will want to revise the formatting of the subtitles for high-quality jobs, e.g. only a minimum time interval can be specified in the dialogue and different speakers are not recognised. Finally, line breaks and timing are not always perfect.

Classification

Last but not least, another delicacy of the AI for audio: it can classify the sound of all clips according to criteria such as dialogue, effects, music or silence. It adds sub-categories to these categories if it has recognised something, such as sirens or dogs barking. These terms appear in the audio metadata under “Category” and “Subcategory” and can of course be corrected or added there. You can then sort the material into “Smart Bins” on this basis – making your work even easier.

Relight

Let’s move on to the image, where the AI has also learnt something new. Relight is intended to simplify tasks that previously required complex Power Windows with tracking and therefore could not take into account the spatial situation in the image. Similar to Depth Map, Relight calculates the spatial constellation in the scene. On this basis, you can then place a directional light source, a point light or a spotlight and subsequently change the lighting.

Once again, this is as amazing as it is difficult. Just as Depth Map cannot replace a green screen, the separation from the background is also problematic here. This can already be seen in the map preview: The AI has recognised the spatial situation correctly, but the mask is not clearly separated from the distant background. If we try to soften the lighting in the foreground, the result is unfortunately a “halo”.

Although Relight offers very versatile adjustments and even a connection to the tracker, this fundamental problem cannot be solved with this alone. You need precise masks or a key again. An additional ‘magic mask’ can help to a certain extent, but it is not always clearly delineated enough. On the other hand, it is clear that there is considerable potential with green screen if the background is replaced anyway and the lighting situation needs to be adjusted afterwards.

Another use case would be “Day for Night”, or rather twilight instead of night. If you can’t replace all the lamps with more powerful ones, they often can’t compete with the rest of the light. In this case, the function is very well suited to convincingly amplifying the light in their surroundings. The advantage here is that the angle to the light, especially in buildings, is correctly taken into account. Because you often need several light sources for this, but the effect is computationally complex, you can use an analysis for several nodes. Casey Faris shows this quite well in a YouTube tutorial.

To understand the function: Relight does not generate a light source itself, but only defines its area of influence in 3D space. The result is a mask with grey scales, which then controls the effect of all the usual grading settings. Incidentally, the surface map is compatible with the method often used by 3D software for the alignment of surfaces. Such image data, usually called “normal maps”, can be exported and imported for corresponding tasks. This opens up far-reaching possibilities for the integration of real video and CGI.

Blackmagic Cloud

Previously, only the project data was exchanged in a cloud account at BM. The transfer and synchronisation of proxies or even raw material had to be done via a Dropbox or Google Drive account. This was obviously not always easy, so BM now offers its own cloud storage (currently still in beta). This costs 15 US dollars a month for 500 GB of storage space. For comparison: Dropbox Plus costs 12 euros per month or 99 euros per year, so you can use 2 TB of storage space, but can only exchange up to 2 GB of data per day.

Google’s cheapest offer initially costs 1.99 per month for 100 GB. This makes the BM Cloud seem quite expensive. But apart from the fact that you can cancel at any time, it is also more convenient. With the right settings, proxies are automatically generated and synchronised in the background. In addition, the desired storage space can be adjusted at any time, with precise billing after cancellation, even for parts of a month. The Project Libraries still cost an additional 5 US dollars.

With a fast connection, DaVinci Remote Monitor can be used to stream a session in DR to another location. The image quality is so good that it is even possible to judge the image on a calibrated monitor.

All computers must be equipped with DR Studio and with Apple Silicon or for Windows and Linux with RTX Nvidia GPUs, other GPUs are currently not supported. You can use the free app of the same name on an iPad or iPhone. In addition, all participants must have a BM Cloud account. Of course, the connections are protected with a session code.

Improvements and additions

So much for the most spectacular functions in the studio version. The AI-based Depth Map previously found in DR is now also available in Fusion. Magic Mask has been given new fine adjustments, similar to conventional keys, with an additional parameter for “Consistency”. This allows you to better define uneven mask edges over the course of the clip. For this to work, a “stroke” must exist over a sufficient number of individual frames.

You will often use the cache for such work, especially on somewhat weaker computers. If any inexplicable errors occur, it usually helps to simply delete the cache and regenerate it. It doesn’t run smoothly yet, especially when combining AI-based tasks and tracking with masks. In addition, you should not work with reduced “Timeline Proxy Resolution”, the other options for smoother work with limited computing power work better.

Timelines can now be quickly saved as individual backups and you can set the colour management for each timeline separately. The cut page has been given many new functions, including subtitling including the speech recognition described above, simpler split edits, as well as cut detection and the optional creation of empty spaces in the main track. Anyone still struggling with interlace video will breathe a sigh of relief, as the cut can be correctly limited to the full frame limit.

The Media Page now offers the export and import of timelines in OpenTimelineIO format. Timelines can be transferred directly to the render queue from here, with immediate selection of a preset and the storage location. When rendering individual clips, the complete originals can be output instead of the edited version. Power Bins can now also be exported or imported. DR also understands the latest XML format from Final Cut Pro.

Fusion now recognises the USD format (Universal Scene Description), which is becoming increasingly widespread and is also supported by the free Blender. There is also a specialised toolset including MaterialX Framework and support for USD Hydra renderers. Multi Merge simplifies the combination of several layers and a native Depth Map can be found in the studio version
the studio version. Clean Plates and Anaglyphs now have GPU support, and the splitter has also become much faster. Many will also be pleased with the 3D extrusion of shapes, with bevelled or rounded edges if desired. Corresponding shapes can also be designed with the sPolygon Node.

New formats

The latest SDKs have been integrated for BM and RED cameras, and XAVC H and HS from Sony now also work. Apple Log is supported, but unfortunately still no ProRes RAW. Playback of AC3 sound is finally available under Linux and Macs decode AAC with low latency. Compression to ProRes, AV1, H.264, H.265, MP3 and AAC is now offered for the MKV container, plus FFV1 in MKV or QuickTime. Customised render presets can be written as XML files and can be transferred to other systems to achieve identical results (provided the hardware encoders play along).

Speed

HDR material can be graded faster. On Apple Silicon, the flat noise reduction has become significantly faster, but the biggest speed gains this time are for the AI tools on the Nvidia GPUs with TensorRT. The first time the current version is started on a corresponding computer, an optimisation run is performed. As this is still a fairly new process, BM has cleverly provided a switch-off option this time. But even with newer AMD GPUs, such as the RX 7900, there are impressive speed gains for the AI tools.

Comment

With so many new features, users should be quite satisfied, shouldn’t they? There’s even more detail, including some long-cherished wishes. But unfortunately this time Blackmagic has managed to unleash an extremely unfinished version on the world. Presumably the IBC was the occasion to snatch a software from the developers’ hands that would have been more suitable as a public beta. Even the hastily delivered 18.6.1 still caused considerable problems. Only 18.6.2, which we tested, runs reasonably smoothly.
For the future, we would like the developers to concentrate a little more on bug fixes and a little less on new features.

Speed on PC hardware
We would like to provide you with a few more benchmark results from PCs for our test of Mac Studio from DP 23:05. Thankfully, these were created with the dedicated help of several members of the German DR forum. They also showed that 18.6 was mostly faster than 18.5.

A Ryzen 9 3950X with 128GB RAM and the RTX 4090 with 24GB VRAM was already way ahead with 18.5.1, with 6:11 at UHD and 15:57 (!) at 8K. The values for H.265 (Setting Master) and DNxHR HQX 10 Bit were almost identical. 

A Ryzen 7 5800x with 48GB RAM and an RTX4070 managed 13:25 in UHD in DNxHR, but needed significantly more at 21:05 due to the lack of a hardware encoder for H.265. this computer did not manage 8K. On the Intel side, a "Hackintosh" based on the 13900KF with 64 GB RAM and the Radeon RX 6900XT was quite fast in UHD with 11:40, 8K was not tested.

Under Windows, an Intel i9-10940X with 64 GB and the GeForce RTX 2080 Ti needed 22:08 for H.265 and 21:22 for DNxHR HQX.  An i9-9900K with 32 GB RAM and the GeForce RTX 3080 was significantly slower. Under DR 18.6, it took an hour and 6 minutes for H.265, and even a little longer for DNxHR. 8K was not tested.
An i7 9700K with 32 GB RAM and the inexpensive GeForce RTX 3060TI needed one hour and 34 minutes for UHD in DNxHR and two hours and 12 minutes for H.265 in 10-bit (8K was also out of the question here).  The detailed results and their authors can be found here: bit.ly/bmd_forum

Another useful benchmark for Neatvideo is here, including the results of common devices: bit.ly/neatbench

The post Blackmagic DaVinci Resolve – the Sequel first appeared on DIGITAL PRODUCTION and was written by Uli Plank.

With so much stuff – where to start? The fine new “little things” that would justify an entire release for some other software manufacturers or the big chunk? Suggestion – in order to be able to devote ourselves undisturbed to these chunks aka the new animation system, Feathers and Solaris/Karma updates, let’s take a look at the “sideshows” first and free up a lot of mental working memory.

Texture Mask Paint SOP aka Houdini 3D Painter

Houdini has long allowed masks and attributes to be painted directly onto the mesh, but until now only on the basis of topology. If you want to paint detailed masks, you need a correspondingly high-resolution mesh. The new Texture Mask Paint SOP, on the other hand, works on a UV basis and is therefore completely independent of the topology.

The interactive tool not only offers the expected paint function, but also brush features that are more familiar from specialists such as Photoshop or Substance Painter. The applied brushstrokes can be softened with Smooth or smudged with Smudge. Texture and Stamp allow you to paint with textures, while Dirt and Cloud loosen up the brushstroke with noises. The whole thing also works symmetrically.

The painting can be output as a volume mask based on the UVs, which can be used for any other purpose with the usual volume tools or simply used as a mask for the new Feather tools, for example. Alternatively, the volume mask can be exported via Heightfield-Ouptut as .exr and co. and used as a texture mask for hairgen/grooming or shading, for example. The appropriate node for scattering points into the freshly painted mask is called Scatter in Texture Mask and is able to understand the opacity or luminance of the paint node as a density multiplier. If you want to go deeper with the brush, you can use the free HPaint HDA(is.gd/hpaint) by Aaron Smith, a Houdiniesque version of the Grease Pencil.

Quad Remesh

Long wished for and finally here – a native Quadremesher. Although it is still in beta, it already delivers very good and, above all, very fast results. As soon as the node has the first remesh after a few seconds, the number of polygons can be changed almost in real time. Guides can currently only be created indirectly, but this will certainly change in the final version. Until then, you can draw a curve on the mesh, calculate the tangent using polyframe or orientalongCurves and then transfer it back as a guide attribute.

Clouds

If the current weather isn’t cloudy enough for you, you can use the completely new cloud tools to create hyper-detailed hero clouds or even an entire skybox full of clouds. A Cloud Shape Noise serves as the basis for the former, which can either be fed with any geometry or create a structure close to the shape of a cloud itself. This basic shape is ripped apart, processed and transformed into a realistic cloud using new cloud noises (Cloud Billowy & Cloud Wispy Noise), which were created from the detailed analysis of real cloud structures. As these are classic volumes, any Houdini tool can be used to further customise the shape.

The clouds generated in this way can be rendered physically accurately using the new cloud shader with multiscattering. If you want to go a little faster and don’t want to be completely precise, you can use the Cloud Ambient Occlusion Shader, which delivers very useful results for background clouds with a much faster rendering time.

The skybox is similarly direct, with a choice between 2D skyfields and 3D clouds. The former are configured with the new Skyfield plus Skyfieldpattern node, which comes with various cloud type presets, and then designed using the Skybox node. The Skybox alone is sufficient for 3D clouds. The cloud nodes mentioned above offer the option of applying further deformations.

Bubble Solver

A by-product of the new cloud tools is the bubble solver, which takes care of the correct interaction of bubbles. Overlaps are avoided and the individual bubbles are pressed against each other and deformed accordingly. As input, the node requires nothing more than a point cloud (particle, cloud generator or simply a grid) with the Pscale attribute set.

Ripple Solver

Another very simple tool (almost too simple for Houdini) for simulating wave movements on geometry, for example the effect of raindrops on water, but also the impact of shock waves on character. The waves can be output as a float attribute for further use in shading or can be used for other purposes, for example as a density attribute for scattering.

Solaris/Karma

The most important innovation here is the gold status of Karma XPU. The beta status has been cancelled and a whole host of new abilities have been added. These include the complete Light Filter Shader Set, Round Edges, Thin Wall (see Bubble Solver section), production boosters such as Cryptomatte and Deep Images, Absorption and Nested Dielectrics, faster First-Time-To-Pixel and generally increased speed, especially for Fur and Volumes. To be able to marvel at the latter, the render gallery now comes with an extensive statistics panel.

To go with the new Cloud Set, there is finally a Physical Sky (Hosek et al), which in principle combines the new Sky Dome Light Lop with a Distant Light under the bonnet and thus offers the usual Houdini flexibility – if desired. A lot has also happened in the area of shaders and material management. The new material library integrated in the material linker is a good way of creating a material collection more easily and assigning it via drag’n’drop. Some presets and access to the AMD Mtx library (online access) are included out of the box, but could be a little more user-friendly. If you are looking for an intuitive alternative, you can take a look at ODTools with its extensive asset library – is.gd/shelf_tools.

The Karma Material Builder has been tidied up and is now equipped with the Geometry Properties Node to save time. A new fur shader offers settings for medulla manipulation, the inner core of the hair, so to speak, which in turn promises realistic lightscattering.

As MaterialX is subject to constant growth and change and some essential (Uber) nodes are missing or have to be put together yourself, SideFX has thankfully supplied a whole host of new custom nodes, e.g. a ramp node. Fog Box (Uniform Volume for fog and other atmospheric effects), Background Plate, Ocean Procedural and Tone Mapping (Reinhardt and co.) also make it possible to work faster with XPU.

The new cloning offers the option of rendering and comparing the scene in parallel at different times or settings, for example to adjust the effect of a light at the beginning and end of a shot – temporally at the same time, so to speak. The Solaris toolset has also been optimised in many areas and offers, for example, the latest ACES support (applies to all areas of Houdini, not just Solaris), new lighting views and the RBD Desctruction LOP, a procedural that transforms RBD pieces at render time. Mplay can now export video formats natively without an FFmpeg installation, and not just from Solaris.

With its seamless integration, Karma can therefore be used very well as a production renderer and not only saves 3D party engines, but also a lot of setup time. Karma can also be used as a standalone Hydra Delegate from next year. Further in-depth information that would go beyond the scope of this article, as well as wonderful examples, can be found in Olaf’s article – already in progress!

Feather

In addition to the Clouds, the new Feather system is another highlight of Houdini 20. To provide an efficient solution to the complex production challenges of feathers, Houdini offers over 20 new Feathers nodes, many of which are GPU-accelerated and interactive as well as perfectly integrated into the existing grooming tools. Individual feathers can be created with just a few nodes or curves, a shaft and outline, and can be brought into the desired (dis)order with the existing grooming nodes such as Frizz and Clump (of which there is also a special Feather version).

In addition to the classic Guidemask Node, the new Texture Paint Mask Node can also be used to mask certain areas. To create as many different feathers as efficiently as possible, shapes can be interpolated to form new shapes. Once the individual feather (sets) have been defined, they can be quickly drawn or scatterned using the familiar grooming tools such as Guide-Groom.

New options ensure clean interpolation between different feather type areas. Otherwise, the Feather Pipeline follows the work steps of the Hair Tools exactly with a few special new nodes, such as the GPU-supported Feather-Deintersect. This node interactively prevents unwanted overlaps and also ensures a fluffier look, for example. The finished feathers can then be easily converted into polygons and resampled for simulation and animation. The feather fun is best rendered with the new Hydradelegate-independent Feather Procedural, i.e. memory-saving high-resolution only at rendertime.

Animation

The big star of H20 is undoubtedly the new APEX animation ecosystem, a direct challenge to the workflows of long-dead empires. Star text: Beta status with many good ideas, most of which are running well in initial tests and then still have to prove themselves in practice. Before we delve deeper into the topic of APEX, let’s take a brief tour of “general” improvements in terms of animation. In the animation editor, the areas of the curves can now be changed directly without having to use handles. A bit like the newer path tools in Illustrator and incredibly practical. Keyframes can now be moved via MMB without the mouse pointer having to remain directly over them.

The timeline has been given a useful companion in the form of the Animation Toolbar. Here, for example, breakdowns, tweens, easing, time offsets, noises or blending to certain frames or neighbouring keys can be set to speed up the animation processes with sliders. Finally, bookmarks can also be created and controlled with new shortcuts.

What is Apex and what has happened to KineFX?

In short, and this would have taken up most of the time writing this article, so I’ll quote SideFX at this point: “APEX, a brand new context designed to deliver an animator-friendly environment that is built using a robust procedural rigging toolset.” For animators, this really does mean a completely new, viewport-centric and interactive experience. The entire animation now happens in the new Apex Scene-Animate Node, which has no parameters other than Reset, only interactive viewport tools. And what tools!

The selection sets make it easier to select and save relevant controls, but it gets really exciting with new ideas such as locators and dynamic motion. The former allow us to set and save pivots on the go, so that we can move our mesh or character over these pivots away from the controls defined in the rig. The movement is not limited to just one mesh, but can also move everything that has been selected via the selection sets without any constraints.

And if you do need constraints, the new constraint tool allows you to create them directly in the viewport with just a few clicks (2: Driven, Driver, Done) – and remove them again or key the constraint state (On/Off). In addition, reference videos can finally be displayed in the viewport. Everything is controlled via the Animate Node tool menu. The ingenious Dynamic Motion Toolset can also be started here. Considering the original strengths of Houdini, it is only a logical consequence to integrate physical simulations directly into the animation tools.

Thankfully for all pure animators without DOP-Network and Co. the workflow is as follows: The animator sets raw keyframes and contact points for his motion paths, such as the trajectory of a ball or a jumping character, and then lets Houdini adjust the animation physically correctly, fine-tune it and bake it back as keys. Everything simulated can be adjusted at any time. The animator also has direct access to the ragdoll system, allowing him to fling his character around without keyframes.

Kine Fx and rigging

APEX does not replace KineFX, but is part of it. Skeletons, skinning and the like are implemented as before and then transferred to an APEX rig (if you want, you can also animate using a rig pose as before and combine the whole thing with APEX). The idea of APEX is basically a further development of the basic idea of KineFX to treat joints and co. as geometry. However, if rigs are too complex, this leads to performance losses, so that the APEX Graph System, which treats geometry as pure data, allows considerably better performance. For more in-depth information on procedural rigging, be sure to watch the Houdini Hive videos, especially the one by Esther Trilsch here: is.gd/h20_rigging.

Conclusion

Lots of new features and tools and still no end in sight – except for this article. If you are interested in the updates in the areas of RBD (sticky collisions!), Vellum, Flip, Muscles, Pyro, ethically correct machine learning, PDG and actually every aspect of the program (including “hidden”, i.e. undocumented changes such as the new “Create Digital Asset” dialogue), we recommend the extensive, albeit often very technical, docs at this point.

In the Content Library(www.sidefx.com/contentlibrary/) you can download sample scenes for many new features. And they are quite something – they show more advanced setups than just the new tool and take a second or two to calculate the node network due to caches that are not included when opening them. Once you have rummaged through the nodes, the examples are a real source of knowledge. As always, H20 is also available as a free non-commercial version.
Take a look and happy rendering!

The post Houdini 20 first appeared on DIGITAL PRODUCTION and was written by Manuel Kotulla.

By the way, all the footage is from my workshop on Nuke at FXPhd(www.fxphd.com)– and if you enter the code “DPFXPHD20” by the end of January 2024, you can get your membership for 20 per cent less – and of course there are many more great tutorials!

PxF – Kill Spill

When it comes to pulling a key, Nuke provides you with a whole cornucopia of tools. When it comes to editing the corresponding fill, however, things get pretty meagre. The Hue Correct Node is the closest thing to a proper despill tool, but it’s anything but intuitive. However, as is so often the case, the commu
nity has responded and there is a large, colourful bouquet of gizmos that can take over the tasks. In the last issue we already looked at “Despill to Colour”, today we’re looking at “PxF Kill Spill” from the Pixelfudger Suite by Xavier Bourque.

This is not only characterised by a really clear, simple interface, but above all by a tutorial that not only explains how the gizmo works in ten minutes, but also the underlying concepts so that you really know what you are doing when you turn the knobs. Pxf Kill Spill” usually delivers a very useful result the very first time you connect the tool.

The tool achieves this by comparing the channel to be despilled with the other two and calculating a corresponding limit. If the tool is now to carry out a green despill, it must not exceed the average value of blue and red. Once this has happened, the luminance is equalised. All this happens automatically. The limiter can then be used to increase or decrease the limit and the suppression can be used to change the weighting of the other two channels, in our example red and blue. If you don’t like the neutral background that has now been calculated, you can use the SpillTweak to adjust this value so that it also matches the background plate.

pixelfudger.com

PxF – Screen Clean

The Pixelfudger Suite offers yet another tool that many of us have probably sorely missed. To explain it properly, we need to make a small digression in the direction of IBK. This “Image Based Keyer”, which is certainly one of the most popular and efficient keyers in Nuke, needs a cleanplate to create a good key
a cleanplate to create a good key. This was of course either created by the extremely responsible team or generated with the help of the IBK colour node.

What happens in the IBK under the bonnet is that all imperfections, dents, shadows and spots are removed from the actual keying plate with the help of the cleanplate. This cleaned plate is then the basis for the next key that is generated. The only annoying thing about this is that you cannot access this cleaned plate to use it in the chroma or primatte keyer, for example.

PxF Screen Clean now fills this gap. Clean Plate and Keying Plate are quickly connected, the Screen Type is filled in and the piercing pink, which is set by default, is replaced by a self-sampled colour. Prepared in this way, the plate can then be further processed to your own taste.

pixelfudger.com

Additive Keyer Pro

I hear there are still some veterans who worked on Quantel equipment. Not only were these boxes bulky and expensive, they also had a feature that was the envy of all shake, nuke and flame artists – the Additive Keyer. In the meantime, however, not only have Quantel Henry and Editbox moved on to the eternal VFX hunting grounds, there are also several gizmos for Nuke that have brought the additive keyer into the twenty-first century.

An additive keyer is not actually a keyer at all – at least not in the sense that it separates pixels and the result then becomes part of the alpha channel. The additive keyer is much more of a very sophisticated merge node that comps parts of the foreground onto the background using min and max operations. This is extremely helpful for getting small subtleties such as edges, hair and motion blur into the comp. Three inputs are required for this: the foreground, the background and – of course – the cleanplate. The cleanplate and foreground are then used to calculate which parts should be screened into the comp and which should be multiplied.

So much for the theory. In practice, there is once again a large selection of additive keyers on Nukepedia, but we recommend the representative from Tony Lions here, as it has a few very nice additional features and is of course also part of the Nuke Survival Toolkit – quite rightly, by the way, this little gizmo has already saved quite a few keys.

These features include, for example, the option of working with a picked colour instead of a cleanplate and the option of packing the calculated difference matte into the alpha so that this matte can then be further processed downstream. On Tony’s YouTube channel there is, as always, a very good tutorial that illustrates the workflow well.

is.gd/nuke_survival_toolkit

Point Cloud Keyer

If you’ve ever had a closer look at Primatte, you may already be familiar with the principle: the colours are plotted in a three-dimensional coordinate system and then you use the GUI to model a kind of bubble that defines which part of this coordinate system will then become part of the mask and which will not. And as powerful as this tool is, it is also very confusing.

Isaac Spiegel has taken the concept but shifted it to a two-dimensional coordinate system. The whole thing works as follows: The artist chooses a colour space. This does not mean ACES, Rec.709 or similar, but actually the way in which our RGB image fits into a two-dimensional coordinate system. To do this, you use the colour spaces that have already mastered this in the days of good old magnetic recording. The default and absolutely sufficient for most cases is yCbCr, or, so that it rolls off the tongue a little easier, Chroma Red/Blue. Based on this selection, a point cloud of the pixels present in the plate is now generated. This point cloud can now be displayed either solo or above the plate or a slapcomp. But what can you do with it? Well, the Point Cloud Keyer has a shape input that can be used to connect a Roto Node. We can now use this Roto to isolate the parts of the Point Cloud that we want to key.

Admittedly, this is a concept that is difficult to explain. But it is fascinating to paint your key with a shape. The tool offers the above-mentioned overlay function as well as scale and position values for the point cloud to ensure that this works well. If desired, this can also be displayed more densely and the axes can be swapped for display. In any case, the gizmo is a new and innovative approach to the problem of how to separate foreground from background.

is.gd/pointcloudkeyer

Check Matte V4

Jonas Almeida brings us this little quality control gizmo. As the name suggests, the whole thing produces a matte overlay, but with a little more functionality than Nuke would like to give us out of the box. The overlay colour can be freely selected, the BG can be switched on and off and all areas of the matte that are below 1 but above 0 are displayed as an edge in a second, freely definable colour.

is.gd/checkmatte

Advanced Keying Template

Finally, Tony Lions once again contributes something to our keying arsenal. This time, however, it’s not a gizmo or a tool, but a template, i.e. a clear pipeline in which we can create our key. The individual steps, such as Core & Edge Despill, Core and Edge Matte, Cleanplate Generation and so on, are not only clearly labelled with backdrops, but the most obvious tools have also already been provided for us. A Primatte is already available for the Core Matte and an IBK for the Edge Matte. Even post processes such as light wraps have been taken into account.

Whether you want to work in such a rigid structure is something everyone has to decide for themselves. The template certainly helps to keep things organised and to take important steps into account. On the other hand, it very clearly defines a certain way of working, which may push the artist in one direction too quickly. But especially for people who are still struggling with the concept of node-based compositing or who like to work within a fixed structure, this template is certainly helpful. Even if you use it as a starting point or inspiration for your own template, it is certainly very helpful. Incidentally, there are two versions of the template: one uses “Stamps”, the tool we already presented in issue 22:06, while the other works without it.

Like the Additive Keyer Pro, the Advanced Keying Template is part of the Nuke Survival
Toolkit. On his YouTube channel “Compositing Mentor”, Tony Lions has published a multi-part and very comprehensive keying tutorial, in which not only the template, but also the exact task of each individual sub-area is explained in detail.

is.gd/nuke_survival_toolkit
youtube.com/@CompositingMentor

The post Nuke UtilitiesVolume 6 – Keying first appeared on DIGITAL PRODUCTION and was written by Christoph Zapletal.

Before we go any further: If you haven’t used it: SynthEyes is a standalone tracking and matchmoving application for camera, objects, geometry and planes – including tools for stabilisation and motion capture. It is used – among many other things – for critter insertion, fixing shaky shots, virtual sets, object removal, stereoscopic production, 360°VR, architectural previews, accident reconstruction, product placement, face and body capture, and many more – and in the 20 years it has been around, it has been one of the premiere trackers in the market – although with a strong focus on features, and not necessarily on ease of use. See the announcement and Dr. Russ Anderson in action here:

blog.borisfx.com/3d-camera-solving-syntheyes

In the current version, a few of the “highlights” are a new “Curves” colour correction tool, a PrepSet Manager, easier roto splines editing, with per-spline Invert and Ignore. As well as a camera view “RotoMask” display, “Greenscreen” Keying panel improvements, “Render Cameras” for independent preview renders and many workarounds and fixes for problems in “adjacent programs” – Syntheyes is well aware that it has to talk to any platform out there.
But enough of the preliminaries – we went to the Beach at the Amsterdam RAI and chatted to Boris Yamnitzki, CEO and Foun­der of BorisFX and John-Paul Smith, CEO Imagineer Systems. If you want to know what else they are doing: borisfx.com

The Boris FX Team at the Dell Booth in Amsterdam: Third from the left John-Paul Smith and on forth from the left Boris himself

DP: Syntheyes has been around for 20 years now. What can existing users expect now that it has joined Boris effects?
Boris Yamnitzki: Syntheyes is joining the Boris FX suite of products, which means that it will be marketed and sold and – what‘s more important – supported by a larger team. Up to this point it was basically a one-person effort – however brilliant and hard-working Russ (Anderson) is. Now Syntheyes is going to benefit from a larger team and wider technical support. In addition, obviously, there is a big road map for Syntheyes technology and we plan to take it further. Russ is of course staying with the product and is obviously going to lead the technology and the science of 3D camera solving, but there‘s a team of programmers who will be supporting him in that – speci­fically, the original Mocha Imagineer Team.
Also, we would like to invest in education and training of this product. We would like to enhance support and basically increase what I call the “product availability” – in other words, the product will be available to a wider range of users.

John-Paul Smith: I think the one important point to reiterate for the existing Syntheyes users is we‘re not planning to stop making the standalone version of Syntheyes. We‘re not planning to step away from any of the really important use cases. This involves growing the house of our industry and I know that there are particular niches where it‘s widely used and so far just our exposure to this entire community is that it is really passionate about the product, and we really want to build on that and, you know, make the most of the community engagement and take the products in the direction that benefits this community.

DP: Can we expect to see a plug-in version of Syntheyes in the future?
Boris Yamnitzki: Well, without commenting specifically on a date, yeah.

John-Paul Smith: And I just add on to that, people should see what happened to Mocha when it was acquired by Boris FX and the kind of direction we took that product – something similar will probably happen with Syntheyes.

DP: You‘ve got a pretty broad line-up now, starting with some pretty easy to use tools like Optics or Continuum to really complex applications like Silhouette or Syntheyes. How hard is it to bring these into streamlined UI/UX?
Boris Yamnitzki: Hard! But we do have a standardised development toolchain that really helps us on every platform and streamlines our software development. One example of that would be Particle Illusion – which originated (similarly) many years ago with one developer and had access to the develop­ment framework. And then it was brought on the more standard framework that our other products use. And of course, the program itself benefited greatly from that. I expect all current and future products joining the Boris FX family to benefit from that – and the developments of the platform in the future!

DP: One of the biggest trends in our industry at the moment is A.I. which is
of course always connected to the fear of A.I. replacing the artist. What is your stance on the issue? Especially looking at the tasks that artists use Mocha, Silhouette and Syntheyes for?
Boris Yamnitzki: Well, first I can mention that Syntheyes currently has an AI component in it, which was developed earlier by Russ Anderson. So, it definitely will move that forward or improve and retrain and enhance that.
In general, the company is investing heavily in A.I. development and essentially rapidly becoming an A.I. company where we spend a lot of effort in building training sets, adopting models, creating more programing models and releasing new features based on A.I. Features that were solved by traditional programming methods earlier. However, having said that, we have not found that it is a “cure all” for everything.
So there is always this final human push that is necessary for any professional pipeline and results in film or broadcast tele­vision – the last 5 percent quality that can only be done by humans. A.I can help streamline the work. It can definitely accelerate and make the projects more cost effective. And it‘s just the technology shift, or rather a technological revolution. That‘s how I would put it bluntly.
I think about it as it‘s like the internet: That was a threshold. It was a changing point for a lot of industries and a lot of markets. And so A.I. is no less and we‘re just in the beginning. I‘m not afraid of A.I., and I wouldn’t discourage anyone from being enthusiastic, but there‘ll be winners and losers, like with every technological shift, like the horse carriages gave way to automobiles. Same thing will happen here.

John-Paul Smith: We don‘t want to replace the creative process with an A.I. process. It‘s very much “assisting”, as far as we‘re concerned. And those are the areas that we‘re investing in – assistive technology. I think it‘s worth saying that our focus on the A.I. side at the moment is very much around things that are currently either awkward, time consuming or bad – meaning where the existing algorithms just don‘t give as good quality results as A.I. can. We‘re focused on making artistic tools, which help create and tell a story or, do whatever it is that they‘re trying to achieve.

DP: So we‘ve seen a lot of integration across the BorisFX line-up. Can we expect similar things for some folks or other products?
Boris Yamnitzki: Again: Yeah, without directly commenting on the roadmap. We
try to take advantage of all technologies that we have in the Boris FX Suite. Mocha made a big impact on Sapphire, likewise on Silhouette. So, you know, definitely something to look forward to in the future.

John-Paul Smith: We‘re demonstrating a kind of experimental data exchange between Mocha and Syntheyes. We showed taking Mocha mesh tracks into scenarios and then solving them in 3D. And that‘s something that we were able to put together in the space of two or three weeks of development – that‘s really promising. It illustrates the kind of thing that we would like to do in the future.

DP: Looking into the rise of smaller content creators, are there plans for simplified versions?
Boris Yamnitzki: That‘s a difficult question. On the one hand, yes, you would want everyone to be able to matchmove. The strength of Syntheyes, though, is in that specia­lised user interface. That‘s not to say that we won‘t be able to offer maybe a simplified version of that.

John-Paul Smith: The depth in Syntheyes is one of the reasons why it is a successful product. Ob­viously, the camera solving algorithm and the 2D tracking process is fantastic, but it‘s the 20 years of deve­lopment of Syntheyes. I‘ve seen so many features added in response to specific customer situations. Anything we want to make sure that we don‘t dumb it down too much. But yeah, there‘s the potential to do something different – we‘re well aware that there‘s a steep learning curve and Syntheyes and we‘re absolutely interested in reducing that. And certainly, as we look at the roadmap of Syntheyes, we are definitely going to be putting effort into the user interface and making it more accessible. Whether it will ever be accessible for a casual user, I‘m not sure.

DP: Anything else our readers should be on the lookout for?
Boris Yamnitzki: By the time everyone reads this, Syntheyes will be included in the Boris FX Suite, which is a one license for all pro­ducts and all programs, and that also goes for CrumplePop audio plug-ins, which was also recently acquired. So this is what I call „a larger availability for a wider availability“. Whilst people may not necessarily go out and buy a license and start learning it, Syntheyes will already be included in the “Swiss Army Knife” that is the Boris FX Suite.

So they will be entitled to it, it is very much available to them and that may encourage them to look into it and use it. And of course, we are working on training materials for everybody in the Boris FX Family, to use Syntheyes to its fullest potential!

The post Look into my Syntheyes, Boris! first appeared on DIGITAL PRODUCTION and was written by Christoph Zapletal.

Assets need to be quickly accessible and follow a consistent pattern so that we can complete our creative tasks efficiently. In this article, we take a closer look at how we can achieve this and what problems need to be solved along the way.

What are assets?

In the context of visual effects, an asset refers to any digital element or resource used in the production process. Assets can include a wide range of digital files and components that contribute to the overall visual enhancement of a project. There are different types of assets that can come from different sources, e.g. from previous projects, supplied by clients or downloaded from the internet by third-party providers.

It makes sense to reuse existing assets so that we can focus on the creative part of our work. If we bring these into an organised structure now, we are well prepared and can react flexibly to future changes.
For example, some VFX companies are working on ongoing long-term projects. Perhaps a second season will be shot in the future, so it is advisable to archive previous work. A well-managed asset library can be reused across projects, saving time and money.

Another example is that more and more tools are coming onto the market that use “AI” to generate images. These models are trained using large amounts of data. The better the training data, the better the result. However, this data only becomes useful when it is put into a format that can be used to learn the machine learning models.

The types of assets range from elements such as film footage or HDR images to animation rigs and 3D models with textures, shaders and simulation caches. Each of these elements needs to be treated a little differently in terms of proxy generation. Let’s take a look at what needs to be considered and how the organisation of our asset library can be achieved.

Naming convention and folder structure

Every studio has different needs and workflows. However, the basic idea is always the same. We have a number of files that we want to store in a clear folder structure that follows a commonly agreed scheme for folder and file naming, also known as a “naming convention”.

If you work on VFX projects, this should sound very familiar. The same concept can be applied to an asset library. Certain details in the file path, such as the resolution of an element, can be helpful to get some basic information. However, it is advisable to store additional information such as tags and metadata in a separate location rather than on the file system. Similar to project management software, the asset management tool also stores most of its data in a database. As an artist, however, it shouldn’t matter where the files are located. Artists just want to interact with a user interface that contains all the information they need.

For example, by separating the metadata of files on the server, we can assign multiple categories to an item. If we tried to map this with a folder structure, we would quickly run into problems. Take, for example, a film file of a burning fireplace with lots of smoke. Do we put it in the “Fire” folder or in the “Smoke” folder? If we abstract tags and categories from the file system, we can easily deal with such cases. It is also important that files are not renamed once they have been added to the library. This prevents the media from no longer being found when older project files are opened.

Single file vs. sequences

Another issue related to the naming convention is the way the file is named on the server. We are often dealing with file sequences that contain the frame counter in the file name. On the other hand, there are film files that do not need this frame counter. Images taken with a DSLR camera also have sequential naming, but the individual images may not be related and must be considered as individual files. In other cases, however, they represent a sequence, such as bracketing images of an HDR or a stop-motion animation.

Unfortunately, it is not always clear which scenario we are dealing with. Therefore, it is complicated to automate this task. At this point, we still need the user’s input to tell the asset library how to handle each of these cases. If we add an asset (e.g. a matte painting) to the library that contains a framecounter in the filename, we may want to remove the framecounter as we no longer need it and to indicate to the artist that it is a single frame. In other cases, we may want to keep the frame counter, even if it is a single frame. Unfortunately, there is no clear
Answer to the question of what is the best solution in these cases, as it very much depends on the company’s pipeline and workflows.

Example for single files:
movie.mov/img_1001.jpg
Example for a sequence:
img.1001.jpg/img_1001.jpg

What is the exact file name of an element? There are different names for files, which can vary depending on the operating system and software application. We suggest the following name:

img_1001.exr

Base name: img
Stem Name: img_1001
File Name: img_1001.exr

If the assets come from our internal VFX pipeline, in most cases we want to remove the frame counter and any subsequent characters such as dots, hyphens or underscores to get the actual base name. If the images come from a DSLR camera, we expect the file name to contain the frame number of the image (img_1234). If this were not the case, the files would all have the same name (img) and we would no longer be able to tell them apart.

In most cases, we want to recognise and display a warning if images are missing in a file sequence. With UDIMs (U-Dimension – more on this at FXGuide under is.gd/fxguide_udim), however, “missing frames” are permissible, as the neighbouring tiles are not named consecutively (texture.1001.exr, texture.1011.exr, etc.). Here too, the user must tell the software how to handle this scenario.

Category vs. tags

We want to have different options for a detailed search for elements. This is where tags and categories come into play. These are the keywords that artists will use to search for assets.
This process is similar in every studio. The reason we want to differentiate between all these categories, tags and synonyms is so that we don’t have to reinvent the wheel every time a studio wants to improve their asset management. Also, by using pre-defined terms, we can avoid duplicate tags and misspellings to keep our systems and databases clean. There is no need for two tags if they both describe the same thing. For example, singular or plural terms are often one and the same, so we can avoid unnecessary duplication.

Category: Categories are fixed classes that are constant and do not change. A category is a class or subdivision of things that have certain common characteristics, such as fire, smoke or explosions.

Tag: We can use as many tags as we want to describe an asset. A tag (sometimes called a label) is attached to a person or thing to identify it or to provide additional information. They can be considered dispensable, i.e. “the more the merrier”, to describe an asset in as much detail as possible. Tags can indicate what you see, e.g. smoke and fire, but also descriptive information such as the amount, direction and speed (“fast approaching the camera”).

Synonyms – same same but different

The beauty of the VFX industry is that we come together from all parts of the world. From this connection, we want to develop a common vocabulary that unites all our languages. The asset library software “das element”, for example, uses descriptions and synonyms for each category from a publicly accessible knowledge database.

Wikidata ID

Behind every Wikipedia page is a specific Wikidata entry with a unique Wikidata ID. The advantages of using data from this publicly accessible knowledge database are additional information such as descriptive texts, synonyms and translations into different languages.
Because each category can be linked to a unique Wikidata page, we can create a common understanding and vocabulary of what a particular category is called. No matter where you’re from or what language you speak, you’ll have no trouble finding the perfect item.
Wikidata (wikidata.org) is a centralised data repository for the various wiki projects. The Wikidata ID for each category looks something like this: Q3196 – This text is an identifier (ID for short) that can be used as a link to a human-readable text in the Wikidata database.

Tree diagrams of the category hierarchy

Since most category names such as smoke, fire and explosions are known to all artists around the are known to all artists around the world, the asset library software “das element” has developed a system that emphasises the importance of this terminology. On the one hand, the hierarchy tree is structured according to the actual order of the categories. on the other hand, it is optimised for the vocabulary commonly used in the VFX industry. One concept is to organise everything into super-categories, which makes the most sense for grouping certain types of elements such as fire, smoke or explosions.

As each category is linked to another, we can visualise the relationships between the categories to create an ontology and a knowledge graph. This can help us in the future not only to find similar assets that are currently available, but also to develop future solutions based on machine learning.

Assets

Let’s now take a look at the assets themselves. The source files can be in different file formats and file types. The spectrum ranges from old film Old film scans such as Cineon or DPX, to modern file formats that are not yet supported by most software applications, to anything that can be found on the internet Internet. In an ideal world, we get everything in the same file format and with the same colour space. We don’t want artists to have to deal with these things. Unfortunately, that’s not the case, and achieving this is one of the biggest challenges to overcome. It’s best to save everything in the same file format and colour space. This will make it easier for us to batch process all files in the future when newer systems like USD and MaterialX are integrated into our pipelines.

Today, a good solution is OpenEXR with the ACES colour space. Converting video files into image sequences makes it easier for artists to use, as each sequence can start with the same frame (e.g. 1001). But be careful! This increases the amount of memory required. This must be taken into account for larger libraries or if we want to store everything in the cloud.

For a quick preview of assets, we also want to create proxies or, for example, a rendering of a turntable. We want to get an idea and preview of what is happening in a large film or project file without having to open it. This speeds up the work process without having to wait for a large file to be loaded with the appropriate software.

In some cases, a project file is an older version of software that a studio may no longer have a licence for. Or maybe the file is stored in an archive on a LOT of tapes and you’re not sure if it’s even usable. A good preview can save money, time and a lot of headaches.

The preview can have a lower resolution, for example an 8-bit image like a plain JPEG (or a newer version like HEIC) with Rec709 colour space. For video files it was common to use the H264 codec in a Quicktime container (.mov). There are some new codecs that require less file size with better quality. E.g. HVV (also known as h266) or AV1. Please note that newer codecs may not yet be supported by all applications and operating systems.

Version & Iterations

You should consider how to deal with different versions and variants. An example of different versions would be a recording of a green screen. Version 0 (version zero) could be the original material, version 1 would be the first key. In some cases, you might want to rework the previous key due to new keying techniques or something similar. Now another artist will render a new sequence, which is called version 2. Similar to versioning renderings of VFX shots, this concept can also be applied to an asset library.

Variations can be different resolutions for a 360° HDR image that is available in 4k, 8k or 16k resolution. They all have the same content, but in order to work more efficiently, we want to switch between the different resolutions. However, variations should not be confused with proxies. A proxy for an HDR image can be an 8-bit jpg file, while the variations are still an .exr or .hdr file. The same applies to different LODs (Level of Detail) for 3D assets.

Metadata

Often we want to store any additional information about an asset, e.g. the lens an element was shot with. If we are archiving it for current projects, we may want to store data from our project management software, e.g. the name of the show, the artist or the location of the studio that created this asset.

When purchasing assets from the internet, it is also important to store the relevant licence information with the asset. Years later, no one may remember if you are allowed to use this asset for another project or if there are other restrictions you need to be aware of. Depending on the file type, the amount of metadata that can be linked to a file is limited. By storing the metadata in a database, we can store as much information as we want.

UUID & ULID

When managing assets from multiple studios within an organisation, we want to keep track of when, where and by whom an asset was added to the library. Using the asset name can lead to problems, as names such as “fire_0001” can occur multiple times in different studios.
We can therefore use so-called universally unique identifiers (UUID – is.gd/wiki_uuid) to differentiate between the various elements. These randomly generated IDs are temporally and spatially unique (depending on the version used) and we can trace back when and from where an element was entered. If we store these IDs in the database, it is also advisable to take a look at ULID (Universally Unique Lexicographically Sortable Identifier – more on this at github.com/ulid/spec). The big difference to UUID is that the timestamp is the front part of the ID, which makes the IDs alphabetically sortable. This can increase the performance of database queries.

Automation

Automation is great. However, there are some specifics that need to be considered. One of the biggest problems with automation is human error and the naming of source files. If a file is misspelled, it is difficult to find a solution with a Python script to catch certain errors. Example: T_Grass_Sppecular.png with the word “Sppecular” with two “p” instead of “Specular”. Or the spelling error “Rougness.png” instead of “Roughness”.

Another problem is that the frame counter can change in a continuous image sequence. For example, from 3 (999) to 4 (1000). This often happens with sequences that are rendered by Premiere Pro or After Effects. In most cases, these frames belong together, but this is not always the case.

Name.998.exr
Name.999.exr
Name.1000.exr
Name.1001.exr

It is also important to recognise corrupted files before adding them to the asset library. When buying assets from the internet that come in ZIP files (or copying a ZIP file), the extraction may fail without anyone noticing and we end up with file sizes of 0kb. A simple file size check should help to detect such cases. Some helpful tools are node-based systems that can be used to create automation pipelines. For example: Gaffer(gafferhq.org), NXT(is.gd/git_nxt) or iograft(iograft.com).

3D assets

Creating an asset library for 3D assets is a complex undertaking with many hurdles, and let’s face it, it’s a lot more work than organising 2D elements. There are a few software solutions for 3D asset libraries, such as Quixel Bridge(quixel.com/bridge) and Kitbash Cargo(is.gd/kitbash_cargo). These tools work great because they use an already organised structure in the background from the vendors and their variety of assets. Using these systems is only possible as long as you stay within their ecosystem and don’t want to worry about where and how the files are stored on your hard drive.

If you need more control or want to organise assets from a different source, these systems no longer work as you have no control over the database. Again, we need to find an alternative and generic solution that gives us the flexibility to add any asset type.

Technology

If you have already gone to the trouble of organising everything, you should also consider doing this just once and being well prepared for the years to come. It’s worth researching what options USD and MaterialX offer.

The 3D models come with different types of textures. These are often named differently, even though they represent similar things. Here too, it is recommended to structure and standardise everything so that it follows the same pattern. This makes it easier for artists to use the asset.
A 3D scene purchased from the internet can either contain a single object with associated textures or multiple assets with variations, e.g. different shapes of pipes that can be used for kitbashing. The supplied folders contain multiple textures, often with cryptic names and different resolutions such as 2k, 4k, 8k.

Organising all these files into a clear structure can be a tedious task, but will save artists time in the long run. Splitting assets into individual files can help us speed up the process. However, the initial time required to set it up needs to be considered. To be honest, most of the elements may never be used again in production. On the other hand, valuable time is saved in the crunch time if the assets are well organised
are well organised. If you have the time and resources, it is advisable to take care of the assets and organise them properly.
Experience has shown that the only way to automate this step is to use (Python) scripts to extract certain search terms and assign the texture files to the corresponding texture channels. In the script you can create some rule sets to process the files in a batch process.

Additional files and previews

It is worth creating previews of 3D assets. This way the artists can find them faster and get a simple preview. Otherwise you spend too much time starting the 3D software only to realise it was the wrong file. A simple and less elegant way is to take a screenshot of the viewport to get an idea of what a particular 3D file contains.
A more sophisticated approach would be to create 3D models with low resolution and textures. Sometimes it is necessary to look at a model from a different angle to check whether it is useful for the current task.

Useful file formats are USDZ (is.gd/usd_usdz) and GLB (is.gd/gltf_glb). USDZ is a variant of USD, with the major difference that all textures are merged into a single file, which makes it easier to manage the files on the server. An alternative is GLB (as a variant of glTF), which is already widely supported by web browsers and other applications.

The big challenge could be to create a low-resolution version of the 3D model or scene. For large scenes, it is often not possible to automatically reduce the polygons of the 3D files as well as the textures/shaders without artists having to edit the scene manually. Again, it’s a question of time and resources and what effort you want to put into the library. There are some tools like InstaLOD(instalod.com) that can help with exactly this task.

Scale of the scene

When adding new 3D assets, it makes sense to standardise the scaling for all assets. It doesn’t really matter which scale you choose, at least it should be the same for all files. Some studios choose a scale of 1:10 (1 unit = 10 cm).

Textures

Similar to OpenEXR sequences of 2D elements, using proxies for texture files allows you to quickly display a specific layer. The texture files can be small jpg or png files, but can also be several gigabytes in size if you are working on high-resolution textures for feature films. It is therefore advisable to familiarise yourself with the open standard MaterialX (materialx.org). The aim is to solve the problem of assigning shading and textures to the corresponding geometry. This approach can be useful if you want to be optimally equipped for 3D in the future.

Caches

There are other additional files such as cache or VDB files that need to be saved along with the assets. In some cases, you may be tied to a specific proprietary software. However, studio workflows change over time and certain software packages may become obsolete. Again, it’s worth looking at open source file formats that aren’t dependent on a specific software to build a sustainable asset library for years to come.

About das Element

The asset library software “das element” was first released in August 2021 by a software company based in Munich. Rather than using one of the existing software solutions that are broadly tailored to as many needs and industries as possible, “das element” is customised to the requirements of VFX studios.
Years of experience in building and maintaining asset libraries have gone into the development of this software.

The focus is on combining the above concepts and taking care of the problems you will encounter when building an asset library. Developed to the highest industry standards, the software works completely offline and is therefore ideally suited to the stringent security measures of feature films and television series. Due to its flexibility and configurability, it can be used in a wide range of applications. From high-end visual effects work to fast-paced commercial productions and even non-VFX related projects such as games or architectural visualisation. If you want to start managing your asset library, you can find more information at: www.das-element.com

The post Hitchhikers Guide to … asset libraries? first appeared on DIGITAL PRODUCTION and was written by Jonas Kluger.

By Hans Jakob Harms

The idea for the short film “S/N:05” (Sinos) came about when I read Catrin Misselhorn’s book “Grundfragen der Maschinenethik” in 2019 and imagined a scene in which a small machine fights a cat, a symbol of freedom. Which position do we take? That of the clunky machine or that of the cute cat? At the time, however, I didn’t realise that the topic of artificial intelligence would experience such a boom with ChatGPT and co. All the more reason to take another look at Catrin’s Reclam book.

Jonas (script) and I worked out the story in detail and convinced the FFF Bayern to support the project with an animatic. It was important to Noah (camera) and me to achieve a dark, cool tone in the visual design, as we couldn’t tell the whole dystopia in nine minutes and with the tight budget. To achieve the look we wanted, we decided to shoot
to shoot on a RED Gemini 5k with anamorphic lenses. The film mainly takes place in two locations, a children’s room and a remote house, which Andreas (set) decorated according to a precise construction plan and colour concept. Then Ivetta (producer) and I planned the shoot before we threw ourselves into the fun. Eight days of filming. With cat, child and CGI – you can find the rest of the key data here: is.gd/sinos_imdb.

Maya, our leading actress, impressed us with her talent both at the casting and during filming. Although she was only twelve years old at the time, she knew exactly where to go. It was a lot of fun working with her. Up until the first day of filming, we were very worried that Gizmo the cat wouldn’t be in the film. But Barbara’s (animal trainer) stars are used to the hustle and bustle on set. We did our best to create a feel-good atmosphere for the little one, and lo and behold, Gizmo delivered shot after shot. Now there was just one more hurdle: Sinos.

Shooting

It was important to us that the film looked like a snapshot from the future, as realistic as possible. To achieve this, we worked on the character of Sinos and the monstrous machine long before filming. Alan (metal artist) built the robot from scrap parts based on a sketch by Michael (illustrator). Using a small scaffold that we retouched in post, we were able to move Sinos’ head and limbs. So we used the real Sinos in the close-ups and a CGI double in the wide shots. As the images in the film often follow each other directly, we needed an exact digital copy.

This is where Markus comes into play. He recreated the figure in Cinema 4D true to the original. He used a scan and the real model as a reference and the result was amazing. In the end, we were even able to mix the real model with the 3D model. For example, we unscrewed the arms from the real Sinos during the shoot and animated them in post.
It was nice to see how the character gained personality with each step during the creation process. Based on my description and some references, Michael drew the first picture of Sinos and gave him the features of an old, limping man. From there we could have gone straight to 3D and made the film with a digital model. But Alan’s sculpture, which
but Alan’s sculpture, which he assembled from metal parts (such as pliers, screwdrivers, pistons, etc.) and which is subject to the laws of physics, breathed a different essence into the little rascal. Now he looks more like a chunky baby who is struggling to hold his screws together. An ideal teammate for Maya, who plays his mother in the film.

Unfortunately, we were unable to recreate the large machine. We did have the idea of modelling the machine with individual elements, but the budget was too tight. This time, Markus created the 3D model directly from a concept by Michael and refined it according to his ideas. During filming, we used tripods as a reference to estimate the size of the machine for the shots. This allowed the actors to visualise the machine with a lot of imagination.

With my background in post-production, I was able to take on a dual role on set as director and VFX supervisor. A static camera and lots of references provided a solid basis for post-production, so that the 3D elements could be combined well with the filmed material. I realised the animation of the machines in Cinema 4D and rendered them with Arnold. The renderings were then applied to the plates in After Effects and played out for Resolve.

The large number of VFX shots and the tight budget made detailed planning in advance essential. Even though the possibilities of digital post-production are becoming easier and more accessible to everyone these days, poor and often unnecessary visual effects can take away the authenticity of a film. We scrutinised the VFX twice for every shot and only shot what was necessary to tell the story. But of course, in this science fiction film with fantasy characters, there were still a lot of images that had to be supplemented with renderings.

To avoid headaches in post-production, it helps to have a storyboard in which the tasks are precisely specified after filming. This makes it possible to estimate and possibly reduce the scope of post-production in the pre-production phase. Of course, it is an advantage if you have mastered the craft yourself or at least have a basic understanding of it. Otherwise, I would always involve an experienced VFX supervisor in the planning.

A film lives from its actors and the roles they embody, and VFX are known to complicate the acting during filming. That’s why I distributed the focal points in the resolution of the scenes. In one shot, I focussed on the actors, giving them the freedom to perform. In other shots, however, the visual effects took centre stage and the entire team concentrated on the clean execution. Of course, it is particularly appealing to merge an elaborate visual design with an emotional play in one shot. But the problems that arise during staging and the high, often incalculable effort involved in post-production should not be underestimated, especially for an independent film. I acquired my post-production skills through my own film projects. After school, I made my own short films, in which I always had to take on several positions. As a few days of shooting were often followed by several weeks of post-production, I honed my skills in the areas of editing and animation in particular. I am first and foremost a filmmaker and like to tell stories that don’t require elaborate VFX. That’s why it’s important to me to work with programmes that produce meaningful results quickly.

I want to keep the technical effort in post as low as possible and emphasise creativity. Cinema 4D offers a user-friendly introduction to the 3D world and is similar in structure to the standard Adobe products. At the same time, it is equipped for most challenges in terms of scope. In special areas (e.g. simulations) there are software solutions that are better equipped. However, when I reach my limits, I usually look directly for artists who can help me with their specialisation.

Marie and Thomas helped me with the organic animation of a deer. Tapan tracked a drone shot that Cinema 4D couldn’t solve. Tanver helped me rotoscope the actors and Nagendra removed a rig from Sin’s eyes that I couldn’t retouch in After Effects. Of course, as with almost every project, there were problems that can get on your nerves. My tip: ask Reddit. The community is full of professionals who are happy to help if they have a solution.
I have kept the animation pipeline as simple as possible. With a rough animation of a low poly model of Sinos on the tracked plates, I was able to determine the timing in the cut and play out a picture lock with preview renderings. I then linked the rig of the low-poly model with that of the high-poly model for the final rendering. In this way, the turnarounds with the final scene were kept within limits.

The nature of Sinos, a robot made of rusty metal parts, suited me for the animation. Simple mechanical movements and the combination with the real model enabled me to create the animation myself. Rigging and animating the monstrous machine was a little more complex, but manageable thanks to the few settings.
Colleagues asked me why I chose Arnold as the render engine. Markus had prepared the 3D model with Redshift materials. But I think GPU renderings are not yet “on par” with classic CPU renderings. Technically, I don’t know much about this area. That’s why I converted the materials from
Markus’ materials to Arnold and compared them with the Redshift renderings. We both liked the look of Arnold better and so the decision was quite easy.
The soundtrack by Giovanni, the sound design by Alexander and Florian and the colour correction by Lukas and Nadir put the finishing touches to the images and brought the menacing world to life. I would like to take this opportunity to thank everyone who was involved in this project and helped make this short film come to life.

Conclusion

We have submitted the finished film to several festivals and are waiting to see if it finds an audience there. It will then be made freely available to everyone on the internet. A trailer is available on the Instagram page. We are currently working on a screenplay for a feature film to continue the story of Sinos in the dystopia.

The post S/N:05 first appeared on DIGITAL PRODUCTION and was written by Bela Beier.

This time we have dug up material from the analogue era. In addition to very poor quality video from amateur cameras, there is also video from a very high-quality Ikegami camera, which was only recorded on Super VHS after a professional recorder failed.

Part of Topaz Labs’ business model is to constantly keep users happy with updates, as these updates are only included for the first year after purchase. After that, users have to pay 99 US dollars a year if they want to keep up. Unlike Adobe, however, your own work is not held hostage, but you can continue to work with the last licensed version. When the software reports an update, it is fair enough to point out if this is no longer included in your subscription. There is also a trial licence for one month, which you can use to thoroughly test the software for your own needs and with your hardware. Since TVAI 3.0.3 from the last test, not only have new features been developed, but a number of bugs have also been fixed. However, similar to DR 18.6, a few other bugs have been added with the very fresh version 4.0, especially in the new user interface and the Nyx 2 model.

Stabilisation

First of all, we would like to tick off the less successful features. These include stabilisation, which was still listed as a beta in our last article. Although this is handled very quickly by both programmes, it doesn’t work miracles. We tested it with unsteady material from a digital handheld camera, whose sensor is not particularly fast.

Although we had activated the treatment of the RS with “Rolling Shutter Correction” in TVAI, quite a lot of image distortion remained in the peripheral areas, especially when set to “Full Frame”. TVAI attempts to reconstruct missing edge areas instead of zooming into the image. Unfortunately, this leads to quite a lot of “jelly” in these edge areas. The results from Resolve were no better. In contrast to DR, rendering processes do not block further work on other tasks in the programme.

Our findings from the article on stabilisation with support from gyroscope data (DP 22:06), where DR gets a perfect grip on the RS (at least for the camera’s own movements) still apply here. We also tested this again with a clip from Sony’s Catalyst Browse, in which vibrations were stabilised with gyro data. Typically, short blurs become clearly visible with each shock. TVAI has a Deblur model called Themis for this purpose. This gets such motion blur under control quite well in some cases, but is still heavily dependent on the subject. It can sometimes have the opposite effect on contours with high contrast, and unfortunately there is no fine-tuning for this.

Slow motion

Both programmes offer the synthesis of intermediate images for subsequent slow motion, in DR the highest quality, but also slowest algorithm is called “Speed Warp”. As this worked best with very high-quality source material in both programs during the last test, this time we also deliberately tried out the lower-quality, upscaled analogue material (see below). In detail, both programmes show almost identical problems with crossing movements, as is also known from Optical Flow. Nevertheless, the results are so good that they can be used for many purposes. Speed Warp showed slightly more artefacts in detail and, in rare cases, slight flickering in textures. Nevertheless, the slow motion alone is certainly no reason to buy TVAI if you already have DR Studio, as the programmes don’t have much in common in terms of processing times either.

Deinterlace and upscaling

If the source material is interlaced and only has SD resolution, conversion to progressive HD actually seems impossible. But here the AI should show what it can achieve by looking at the neighbouring images. There are two models recommended in TVAI for this purpose: Dione and Iris. With Dione we tried Dione TV and Dione Dehalo, with Iris we simply used the LQ version. Although this means “Low Quality”, it refers to the source material and not the result. Dione Dehalo is already able to achieve a more appealing look by reducing the ugly edge sharpening of the amateur camera.

Unfortunately, just like DR’s neural de-interlacer, it still struggles with staircase artefacts on critical curves in the image. The current version of TVAI now also offers inverse telecine, but there are still problems (at least they are correctly documented). In addition, many editing programmes can do at least this as well. A new option is the setting to pure black and white, so that the AI does not hallucinate colour with corresponding sources.

With Iris LQ, on the other hand, we were somewhat taken aback, and not just in terms of the perfect de-interlacing. The TVAI version looked much better than the DR version, despite the activation of “Superscale 2x Enhanced” and neural deinterlacer. Although we had not yet done any fine-tuning here, but had processed with the standard values, the edge sharpening (the halos) was also successfully removed (this did not play a major role with the professional camera above). We have never seen better upscaling of analogue material with a stand-alone program.

In contrast to some online offerings with AI processing on a third-party server, nobody has to hand over their material to TVAI because the AI models are still downloaded to their own computer.

In addition, the field sequence was reliably recognised by the automatic system, but can be set manually if required. Even if the result sometimes looks a little flat, it is not embarrassing as archive material in a modern environment. Faces are reconstructed better by Iris LQ than by Proteus. TVAI can also add a little ‘grain’ if desired, which makes the images look a little more natural again. The other big surprise was the speed. Until now, one of the criticisms levelled at TVAI was the sheer endless computing time, but even on a modest Apple MacBook M1 Pro, the better IrisLQ was generated at 5 frames per second. The less successful DioneTV model even managed almost 10, while DR only delivered 2.3 frames per second.

For the first time in our tests, TVAI was clearly superior to DR’s on-board tools, not only in terms of quality but also in terms of speed. Having become overconfident, we even dared to blow up the material to UHD with Iris LQ. However, the AI then invented a little too much small detail, and slight interlace artefacts returned at the archway. We had already tried to convert HD to UHD in an earlier test. However, the recognisable quality gain compared to DR shrank with high-quality source material, such as the conversion from good HD to UHD. We then needed “pixel peeping” to recognise the differences. If there is only a small amount of material to convert, many people will not want to change the programme. A suitable use case would be the upscaling of clips from a modern camera if the camera is not capable of full resolution in slow motion. Some CGI studios use it to upscale material in 2K to 4K because they can avoid gigantic render times.

Input and output

Incidentally, one of our sources was available as MPEG, which was read by TVAI without any problems, whereas we first had to have it repackaged as MOV for DR. This is because TVAI is based on ffmpeg and therefore understands an enormous number of formats. It can also deliver the ProRes format often requested by customers on PCs, which is not possible with DR. Although this is not approved by Apple, it works perfectly everywhere.

On the Mac, however, TVAI uses the system routines and should therefore produce completely “legal” ProRes. In the new version, it offers the setting for adaptive compression for H.264 and H.265. However, the three settings for this are somewhat unfortunate: High delivers extremely large files, while Medium compresses far too much and is not very different from Low. Here we would rather recommend selecting a constant bit rate until Topaz has optimised the values. FFV1 is also available for the output of an archive copy. The export settings are now directly accessible and with “Export as…” you can assign more meaningful names.

Improved preview

With a program of this type, it is essential to optimise editing for your own raw material and customer requirements. Until now, it was quite cumbersome to compare the results of different models and manual adjustments. Now two previews or original and preview can be played side by side or in split view. As usual, not all veteran users are happy with the new interface. In our opinion, however, there are significant improvements, e.g. TVAI now remembers the settings for zoom, split and position of previously edited views when the original is changed. Unfortunately, it is not possible to save entire projects: When you leave the programme, everything is gone. The 4.0 version is not yet completely free of bugs, but you can always find the latest versions of version 3 for download.
Some tools now work completely differently, such as Trim, or are subject to a few restrictions in the direct display, such as cropping. Some will look in vain for the crop tool, as it has moved to the top right and is only available when the original is displayed in full screen. The timeline now offers a zoom and the trimming tool also determines the playback area for previews. So don’t forget to adjust this for export if necessary. Version 3 users should look in the small menus with the three dots if they are missing something. The option of having two models edit the same clip one after the other must first be activated in the preferences.

Performance requirements

We also wanted to know whether it would make sense to clean the material with TVAI only from the interlace and edge sharpening in order to then scale with DR. With pure de-interlacing, IrisLQ even manages 9 fps, whereby the laptop did not appear to be fully utilised, while with DR the GPU was running at full capacity.
However, this method was somewhat weaker in terms of quality, more cumbersome and not any faster. In contrast to our last test, TVAI has now been optimised for Apple Silicon and significantly outperforms older Intel Macs, even if the minimum hardware requirements for both PC and Mac remain modest and date back to around 2014/15 (AVX2 instructions are required for PC). For occasional, short restorations with overnight rendering, most computers that can work with video at all should be sufficient. They can also be better utilised if you allow multiple instances; our GPU was almost fully utilised with 2 processes.

Unfortunately, even Mac Studio Ultra cannot compete with TVAI on the best PCs for more intensive use. If you regularly have such restoration work in your home, you should rather buy a PC with the best Nvidia GPU that is financially within reach. For example, an AMD Ryzen 9 5950X with 16 cores and the NVIDIA GeForce RTX 4090 can convert from HD to UHD at over 13 frames per second. CPUs with more cores or multiple GPUs provide even more of a boost, but the latter doesn’t always seem to work. In addition, not all models work with Intel ARC GPUs yet; these will be included in the coming weeks. Details can be found in the user forum at: bit.ly/topaz_bench. Please note that the output format in the list is already full HD, in some cases even UHD. As one licence can be used on two devices, whereby this applies to PC and Mac, such a bolide can also work separately.

Comment

Obviously, really bad analogue material is TVAI’s true domain. With such sources we were able to observe a clearly visible superiority in de-interlacing and upscaling compared to DaVinci Resolve. The speed is now also comparable, in some cases even better. The only wish that users could possibly still have if they only rarely have a job with critical archive material to process: How about a low-cost monthly licence?

The post Topaz Video AI Revisited: V4 first appeared on DIGITAL PRODUCTION and was written by Uli Plank.