Table of Contents Show
By now probably all of us have been both impressed and also scared about AI identifying visuals in our smartphones. Well, maybe rather the latter, in the light of recent events at OpenAI. But let’s think positive and see what it can do for us in everyday video production. Blackmagic (BM for short) is running with the pack and added heaps of features with the buzzword in DaVinci Resolve 21 (DR for short), even if they used the expression “neural network” for most of such functions not so long ago. So, how well integrated is AI and how capable is it?
Is it safe?
Well, generally speaking, nobody knows. But being respected in professional production, BM’s AI is not sending your content to a massive datacenter, devouring precious energy, water, and storage. They do not train it on your work, like others, eventually returning some results. They would be out of business soon if they did.
Instead, you have to download the respective models for the function you need, which were probably trained needing massive resources too, but not your local ones. Nevertheless, the AI works on your machine and eats your storage (and GPU). Bad enough at current prices, but at least you can delete models you don’t need any more. You could even pull the plug if you own the license dongle, and it’ll still work (if you are not too paranoid, you won’t expect it to find an escape door).
The editing assistance
Having spent so much of our lifetime just cataloguing footage, this function caught our attention early: AI IntelliSearch. It’s supposed to identify people, objects, and even slates. The downloader gives you a choice between a smaller collection of models named “Faster”, a tad over 1 GB, and “Better” with over 4 GB to be downloaded. Unfortunately, you can’t receive them in the background; DR is blocked during that time.
Resources and speed
We tested IntelliSearch on a Mac mini M4 Pro with 24 GB of unified RAM, which is neither a speed demon nor does it offer a surplus of RAM. BM recommends a GPU with 16 GB (!) of VRAM for demanding AI functions and 16 GB of RAM under Windows (more for Fusion). If we add those numbers up, a Mac should have 32 GB. Well, BM says a Mac should run with 8 GB only, and some folks report editing on a MacBook Neo. But don’t try advanced AI on that poor little thing. While BM considers 16 GB sufficient on the Mac, we wouldn’t suggest doing that, according to our results. Rather get 48 GB, or even 64 for some future proofing, even if it hurts the purse.
We tried a project with 580 HD clips of 1.5 hours total, and analysing only for faces took 7 minutes when set to Faster, while staying well away from the memory limit all the time. Now we dared to aim for Better and it got dead slow, needing about 1 hour and 10 minutes while using twice as much RAM and getting dangerously close to the limit of those 24 GB without running any other unnecessary software. The fps value is going up and down over the different phases of the analysis, but after a while the estimation under the progress bar is not too far off.
In both modes the GPU cores are used to their limit, while the CPUs are close to idle. Obviously, even HD resolution when operating for Better would get still slower due to swapping with only 16 GB and ruin your SSD in the long run. BTW, it’s always a good idea to restart your machine and have nothing else running than DR before starting demanding AI tasks on hardware with tight RAM limits. If you need to cancel the process, don’t trust the little icon labelling all clips as analysed, you’ll need to reset and start over. Alternatively, you can select a bunch of clips, let the AI loose on those alone, and tackle the rest later.
Of course, we also wanted to know how well it handles higher resolutions than HD. A project with a mixed bunch of footage in HD and UHD totalling 2:16 hours was analysed, set to Faster, with about the same speed as HD only. But with Better the temporary step of Indexing Faces went into swapping for UHD, even if speed went close to 600 fps in that stage. So, it’s not a bad idea to run the analysis on HD proxies if the majority of your footage is UHD or beyond. The AI is capable of generating Smart Bins for what it found under separate search criteria. If you’re short on time and RAM, check if Faster isn’t good enough and link to the high-resolution originals later.
Quality of the results
After using only Faster we were impressed by the precision of face recognition, including critical lighting conditions. In the Face Gallery you can give names to people who were grouped as being the same person. You can also improve the grouping manually if the AI is not sure a clip is showing the same person, but even with a motion blurred or unfocussed face it was quite good at it.
While faces are identified as such and can be named, what else will be found? First of all, you need to know that this AI right now only understands English for queries (we tried some German and French). It’ll normally find well defined objects, e.g. smartphones, books, shoes, weapons, cars, or motorbikes, and you may add a color to narrow results down. In most cases there are more false positives than missing things. Parts of a landscape like mountains or forests are usually found, just like clothes of a specific color, even if the AI doesn’t know a shirt from a jacket or coat. Of course, we had to check for cats, which works very well ;-) Sorry, no dog around.
Clapperboard aka (sync)-slate
While you should normally know who’s present in a take for a well-scripted fiction work, a slate will help to identify the takes. And then, not everybody is always using proper timecode devices for separately recorded sound. One should at least use a good, old-fashioned slate instead, though, and call the numbers. If not, a hell of a lot of work is waiting for you, and no current AI can save you. But if you have used slates, IntelliSearch is coming to the rescue.
If you are noticing that even more data needs to be downloaded for this function than for any other, you may expect a lot from it. While a proper electronic clapperboard is easy to read, folks tend to use hastily handwritten versions too, and not every slate is organised in the same layout either. This is where IntelliSearch shines. We were truly impressed how well it can read slates that are even hard to decipher by eye. It also handled an upside-down slate close to perfection, which is normally used if you forgot it at the start, or didn’t want to bother the actors or demand refocus. Such a tail slate is then recorded at the end of the take before the camera stops.
Limitations
Sure, there are limits, like ambiguous data or really bad marker writing. So you may need to check these after processing. This is where we noticed two shortcomings: you have to click that Slate ID icon one by one to get the results, and it always takes a few seconds. And then, you can’t edit directly in the transfer window. The text needs to be saved first, including any mistakes, to the metadata, where it can finally be edited after closing the overlaid window. Sync or MOS are only identified if the other is taped over; circling doesn’t suffice. Entries for Day/Night or Int./Ext. are not registered. And a rather general remark: the functions for IntelliSearch are distributed all over the GUI, which is not really intuitive. Definitely room for improvement here!
Conclusion
Depending on your specific needs, IntelliSearch might be the most important one among all the new AI functions in DR. Searching for people, cats or objects should save documentary workers hours when organising their sources. Slate ID, together with the AI transcription (if the text was clearly called-out by the clapper) will assist those with less than perfect recording equipment. Until now, final sync still needs to be done by hand, though, if there is no TC.
We’ll look at the other new features soon.
