In nuce: At the CES 2023 (Consumer Electronics Show), graphics card manufacturer Nvidia announced various innovations for its Omniverse collaboration platform. This is according to an announcement from 3 January 2023, published by Omniverse Vice President Frank DeLise on blogs.nvidia.com. The announced highlights include: AI tools for 3D artists, an extension for Blender and the early access of RTX Remixer (for the purpose of ray-tracing and DLSS-based remasters of gaming classics).

In toto: Below is an outline of the announced highlights for Omniverse – and the corresponding video of Nvidia’s special address at CES 2023, moderated by Jeff Fisher, Senior Vice President at Nvidia.

NVIDIA Special Address at CES 2023

• New, experimental AI tools for 3D artists: This concerns the tools Audio2Face, Audio2Gesture and Audio2Emotion. These are tools for 3D character animation, all of which have received a performance update.
• Extension for Blender: The Blender Alpha version is now available for the Omniverse Launcher. This allows geometries to be repaired, UVs to be generated automatically and high-resolution CAD files (computer-aided design) to be downsampled to a more practical polycount. Users can now easily transfer shape keys and rigged characters – or attach the head of one from Omniverse Audio2Face. Audio2Face is an AI-powered tool that automatically creates realistic facial expressions based on audio files.
• Nvidia RTX Remix will soon be available as Early Access: Such classic games that caused a sensation with RTX remasters from the modder community include Half-Life 2 and Max Payne. RTX Remix is a modding platform that – as stated on nvidia.com – utilises “powerful AI tools” to “enhance classic games and quickly create stunning RTX remasters with ray tracing and DLSS (Deep Learning Super Sampling) .”
• Update for Canvas: This will be available to users of Nvidia’s RTX graphics cards. Canvas is an AI tool that can be used to create low-threshold landscapes.
• Omniverse will soon be pre-installed on the first Nvidia Studio laptops: As Nvidia mentioned, laptops from Taiwanese computer hardware manufacturer Gigabyte Technology will already ship with the Omniverse Launcher this year (this also includes laptops from Gigabyte’s gaming division Aorus)
• New, free USD assets: The number of these should be in the thousands.

Turn the page: In our previous three-part advertorial series on Nvidia’s Omniverse, we reported on the basics of the collaboration platform (“Omniverse” on page 56 of DP 03 : 2022), how it can be set up professionally (“Getting started with Omniverse” on p. 84 of DP 04 : 2022) and how a sensible workflow can be set up (“Working in Omniverse” on p. 87 of DP 05 : 2022; the issue is also available as a free trial copy!).

Clicked further: Nvidia’s announcement, published in the course of CES 2023, can be viewed at blogs.nvidia.com . On 9 January 2023 , Digital Production recently reported on Omniverse – at that time, an extensive update for Omniverse Enterprise was worth reporting.

Sources: 80.lv ( announcement by Gloria Levine), blogs.nvidia.com ( announcement by Frank Delise)

The post Omniverse updates at the CES 2023! first appeared on DIGITAL PRODUCTION and was written by Patrick Poti.

In nuce: The technology company Rokoko has released free, AI-supported software: Rokoko Video. This software, supported by artificial intelligence, is tailored to motion capturing – and can be controlled via the web browser. Rokoko announced the release of Rokoko Video on 6 December 2022 via the website 3dvf.com .

Start animating with our free AI mocap tool Rokoko Video I Tutorial

In toto: According to the announcement on the company website, Rokoko Video is comparatively easy to use. The first step is for users to upload a video using a webcam, smartphone or comparable moving image recording device. It should be noted that the entire body should be recorded, which users can then edit using motion capturing. The motion data is generated automatically by the software. If required, the same data can be edited using Rokoko Studio (also from Rokoko)(Manuel Skroblin reported in detail for Digital Production on 1 March 2019 about how character animation can look with Rokoko Studio and associated hardware and software solutions). Finally, users import the finished motion capturing into common DCC software such as Maya, Blender, Cinema 4D or Unreal Engine. According to the company, a free version of Rokoko Studio is also available.

Click further: Further information is available via the aforementioned website at 3dvf.com . An alternative to Rokoko Video is Maxon Moves, an app for mobile devices, which Digital Production reported on 5 March 2020 .

The post Motion capturing with Rokoko Video? first appeared on DIGITAL PRODUCTION and was written by Patrick Poti.

In nuce: After more than ten years of development, the full version of the AI-supported keyframe animation software Cascadeur will be released within the next few days. In the course of this, a five-minute video has been published which presents the functions of the full version. This is based on information available to the editorial team.

In toto: The release marks the end of an almost three-year beta testing phase in which more than 100,000 industry professionals and hobbyists were involved. The video is linked below for those interested.

Click further: Digital Production last reported on Cascadeur on 20 August last year , when the software went into early access. In our print and e-paper issue DP 02 : 2021 , our author Ralf Gliffe took a closer look at the Cascadeur open beta on page 6.

Cascadeur release version product demo

The post The Cascadeur animation software is available as a full version! first appeared on DIGITAL PRODUCTION and was written by Patrick Poti.

In nuce: Russell Midfield, a Blender fanboy with a focus on creating characters, has published a ten-minute video in which he compares four motion capture programmes. The highlight: all four programmes work on the basis of artificial intelligence.

In toto: For his video, Midfield takes a look at the programmes Plask, Radical, DeepMotion and Pixcap (Plask was presented by Digital Production on 26 January this year ) – and uses iClone 8 to solve some of the typical teething troubles of motion capture software, such as posture correction, shaky movements and slipping feet.

Click further: Digital Production reported on the release of iClone 8 on 7 June 2022. Russell Midfield’s entire video is linked below.

Russell Review: AI Motion Capture – Who is the best?

The post What can iClone 8 from Reallusion do that the competition can’t? first appeared on DIGITAL PRODUCTION and was written by Patrick Poti.

In nuce: In October of this year, software developer Topaz Labs released the latest version of Video AI (previously known as Video Enhance AI). This was announced by Product Marketing Manager Brian Matiash on 19 October this year. Video AI 3.0 is software that can be used to perform upscaling, de-interlacing, interpolation and image stabilisation. The same announcement also states that Video AI has been “developed from scratch” or rather a new code base has been developed, which – according to the announcement – is also responsible for many of the newly added functions.

In toto: Below are the most important new features that come with Video AI 3.0.

• Video stabilisation: This can be used to compensate for shaky camera movements. See the first video below.
• Apollo slow motion mode: Can be used for non-linear movements and motion blur. A practical example is shown in the second video below.
• Stacking AI models: More than one AI model can be accommodated in a task. In practical terms, this means that the screen resolution can be scaled up, for example, while the image is stabilised at the same time. The third video below this news item uses three application examples to demonstrate how to find the right AI model for the creative work in question.
• Improved user-friendliness: Sometimes a command line interface can also be selected.
• General improvements in terms of performance and stability

Click further: You can find out everything else about Video AI 3.0 at topazlabs.com . On the Topaz Labs YouTube channel there are further handouts that make it easier to get started with the software. Digital Production last reported on software developer Topaz Labs on 19 May 2022 – at that time, version 6.0 of the Gigapixel AI upscaling software was worth a report.

Sources: topazlabs.com ( announcement by Brian Matiash), 80.lv ( announcement by Gloria Levine)

Topaz Video AI v3.0 – Stabilisation example

Topaz Video AI v3.0 – Apollo 400% slow motion AI model in action

Topaz Video AI: Finding the right AI model

The post Switch several AI models in parallel: with this moving image software! first appeared on DIGITAL PRODUCTION and was written by Patrick Poti.

In nuce: The Image Creator is now available in selected countries via the Microsoft Bing internet search engine – and will soon also be made available for the Microsoft Edge web browser. This is according to a post from 20 October this year on the Microsoft Bing blogs. Image Creator is a tool that uses the artificial intelligence of Dall-E 2 to create images based on text input (also known as prompts in technical jargon). Image Creator is comparable to the text-to-image AI Stable Diffusion, which Digital Production reported on 6 September this year.

Image Creator in Microsoft Bing

In toto: According to the blog post, Image Creator allows you to create images based on textual input. The post continues: “Simply enter a description, for example contextual information such as location, art style or type of activity, and Image Creator creates matching images.” The post also states that it has always been possible to search for images on the Internet using Microsoft’s search engine – but Image Creator now also makes it possible to generate customised images.

Image Creator in Microsoft Edge

How does it work? Image Creator can be selected via the Microsoft Bing sidebar. The 40-second video above demonstrates how Image Creator can work. In the application example, a user called Adam Curtis gives his friends a status update via a social network – which reads: “I’ve found my dream home!” In the text input field, the user right-clicks to open a drop-down menu where he selects the “Binge Image Creator” option. The sidebar then opens, where the user types into an input field what the dream house should look like (in the example it is the prompt: “Watercolour of a house on a hill overlooking a beautiful ocean with the setting sun”) – after which several images are created according to the user-defined specification. The user now selects the image they like best and posts it on their profile.

Click further: Image Creator can be accessed via bing.com . There are also tips and tricks on how to use it. The aforementioned blog post alsooffers tips on which commands are suitable for achieving a particularly good result in the image display.

The post Generate images with the text-to-image AI Dall-E 2 in Microsoft Bing! first appeared on DIGITAL PRODUCTION and was written by Patrick Poti.

In nuce: Developer Travis Hoppe has presented his new tool – its usefulness: texturing is easier. The cloud: Travis uses Stable Diffusion for his tool (one of several image synthesis tools that are currently causing a furore. Digital Production reported on 6.09.2022). Travis describes the textures he creates with Stable Diffusion as infinitely expandable.

Click further: Travis Hoppe’s Stable Diffusion tool on colab.research.google.com. See also Travis’ Twitter posts below.

https://twitter.com/metasemantic/status/1568997322805100547

https://twitter.com/metasemantic/status/1568997328085729283

The post Texturing ad nauseam dank Stable Diffusion first appeared on DIGITAL PRODUCTION and was written by Patrick Poti.

In nuce: Topaz Labs recently released version 6.0 of Gigapixel AI – a tool with which you can increase the image resolution of photos and renderings. The tool works using artificial intelligence.

The new highlights include:

• Native support for Apple’s M1 processors: This should make these CPUs significantly more performant. This affects users who utilise the M1, M1 Pro, M1 Max and M1 Ultra processors. The support should result in up to 50 per cent faster loading times compared to Rosetta emulation. At least that’s what the statement from Topaz Labs says. The previous version still relied on a combination of Rosetta emulation and Intel architecture.
• New download function: You can now also make batch downloads based on the settings of your system. This should significantly speed up downloads.
• No pixel shifting: In the previous version, heavily compressed models caused pixel shifting. This should now be a thing of the past.
• New user interface: This should make handling easier. The improved clarity is said to have been achieved through the use of strongly contrasting colours for the control elements.
• And of course: various bug fixes, which should increase the usability of the tool.

Click further: You can find out everything else in the corresponding blog entry on topazlabs.com.

The post Gigapixel AI 6.0 released! first appeared on DIGITAL PRODUCTION and was written by Patrick Poti.

In nuce: 3D generalist and character artist Hirokazu Yokohara shares his deep-fake demonstration on Twitter, in which Tom Cruise pulls a face, grimaces cheerfully and otherwise puts the corners of his mouth to the utmost.

Check out this post on Instagram

A post shared by Hirokazu Yokohara (@hirokazu_yokohara)

How does it work? According to Yokohara, DeepFace Live and Autodesk Maya were used for his demonstration. Yokohara was inspired by software developer (specialising in augmented & virtual reality) Brielle Garcia, who herself works with deepfakes.

What is DeepFace Live? The software allows you to replace your face with that of other people – for example Hollywood beau Cruise or other people from film history. This is a great way to get up to mischief at your next video conference, as DeepFace Live is explicitly designed for streaming and video conferencing.

Click on: If you want to keep an eye on Hirokazu Yokohara’s creative work, it may be worth following the 3D generalist and character artist on Twitter, Instagram, Artstation or 3Dtotal . Also pay a visit to AR/VR developer Brielle Garcia, on whose work Yokohara’s Deepfakes are based, on Twitter or tacolamp.com . Finally, if you want to make sure that the computing machine does the learning for you, then head over to github.com to try out DeepFace Live.

We hope you enjoy your next video call (with the AI-edited faces of your film and series stars)!

The post Deepfake with Autodesk Maya! first appeared on DIGITAL PRODUCTION and was written by Patrick Poti.

• Plask is a 3D animation editor and motion capture tool
• What exactly is the Plask tool? The tool is web-based and powered by artificial intelligence
• The killer feature of the Plask tool? The ability to animate your character based on video recordings – and practical motion capturing

In nuce: The Plask team releases its two-in-one tool: 3D animation editor and motion capture tool.

What is the Plask tool? The tool is web-based and powered by artificial intelligence. Plask allows you to record, edit and animate your 3D projects without having to leave your internet browser. And, let’s be honest: who leaves their internet browser these days (unless the user is forcibly removed from the display with a crowbar)?

The killer feature? The ability to animate your character based on video recordings. Plask owes this practical motion-capturing experience to the AI mentioned above. Plask automatically sets keyframes within your videos. It’s even easier if you use your webcam to record your movements in front of the camera (and, let’s face it, who doesn’t record their body claus via webcam these days?)

Click on: But enough about crowbars and being overconfident in front of webcams. You’re just one Plask login away from animating in your web browser ! The video below shows you how Plask works. We hope you enjoy this groovy tool!

Meet Plask – Plask – AI Motion Capture and 3D Animation Tool

The post Plask | Great tool first appeared on DIGITAL PRODUCTION and was written by Patrick Poti.

Alex Audio Butler (AAB for short) is a small series of VST plug-ins that plug into popular video programmes and promise to handle sound mixing independently. Currently these are DaVinci Resolve, Premiere Pro and Adobe Audition. Others are in the works. AAB runs under Windows 10 and Mac OS 10.13 or higher, and even natively with M1 on the Mac, but it does not work with the AppStore version of Resolve. (But who wants that?)

My expert colleague Björn Eichelbaum gives you a general overview of AI in the audio sector in this issue from page 72 onwards. I, on the other hand, am a tester without any audio expertise; at best, in my younger years I was able to use the equaliser on my stereo system to crank up my music to disco levels. In demanding film projects, I have always respected the important role of sound and left this work to the professionals.

Final mix from the robot

But what about a small, underfunded project with no budget at all? We carried out a practical test on a video portrait with which a university from an emerging country wanted to introduce itself. Alex Audio Butler from the Dutch company Unimule was asked to control the volume, use a compressor if necessary and take care of the audio ducking (i.e. reducing the background music or effects when speech starts).

We worked with Resolve, where AAB can be found after installation under the filters for Fairlight in Uncategorised > VST. When editing, we naturally made sure to assign the sounds to individual tracks according to their function. AAB differentiates between Voice, Music and Sound FX (although the latter were not used in our project). However, you should not only place the little helpers on the individual tracks, but also on the master.

After selecting the function for the respective track, you don’t have to deal with hertz, decibel or attack and decay times in the settings. Instead, the respective task is selected from a few generally understandable terms, which are also provided with explanatory texts. Even a video enthusiast should have no problem understanding them – if necessary, it helps to try them out. The PDF manual is limited to installation and licensing. In itself, this is sufficient, but you can also find additional videos under “Alex Audio Butler” on YouTube.

Push

If you try to output the complete work straight away, you often get an error message saying that Alex hasn’t finished analysing it yet. There’s no point in waiting, instead you have to give the little guy a little help. You should either listen to the entire film once or simply output the sound alone two or three times, which is usually quicker than calculating all the images.

You will then receive a message that the analysis was successful and you can now render the entire film with video. However, the screen messages should be a little larger – we could hardly recognise them on a 27-inch monitor with 2,560 x 1,440 pixels, although the fonts in Resolve were still clearly legible.

Result

The result is certainly worth listening to – a complete audio amateur could hardly do better. In our case, there was essentially the usual problem with inexperienced speakers, who often start loudly and then get quieter and quieter or slowly increase in volume. AAB manages this quite well with internal keyframes, without raising the background noise during pauses, because it recognises the human voice. A compressor can also be used in several stages. Background music from two completely different sources on one track was well regulated to the same subjective volume. However, you can also use several tracks with different settings for the music. You only specify the volume for music in the foreground and the auto ducking ratio. When assessing compression and audio ducking, it also depends on the monitoring system. We were quite satisfied with the result via loudspeakers on the computer and also via high-quality headphones. However, we were criticised after the demonstration on a sound system. No problem: we quickly changed the compression and ducking, gave it a new push and the university was also satisfied.

Comment

What AAB cannot do at all is repair errors and faults in the recording. This has to be done by specialists with trained ears through precise adjustment of appropriate filters (which nowadays can also be based on AI). Alex will also not deliver a perfect feature film mix, where the emotional impact depends on very subtle factors. But the Butler can handle simple projects with interviews, voice-over and background music with technically clean recordings – currently for the introductory price of 79 euros.

The post Does Alex make Toni unemployed? first appeared on DIGITAL PRODUCTION and was written by Uli Plank.