Archive notice: This article was originally published on January 7, 2019. Links and embedded videos are preserved as part of the historical record.
Not only developers and research teams, but also more and more 3D artists are now considering the extent to which new advances in AI will change the 3D industry – including “Andrew Price”, known from Blender Guru, with his talk at the Blender Conference 2018. Nvidia presented a summary of its most exciting research projects from 2018 on the topic:
Interactive 3D Worlds
In the demo shown here by Nvidia, Unreal Engine 4 is also used, among other things, to create semantic layouts – maps that show colour segmentation of objects. A neural network trained with videos from the real world then creates the details: an urban environment with buildings, cars, roads and other objects.
The demo could also be driven interactively at the NeurIPS Conference in Canada. In the past, Nvidia had already attracted attention first with pix2pix and then with Vid2Vid. With the technique used here, the team had not only cities but also faces and poses created synthetically. You can find more information about the project here.
Super SloMo
With the aim of creating high-quality slow-motion videos from 30 fps footage, the team trained its system with 1,132 reference videos (240 fps) of everyday and sporting activities. The paper describes variable-length multi-frame video interpolation to generate one or more frames between two consecutive images.
The result is intended to be a video sequence that is credible both spatially and temporally. Here you can see several examples; slowed-down footage from “The Slow Mo Guys” was also already used, among other things. You can find further information about the project here.
Further AI Research Projects from Nvidia
Until the techniques shown mature and find their way into commercial software tools, some people are working on unofficial implementations, such as for the “Noise2Noise” project. Nvidia itself frequently publishes the respective source code and further information, as a non-commercial version, on the official Nvidia Github account.
