Table of Contents Show
The first speaker is Anton Palmqvist, Head of Real-Time at Goodbye Kansas. He starts the talk with the project “Troll”, which is presented as a cinematic trailer on the one hand and serves as a tech demo for the real-time raytracing features of the Unreal Engine on the other. The half-hour presentation will focus on the use of real-time ray tracing in a modern VFX pipeline and how game tech can be combined with a classic VFX pipeline. Anton Palmqvist answered DP’s questions in an interview and went into further details as well as possible future prospects for the technology, which is still in its infancy.
The demo project
As you can see from the pictures in the article, it revolves around a young lady called Bianca, who is sitting in a crown in the middle of a forest by a small lake. Frogs croak and the wind rustles the leaves. Three fairies appear below the surface of the water. They circle at first, glow, light up the area around them and rise out of the water. They dance around, play, fly around with their crowns. Soon muffled noises are heard, leaves rustle. The camera jumps back into the forest, a dark silhouette rises with muffled noises. Startled, the fairies disappear, Bianca’s face full of displeasure, darkness sets in, the crown falls into the princess’s hands and her gaze is directed towards the dark forest and the silhouette – fade to black. The audience applauds.
The underlying story was directed by Björne Larson, Head of Studio at Deep Forest Films in Los Angeles, with a musical score by Academy Award winners Ludwig Göransson and Joseph Shirley. To ensure that Princess Bianca’s emotions are as convincing as her movements, 3Lateral was involved in the project. 3Lateral is now part of Epic Games and specialises in the natural and realistic representation of digital creatures.
Game tech in VFX
Just under a minute and a half of film footage shows the latest real-time ray tracing features of Unreal Engine version 4.22 in action, and the quality is very convincing. The differences to final frame rendering with sophisticated path tracers are recognisable to professionals, but from the perspective of real-time computer graphics, the trailer was a visual treat and definitely forward-looking. Especially because the trailer was played at a constant 24 fps with an Nvidia RTX 2080Ti. The highlight is when Palmqvist pauses the trailer and zooms in on the respective areas to show the features.
“Troll” is a joint project between Goodbye Kanses, Epic Games – also 3Lateral – and Deep Forest Films. The VFX studio Goodbye Kansas, based in Stockholm, is known for creating high-quality game trailers, films, TV shows and adverts. Palmqvist says that Goodbye Kansas covers both areas, games and film. The VFX studio has an offshoot in Los Angeles that is dedicated purely to realtime. As a result, the studio has experience in creating products in both areas, and it was obvious that real-time ray tracing should also be examined more closely for production suitability.
As Palmqvist is at home in the field of games and had a great deal of prior knowledge of the Unreal Engine, he was predestined for the role of Head of Real-Time, which he took up in April 2018. He worked on the “Star Wars” franchise “Battlefront” and contributed his skills as a 3D and lighting artist to “Mirror’s Edge Catalyst”. The “Troll” project was intended to explore the potential of the Unreal Engine and compare it with the classic VFX pipeline in order to identify possible intersections. According to Palmqvist, a general review was planned and the real-time ray tracing features were the cherry on top.
Game tech workflows, as we all know, are different from offline rendering workflows. In order to create an intersection between the two worlds, attention was already focussed on the real-time ray tracing features during the preview version of Unreal Engine 4.22. According to Palmqvist, one major advantage is the support of real-time reflections. In game tech, there are ways to realise visually appealing reflections in one way or another, usually using special tricks or hacks. There are also common features such as Planar Reflection and Screen Space Reflection. However, the latter creates a certain type of artefact that detracts from the overall picture, precisely because the large water surface reflects the forest and all the surrounding objects, Bianca with her crown and the fairies. The same applies to the crown – an asset predestined for real-time reflection.
Especially in relation to the water in the trailer, it was also a relief that real-time translucency is supported. This gave the shot in which the fairies move in a circle beneath the surface of the water in front of Bianca a whole new dynamic. Translucent objects such as water or glass surfaces behave more realistically in the first instance. In conjunction with the lighting features that are also supported, the working method in Game Tech is slowly leaning towards the offline rendering VFX pipeline.
We are talking about area lights, which can be used in the Unreal Engine to bring more dynamics into the backdrop lighting and create soft shadows. To back up the example with visual material, Palmqvist repeatedly shows certain excerpts from the trailer. He presses the pause button, so to speak, and zooms in on certain parts, such as Bianca’s face, which is shown in close-up while the area light moves. The real-time reflection is shown by zooming in on the crown, in which the silhouettes of the glowing fairies can be seen.
Global Illumination
In addition, it was possible to use real-time global illumination in the area of lighting, which offers artists a completely new degree of freedom. Palmqvist took up a standard working method from the game tech sector, in which UV channels have to be created for all environmental objects as part of the light baking process, with sensible UV arrangements so that the respective light map is given the most economical dimension during baking. Then the process of light baking begins, which takes time and, depending on the resolution, is similar to offline rendering times. The work steps would be virtually eliminated and create new freedom. He added that there may be areas that do not benefit from real-time ray tracing and that can be properly given a light map channel with semi-automatic tools of the Unreal Engine for light baking so as not to waste samples unnecessarily. Nevertheless, the real-time ray tracing features in the area of lighting are another reason to combine game tech with the classic VFX pipeline. After all, the features would allow artists to adjust atmospheres in real time without having to worry about asset properties and re-baking. The DP editorial team was curious and questioned the real need for real-time ray tracing.
Palmqvist grinned and said quite frankly that the real-time teams at Goodbye Kansas in Stockholm and Los Angeles would have been able to create visually similar visual effects, but only with the aforementioned tricks, hacks and tricks, and with a significantly higher time investment compared to the real-time ray tracing features of Unreal Engine 4.22. And not only the time expenditure would be a problem, but the abundance of interfaces that would still have to be implemented to dock the classic VFX pipeline to the game tech.
Palmqvist also briefly touched on the Unreal Engine’s Alembic support and praised the feature, as Goodbye Kansas, as an established VFX studio, uses the Alembic format for all assets as an exchange format. In the production of the “Troll” trailer, for example, Princess Bianca’s dress, which has around 100,000 polygons, was simulated in Houdini and transferred directly to the Unreal Engine via Alembic Cache, which was a great help. Support for industrial data formats is necessary.
In order to bridge the gap between game tech and the classic VFX pipeline, additional tools for artists had to be created in the Unreal Engine. The first tool was an uber-material. A large shader builder that allows artists to easily create a variety of different surfaces without any software-technical fiddling. Palmqvist explains that it is important to keep the material system as close to the classic VFX pipeline as possible in order to make it easier for new artists to work with game tech and reduce learning curves. The teams have therefore attached great importance to ensuring that UDIM support is available and that assets from production can be imported directly into the Unreal Engine.
To make lighting and look development more standardised and intuitive, a scene was developed for Look Dev that artists can use to test their assets. A major advantage is that lighting and material adjustments can be made live. In addition, a build of the scene can be generated and exchanged with the customer. This allows the customer to interactively inspect the assets and test different lighting conditions and surfaces.
Is Game Tech with real-time ray tracing capabilities ready for use in a classic VFX pipeline?
Palmqvist assessed the new possibilities as positive and spoke of a rethink in production planning: “The classic VFX pipeline model naturally has iteration cycles at one point or another, but under the bonnet it is a linear production model – even if modern renderers are equipped with interactive rendering. Modern denoisers are usually added to approximate a specific pixel colour. When modern game tech is integrated into VFX pipelines, the linear production model is replaced by one that works in parallel. As an example, he pointed out that adjustments to the light setup can be made very early on in the production – fine adjustments and nuances as well as the elaboration of intricate lighting details can be carried out until shortly before the end of production. This results in a new dynamic in production. The customer can be involved in the initial Look Dev. He is at the forefront.
Conclusion
We still need to investigate further where the limitations of the new features lie. The “Troll” tech demo ran at 24 fps with an RTX 2080 Ti and there were no performance lags that could have jeopardised the project. It would also have to be examined how studios could develop USPs using game tech. If the customer wants a project for which classic VFX with offline rendering is too expensive, it can be converted to Game Tech. However, the advantages of asset exchange also need to be analysed more closely within the studio. It may be necessary to develop an over-pipeline that allows a raw data set to be integrated into offline or real-time rendering in just a few steps. A similar situation already exists with the character pipeline in Goodbye Kansas. The full potential still needs to be uncovered through numerous tests, trials and further projects.
