For those who don’t know the tool: Nuke from Foundry is a node based compositing application used in feature film and episodic VFX. It ingests multi pass renders, deep data and USD scenes, and sits downstream of layout, animation and lighting while feeding DI and delivery.

Gaussian Splats go native

Foundry has released Nuke 17.0 with native support for Gaussian Splats. Artists can now import, view, manipulate, render and export splat data directly inside Nuke without third party conversion tools. Gaussian Splats are point based scene representations in which radiance is encoded as collections of anisotropic Gaussians rather than polygonal meshes. In Nuke 17.0, splats can be imported via GeoImport or GeoReference nodes and visualised in the Hydra based 3D viewer.

Rendering is handled through the SplatRender node, which supports depth output, deep output and motion blur. GeoDeletePoints enables non-destructive point removal, while GeoGrade allows colour grading operations on splat data. Field Nodes integrate with splats, enabling combination and transformation of point data. According to Foundry’s documentation, these nodes allow non-destructive masking and manipulation of 3D data, including Gaussian Splats.

USD 3D system leaves beta

The new USD-based 3D system is now fully released. It is built on Pixar’s Universal Scene Description and is intended to bring structured scene management into Nuke’s compositing context. A new import dialogue enables selective loading of data from complex USD scenes, including payload control and item-activation toggles. GeoImport supports USD with scene graph metadata as well as Alembic cameras and geometry. Axis nodes now include new constraint input pipes such as LookAt, Translation and Rotation constraints, alongside a Snapshot function to freeze transforms.

Camera workflows have been extended. Viewer locator visibility is improved, GeoCamera nodes can be created directly, and GeoEditCamera allows direct editing of USD camera data. Projection tools have moved into ScanlineRender. Cylindrical projection and UV Unwrap are supported, including UDIM tile outputs.

Material workflows have been aligned with USD and MaterialX. MtlXStandardSurface supports MaterialX Standard Surface preview and render in the Hydra viewer. New shader nodes include ReflectiveSurface for reflections and refractions, BasicSurface and WireframeShader. Legacy equivalents such as ConstantShader, FillShader and MergeLayerShader are retained.

Material binding is handled through GeoBindMaterial, which includes Binding Strength and Purpose controls to balance performance and render fidelity. Lighting nodes have been expanded. USD light attributes can be pulled directly into Nuke without baking. Updated DirectLight, SpotLight, PointLight and EnvironmentLight nodes include shadow toggles and output channel controls.

New USD aligned light nodes include GeoEditLight, GeoDistantLight, GeoDiskLight, GeoSphereLight and GeoDomeLight. Controls extend to intensity, exposure, falloff type, focus and colour temperature, with terminology aligned to physically based workflows. ScanlineRender now exposes explicit ray tracing controls, including Max Ray Depth and separate depth limits for reflection, refraction and diffuse rays. Motion blur controls include Shutter, Shutter Offset, Shutter Segments and Shutter Bias.

AOV output control has been expanded with Z depth, motion vector variants and independent surface AOV toggles. GeoMask and GeoClearMask nodes enable 3D masks to cascade down the node graph with visible indicators, improving readability in complex setups. GeoPython provides direct Python access to USD prims and schemas inside the scene graph, allowing scripted edits to USD data within the comp context. Graph Scope Variables can now operate inside USD scenes to swap geometry, cameras and textures based on variable logic.

BigCat scales machine learning

Nuke 17.0 introduces BigCat, extending the existing CopyCat machine learning toolset. While CopyCat focuses on shot level training, BigCat is designed for large scale dataset training across tens or hundreds of shots. BigCat supports data augmentation to generate variation for lighting and motion conditions. Validation tools include validation loss graphs and a validation contact sheet. Custom loss functions such as LPIPS are supported to encourage structural and semantic alignment beyond simple pixel difference metrics.

Graph Scope Variables mature

Graph Scope Variables receive a production ready framework update. Python callbacks and API integration allow custom code to be triggered on variable events. Root node knob expressions such as frame range and fps can drive dynamic variable values. Variable names can now be displayed in node labels for improved script readability. Foundry reports performance improvements for large scripts using Graph Scope Variables, though no numerical metrics are published.

Annotation and UI refinements

The annotation system has been overhauled. Drawing tools are more responsive, brushes are redesigned and a dedicated comment panel is introduced. Visual indicators aim to clarify review notes within the node graph.

In the 3D workspace, node colour coding, mask icons, scene graph filtering and live read indicators have been updated to improve clarity. Error and warning visibility in the viewer and node properties has also been enhanced. By the way, if that person looks familiar: You’ve read a whole story about that project in Digital Production:

We Hunt Giants

Core performance and colour

Core image processing updates include significant acceleration in TVIScale. Foundry states that GPU upscaling can be up to 98 times faster and CPU upscaling up to 26 times faster than previous behaviour. Deep composite rendering is reported to be up to 1.88 times faster to viewer and disk. Test conditions are not detailed in the documentation.

OFX plugins are no longer artificially resolution-limited. Plugins, including Furnace OFX, can now run at full hardware resolution. HDR MOV reading and writing now preserves YCbCr matrix, colour primaries and transfer functions. Native ACES 2.0 configurations are included. NotchLC codec support for MOV ingest and export is available on Windows and Linux.

Platform alignment and SDK updates

Nuke 17.0 supports USD 25.08 and aligns with VFX Reference Platform 2025. Third party libraries updated under this platform include Boost, OpenColorIO, OpenEXR, TBB and OpenVDB. OpenAssetIO and OpenAssetIO MediaCreation libraries are upgraded to version 1.0.0, with MediaCreation at 1.0.0 alpha.12.

Camera format support is updated via the latest Sony SDK and SMDK, including support for Burano firmware version 2 and 3.8K and 4K formats. Monitor out updates include AJA NTV2 17.5x, DeckLink 14.4 and NDI 6.x SDKs, with 10-bit support.

Parallel release for conservative pipelines

Alongside Nuke 17.0, Foundry has released Nuke 16.1. It includes most of the same feature updates but is built on the VFX Reference Platform 2024, allowing studios additional time to migrate their infrastructure.

Nuke 17.0 is available to customers with an active subscription. Pricing details are available from Foundry’s official product pages.

As with any major release that modifies 3D scene handling, machine learning workflows and core processing behaviour, facilities should validate performance, memory usage and pipeline compatibility before deploying to live productions. New tools and innovations should always be tested in controlled environments before use in production.

// Foundry Nuke 17.0 release notes
// https://learn.foundry.com/nuke/17.0v1/content/release_notes/nuke_17.0.html

And, if you want, you can compare it to the release of the Beta version in November:

Foundry Opens Nuke 17 Beta: New 3D System, New Variables

The post Nuke 17.0 rewires 3D and adds Gaussian Splats first appeared on DIGITAL PRODUCTION and was written by Bela Beier.

Placement in the Ecosystem: glTF from the Khronos Group is a lightweight asset format for realtime engines, viewers, and DCC export, commonly used between tools rather than inside them, and now it wants splats too. Visit their site here.

A careful step into splat territory

The Khronos Group has published a release candidate for a new glTF extension that adds support for Gaussian splatting. The proposal, named KHR_gaussian_splatting, defines how 3D Gaussian splats can be represented inside glTF 2.0 assets using a common, vendor-neutral schema.

Gaussian splatting is a scene representation technique where geometry is described as large numbers of oriented 3D Gaussians rather than polygon meshes. Each splat typically carries position, scale, rotation, colour and opacity. Rendering relies on blending these splats in screen space to approximate surfaces and volumetric detail. The approach is increasingly used in photogrammetry-derived captures and radiance field research, particularly where dense detail and rapid rendering are required.

Until now, Gaussian splats have circulated in a growing number of ad hoc file formats. Khronos is explicitly positioning this extension to prevent early fragmentation by defining a baseline encoding before incompatible variants become entrenched.

What the extension actually defines

KHR_gaussian_splatting introduces a standardised way to store splat data in glTF without redefining the rendering algorithm. The extension specifies how splats are described as primitives and which attributes are required or optional. These include spatial transforms and per-splat appearance data, mapped onto existing glTF data structures where possible.

The proposal does not mandate a specific splatting technique or optimisation strategy. Khronos describes the design as algorithm-agnostic, with the goal of supporting multiple implementations rather than locking vendors into a single approach. This restraint is notable given how quickly splat rendering methods are evolving.

Viewers that do not support the extension are expected to fall back gracefully, typically treating the data as a point cloud. This preserves basic visibility rather than failing outright, which aligns with glTF’s long-standing emphasis on predictable degradation.

Why Khronos is getting involved now

Gaussian splatting has moved from research into production-adjacent tools at a speed that has made standardisation awkward but necessary. Khronos frames this release candidate as a response to real usage rather than speculative future demand. The organisation is inviting feedback from developers and vendors before final ratification, signalling that the format is still open to adjustment.

The work builds on earlier collaboration between Khronos and geospatial and platform vendors exploring splat-based representations for large-scale scenes. While those efforts are not detailed in the extension itself, the context suggests a desire to make splats a first-class citizen alongside meshes, animations and textures in glTF.

glTF is often described by Khronos as a transmission format rather than a working format. Adding splats does not change that positioning, but it does acknowledge that realtime pipelines are absorbing data types that were previously confined to research or proprietary viewers.

Interoperability over novelty

There are no promises of faster rendering, smaller files, or better fidelity. Instead, the emphasis is on interoperability and predictability. This is a conservative but welcome stance for a standard that already underpins web viewers, game engines and real-time visualisation tools.

Compression is not part of the core proposal. Khronos notes that companion extensions may be introduced to improve storage efficiency, but they are not part of the current release candidate. No compression scheme is ratified as part of KHR_gaussian_splatting yet.

This separation is sensible. It allows the base representation to stabilise before additional complexity is layered on top, and avoids repeating past mistakes where format and codec decisions were entangled too early.

A positive but cautious signal

As proposed, KHR_gaussian_splatting does not attempt to redefine how Gaussian splats are generated or rendered. It simply gives them a place to live in a widely adopted container format. That restraint is arguably its strongest feature.

For production teams experimenting with splat-based capture or visualisation, a common interchange format reduces risk. For tool developers, it provides a reference target rather than another reverse-engineered file. For Khronos, it reinforces glTF’s role as a neutral meeting point rather than a battleground of features. As always, new formats and extensions should be tested thoroughly before being relied on in production environments.

// Khronos Group press release on glTF Gaussian Splatting
// https://www.khronos.org/news/press/gltf-gaussian-splatting-press-release

The post glTF learns to splat without losing the plot first appeared on DIGITAL PRODUCTION and was written by Bela Beier.

For those who don’t know the tool: D5 Render from Dimension 5 is a realtime rendering engine used across architecture, design, and visualisation pipelines. It connects with 3ds Max, Blender, Cinema 4D, Revit, Rhino, SketchUp, ArchiCAD, and Vectorworks via direct sync plugins.

Oceans, clouds, and skies

Version 3.0 introduces two major environment tools: an Ocean object for animated water surfaces and a Volumetric Clouds system with multiple presets such as cumulus and cirrus. The ocean object allows control of wave scale, wind speed, and interaction near shorelines.

Volumetric clouds integrate with improved fog and light scattering options for more natural outdoor illumination. Enhancements to the existing Geo Sky model provide finer control over sky intensity and sun balance, improving consistency between time-of-day setups.

Rendering and material control

A new True Displacement option replaces parallax mapping with physically accurate surface relief. The trade-off is a higher render cost, but it yields more convincing depth in materials such as stone or concrete. Additional material updates include CMYK texture support and a toggle for HDRI resolution, allowing lighter scene loads when full 16K environment maps are unnecessary. Users can now also set a custom viewport background colour, aiding previs or presentation renders that do not require a full environment.

AI generation expands inside D5

Dimension 5 continues its exploration of integrated AI tools. The new Image-to-3D feature creates simple 3D models from one to three reference photos, processed directly inside D5. In the AI Agent suite, Scene Match allows textual adjustment of scene elements, while Asset Recommendation searches the asset library based on descriptive keywords or reference images. At press time, these AI tools remain early implementations, and output quality varies depending on subject complexity.

Experiments with Gaussian splats

D5 Render 3.0 introduces experimental import of Gaussian splats, a point-based 3D representation format derived from neural rendering research. The implementation is explicitly labelled experimental, and Dimension 5 advises users to expect incomplete material or lighting behaviour.

Workflow refinements and licensing

Navigation within scenes has been overhauled with a free-camera mode and refined controls. Several features previously restricted to Teams subscribers, such as Caesium geographic data and XR Tours, are now available in the Pro tier. The procedural city generator is also promoted from Teams to Pro.

D5 Render continues to be offered in three tiers: a Community edition that remains free, plus paid Pro and Teams subscriptions.

D5 Lite: a lightweight sibling

Alongside the main update, Dimension 5 announced D5 Lite, a compact edition designed for early-stage design work, initially integrated with SketchUp. D5 Lite runs in two modes: Real-Time Rendering for viewport previews and AI Generation for cloud-based image synthesis from screenshots. Future versions of D5 Lite are planned for 3ds Max, Revit, and Rhino, though no release schedule has been released.

Compatibility and system support

D5 Render 3.0 supports Windows 10 and later. The recommended GPU baseline is an NVIDIA GeForce GTX 1060, AMD Radeon RX 6400 XT, or Intel Arc A3 series. Integration plugins remain available for all major DCC and CAD tools. As with any major rendering update, users are advised to test D5 Render 3.0 and its AI features on non-critical projects before integrating them into live production.

The post Ocean, clouds, and splats: D5 Render 3.0 sets sail first appeared on DIGITAL PRODUCTION and was written by Bela Beier.

The release of Corona 14 by Chaos introduces a broad set of enhancements for artists working with 3ds Max and Cinema 4D. According to the official “What’s New” overview from Chaos, the update emphasises three areas: smarter AI-assisted workflows, advanced procedural materials and greater environmental and scene-context realism.

Gaussian Splats

One of the standout additions is support for Gaussian Splats, a novel method for scene reconstruction designed to accelerate the placement of buildings and objects within realistic contexts. Chaos describes this feature as enabling memory-efficient “splats” derived from real-world environments, which provide smoother surfaces, richer volumetric detail—meaning more accurate light and particle interactions with depth and atmosphere—and a more natural sense of depth in renderings.

Nighty-Night!

In the domain of night-time and atmospheric visualisation, Corona 14 expands its Sky system with the Night Sky feature. Artists can specify location, date and time to simulate moonlight, stars and the Milky Way, or manually adjust celestial elements for artistic control. The feature is fully animatable, offering potential benefits for time-lapses or cinematic render sequences without relying on external HDRIs (high dynamic range images).

Materials

Procedural material creation receives a boost with the Fabric Material, enabling the creation of realistic textiles via native control over weave structure, threads, opacity, bump and displacement without the need for external bitmap textures or complex node graphs. A set of presets is provided for rapid deployment.

On the AI-tooling front, Corona 14 introduces an AI Material Generator that takes a photo of a real surface and transforms it into a fully render-ready PBR (physically based rendering) material. The goal is to expedite creation of secondary materials that need little artistic input. An AI Image Enhancer improves detail and texture in supporting scene elements such as vegetation, people or terrain while preserving the core design; this tool supports sending LightMix results (lighting scenario composites) directly into the enhancer. An AI Upscaler enables conversion of low-resolution drafts into high-quality presentation imagery, potentially saving render time while maintaining crisp output. Users may toggle AI features on or off, giving full control for studios with strict AI policies.

And more ….

Performance and workflow refinements accompany these major features. Volume sampling for render-intensive volumetric effects such as fog, light rays or smoke is optimised. A new option enables rendering at double resolution and automatic downscaling on save, which, according to Chaos, may allow higher noise levels or fewer passes and thus reduce render time. The UI has begun a rewrite using Qt (initial phase), starting with the Physical Material interface.

The Virtual Frame Buffer (VFB 2) has been updated with a cleaner, more compact interface, simplified navigation and clearer icons; the Corona Image Editor now mirrors the VFB 2 look and feel, enabling tone mapping, LightMix variants and other post adjustments outside the host DCC (digital content creation) tool. Additional workflow improvements include triplanar controls inside the Corona Bitmap (no separate node setup required), multiple-camera export to Chaos Vantage (initial implementation) and docking of the VFB window in 3ds Max.

Corona 14 is listed as compatible with Autodesk 3ds Max versions 2018–2026 (64-bit) on Windows and with Maxon Cinema 4D R17–2026 on Windows and macOS. Subscription licensing is offered on monthly or yearly payment terms.

Get your Corona here

However, the true impact of features such as Gaussian Splats and the AI modules will depend on how they perform in real-world production environments. Testing before deployment remains essential.

The post Corona 14: Gaussian Splats, Night Sky & fabric-material first appeared on DIGITAL PRODUCTION and was written by Bela Beier.

When Chaos Labs first showed off Chaos Arena and Chaos Vantage together, the pitch sounded almost heretical: a real-time virtual production suite that skips game engines entirely. To some, that sounded impossible. To others, overdue. We spoke with Chris Nichols, Director of Chaos Labs, about what it means to go “engine-free,” why USD and MaterialX matter when the lights go up, and how Gaussian splats actually behave outside a SIGGRAPH paper. *¹

DP: Chris,before we jump into the tech: Chaos Arena and Chaos Vantage together are pitched as a virtual production suite without the usual game engine baggage. How would you describe the idea behind this approach in your own words?

Chris Nichols: One of the biggest issues with virtual production is rendering. There are options that have existed for a while, including working with game engines, but they require you to move to an entirely different platform, use an entirely different ecosystem, and switch to an entirely different pipeline. What we’re doing is focusing on the rendering part in a way that lets you keep your current ecosystem and pipeline. 

Using Chaos Vantage and Chaos Arena simplifies the entire process. You can basically just take what you already do and work with real-time content. You don’t need to bring in additional experts or learn new systems, you just do what you’ve always been doing and take it a step further. 

DP: Arena and Vantage now support USD, MaterialX, and even Gaussian Splats. Could you walk us through why these formats are important?

Chris Nichols: A lot of times, people are forced into using a specific system or format that puts them in a walled garden. It makes them take additional steps, which just slows everything down. We’re trying to break out of that by adopting open source formats and making things as easy as possible, and the visual effects companies that have begun to accept and support USD and MaterialX have an advantage. With USD and MaterialX, no matter what tools they are using – Blender, Maya, Houdini, 3ds Max, etc. – they are directly compatible with Chaos Vantage and Chaos Arena without needing to convert anything. 

With gaussian splats, that’s just a general demand that people want. It also works really well with some of the things we specialize in, especially ray tracing, and it allows us to utilize them more than just a 3D point cloud. 

DP: One of the bold claims is that you can work without pre-processing, no baking, no data wrangling. From a technical standpoint, how are you pulling that off?

Chris Nichols: This is a very interesting question. If you’re looking at game engines, which use rasterized rendering, that does need pre-processing, and you do need baking and data wrangling. But if you skip rasterization and go full ray tracing, none of those things are necessary. So they are essentially two separate things that seem similar, but are not. It’s kind of like asking how many miles per gallon an electric car gets, which is an understandable question given that we’ve been thinking in those terms for years, but it’s really two different technologies that work differently. 

With rasterized rendering, pixels are free. No matter what the resolution you’re working in, the render times are not affected that much, but adding more polygons slows it down. With ray tracing, polygons are free. So no matter how much geometry you throw at the system your rendering isn’t affected all that much. Alternatively, when you start adding more pixels you need more rays, which slows things down. 

What has really changed that allows us to bring it all together is recent technological breakthroughs. That includes AI, which lets us denoise and upscale our renderings at very, very high quality, using something called “ray reconstruction,” which was first introduced in DLSS 3.5 and has since been improved. Ray reconstruction allows us to produce very good renders using smaller resolution bases with fewer rays, while keeping the speed. Add in AI and you can get clean, high resolution results with far fewer rays cast, and significantly fewer pixels. AI fills in the gaps. 

With ray tracing, there is no baking. That’s the point. We don’t have to pre-render the lighting, it’s full global illumination in real time. Move lights and move objects, all the lighting follows in real time, no rebaking. There’s also no need for data wrangling, because we can just throw the maximum geometry at everything all the time. We don’t need to create levels of detail, we don’t need to do anything else. We can just say the final asset is in real time and enable that, which basically means one asset for everything across the board. 

DP: If I’m an artist walking onto a stage with Arena and Vantage for the first time, what does the workflow look like from setup to shooting?

Chris Nichols: It’s actually very straightforward. Vantage itself is a simple program. You can just load your asset into Chaos Vantage as either a .vrscene file or as a USD with a MaterialX system. We also support OBJ and a couple other formats, but those are the most common.

Navigating around is then very straightforward, like most 3D applications. Chaos Vantage is not trying to be a game engine, and it’s not trying to add all kinds of complicated systems. It’s just meant for real-time visualization. So you put your cameras down, move your lights, and move your objects along with a couple of other things. Arena manages multiple instances of Vantage and enables them to operate simultaneously, synchronizing multiple render nodes across a large LED wall. Essentially, it’s saying “here is a master Vantage scene, here’s all the slave nodes that are working in the background.” Arena makes sure everything is synchronized and rendering together. 

Arena also adds the camera tracking system you are already using, and we support multiple camera tracking systems. We just enable the protocols. It could be OptiTrack, stYpe, Mo-Sys, or any of those systems, and it enables where the camera is looking. It’s very straightforward and there’s not much of a learning curve. It’s essentially a viewport. This makes it much easier for people to adapt to the system. 

That means that a Maya user can easily jump into Arena within a half an hour or less. Same with a Houdini user, a Blender user, or any of those types of tools. Those are the types of artists we are specifically targeting, giving them the ability to continue in the real-time world. That also includes people like directors of photography, because it operates like how a real camera operates. It’s the same language they would use. 

DP: Virtual production workflows often involve deciding what to solve in real time and what to leave for post. How does your toolkit support that balance?

Chris Nichols: In the past, there’s always been this break between what you need for offline rendering, and what you need for real-time rendering. It’s two different pipelines, two different teams, sometimes two different companies doing completely different things. We’re looking to make one system that can cross many departments and many avenues simultaneously. We want to offer the same asset that can be used from previs to virtual production to post-production, and make it as seamless as possible. With USD and MaterialX, you are free to use the same asset across the whole production cycle, no matter what DCC, what renderer, and what pipeline you use. It’s your pipeline, and your choice.

The transition from virtual production to visual effects to post-production should be as seamless as possible. That’s one thing that we feel that we support very well. The best example of that is the asset we used for the short film we created to announce Arena, Ray Tracing FTW. You can watch it here in full:

That was just one massive asset that had the entire town. That included the interior of the bank, five kilometers of train tracks, the tunnel, the bridge, every blade of grass… everything. In total, the asset contained around 2.8 trillion triangles, and the whole asset was generated in a matter of days. We used it for everything, then we’d take that same asset and use it for post-production, because we still had all the detail that we originally had from the beginning. That’s how you save a lot of money. So basically, why build it three times when you can build it once?

DP: You tested Arena extensively on the short film Ray Tracing FTW. What were some surprises?

Chris Nichols: Our assistant director on Ray Tracing FTW was Ben Hansford, a well-established commercial director that has experience with virtual production. He told us to plan for an hour to an hour and a half of crashes every day, because that’s what’s expected when traditionally working with virtual production. We ended up completing the shoot in three days without a single crash.

In fact, I don’t know of any crashes that have ever happened on set with Arena, at least nothing that was not recoverable in a matter of a few minutes. We didn’t expect it to be as stable as it was, especially in those early days when it was barely a product and the shoot partly meant to be a test for it. 

DP: Real-time changes on set are often the stress test for virtual production. How did Arena and Vantage handle that during Ray Tracing FTW?

Chris Nichols: We aren’t doing any baking or anything like that, so real-time changes are trivial to us. That’s exactly the way it should be. Rasterized rendering can look very photoreal, but you have to do a lot to make it look like that, and some of the things you have to do to make it look photoreal involve not being able to edit it later. For us, if you’re just ray tracing the whole time, none of those things matter. 

DP: Vantage has always been about pure ray tracing in real time. How does Arena extend that, and what does the combination unlock that wasn’t possible before?

Chris Nichols: When you’re done ray tracing, you’re done. You don’t have to solve any of the other problems, so you massively simplify your process. If you go back to 2006 when VFX started to adapt full ray tracing – not just the hybrid ray tracing that most game engines use – teams were still trying to rasterize things, but they started to realize they could simplify the process. They didn’t need to have things like full teams of shader writers, instead they could use ray tracing and that essentially solved the problem. It made things much, much faster and more efficient. 

The same philosophy is true here when we’re doing real-time ray tracing. It gives us a big advantage, and it’s not just in the quality of how things look, it’s how fast you can make something look good, and how easy it is to get there. So you’re actually saving a ton of money by ray tracing. 

In terms of what new technology has made all of this possible, when the 2080 graphics cards came out in 2018, it was the first time ray tracing cores were enabled on NVIDIA cards, and that let us design around that technology. But we weren’t able to achieve that super high level of quality resolution needed for virtual production until DLSS 3.5 came out. More recently, AI has made a lot of things possible. The best way to think about it is one out of every eight pixels is real, the rest are AI generated.

DP: For artists who are used to Unreal Engine or Unity, what’s the biggest mental shift when moving into an Arena/Vantage-based workflow?

Chris Nichols:  Unreal and Unity are fantastic tools for making games, and essentially what they’ve done for virtual production is to try to gamify the film making process. Game engines turn everything into a game control system, which can work to some extent, but it completely deviates from the other systems that the movie industry has been using for a long time, especially visual effects. What we’re doing is we’re making it easier to streamline that process across all departments. 

For people that have been using game engines, Arena is much simpler and you’re not sacrificing anything to get there. There are a lot of tricks and processes that you need to know for a game engine, and that’s just the way it has been. With Arena, we’ve had people ask if we can recreate a unique process they’ve been doing, but we don’t need to implement their workarounds. So their very clever tricks are completely unnecessary in the way that we do things.

DP: Gaussian Splats are hot right now. How do they behave inside Arena and Vantage compared to the hype we see in research papers?

Chris Nichols: Gaussian splats seem like they are here to stay, plus they are simple enough that it was relatively easy to support them compared to other standards in the past. The technology is also fairly familiar, and builds on the older light field technology that we experimented with years ago, first with companies like Lytro, then Google. Gaussian splats are essentially a representation of color and light from multiple different angles that change based on the angle you are viewing it, which is similar to what a light field does.

The problem with light fields is that it involves massive amounts of data, and lots of redundancy to get it there. So much so that they were impractical to use. But if you use neural fields to represent all the redundancy, it changes things. That’s basically what neural radiance fields, or NeRFs do, they are essentially light fields with a neural network. It’s a really cool system, but still very cumbersome. 

So when gaussian splats came along, it solved a lot of issues. All we really need is a point in space and a shape on that point – typically a disc or an ellipsoid – that changes colors depending on the viewing angle. It’s such a simple idea that began as a concept and grew from there. And because they’re so lightweight, because they do so well, it just became so easy for us. 

The benefit that we get from gaussian splats, and the way that we’re approaching it by actually ray tracing it – as opposed to rasterizing it – is that we can put 3D objects inside of a gaussian splat and they will ray trace that gaussian splat directly. The best example I can give is that you can take a gaussian splat of a city, and then you put a CG car on a street within the city. Then when the car travels through the city it will accurately reflect the environment within the gaussian spalt. If you did it as an HDR, every light point is at infinity, and that’s not how it really should be.

DP: From a director’s or DoP’s point of view, what kind of creative control does this system provide on set?

Chris Nichols: Everything that a DP would do manually, we can offer virtually. You can move lights, you can change colors, you can do all kinds of things. You can manipulate all of those different areas as well. We’re actually in the process of working on getting a DMX control for what we’re doing, so the digital gaffers can work together in terms of all the lighting controls you would have.

Another thing that we’re working on for the future is a new way of doing depth of field so that we can control the ability to figure out the distance between the camera and the wall, and add the appropriate compensated depth of field postwall. When the camera and wall defocuses are combined, it is the correct level that you would need for your set. 

DP: Are there particular hardware setups that Arena and Vantage thrive on, or can it scale from laptops to full LED stages?

Chris Nichols: Most LED stages I’ve worked on have, at a minimum, an A6000 card, if not two, and that’s perfectly reasonable for most setups. That’s one of the practical advantages of Arena – it can be installed on an existing LED wall in a matter of a few hours, without changing any hardware. In fact, we do that all the time. 

Another benefit of using this type of setup is that Chaos Vantage is a standalone app that can be installed on any laptop running an NVIDIA card. So a virtual production supervisor, a VFX supervisor, or even a director can open the scene and look through the 3D environment on their laptop.

DP: What role does MaterialX play in enabling consistency across tools, especially when moving assets between DCCs and Arena?

Chris Nichols: When someone uses Arena, they may work on an asset that ends up going to several different visual effects studios, each of which has their own pipeline and tools. Some might be using V-Ray, some might use Mantra, some might be using Arnold, and on and on. We want to make sure all of those studios can continue to use the same asset without any compatibility issues. And for shaders to translate, the best thing we can do is support MaterialX. As long as a studio supports MaterialX, they can use the same asset as everyone else without issue. 

DP: If you look at the ideal future workflow, what would be the hand-off between on-set capture with Arena and finishing in post with Vantage or other Chaos tools?

Chris Nichols: Really, it’s just a matter of being able to use the same asset. You can have an asset at a post-production house, then hand it off to a virtual production studio and keep doing what you’re doing. The asset might receive some changes along the way, but when you get it back, it’s essentially the same asset. 

When we were working on Ray Tracing FTW, there’s a scene in a Spanish hacienda. We were busy getting everything ready, so we didn’t have time to build one ourselves. We just went to Evermotion and bought one for around $100, and within 10 minutes it was on screen. It just worked. There are a ton of great assets out there that are Arena ready, and you can find comprehensive libraries filled with virtual production-ready assets from sites like Turbosquid, Evermotion, and KitBash3D to name a few. Hopefully, future workflows will keep things easy. We’re not trying to overcomplicate things. Just use one asset and use it across everything.

DP: Finally, what’s next for Chaos Labs in this space?

Chris Nichols: With Chaos Labs, each project builds on the success of the last, and each new experiment opens the door for the next. For our film Ray Tracing FTW, we wanted to know if Arena could work as a virtual production tool, and it did, because we could see how easy it was to go from one asset to the next. Right now, we’re building on that and testing to see how we can improve the complete filmmaking workflow, from the earliest stages of development to the final pixel, all using a single asset that will be developed throughout the lifecycle of a film. 

One of our focuses with Chaos Labs right now is on creating a previs system for Vantage that enables camera tracking on a bluescreen environment. Using the same asset for previs, we can then translate it for virtual production for in-camera VFX, and then go on to be used for final production. So we’re really looking at the full lifecycle. 

DP: What would be your recommendations for people getting into it?

Chris Nichols: Arena is designed to be easy. It’s virtual production made simple. You don’t need to learn any complicated new techniques or workarounds, you just use what you already know. Your pipeline, your choice. Load the free trial and give it a whirl!

DP: What good resources are out there to learn more about it?

Chris Nichols: There’s a lot of great documentation out there. We make it a point to offer as many tips and documents as possible. There are also tutorials and things like CG Garage. I recently recorded an episode specifically about this. 

DP: What is the next project you are working on ?

Chris Nichols: There is a feature film in development that we are using as a major case study for Chaos Labs, designed to showcase how this can all work. We feel strongly that Arena and Vantage will debunk the idea that virtual production is only for big, expensive movies. It’s a tool that can be used for everything, even low budget independent films. They can have the same power and process as a major production, at a much lower cost. That’s what we’re trying to do within Chaos Labs. We want to show people that you can do your $5 million movie with virtual production and get the same results as films with huge budgets.

1. Editorial Comment: Did Chaos Lab want their name in every sentence? No. But we find it funny to have a highly technical topic, talking to a top-shelf expert, and almost every sentence has the word “Chaos” in it? Yes, because I am somehow still cursed with the humor of a 5-year-old. ︎

The post Keep Your Pipeline: Chaos on Skipping the Game Engine first appeared on DIGITAL PRODUCTION and was written by Bela Beier.