Table of Contents Show
Archive notice: This article was originally published on November 15, 2018. Links and embedded videos are preserved as part of the historical record.
Double Negative (DNEG) was not consulted by Marvel at the actual start of the show’s project, but was only involved in the production when some kind of direction was already in place and several studios were already working on the show. According to Franzen, it is common for different studios to co-operate with each other on big shows, which creates further challenges.
DNEG’s task was to create two large sequences. Both are set on the planet Sakaar. Sakaar is truly a special kind of planet. Mountains of rubbish as far as the eye can see. There are countless wormholes in the sky, all of which dump space debris, and in the middle of the countless smaller wormholes is a massive black hole. The planet resembles a rubbish dump, so to speak. In the centre of the landfill is a large city with a leader, the dictatorial Grandmaster. According to the stories in the film, the Grandmaster was the first person on the planet and, according to Marvel Fanwiki, he also created the planet.
Filling the digital set with computer-generated mountains of rubbish presented the DNEG team responsible for the show with challenges that had to be overcome. Marieke Franzen, Rendering Supervisor at DNEG, and Illaria Introzzi, Pipeline TD at DNEG, explained within the Clarisse Company Suite how Sakaar came up with his immense mountains of rubbish.
The biggest challenge, item 1 on the list so to speak, was the extremely short production time available for the realisation of the mountains of waste and the city. Shortly after DNEG joined the production, trailer deadlines and preview screening deadlines were submitted. The team was told that processes should ideally be finalised yesterday. The presentation primarily focussed on the use of digital distributors – the scatterers.
According to Franzen, the aim was to put their heads together and find out how the extremely detailed objects could be sensibly distributed, what material from other studios working on the show was already available and whether there was still room for creative manoeuvre. In addition, how entry points could be provided for the art direction in order to make the creative leeway as simple as possible if required.
Last minute help
In total, two immensely large environments had to be realised. Both are located on the aforementioned planet Sakaar. One was the industrial city of Sakaar, which is visible from the living area of the Valkyre and was filmed with bluescreen. The live action plates were present here, along with concepts of the city backdrop.
The city backdrop as well as the piles of rubbish were filled with a variety of dynamic elements. In addition, there were animations of characters and vehicles of various kinds. Not an easy endeavour.
The advantage was that the team at DNEG did not have to start from scratch, as other studios were already involved in the show. Accordingly, assets were taken from other studios and incorporated into the in-house pipeline. A process that had its pitfalls, as the look development had to be similar to that of the other sequences and the other studios had different approaches to asset creation. In some cases, objects and textures had to be adapted to the in-house conditions in a very time-consuming process.
To ensure greater clarity, Sakaar was divided into different areas. During production, we spoke of the city centre, the outer city and the Wastelands. The latter presented a particular challenge. Millions of tonnes of scrap, but where from? According to Franzen, suitable assets from DNEG’s asset database were used and cut up, and the spaceships were turned into scrap from the assets supplied by the other studios. Every step taken by DNEG was well thought out. In the end, there was hardly any time. Once the issue of the source of the scrap was more or less resolved, the next step was to work out the best and quickest way to line the Wastelands with scrap.
Scrap islands were listed in the presentation: Clods of scrap spread around the city to simulate slums. Water was visible between the islands to enhance the effect of the islands. Slums in which numerous gangs from the lower social classes are up to mischief, searching for valuable scrap metal and not above taking prisoners. The shot of Thor’s landing shortly before his rescue ends on a scrap island among many different scrap islands. Occasionally the scrap was real scenery and integrated into the live action plates, but for the most part the scrap elements were distributed using an in-house scattering system.
DNEG builds on Clarisse
It has probably been known since 2014 that Clarisse is used as the main renderer at DNEG. In 2014, DNEG acquired a global site licence from the French software developer and has since presented the latest sequences at various trade fairs such as SIGGRAPH and FMX. For the project, this meant that the layout of Sakaar was set up directly in Clarisse. The first sequence to be realised was Thor and the Valkyre looking out of their flat window towards the giant wormhole above the industrial city of Sakaar.
The layout of the industrial city was already predetermined by a matte painting and had to be recreated accordingly. Buildings were placed on the matte painting and their shape determined. Franzen spoke of the great importance of careful work, as the Marvel universe has loyal fans and inconsistencies would be immediately apparent to viewers. Recreating the industrial city of Sakaar through geometry was like a building block system. Added to this was the challenge of staging the gigantic wormhole. Apart from that, the Wastelands posed a further challenge.
The second sequence was dedicated to the Wastelands, affectionately called Kitchen Sink by the team: the mountains of rubbish that grow out of the wormholes due to further rubbish and are virtually piled up in islands. When the team saw the concept drawings of the Wastelands, some of the people involved felt a little queasy, said Introzzi.
The problem was not only that the deadlines were breathing down the necks of those involved, but also that every scrap part was equipped with complex details. Complex assets in a print run of millions with a polycount of trillions, if not trillions of faces, complex geometry, lots of details and trailers or screening deadlines directly ahead – a hardship case.
Clarisse is known for displaying trillions of polygons in the viewport and also in progressive rendering mode. But Clarisse is also known for supporting very efficient scattering methods. Introzzi said openly that scattering systems on this scale have never been used on a show before and that the work on Thor: Ragnarok laid the foundations for the efficient development of a general scattering pipeline using Clarisse Point Clouds in conjunction with the native scatterers.
The fact that Clarisse has the aforementioned native Point Cloud nodes, which can be efficiently connected to the existing in-house open vdv scattering pipeline and Houdini from SideFX, was a great advantage. In addition to the actual distribution, this allowed adjustments to be made to the art direction.
Dan Bailey, who developed a special script and integrated it into a specially developed node, came to the rescue. This node made it possible to pick out certain assets from the point cloud of assets and exchange them for others, so to speak, to enable a point-by-point exchange – in addition to other customisation requests from the customer.
Controlled distribution
The principle behind the script is relatively plausible. Points are distributed over a certain area and assigned IDs. The IDs stand for a specific
Asset type. For the sake of simplicity, one could also speak of a grouping of a number X of points.
The points were processed in Clarisse. The existing islands and their geometry were used as the basis for the point clouds. Point clouds were generated from the surface of the various island surfaces.
The various scrap assets were distributed to the individual points within the point cloud using the scattering system. Before the actual distribution, intermediate steps took place to allow the artists to intervene.
Fully automatic distribution of the objects to the points based on group membership was therefore possible. What was still missing was the possibility to change the appearance manually afterwards, not globally but also locally. In other words, leaving doors open to the different points for the art direction to make visual adjustments or refinements.
A Wrangle Node was developed for this case and provided with a script. Franzen went into more detail about the Wrangle Node in the presentation and used a script example to explain the basic function of a group.
For the first pass, the script assigned random values for the size of the distribution. The VFX director or the customer could then say what changes were required and these were made.
The script was equipped with additional functionalities for further actions. If a particular object was to be made larger or a series of objects had to be reduced in size, it was possible to filter out and select the ID of the underlying point(s) or even bring entire groups into focus. This isolation of the points allowed free editing such as changes in position, rotation and scaling and the subsequent replacement of the assets by intervening in the groups of assets themselves.
After the wrangle pass, after passing through the wrangle node, an additional filter was applied to the points. The aim of the filter was to delete unwanted or unnecessary points in order to fine-tune the visual result. The clear isolation of individual points or groups also has the advantage of breaking patterns and serving as a digital eraser. This is also referred to as point cloud decimation. A feature that is also part of Clarisse’s standard repertoire. The remaining points in the point cloud were transferred to the scatterer, which distributed the objects to the points.
Introzzi spoke of 500,000 points per island in the presentation, and a total of eleven islands are said to have been present in the sequence, which corresponds to 5.5 million detailed assets. If you were to add the polycount at this point, then seasoned VFX veterans would feel the pressure on their chests. One of the supervisors in charge is said to have shed a tear or two when looking at the islands. According to Introzzi, the reason for this was the speed with which the team was able to work within Clarisse.
A question of the version
What was impressive about the presentation was not just DNEG’s approach to production, but also the underlying version of Clarisse. It emerged that Clarisse version R4 SP9 was used at the start of production. Franzen and Introzzi were all the more excited about upcoming shows and the use of the latest scattering features of Clarisse 3.6 SP1. For Franzen and Introzzi it was important to develop a pipeline that makes it easy to use this scatterer – based on point clouds. The network in which Thor was caught by the gang was also realised using a scattering system that linked connection and node points to the respective assets in the specific groups. This meant that the team only had to animate the point cloud and the geometry followed the points perfectly.
