We spoke to Andreas Giesen (andreas-giesen.com), who has been with RiseFX (www.risefx.com) since 2011 and has worked on X-Men, The Matrix, The Last of Us and the various entries in the Marvel universe, among others. He was involved in Fallout as VFX supervisor – and we asked him again after his talk at FMX.
DP: What shots did you get? And how did you get involved with Fallout in the first place?
Andreas Giesen: We worked on just under 400 shots in total and were lucky that they were also very varied. Classic set extensions for the destroyed cities and the Wasteland, the iconic Vertbirds and Powerarmors, right through to the effects-intensive sequences at the beginning and end of the show.
We had already worked with Lisa Joy on the film Reminiscence and for Jonathan Nolan and Jay Worth on Westworld, where there were also gigantic set extensions. Of course, it was a great honour for us to be on board Fallout from the very beginning.
DP: I assume you played Fallout again as Research and Development?
Andreas Giesen: Yes, we had quite a few people on the team who spent countless hours in the various Fallout games. I personally played Fallout 4 extensively as a sniper. But we also had veterans from the early days whose input was incredibly helpful in making sure we stayed in the canon of the series.
I think that’s what characterised this project overall, that everyone involved really had great respect for the original and that a lot of heart and soul went into this implementation from all sides. Practically by Fallout fans for Fallout fans! I think it’s remarkable that the creators have also managed to get newcomers to the series on board..
DP: One would think that it would be easier to develop a film from a game because everything is already in 3D – were you able to use assets from Bethesda and the various studios?
Andreas Giesen: Of course, the Fallout universe offers an incredible amount that we orientated ourselves on, which is why we ordered a few artbooks before the start of the project. We only received complete 3D models of the Vertibird and the Dirigible, but these were more for inspiration than anything else, as they were quite simple in terms of the level of detail (the game is already a few years old).
An elaborate version of the Powerarmor and the Vertibird was also built on set, of which we then received lidar scans. While the Powerarmor could be used as is, apart from a few extensions, we had a little more freedom with the Vertibird, as only the front part was built on set and we had to add the rear part accordingly. Although some of the Powerarmors were of course completely replaced with CG in the end, I think it was important for the look and feel of the series to have this realistic, real armour on set.
DP: How many people were involved in your shots and what pipeline are you working in?
Andreas Giesen: In the almost 1 ½ years that Fallout ran at RISE, we had about 120 different artists working on the project. At the peak, which was towards the end of the project, there were around 60-70 at any one time. Of course, it wasn’t just artists, but also people from other departments such as production, coordination, editorial and pipeline.
The central tool in our USD-based 3D pipeline is Houdini, where everything from shading/lookdev, layout, lighting, FX and rendering (Karma) takes place. The same applies to the procedural creation of individual objects, landscapes or entire cities, which is now also playing an increasingly important role. We mainly use Maya in conjunction with Zbrush for classic modelling and Mari or, in some cases, Substance for texturing. Compositing is done exclusively in Nuke.
DP: And what was the first shot you worked on from these?
Andreas Giesen: We started with the sequence where the “Dirigible”, the zeppelin-like airship, arrives at the Brotherhood of Steel airbase for the first time. These shots were perfect for the start, as we were able to directly utilise our asset, which had been built over weeks, and they were relatively simple and clearly defined overall. At the beginning, we also received a package from the client with lidar scans of the locations and concept art for some of the assets and locations.
The fact that the entire series was shot on film with anamorphic lenses presented us with some particular challenges when it came to integrating the CG objects, but in the end, after successful integration, it contributed enormously to the credibility of the shots and the original look of the entire series.
DP: How much fun is it to detonate a nuclear bomb?
Andreas Giesen: An effect like that is, of course, every effects artist’s dream. So seeing the finished sequence in the series certainly makes the hearts of many VFX artists beat faster, including mine!
For us, however, this sequence was also one of the most challenging, as we had to create a retro-futuristic version of the whole of LA, combined with very complex and long large-scale effect simulations. We also had little to no room for classic matte painting, as large parts of the city had to be dynamically destroyed and the huge pyroclastic cloud had to interact with everything.
As a starting point, there was a concept art image from the production that we could build on. However, this had probably been created before the shoot, so the layout in the shots had to be changed considerably in some cases. Nevertheless, it gave us a direction in terms of look.
As a basis, we then extracted the real topography of LA from online mapping tools. However, these buildings, or rather “blocks”, were nowhere near detailed enough, so we used procedural techniques in Houdini to distribute countless details: Windows, balconies, fire escapes and roof superstructures. These elements were scattered based on certain rules, which we first tested on a single block of buildings and then refined. Colour variations and colourings based on the original colour of the building were important to create enough variation and avoid repetition even within these perhaps 30 elements at a distance.
In addition, our asset team, led by our CG supervisor Jenny Leupold, modelled several hero landmarks and buildings in the classic way, which then contributed significantly to the retro-futuristic LA, such as the Town Hall or the LAX airport. Since LA also has many parks, we have tens of thousands of Speedtree trees as instances scattered throughout the city.
Even though the overall topology is close to the real LA, the scaling of downtown was a bigger issue as it was very important to the client that it remained visible despite the distance we had to keep. If we had simply scaled the existing buildings larger, the entire relationship would have been destroyed, as these buildings would have had much larger windows than those in front, for example.
So in the end we recreated the silhouette of downtown with new, taller retro-futuristic buildings. This meant that everyone could immediately identify LA and yet the correctly scaled details made it possible to recognise the great distance. As already mentioned, we also had to adjust the layout of the city or the orientation slightly depending on the shot so that the important elements were always in the picture.
DP: And how easy is it to blow up a city so that it looks good in the series?
Andreas Giesen: I think our main explosion ended up being version number 273, I would say that answers the question of how easy it was. But of course it’s not unusual for effects of this size to go through that many, or even more, iterations.
There were several technical challenges in this sequence. Firstly, the first explosion and the shockwave extend over several shots, so we had to create particularly long simulations to maintain continuity from shot to shot.
Explosion shots in films are often only two to three seconds long, in this case all the shots added together were more like 30 seconds. Our FX supervisor Kristian Kück and his team have probably created some of the greatest simulations ever created at RISE. I think Fallout used more memory on our servers than the three projects I worked on previously combined.
Thanks to our USD pipeline, it was possible to make all the timing and layout adjustments for the individual shots of the sequence from a Principle scene by always moving a large cache. The timing and placement is then “pushed” from this Houdini scene so that the lighting artist always works automatically with the current version, depending on which shot he is looking at in his Principle lighting scene.
Another challenge was the sheer number of layers that had to be simulated for each of the four explosions in the final shot. While the explosion itself already consisted of several layers, there were also collapsing buildings, swirling dust, various shockwaves, shattering windows and vegetation in the city and also in the foreground. This vegetation was particularly interesting as we had to find an efficient solution for this incredible amount of trees.
A complex simulation for every single tree was definitely not an option. Our FX-TD Jonas Sorgenfrei has developed an interesting approach that uses the Speed-Tree Skeleton to skin the trees. In the actual simulation, only curves are simulated in Houdini, which is very efficient. Using KineFX, the curves can even be easily adapted completely without any simulation. In the final step, the movement is then transferred from the curves to the trees again; a point deform or similar computationally intensive method is not necessary, as the trees are already skinned and we were therefore able to use the joint deform node, in principle as with characters.
The third and perhaps biggest challenge was the look of the main explosion. Of course, you can find some references to nuclear explosions online, including the typical mushroom look. However, Jonathan Nolan, the director, had a slightly different idea in this case. In the first shots, when the little girl looks towards the explosion, it should still be unclear whether the explosion is really a nuclear explosion or perhaps just a large cloud of smoke from a fire.
In the references of real explosions, however, you can see the typical mushroom cloud almost immediately after the initial flash of the explosion has died down. So we had to find a way to make it visually believable that the mushroom cloud practically comes out of the smoke cloud.
This caused our FX team a lot of headaches, especially as these two layers could not simply appear “stuck together”. In the end, the decisive factor was the balancing of the different velocities in the simulations, which influenced each other so that the mushroom rising later slightly displaces the other smoke without the scale suffering from too fast an advection.
DP: And what was the last shot you completed?
Andreas Giesen: We actually edited the series pretty chronologically and also handed it in, as the editing room also worked that way and so we had the first episodes ready for final editing earlier.
So we finished the big attack in episode 8 last, although there were also some complex shots from the earlier episodes that took longer.
We also received the sequence with the cold fusion from the last episode relatively shortly before the deadline. Completing this in the remaining time was a particular challenge, as the time frame for complex FX simulations was no longer sufficient. But as always, Jonathan Heine and his compositing team still managed to conjure up absolutely convincing shots, with an exciting combination of some FX layers but also a lot of Nuke particle magic.
DP: I assume some of the shots – especially for the VP stage – were flexibly constructed and realised in your USD multi-shot workflow?
Andreas Giesen: Yes exactly, actually all of our shots go through the multi-shot workflow. We assign all the shots in a sequence or even several sequences, as long as they are playing at the same location, to a principle shot.
For example, we had the “Airfield Principle Shot”. All shots that take place on the Brotherhood of Steel airbase were assigned to this, including those in which the Vertibird flies at the beginning. So we don’t create a lighting scene for every single shot, but manage the lighting for e.g. 40 shots in this one Principle scene.
Of course, adjustments can be made for each shot using a so-called shot switch, which rotates the direction of light of the sun in a shot, for example. Rendering or sending for rendering on our render farm also takes place from this scene. Using a separate submitter node, you can then select in Houdini which shots are to be rendered overnight and send any number of shots with just one click.
We already had a multi-shot workflow before USD, but USD has simplified things and made it even more efficient, at least now that the rather complex changeover has been completed. Whereas it used to be much more complex to combine all elements of a shot, there is now only one node that loads practically all USD layers for a shot and it is not necessary to load cameras, layout, animations and FX caches individually.
As we work with a push pipeline, the subsequent department always has the latest version of a layer, so manual updates are no longer necessary. In combination with our database, the Risebase, we also have seamless dependency tracking, which greatly simplifies the overview and work of the artists. Of course, these Principle scenes are not only available for lighting, but also for all other CG departments such as assembly, layout or FX.
DP: With such a large production, everything is often very precisely planned in advance – were you able to get creative with Fallout, and if so, with what?
Andreas Giesen: On the client side, we worked directly with the VFX supervisor Jay Worth and his team and the collaboration was very pleasant and creative. Of course, there were certain specifications and visions from him and also Jonathan Nolan, the producer and director of the series, as I mentioned before, we initially received concept images for LA City, for example.
All in all, however, we had a lot of creative freedom when it came to the design of the destroyed environments, such as the bombed-out city of Shady Sands or the surroundings of the Griffith Observatory in the last episode. Of course, it was important for it to fit in perfectly with the tone and look of the series, but there were rarely exact specifications as to where which building should be located or anything like that.
The thrusters/nozzles on the power armour were also interesting. The basic model didn’t have these at all and we then made various animated designs with some ideas of how these thrusters could look and also how the mechanism works in detail when unfolding. The most important thing was that it shouldn’t look like Iron Man!
Joshua Koilpillai our asset lead and the other artists did a really great job, so Jay and Jonah quickly decided on a design, which we then refined a little. I think this also allowed us to build trust quickly, so that we were able to get creative again and again, which also made this project so exciting for me and the whole team. Our producer Annelie Dangel and her production and coordination team kept the backs of all the artists perfectly free so that everyone could concentrate fully on the creative work.
DP: Let’s talk about the Power Armors: As one of the most recognisable images in the Fallout series, it must have been a pleasure for the modelling/team?
Andreas Giesen: Definitely! This project was highly coveted at RISE, as we have a really big Fallout fanbase. Fortunately, the project ran for a long time and the tasks were so varied that many artists at RISE were able to work on it.
We received the basic model of the power armour from another vendor and then refined it with a number of details, as it was clearly visible in some of our shots, for example when Max crushes the bandit’s head with the hand of the power armour. A few scratches had to be added, especially in the texturing. We also modelled and textured the entire interior of the armour based on the lidar scan, as we had some shots in which it opens up completely.
Interestingly, they actually had a stuntman on set with a jetpack on his arms, who then flew through the air. A bit unconventional, but we were able to use the movement very well as a reference for the animation and also some of the dust and leaves that were whirled up are still real in the final shots. The integration of the CG armour made this approach a little more complex, but I would say the advantages outweighed the disadvantages.
For the shots in which one or more CG power armours are running, we did a motion capture session with our Xsense suits, which took a few attempts. Apparently it’s not so easy to walk as if you were wearing bulky and heavy armour..
DP: One of your shots was again a sunken city with dust, craters, in one case the sea and all kinds of broken buildings – were you able to use your experiences from Reminiscence here?
Andreas Giesen: Yes, we were certainly able to build on that experience, but with Fallout we tried to rely even more heavily on procedural workflows in Houdini, as the number of different environments was significantly larger.
We also made greater use of heightfields in order to have a realistic basis. In Reminiscence there was a lot of flooding, in contrast to Fallout where the wasteland, i.e. the ground, is an important element. The heightfield tools in Houdini are very flexible, so we were able to create landscapes hundreds of kilometres in size with entire mountains, as well as extreme detail on a small scale for close-up shots.
For the New Vegas establishing shot at the end of the series, we had to split the heightfield into about 10 different chunks to get enough detail for the front area. As it was clear for this shot that there would only be this one camera position, we resolved the heightfield chunks in the foreground significantly higher than those in the background. The mountains were then separate heightfields that we could move flexibly.
We proceeded in a similar way with the establishing shot of the Griffith Observatory during the day. As we were at a real location in LA in this shot, we first extracted the height data from map services and then projected this information into a heightfield. We then scattered procedurally destroyed buildings in Houdini, interspersed with LA Hero buildings that we had also built for the non-destroyed version of LA. Since they were destroyed in the first episode in elaborate simulations, we were able to use practically the final state of these simulations for the post-bombing shots.
In addition to the buildings, countless other details such as piles of rubble, destroyed cars and remnants of vegetation were distributed. This, combined with another heightfield layer simulating the sand piled up around these details, resulted in a realistic landscape. Even though this shot was based on a drone plate, you can see in the before and after comparison that not much of the plate remains, except for the sky. Nevertheless, it was incredibly helpful, especially for compositing, to have the plate as a reference, particularly in terms of the depth of the haze.
DP: The Vertis “feel” very close to the game – but the series stands between the games and the physical maquettes/toys/models that inevitably appear – have you already prepared anything here?
Andreas Giesen: We’ve actually given our detailed 3D model of the Vertibird, including all the textures, to a British company that specialises in such replicas. If you look at The Wand Company’s website, you can even see the first pictures.
A 35cm Vertibird made of metal with movable wings and rotating rotors. I have an idea of what the people in our asset team will want for Christmas..
DP: Will it be possible to throw characters into the rotor?
Andreas Giesen: That “accessory” probably won’t be there unfortunately, but I think that shot in the last episode where someone just gets shredded by the rotor is one of those details that makes Fallout so special.
Our FX artists were really able to let their hair down with this series, even apart from the large-scale effects like the atomic bomb explosion. Whether it was the dust clouds from the Vertibird, crashing power armour, the head smash or the spectacular attack in the last episode, there was never a dull moment..
DP: Looking at the whole project, what would you do differently if you were starting from scratch?
Andreas Giesen: You learn with every project, of course, and especially when it comes to creating landscapes of this scale, we were still able to optimise our workflows quite well during Fallout.
If I think of a specific shot, it’s probably the headsmash shot in the third episode. We changed our approach several times here, starting with a fairly simple solution that turned out to be inadequate. I think both the client and we were hoping that we would get by with it. But if we had started directly with the more complex approach that we ended up with, we would certainly have saved a lot of time. But as the saying goes: you’re always smarter afterwards. And for the next project.
DP: And what was your favourite shot?
Andreas Giesen: Not an easy question! Perhaps we can agree that I can choose a favourite shot for each category? My favourite environment shot is the Shady Sands crater establishing shot in episode 5. I really love the reveal moment and the detail that went into that shot. Our layout team did a great job here, and the compositing team really nailed the integration of the CG layers with all the lens effects.
In terms of our vehicles, it’s the shot in the second episode where we see the Vertibird take off from the Brotherhood base for the first time and make a turn. Here you can see the Vertibird for the first time with all its details, almost like a turntable.
My favourite FX shot is in the last episode, the Vertibirds fly the attack on the observatory, we are with the camera inside with Max and through the window we see how the Vertibird in front of us is hit by a missile and explodes. Every department, from animation to FX, lighting and compositing, really went all out for this shot. The fact that the explosion and the Vertibird don’t stay in focus the whole time makes this shot feel particularly realistic, I think. Of course, we could only influence this to a limited extent, but well-shot source plates are naturally motivating.
We also had a hard time choosing the showreel shots, the whole team did an incredible job, which I’m very proud of!
DP: And what are you working on now?
Andreas Giesen: As far as new projects are concerned, I can’t reveal much at the moment, but the next few months will definitely be exciting and of course we always use the time to improve our pipeline and try out new techniques, such as the use of Gaussian splats at the moment. The switch to Houdini 20.5 will also take place soon and we will continue to work on XPU/GPU rendering.
