Archive notice: This article was originally published on April 17, 2015. Links and embedded videos are preserved as part of the historical record.
For the DP issue 05 : 2012 the VFX supervisors Juri Stanossek and Sven Martin gave us a making-of interview at the Animation Meeting.
Game of Thrones in Munich and Stuttgart
Summer, sun, fantasy. The effects company Pixomondo was recently honoured with the Effects Oscar for its work with Martin Scorsese on “Hugo Cabret”. Another highlight, however, is the work on the US fantasy series “Game of Thrones”, for which a whole host of set extensions, fire-breathing dragons and a nasty shadow creature were created. DP met Pixomondo VFX supervisors Juri Stanossek and Sven Martin at this year’s Animation Meeting, which was organised by MedienCampus Bayern and filmtools Consult. Among the participants in 2012 was Jonathan Weber from Rise Visual Effects Studios, who took the audience behind the scenes of the elaborately staged TV series “Borgia”.
The majority of the event was organised by Pixomondo, firstly with the effects for “Hugo Cabret” (see DP 2 : 2012) – and for the now acclaimed and multi-award-winning TV series “Game of Thrones”. The TV series is based on the novel cycle “The Song of Ice and Fire” by George R. R. Martin and was produced by HBO. No expense was spared on the size of the set during the realisation, which was shot in breathtaking original locations. In addition to creatures, fire and water, Pixomondo naturally also focussed on environments. And time. Producing for a US TV series meant an absolute deadline every fortnight. Special attention was paid to the Shadow Creature and the dragons. For the shadow creature, Pixomondo used the fluid simulation Naiad to get away from the familiar RealFlow look. A lot of time was invested in researching the design of the young dragons. And without further ado, a chicken was purchased from the supermarket. The anatomy course on the real object was helpful for the artists to get a feel for the movement sequences.
DP: What were Pixomondo’s tasks for “Game of Thrones”?
Juri Stanossek: We did all the effects for the second season – including the set extensions and the CGI characters. It was interesting that we had every difficulty you can have with this project. So from full CG characters to water simulation to destructions, everything was included. I think this versatility was the challenge.
DP: In direct comparison, what was the main difference between TV and film for you?
Juri Stanossek: The only difference is that you have an absolute deadline every 14 days for 10 weeks in a row. In the end, we were allowed to work in film quality for HBO – so we also worked with the film tools in the film pipeline and produced in film quality. Sven Martin: I also believe that the quality of the high-end series from HBO or the BBC has increased significantly and is in no way inferior to many film effects. That’s why there was no special “TV pipeline” for “Game of Thrones”. The colleagues in our Pixomondo office in Burbank cannot apply the departmental film structure in all cases, as series such as Hawaii-5-O are also processed there with extremely short deadlines. In comparison, we had almost luxurious delivery times for “Game of Thrones”.
DP: And how did you coordinate the sheer volume of tasks?
Juri Stanossek: Well, based on the amount of sleep we had, it was a 24-hour pipeline. We actually worked with our facilities in China and with our colleagues in London and Burbank. The coordination and most of the final shots came from Germany. Here we mainly utilised the Pixomondo locations in Stuttgart and Frankfurt. Stuttgart took over the coordination and the work was distributed from there. In addition, we mainly did the matte paintings and the shadow creature in Stuttgart. Frankfurt was responsible for the dragons. Sven Martin: It’s actually the case with us that every location can do everything – it’s just that over the course of time, teams with different preferences have come together at the various locations. For example, many artists for environment and matte painting are based in Stuttgart, the FX artists who work with water, fire and smoke are in Berlin, and the character animation team is based in Frankfurt. But all locations can actually do everything.
DP: What was it like working with HBO?
Juri Stanossek: Very good. For the first season, two studios were still responsible for the effects – and it was theoretically a risk for HBO to give all the effects to one studio. But since the VFX producer at HBO had already worked with Pixomondo on Roland Emmerich’s “2012”, there was complete trust right from the start. And in terms of the task alone, it really is something that most VFX people lick their fingers for.
DP: How did the coordination with the makers go?
Juri Stanossek: It went well – the producers were quick with their comments and very enthusiastic. The collaboration worked very well. Our main contact on the production side was VFX supervisor Rainer Gombos. In addition, in contrast to a cinema film, the producer, or rather the producers, are the linchpin of a series. Because with a series, it is much more important that everything is cast from the same mould. And the first shots – especially the very long one with the dragon, which comes in the first episode – went through relatively quickly and well. We then adjusted the pipeline and our workflow a little and then prepared episodes 4, 5 and 10, in which the dragons appear, practically straight away.
DP: And the first big shot of the dragon?
Sven Martin: That was a tough start – it’s a shot with almost 700 frames in total, which is far more than you would normally show in one piece. The shot was also a good start because the producers were so excited that they could show a digital figure so lifelike without constant cuts. The shot was very sophisticated – the actress walks through the landscape with the dragon on her shoulder, feeds it and talks to another person. The matchmove, i.e. tracking the camera and the objects, was also a real challenge for the matchmove team. And when you have such a long period of time, it’s a challenge for the animators to fill this time sensibly. Especially when you have something like a dragon, which is a cross between an animal and an intelligent being.
DP: How did you research such fantasy creatures as a dragon?
Sven Martin: There are very different approaches to dragons. There is the lizard faction, which is four-legged, in the style of “Dragonheart”. This also had a very solid, massive body – basically a lizard with wings. On the other hand, there is the two-legged version, which resembles a raptor (see “Jurassic Park”, editor’s note). In other words, a “light” animal that is almost a bird in terms of its movements and proportions. We decided in favour of the bird variant because a heavy, front-heavy animal cannot credibly fly. For our research, we looked at bird skeletons with their special features and proportions, for example, as well as the real proportions of wings in relation to the body, and also how such animals balance. From this we developed the kites.
DP: And how did you realise the dragon?
Sven Martin: We always work with Maya for the animations – we have a very robust pipeline, because the technology must not fail, especially with tight delivery times. The special thing about our pipeline is that the artists can work in parallel. While the model is still being worked on, the rigger can work, the shaders are already being developed and the animator is already building a first version – and we can flexibly change the individual aspects at any time. This is one of the key points in meeting the tight deadlines, which are also getting tighter and tighter in feature films. Nine-month processing periods are a thing of the past. In most cases, you only have three months or, as with “Super 8” for example, just six weeks until submission. In other words, from the arrival of the shot plates to the finished shot.
DP: Which of the shots do you like best?
Sven Martin: I really like the very first one – the introduction, when the camera pans up to Princess Daenerys, to the dragon sitting on her shoulder, and it turns towards the flesh in her hand, screaming. The dragon is bird-like and totally believable. I generally like anything that has to do with interaction. I think that’s when it starts to come to life – when the dragons and the humans interact with each other, so it’s almost like a split screen.
DP: How did you do the set extensions in Stuttgart?
Juri Stanossek : They are generally 3D environments – more than just two-dimensional matte painting. We have a very strong matte painting team under the direction of Thilo Ewers. The workflow was that we modelled them, mat-painted them and then projected them. We unpacked the “big” 3D tools for very few shots – mostly the expertise and skills of the matte painters were decisive, and the result is a 2.5D matte painting, so to speak. I’m very happy with the environments and it worked wonderfully, because thanks to the concept paintings and the textures that Rainer Gombos photographed on set, we knew from the start exactly what the result should look like. The data and additional information were ideal. Good preparatory work is essential for a TV series with such a tight schedule. The rugged rocks of Pyke or the scenes “Beyond the Wall” – these are also grateful images for set extensions. But what was special about “Game of Thrones” was that it was filmed in Iceland and Ireland and not in front of a green screen in a studio in L.A. I think you can feel that in the images. The landscapes are vivid. Castles and beaches were always shot in the best locations and not where it’s easy to get to. The extension is then digital, but the combination of location and extension creates a mood that is so rare.
DP: Wouldn’t a lidar scan have been easier?
Juri Stanossek: No, not in this case. We didn’t miss it for two reasons: firstly, we have an excellent Matchmove team and they get all the measurements. Secondly, scans of larger areas are overrated in my opinion. It has to be matchmoved anyway, and if the set changes, it doesn’t help in the end. We have often worked with lidar scans and, like any technology, they don’t always make sense everywhere. The effort involved is huge, the resulting data volumes are enormous and if you shrink the data to practicable quantities and convert it into polygons, it only gives you a rough idea of where things need to be. The extent to which this helps or only hinders must be checked in advance.
DP: Another complex character was the Shadow Creature. How did you build it?
Juri Stanossek: The Shadow Creature is really something special. We also got more time from production for this one, because we had to create all the concept art. The character is not described in the books and we had to approach the right look separately for each scene – the character has two appearances. Basically, we worked with emitter objects and Naiad as the basis of the simulation. We used fume elements and hardsurface shading for this, as well as various objects on different layers – but we rendered them all together in order to ultimately show every shadow on every other part.
DP: Was there anything technically that was new territory?
Juri Stanossek: Making a character – the Shadow Creature – out of liquid was something special. Especially the combination of the different layers and passes, the iterations until the design was right, and rendering the whole thing – that was really adventurous. An emitting character setup that is so flexible that the character must and can look different in every shot.Another challenge was that the concept art was not precisely defined in this case. In the book, it’s an undescribed light-eating shadow figure that you can’t grasp. I realised relatively quickly that we had to use a liquid implementation as the basis for this. We had previously simulated animations running backwards forwards and played them backwards again, so that the figure was basically composed of air. That looked quite good, but in the end it wasn’t good enough.
DP: How many of the developments can you use for other productions ?
Juri Stanossek: The knowledge grows with every production and stays with you, that’s the most important development. I think we’ve learnt a lot from dealing with the sheer number of shots. Sven Martin: Several new functions have been added to the character pipeline. We learn something new with every project. For “Game of Thrones”, for example, we tested which of the so-called secondary animations we have to animate by hand and which movements the computer can do for us. This included, for example, muscle deformations, the wing wings, but also the ground contact of the tail and wings. Even the chain simulations, which actually sound simple, were not so trivial. Because when three chains have to collide with each other, with the table, the kites and the simulated wing fingers and skins, it’s a tricky task even for the FX department.
DP: Where was the point in the creative process where the look was finalised ?
Juri Stanossek: Well, I would say the creative process was consistently exhausting until the final look was finalised – the development goes towards the finished material, not to the point where you are sure about the look.
DP: Did you also read the books or did you just rely on the concept art?
Sven Martin: We deliberately ignored the books. That’s usually the case, because the producers or the director naturally want to make something of their own from the book. Films are never filmed books. That’s why foreign ideas are very often taken up. It’s similar with actors. The “Terminator” actress Summer Glau in the “Sarah Connor Chronicles” has never seen the original films and didn’t even know what the character had done before in order to see it independently. Juri Stannosek: We really don’t know in part because the books aren’t all available yet. We’ll have to wait and see what else comes our way.
DP: And the scenes in Blackwater Bay and Pyke? How did you do the water?
Juri Stanossek: We put the entire fleet in full CG water – with water displacement. We now have close-up quality. You always have to bear one thing in mind: a sea surface or a stormy bay is different to a glass of water. And nowadays every studio is confronted with digital water at some point – and then you have to have solutions ready. At Pixomondo, we put a lot into development right from the start – people who test different solutions, who develop the corresponding shaders, the renderers and everything else we needed. However, this development runs parallel to the projects – and has been doing so for several years because the requirements are becoming ever more stringent. And then there is always the step of linking the whole thing with a standalone tool such as Naiad, and linking this in turn with our own software and the respective pipeline.
DP: Can you imagine – hypothetically speaking – making the software available to others?
Sven Martin: It’s important to remember that if you not only use software yourself, but also pass it on to others, support becomes necessary. This is actually a company within a company. I don’t think that’s what Pixomondo wants at the moment. We would rather do the effects for lots of great projects.
DP: What have you been working on since then?
Sven Martin: The last project – which I can talk about – was “Spider-Man”. Among other things, we did the facial transformations of the villain, who turns from a human into a “giant lizard”. That really kept us busy, transferring real facial expressions to a completely different geometry using matchmove. Perhaps the author will write something similar for “Game of Thrones” in the coming volumes, then we could also use the techniques developed for Westeros.
You can still submit your own entries for the animago AWARD until 30 June, all further information on the animago AWARD & CONFERENCE can be found here: www.animago.com
