A wild, prehistoric landscape – herds of dinosaurs roam the frosty plains, or sail through the snowstorm over the mountain ranges. In the undergrowth stirs a completely different, small group – primitive humans, but in this world they are among the hunted. They lie in wait for a creature that haunts their tribe, hoping to defeat it once and for all. But nature has other plans: they are surprised by a gigantic T-Rex and are forced to fight. However, they are powerless against the monster; their primitive weapons have no effect. In the end, only one of them remains, who bravely confronts the giant with his daggers..
This is how the short pilot film “We Hunt Giants” ends in terms of content; but the production began much more harmlessly.

The beginnings

Although Titus took the lead on the script, it was important to him to involve me from the first version of the book. On the one hand, I was able to think about the quantity and realisation of the VFX shots at an early stage. At the same time, I was able to give him critical feedback on the contrast with my own perspective on film and offer further creative suggestions from my side. One of these suggestions was, for example, to introduce the film with a vignette of different dinosaurs for a better worldbuilding – even if this would mean creating several additional dinosaurs in CG, which would be counterproductive.

While Titus was making revisions from his side, I began to create the first storyboard sketches and to build and cut a 3D pre-visualisation based on them. This gave me creative freedom to design the resolution & composition, as well as giving Titus the option to plan the shoot in a more focussed way. As the 3D scenes & cameras were scaled to real-life specifications, this allowed us to plan the technical side of the work on set in advance and identify any potential obstacles that could cause problems for the VFX work. This was particularly helpful when talking to cameraman Marcus Möller, as he not only got a better idea for the shoot on set, but was also able to contribute his own expertise. With a finalised 3D previs, shooting schedule and a well-rehearsed core team, nothing should now stand in the way of production.

The shoot

A few days before the shoot in February 2022, cameraman Marcus unexpectedly showed severe Covid symptoms and tested positive. However, giving up was not on the cards, as the equipment and transport had already been booked, and fortunately Andrés Rignell was found at the last minute to stand in for him. He had also already gained experience with VFX as a professional on various shoots and was very familiar with the camera system (RED Gemini). So we found ourselves for a weekend in Tyresta National Park south of Stockholm, and perfect for the first morning we were presented with an untouched winter wonderland, as it had snowed freshly overnight. So deep that even some of the rental cars got stuck in the snow on the last few metres to the film location.
To gain time, Andrés & I went off separately and started shooting various VFX plates while the rest of the crew freed the cars and prepared the actors. This allowed us to find a rhythm on the fly to gather references & camera data alongside the main takes. This was also important as we had to work with less daylight due to the winter season in the north, so some scenes were shot in a rush.

As soon as director Titus & I were happy with the takes of one shot, the actors were prepared for the next shot or recorded sound. Meanwhile, we were shooting quick cleanplates, noting camera specs, taking shots of lens grids, chrome & greyballs and a toy dinosaur for reference. This initially attracted some strange looks, but after a few shots there were always a few interested people around the camera monitor who wanted to catch a first “preview” of the prehistoric star. In every free minute or general break in filming, I also had to take separate photos of all possible references – actors, props, the surroundings & set design (a mini attrape for the dino trap) as well as additional Chromeball photos at different times of day so that an appropriate selection would be available depending on the shot.

Another result of the tight schedule was that almost all the shots were shot on location without any additional equipment – a blue screen was only used for a few specific shots where the VFX work would otherwise have been enormously complicated. After two intensive days of filming in the snow, I flew back to London to start preparing the VFX, while the material went into editing for the next few weeks.

Plate Prep

Finally, all the footage from the shoot was sent to London in the post as RED raw files, along with the first rough cut. Nuke Indie allowed the EDL of the cut to be imported directly and the individual shots to be exported in separate Nuke scripts. For further processing, the plates were reduced from 5K to 2.5k and saved locally as OpenEXR sequences. As the editing was still in progress in Sweden, the sequences were exported with extra leeway to be on the safe side and all tracked in 3D in Nuke Indie for the matchmoving. Thanks to the quality of the shots and the rich texture of the environment, an automatic 3D track usually produced a high-quality result, and it was rarely necessary to intervene with manual tracks or further manual work.

Based on the 3D track, 3D meshes of the environment could also be generated in Nuke at the same time using point clouds. Fortunately, despite initial fears, only a few shots had to be retouched to remove isolated references to the park such as fences or signs. However, the small amount of bluescreens would mean that every interaction with VFX would have to be rotoscoped by hand – some of the shots would end up spending several days or weeks exclusively in rotoscoping. Luckily, a friend and former colleague, Dominik Platen, stepped in towards the end of post-production and took care of the rotoscoping for some shots.

Dinosaur assets

Before filming began, thorough internet research was carried out for each dinosaur. Drawings & photos of skeletons, artistic reconstructions & screenshots from films, as well as material from related species were collected to provide further references. For the main dinosaur in particular, we initially considered a range of large theropods, but Titus was keen to use a T-Rex as a homage to Jurassic Park. In the end, we allowed ourselves a little variation – the bony dorsal crest was inspired by another theropod, the Acrocanthosaurus.
All 3D models were first roughly pre-built in 3ds Max, and then detailed and textured in Mudbox. A method that had proved successful in previous work was again used for the texturing:
For each dinosaur, a small library of base textures, masks and further grunge maps were painted onto the surface, which were then combined together with normal & displacement maps in Nuke via compositing to create a finished texture.

This process, although hardware-hungry, allowed free experimentation with various options & details, all simultaneously on multiple UDIMs in a non-destructive environment. In addition to the main texture, maps for shader attributes could also be “combed” in this way.
This was particularly important for the T-Rex: the model should give an idea that the dinosaur led a rough existence, in the form of scars and wounds from past confrontations. Other ailments such as frostbite & inflammation should indicate that it is actually not welcome as an invasive species in the cold.

With the texture compositing in Nuke, a suitable balance could be found quite quickly over a few iterations. At least one dinosaur was spared all the work, as the CG asset from “Tucki”, a past project, could be directly reused for a small guest appearance. You can read all about Tucki in DP 21:02, or download it for free here is.gd/tucki_pdf.

Rigging & Animation

For the rigging we went back to 3ds Max. As a tried and tested module, each dinosaur was given a special rig in the Character Animation Toolkit, CAT for short, which could be saved as presets for the subsequent animation process.
To save time during rigging and animation, shortcuts were built into the rig for each asset. For example, the flight membranes of the pterosaurs were animated using a cloth simulation, and the tails of all dinosaurs were procedurally controlled with spring constraints to create a flowing organic movement without further manual intervention.

In turn, the T-Rex received an additional upgrade due to its presence in the film: a simplified skeleton and part of the musculature were modelled for the asset, which in turn formed the basis for a tissue simulation in Houdini using the newer Vellum-based Tissue System. The result was subtle, but provided better deformation and volume on the final animated asset. One advantage of working via CAT rig presets in 3ds Max was that the animations could be saved as separate clip files and transferred to identical rigs later. In this way, the animation scenes were kept very performant without having to load the entire asset. Possible revisions could also be quickly transferred to the corresponding main 3D scene later using a new clip.

For the animation, which was done exclusively with keyframes, it was important that all dinosaurs should reflect natural behaviour. In addition, the T-Rex should appear more aggressive than usual, which may also be due to the injuries. Smaller tics such as snorting, snarling and more impulsive movements helped with this. The animation would also have a strong influence on the editing; scenes were often extended to give the dinosaurs more time to perform.

For one particular scene, which features a herd of 20 dinosaurs, a crowd system was not used. Instead, I spent a weekend animating various actions with excess length – eating, head shaking, sleeping, looking out, running, etc. – which were then placed by hand using animation caches and staggered in time to get the appropriate variation. For a quick turnaround, it was advantageous that the individual animations did not have to be too detailed, as they would all blur together in the mass.

FX simulations

A critical point in the project was the FX simulations, as all the dinosaurs had to interact with the cold snow world. A few systems could be prepared directly in 3ds Max & Particle Flow, but Houdini was used for the rest.

For the most part, snow tracks were created procedurally using geometry; a few shots were even given a dynamic simulation with Houdini Grains, insofar as the interaction had to be represented much more subtly. An important component was also frozen breath for the T-Rex and finely swirled snow dust, which were simulated with PyroFX in Houdini. All Houdini project files were created in such a way that, depending on the shot, only the
correct assets for dinosaurs & environment had to be loaded & exchanged, with a little adjustment afterwards.

Otherwise, separate simulations were created for one or the other shot in the respective tool where it was best suited – such as a flock of birds in Particle Flow (3ds Max) or water splashes with fluids in Houdini. For simpler particle effects such as atmosphere, snowflakes, sparks and more, we even used Nuke Particles, which were incorporated as part of the compositing process.

In contrast, the most prominent FX interaction – where the T-Rex violently squeezes through two trees in one moment – was primarily animated by hand. Springs were again used in the rigging so that the bending of the branches did not have to be simulated. A secondary layer of different FX sims from 3ds Max & Houdini combined gave the moment the finishing touches.

Rendering & compositing

All elements were lit & rendered in 3ds Max with V-Ray; caches from Houdini were exported and imported as either VDBs or Alembics. All dinosaurs & snow elements were rendered directly with surface scattering, whereby the intensity of the scattering was slightly exaggerated in order to benefit from the additional details in the shading.
The lighting rig was kept quite simple to limit the amount of render passes – after all, all shots had to be rendered on a single desktop PC. For each CG element there was an average of 3-4 setups, one for each light and a separate data pass with geometric or vector-based render layers such as Zdepth, Velocity & Position.
As each element in this multi-pass setup was rendered separately, this resulted in countless renders, but only in rare cases did a pass exceed a render time of a few minutes per frame. The immense advantage of this setup was the compositing; the lighting & shading could be manually adjusted for each individual CG element without the need for a second render
a second rendering process was required. A comp template in Nuke, which had been refined over the years for V-Ray passes, helped with this, and changes could later be copied directly from one shot to another. Additional details such as dirt, blood and snow were often projected onto the surface of the dinosaurs using the geometric render layers, either as procedural setups or with prepared textures.
Thanks to the references & cleanplates on set, the integration process for the dinosaurs was quite effortless and the compositing template proved to be very robust.
Some shots also required a few more invisible compositing tricks: for example, in the final shot of the film, one of the hunters faces the T-Rex, but it was not possible on set to make the fur jacket flap in the wind as it roared. So a CG plane was generated in 3ds Max via Cloth-Sim and exported to Nuke, where it served as the basis for a 2D projection of the rotoscoped fur. Additional distortion filters created the impression of turbulent fur.

Environments

Environments also played a major role in the VFX. To keep the shooting schedule efficient, all shots were filmed in the same location – for example, during the first half of the film, the actors stared at the same hill in the distance where they would later be chased down by the T-Rex.
So with the help of digital matte painting, the hill had to be transformed into a flat plane, which was primarily done with alternative plates & more photographs. In addition, a somewhat more threatening sky and more atmospheric fog from the snowfall were added. The forest in the background, on the other hand, could be partially retained and was merely expanded; plates from the shoot with shaken trees helped to give the impression that a larger animal was in the background
to give the impression of a larger animal stalking around in the distance.
The trap was also collaged together from photographs of the smaller set on set, and a CG pole with a chopped off leg from one of the existing dinosaur assets added the finishing touches. In view of the fact that the DMP would be visible over an entire sequence, it was designed from the outset as a 3D projection in Nuke with simple geometry. Depending on the shot, the Matchmove camera or the DMP could be easily moved into position.

The opening shot of the film, in which a couple of pterosaurs fly through a snowstorm, was based on a drone shot of a mountain range at sunset, and followed a group of sled dogs. A new stormy sky was added, the dogs and other anachronistic objects were painted out, and Nuke Particles were used to generate a completely new atmosphere for the pterosaurs to freely interact with.

Digital double, weapons & wounds

During the confrontation, one of the humans, Hunter (played by Urban Bergsten), is scooped up by the T-Rex and thrown around. It was clear from the first draft of the script that a digital double would be needed for this moment, and during the shoot there was an extra photoshoot with the actor Urban in costume. The corresponding asset was built quite simply, as it would only be visible for a few seconds with a lot of blurring.

A male figure from the open source programme ‘MakeHuman’ served as the basis, with the model being slightly adapted to Urban in advance. The photos from the shoot were projected directly onto the models of the body and clothing; to suggest more details, while the model could remain quite simple. Instead, cloth simulations & a simple hair groom helped to make the character look more organic, animated with keyframes.
All the weapon props were also photographed on set and recreated quite simply on the base. With a little secondary animation, it was possible to create the impression that the weapons were dangling from the body and could be detached if necessary.
The digital props were also used in other shots. In one case, a digital bow was placed in the hand of one of the characters to cover up a mistake in the editing; another time, a spear throw had to be digitally replaced so that it could interact with the dinosaur.
Plates were originally shot for another shot in which Urban is grabbed directly, but because the moment was redesigned in post-production, the digidouble was also used here. With the help of a new camera, an environment assembled from various cleanplates and a few animation tests, the desired shot was found and was now realised entirely in CG. After the high-altitude ride, Hunter/Urban lands violently in the snow, and the (graphic) consequences are clearly recognisable – his body is torn open from the bite & the fall, and he is missing an eye. Despite a bit of practical facial make-up on set, the result wasn’t scary enough, so a bit of a rush job had to be done towards the end. After an initial (stomach-churning) online search for references, digital DMP patches or projections for wounds, blood spatter and torn clothing were created directly in Nuke for each individual shot. Depending on the situation, individual parts of the body were tracked in 3D or analysed using Smartvector in order to project the elements. Small variations were incorporated into the track for each level, giving the impression, for example, that wounds/skin and clothing moved independently of each other as the person squirmed.

The hunt continues.

In mid-May 2023, after around a whole year of work, all the VFX shots were finally finalised and delivered. The film went into colour correction & final mixing for another month, followed by the premiere in early June in the heart of Stockholm.
As the short film was also conceived as a pilot/pitch, they are already striving for higher things – while the film is enjoying initial success at festivals, the makers are already working on the first script version for a feature-length film. It is still far too early to say what the outcome will be. But we can only hope that the journey into this primeval world is not yet over, despite its dangers.

The post We Hunt Giants first appeared on DIGITAL PRODUCTION and was written by Andreas Feix.

“Sleep Tight” is a homage to the horror films of days gone by, but at the same time it also breaks with convention. Our aim was to capture this dichotomy visually. After a few brainstorming sessions and mood research, the setting was found: The world was to be dark and foggy, the images diffuse and rich in contrast. Our professor Michael Coldewey gave us a lot of freedom to come up with ideas, and so the first very ambitious script was soon created. That’s how we came to want a creature as the main character: a werewolf.

by Malte Pell, Tobias Sodeikat and Jonas Potthoff

With this basic idea, further meetings took place in which the story began to take shape: a short, menacing sequence – a werewolf breaking into a house – told full of suspense and with a twist at the end that bypasses the usual tropes of the genre. Here, for once, no one is eaten; the werewolf creature is itself an inhabitant of the house it appears to be breaking into.
But above all, the werewolf becomes a werewolf in the course of this process. This variation is not common in film history, and it was precisely this break with the traditional that brought us a huge step forward in the conception. Based on this premise, the rest of the setting was also modernised under the supervision of Prof. Jürgen Schopper; the originally planned, run-down forest hut was replaced by a modern flat that now stands on the outskirts of the big city. The first storyboard and character drafts were created in November 2020, which were later incorporated into a simple animatic. This rough preliminary version gave us an impression of the most important story beats and, above all, gave us an idea of the timing and pacing, so that we were able to further develop the rhythm and camera angles under the supervision of our artistic collaborator Rodolfo Anes Silveira.
The conception phase lasted a total of 3 months, after which the entire project was set up on the project management platform ShotGrid (formerly Shotgun). A particular challenge in times of coronavirus was that we were often only able to meet online. Nevertheless, we were supported by weekly Zoom meetings with professors and artistic staff from the HFF, as well as experts from the industry – from topics such as concept art and storyboarding to rigging, modelling, texturing, animation, simulation and compositing.

Horror on velvet paws – the creature design

The creature design in particular posed various challenges that had to be taken into account right from the start: In terms of content, we had the requirement that the werewolf could not be too big and massive, after all she still had to fit through the doors of the house. At the same time, she had to move on velvet paws so that her husband wouldn’t wake up in bed when she crept through the building. Nevertheless, she still had to look scary and command respect. Various references from nature were therefore incorporated into the design process: The teeth are arranged in several rows, similar to a shark. We modelled the fur comb on the back on the hump and fur of a hyena.

An additional challenge was that the creature had to walk on both four and two legs. It only reaches its maximum size and dominance at the very end, when it stands up and steps into the moonlight. Before that, it only appears in reflections, in details or as a shadow, so it is always veiled. Only in this shot do we see it clearly and distinctly in front of us.

To ensure that this straightening works, we carried out a number of tests with a motion capture suit from Xsens. We realised early on that the three of us would never be able to animate the entire character by hand. So we took inspiration from films like “Planet of the Apes” and built our own forearm extension from sawn-off crutches. These were adapted precisely to the proportions of the werewolf model so that the sensors of the suit were located where the werewolf’s wrists had to be. This enabled more precise retargeting of the movements of the (real) motion capture performer to the rig of the (digital) werewolf. For her skeleton, the lower body of a dog was fused with the upper body of a human; the creature’s outer skin was modelled around it.
In addition, the extensions of our actress’s arms helped her to feel her way into the role of a 2 metre tall, four-legged monster. We were able to recruit the actress and dancer Kathrin Knöpfle for this physically demanding task.

Digital film set – motion capturing

After this intensive preparation phase, the time had finally come. The model of the werewolf was optimised for capturing, the environments were prepared so that we could create and measure a floor plan of each digital set. Then the four-day shoot began in the HFF film studio. The sets were recreated using movable walls. If there were to be interactions between the werewolf and the environment, they were supplemented with props such as moving doors, branches or a mattress, representing the bed in the bedroom. Various versions were recorded for each shot, which were analysed on set and fed into the existing scenes in Maya via retargeting. This allowed us to view all takes in the form of grayshade playblasts after each day of shooting and decide whether shots needed to be repeated if necessary. Almost like a classic film set! To recreate the mood of the scene as well as possible on set and thus support Kathrin’s performance, we also used a bright light source from the direction of the (digital) moon. At the same time, all motion capture takes were also recorded with a real camera in order to obtain as many movement references as possible.

Bringing the dead to life – the animation

A more detailed, three-dimensional animatic, the first real rough cut so to speak, was cut together from the playblasts of the motion capture shoot. This allowed us to reassess the narrative necessity of each shot, after which we significantly shortened the entire film. Now we could start with the detailed work. In almost every shot, errors that had occurred during motion capturing or retargeting had to be corrected by hand animation. The hands, ears and the blendshapes on the werewolf’s face also had to be animated by hand and any connection errors between the shots had to be corrected.
The two words that give the film its title, “Sleep Tight”, also posed a challenge, as they are spoken by the reverted woman at the end of the film. She turns directly into the camera and speaks to us. Her eyes glow discreetly in the darkness. In order to avoid the Uncanny Valley as much as possible, we had initially thought of a more distant shot. However, for the emotional impact of the final scene, we realised that we couldn’t avoid including a close-up of the woman. This meant that the lip synchronisation and facial animation had to be as detailed and believable as possible. To do this, we not only recorded the audio of the speaker in the recording studio, but also filmed her face at the same time to obtain reference material. This helped immensely with the facial animation. It added nuances to her performance that we would probably never have realised when animating by hand without a reference. We tried to recreate her acting as well as possible digitally. Our lecturer, Prof Melanie Beisswenger, was a great support in this, regularly assisting us online with all questions relating to the animation.
Furthermore, the animation of the camera was fine-tuned afterwards to ensure that the camera movements meshed as well as possible. At the same time, our concept from the outset was to limit ourselves to camera movements that could actually be realised and not to incorporate any illogical or exaggerated movements. We also wanted to do justice to our role models, the early horror films.

Cosy blankets and lumpy fur – the simulation

Once the animation was complete, we moved on from Maya to Houdini, because another major challenge followed: the simulation. Our aim was to make the look as realistic as possible. So with a creature full of fur, we had no choice but to delve into the topic of grooming. Various attempts and several hours and Gbytes of simulation cache finally led to a result. In the final version, the setup consists of three different grooms for different parts of the creature’s body, each of which is simulated with its own parameters in order to get as close as possible to real fur behaviour.
In addition, the woman lies down in bed with her husband at the end of the film, so an elaborate cloth simulation was necessary for the interaction of the two characters with the blanket. Finding the right stiffness with a natural drape took time. But the specialist Felix Hörlein helped us enormously with all the simulations.

Leathery to shiny – the texturing / shading

Underneath the fur, our werewolf naturally also needed a detailed texture for her skin, nose, teeth, eyes, etc. All the texturing was done in Substance Painter. Here we made sure to give the skin a leathery look and to include lots of details such as abrasions and scars, especially on the face. The shader itself was then created in Houdini, with various masks for individual areas such as the skin, nose, teeth and eyes. As with the modelling, we received a lot of support from asset specialist Dirk Mauche. The fur was deliberately not simulated in scarred areas so that the skin comes through in some places.
The remaining shaders for the environments and objects were also built in Houdini. Here, we paid particular attention to matching the colour scheme to the black and white final image from the outset, so that the greyscale supports the focus on the respective image section wherever possible. There was also a further element to help focus on certain areas of the image: a special feature of the creature is that its eyes constantly emit a menacing, subtle glow.
The glowing eyes in the darkness of the house were an element that we wanted to have in the film from the outset. It can be found in early mood boards, but also in the very first storyboard. And it closes the film as a bracket, because the eyes of the reverted woman in the bed also glow in the dark. For this, we created the shaders with an emission map in Houdini. In this area, the texturing went hand in hand with the lighting.

Telling darkness with light – the lighting

Just like the visual design of the camera and set pieces, our lighting concept was consistent throughout the project: Light that is as hard as possible with clear edges that emphasise certain parts of the image and thus support the suspense of the film at the end. At the same time, we hide the exact appearance of the werewolf until the last moment – and thus (hopefully) also increase the suspense.
To achieve this look, we tried to limit ourselves to as few light sources as possible and orientate them on real-life models. The moonlight in the outdoor shots was mainly created using HDRIs supplemented by hard panel lights. In the interior shots, mainly strongly focussed panel lights were used, which were supplemented with moving gobos in some places. These moving, very natural shadows also helped massively to give the images a more realistic look. The lighting design process was supervised by creative director and CGI artist Kathrin Hawelka and DoP Moritz Rautenberg.
Rendering was done with Redshift from Houdini. Pipeline TD Jonas Kluger set up his own render farm at the HFF, which is operated via Deadline. Despite the computing power, up to 5 days were needed for some of the renderings. The fine-tuning of the images only came afterwards, in compositing.

Digital to analogue – compositing

As each shot from the lighting was rendered as a multilayer EXR with 32 bits, we still had a lot of leeway in compositing to adjust the light across all shots and to further harmonise the atmospheric elements with each other.
For the establishing shot in particular, in which we see the modern house and the werewolf’s leg for the first time, some layers were combined as matte painting in Photoshop and Nuke.
In compositing, we also further developed the aesthetics of the film under the guidance of senior compositor Heike Kluger. In order to simulate older lenses, we created a sharpness drop-off towards the edges of the image and a separate lens distortion for each focal length based on old Angenieux zoom lenses. The corresponding lens grids were shot by fellow students from the HFF camera department when they had a seminar on handling green screen and VFX on set at the same time as our motion capture shoot.
In Nuke, we also added a subtle glow to the highlights and particles, similar to the properties of a BPM filter. This stylistic patination of the otherwise clean, digital images contributes to the impression that the footage was actually shot, and we also wanted to get even closer to the aesthetic of old horror films. In colour grading (colourist: Claudia Fuchs), we then reinforced this impression even further by adding film grain and high contrasts. Here, too, the large headroom of the EXR files paid off and we were able to give the images the perfect finishing touches.

Lost in the sound? – The sound design

The final step was a sound mix in 5.1 under the direction of Gerhard Auer and Rodolfo Anes Silveira. For the sound design, we focussed on atmospheric sounds and music beds that didn’t sound too melodic and intrusive in order to draw the suspense from the sounds of the creature and the house. Only at the big climax, when the werewolf stands up, do weird violins enter and the music comes to the fore, getting louder and louder to make the atmosphere as uncomfortable as possible and the twist afterwards all the more effective. We even recorded the squeaking for this ourselves at home.

On velvet paws into the future – an outlook

Once the sound mix and the edited material had come together in the form of a DCP, an internal team premiere took place in the HFF’s own cinema. Guests included the department heads and supervisors from the industry, with whom 9 months of work on this film came to a provisional conclusion. We are very proud of the result and would like to thank everyone involved for their support. The film will be released in 2022, and perhaps our Werwölfin will then sneak into one or two festivals. Until then, you can watch a making-of at is.gd/hff_sleep_tight.

Team

Cast

Motion Capture Actress: Kathrin Knöpfle
Voices: Lisa Hagleitner, Hendrik Ehlers

Crew

Directors, Script, Editing, Camera, Animation, Simulation, Compositing: Tobias Sodeikat, Malte Pell, Jonas Potthoff
Producer: Luisa Eichler
Project Supervision: Prof. Jürgen Schopper, Prof. Michael Coldewey
Project Consultant: Rodolfo Anes Silveira
VFX Pipeline TD: Jonas Kluger
Line Producer: Ina Mikkat
Assistant to Line Producer: Jenny Freyburger
Team Assistant: Petra Hereth
Colour Grading: Claudia Fuchs
Re-Recording Mixer / Sound Design: Gerhard Auer, Rodolfo Anes Silveira, Stefan Möhl
Postproduction Supervisor: Christoffer Kempel
Scheduling: Beate Bialas, Sabina Kannewischer
Editing Support: Christine Schorr, Yuval Tzafrir
Technical Support: Benedikt Geß, Florian Schneeweiß
Rental HFF Munich: Rainer Christoph, Boris Levine
Studio Management: Peter Gottschall, Andreas Beckert
Conforming: Martin Foerster
Consultants: Dirk Mauche, Kathrin Hawelka, Moritz Rautenberg, Felix Hörlein, Melanie 
Beisswenger, Heike Kluger

Production

University of Television and Film Munich
Supervising Professor: Prof. Jürgen Schopper, Prof. Michael Coldewey
Technical Details
Frames: 4,608; Resolution: 2,048 x 858; Aspect Ratio: 2.39:1; Renderer: Redshift; Compositing: Nuke, After Effects, Photoshop; 
3D software: Houdini, Maya, Blender, Substance; Sound: 5.1
Website hff-muenchen.de/

The post Sleep Tight – A modern horror tale first appeared on DIGITAL PRODUCTION and was written by HFF Hochschule für Fernsehen und Film München.

A confession first: I have been a fan of the books for my whole adult life and I even slogged through all of the extended universe. So when I heard that the dude who made “Arrival” – Denis Villeneuve – was giving the first book a movie treatment, I was quite excited. But with the extended waiting time (the C that shall not be named) and an unhealthy amount of discussion on certain platforms (without any information – welcome to Social Media), the expectations were exceptionally high. So after we saw it in cinema (twice), we jumped on a call with VFX Supervisor Paul Lambert, and Tristan Myles and Brian Connor from DNEG.

Paul Lambert (Bottom Left), VFX Supervisor, had been involved both in Denis Villeneuve’s last project, “Blade Runner 2049”, as well as a few of the biggest VFX-movies of the last two decades including “Tron”, “I,Robot”, “Harry Potter”, “Benjamin Button”, “Tomb Raider” and 30 more. For “Blade Runner 2049” and “First Man” he received Academy Awards.

Brian Connor (Top Left) , Visual Effects Supervisor from DNEG, is no newbie either. His filmography includes everything from “Star Wars” and “Star Trek” to “Transformers” as well as Marvel & DC movies, “Jurassic Park” and the Godzilla-Monsterverse (including the 1998 version by Roland Emmerich).

The third supervsior on the call was DNEG’s Tristan Myles (Bottom right), who (along with Paul Lambert) won the Oscar for Best Achievement in Visual Effects for „First Man“. Besides that, he was a supervisor on “Fantastic Beasts”, “Interstellar” and many more, including favourites like “Hellboy 2”, “Kingdom of Heaven”, “Children of Men” and “The Dark Knight Rises”.

Another note: Between the interview and going to print, it has been announced that the second part has been greenlit and should hopefully be released in 2023. But we didn’t know that, with the film not being released at that point.

DP: When we first got to see “Dune”, I was amazed by the set extensions. How did you bring Arrakis, Caladan and the sets to life?

Paul Lambert: We built a fair amount of them (laughs). Denis Villeneuve and Patrice Vermette, the Production Designer, spent a year prior designing the worlds of “Dune”, the spaceships and the sets.
Usually, what concept art does for VFX is serve as a springboard into different ideas. But on this, Denis was so happy with the concept art that it became a solid reference. We built the sets in Budapest, and the 3D-assets extended that (“Dune” was partially shot in the Origo Studios in Budapest; origostudios.com).

In VFX, there is often some deviation from the concept art with new ideas or things that don’t match exactly. But Denis felt that in previous movies some things had gotten away, and when it goes down the wrong path, it takes a lot of money and energy to drag it back. This time, he was adamant, and – Brian can attest to that – the assets were as close as possible to the concepts. Any changes were marked and got approved. With the ships, we would A/B-test against the designs.

In a way, that helped a lot with the look – we knew what everything would look like. Instead of putting everything against a blue- or greenscreen and then figuring it out, we never had the „We’ll fix it in post“-attitude. The phrase wasn’t even muttered, as far as I can tell.

DP: Could you give us an example for that?

Paul Lambert: The interior of the ornithopters, for one. Traditionally, you’d shoot that against a greenscreen in a studio, on a gimbal. And after shooting, you’d replace everything. But together with Greig Fraser, the Director of Photography, we decided that we would not try to replicate daylight in a studio. Arrakis is this hot desert environment, so everything that would happen outside, we would shoot outside. You can’t replicate the strength of the sun.

DP: But a virtual production environment with LED screens?

Paul Lambert: I had experience with LED screens, and Greig Frasers worked on the first season of “The Mandalorian” (DOP for episodes 1, 3 and 7). So, with that extensive experience with virtual production, we agreed that you can’t get enough light from LEDs to get the desert feel. If the movie had played during sunset, it would not have been a problem, then it would have been perfect. But the noon sun in Arrakis needs the actual sun. Because of that, we didn’t try to light it, but we found the highest hill in Budapest, put our gimbal on top so we could get a nice horizon, and surrounded it with an eight-meter screen, colored like sand. On a hot day, the sun would bounce off the screen and enter the cockpit. Even when we looked at the dailies, with Greigs camerawork, it already felt like you were in the desert.

And then the compositors from DNEG added their magic. We shot hours of footage flying through deserts in the United Arab Emirates, with a six-camera-rig under a helicopter flying through the dunes. With that, the compositors could do a blend of that with the footage. Rather than a full extraction, like the classic „foreground and completely different background“, our foreground already had very similar tones, so we could mix. And honestly, it felt immediately real, and we didn’t have the usual problems with edges and a lack of believability.

For example, the glass dome of the ornithopter had reflections, and reflections of reflections on the inside. Shooting that on a greenscreen would have been problematic. Obviously, there are times when you have to rebuild that, and I think, the more you hit a plate, the lesser the credibility. And with our approach, we could be seamless. Also, we had a lot of time in preproduction to find ideas and to think about what we want in the end. Visual effects goes hand in hand with the on-set experience and demands, if you do it properly. And when you come up with a good basis, the VFX artists have something that can succeed.

We all know: If you have a foreground that isn’t corresponding to the background in terms of lighting, there is not much you can do about it. The more you pull and push and grade it, the less believable it becomes. Yes, you can get the perfect seams, but it still doesn’t look natural. So, with the time we were given in preproduction, we could avoid bluescreen.

An example: When the story moves to Arrakis, whether it is in the desert or the city of Arrakeen, we shot a lot of it against a sand screen, a sand colored background. Which is funny – because if you invert the colors, you get a bluescreen. Obviously, there are issues with skin tones and the like, but it gives you a very good basis. And let’s be honest: At this point, we can come up with a process for extracting parts of the image for any color, as long as it isn’t a complex background.

I was having this conversation with the DOP: We can remove any background or foreground, but if the lighting between the two doesn’t correspond, there isn’t much good we can do. LEDs and virtual production help with this particular challenge, but we already decided not to go down this path at this point.

DP: When did you get involved in the project?

Paul Lambert: Concept work, especially the interiors, were pretty much finalized, except for a few changes. We came in basically when the storyboard phase of the preparations started. We also had to previs a couple of scenes, which is not a thing Denis likes. But for some scenes we needed every department to know what we were about to do. For example, the sandworm attack on the crawler needed extensive preparation.
Also, from day one we knew that one of the most computationally expensive things for “Dune” was going to be the sand. And the sand around the sandworm in particular would be extremely important. When Tristan joined about a month later, one of the questions was: “How the heck are we going to displace all of that sand?”
The key to a good effect is having a visual reference which an artist can use to make informed creative decisions and even copy to. But we naturally couldn’t find a sandworm or something working in this way anywhere. I asked production if we could get at least some explosions in the desert for reference, while we were filming deserts in Jordan and the United Arab Emirates, but I was told that would not go down well in the Middle East.

DP: So how did you do it?

Paul Lambert: Tristan and the guys at DNEG Vancouver went through iteration after iteration. In an ideal world, you would just simulate every grain of sand by itself, but who has the processing power for that? So, you clump things together and hope the render goes through. But with that, sometimes things appear to not have the right scale or speed. So, during preproduction we did the R&D, so in the edit we could deliver the shots quickly. At the same time, Brian had a similar problem with scale: the ships coming out of the water on Caladan, which is a massive structure. Nobody has planes like that.

Brian Connor: Well, thankfully Paul found footage of icebergs tipping over which were about the same size. When they melt, they roll over, and that helped us to understand how such a massive structure behaves and displaces water on that scale. It was one of those shots that you work on pretty much until the end (laughs). You have to give it the love and the time and the disk space it needs for the simulations.

DP: You mean the scene where the Atreides flagship surfaces?

Brian Connor: Yeah, we had a couple of iterations on that. One of the first ones was with a person in a boat next to it, for scale. But we ran the simulations over and over, even though some looked odd. Remember: If it looks odd, it does so because you don’t know what it would actually do, for example, the huge amount of water piling up on the top of the ship.
If you get in there and change too much of what the dynamics of the simulation are telling you, you run into the same problem of over-processing that Paul talked about earlier. We just put all the distributed rendering power of the DNEG farm to use on this. Strategically, of course – when things slowed down, we took all the resources. It takes a lot of time to figure out the iterations. Same thing with the sandstorm. That was also a computationally heavy piece, but we were lucky that we had massive sandstorms from Africa as reference. So that was a bit easier.

DP: Can you talk a bit about your simulation pipeline?

Tristan Myles: We used Houdini and pushed it beyond its limits, I think. We came aboard early and tried to figure out how to show sand behaving at this massive scale – same as the water. In the beginning we put Fremen in the sand for scale, but we had to make scaled down versions for the edit, so it wouldn’t distract from the story. It couldn’t look distracting, like a visual effects scene – it is the environment for the story.

DP: So you manged to keep the scene files reasonable?

Tristan Myles: I can’t remember the exact file sizes, but Vancouver reserved three servers – we were in the petabytes-range. And 60 percent of that were the caches and the geometry. But with the types of things we were doing that was acceptable. The destruction, the heavy explosions, the sand simulation and the worm itself all were beasts to wrangle through the farm.

DP: So, the worm – how did you bring him to the screen?

Tristan Myles: We did model the whole worm including all the plates along the sides of its body. Those are all moveable, and the skin in between had a little bit of ‘give’, so it’s not fully rigid and organic.

Paul Lambert: We wanted something alive but prehistoric. One reference we had for that was elephant skin. Rigid plates over spots and areas with soft membranes in between, folding like an accordion. Obviously not super agile on this scale – the turning circle of a being like that would not be small. And a beast like that affects the whole environment it moves through. Robyn Luckham, the Animation Director, spent a long time figuring out how it moves.

It’s more about the sand displacement when the dunes ripple and rise like water, almost. And when it goes faster, it becomes almost like an explosion as it is traveling towards whatever source. And in keeping that idea of water: The actual worm’s mouth has baleen as you would see in a large whale. Because like a whale sifting water and catching krill, the worm would sift through sand.

The thing is: On that scale, it is a force of nature, affecting the whole environment. When it appears, we show the scale, adding things like camera shaking and rumbling and little explosions when it approaches. But you don’t get a lot of screen time with the worm. This is not “Jaws”.

DP: The bubbling of the sand in the sandcrawler scene was like a whale coming up from below?

Paul Lambert: Yes, everything around them is influenced by the worm. Funnily enough, when they sink into the sand, that was done in camera. Gerd Nefzer, the Special Effects Supervisor, built a vibrating plate, which we buried under the sand. And when you dialled in the vibration just right, the sand looked like bubbling water and you would sink into it, just like you see in the movie. Tristan was able to replicate that on the larger environment.

DP: For the next part of “Dune”, the worm is ready and roaring to go? (At the time of the interview, no information about Part Two was available.)

Paul Lambert: Well, I assume for the bidding procedure for “Dune: Part Two”, having a sandworm on hand will be relevant (laughs). But so far, there hasn’t been any prep for “Dune: Part Two”. If so, I’d love to know!

DP: You mentioned that you had a lot of concept art. How detailed was it?

Paul Lambert: Extremely! But there are always some things which you need to actually see. For example, Denis wasn’t really sure about the shape and texture of the Guild Heighliners (the massive ship that transports other ships for example from Caladan to
Arrakis). And Brian had a lot variations and iterations of its main docking port and the shape of that. When that was final, the texture was also important. These ships are old and have been around for a long time. Still working, but with bumps and scratches accumulated.

Brian Connor: I would love to use those in “Dune: Part Two”. The ships are so detailed, and with the structured insides and their scale, you could do wonderful things with that, interesting camera angles, composition and showing all that in relation to each other. I hope we can show it off!

DP: Another story beat that needed a lot of CGI was the shields. How was that done?

Paul Lambert: That was surprisingly straightforward. And it came from having artists involved in preproduction and on set, so you can do tests and inform the shoot. We had a list from things that we would have to figure out – among those, the shields. For example, would the shields add additional light? If yes, that would mean additional lights on set.
But we came up with a “past and future frames” approach which works really well when there is a lot of movement. Which there usually is in fight scenes. When there wasn’t a lot of it, we had to fake a bit of it.
What was very important was that we didn’t just procedurally grab frames – two from the back, two from the front and be done with it. We needed an artist’s trained eye and actual people who painted the frames out or in to get a look approved very early on. It shouldn’t feel digital, and DNEG has a few artists who are good at that kind of work.
We tested it with fight scenes from other movies, and it worked for everybody. It was just in DI and the edit that we saw that it became confusing – especially the fight between Paul and Gurney with its quick angles and cuts. There the idea of color came in. Blue for the normal state, and red for penetration.
Also, it was a bit of an homage to the first film, where people scratched the shields into the frame. And when we had it down for the fights, we had to recreate it digitally, for the ships – you’ll see it prominently in the attack of Arrakeen.

DP: Which techniques from “Dune” will you be carrying over to the next show?

Tristan Myles: Well, some of the tools we have written to manage the large amounts of data and bring them together at rendertime will be useful in the future. It’s part of visual effects that you always learn without trying to reinvent the wheel, although you generally end up reinventing the wheel anyway (laughs).
On “Dune”, we learned about large-scale effects simulation and what impact that has on the renderfarm, how to mitigate that and different setups to display what the final image is going to look like.
The real trick there is to work in lower resolution, but not making it look like low resolution. We had a setup for that which was called Ultra Res – once the simulation was signed off by Paul and Denis, then it went through the farm and we could wrangle every grain of sand. The backend of it all – it’s boring to write about, but is an essential part of heavier VFX.

DP: When we say „computationally heavy“, how did you plan for that during production?

Brian Connor: Usually, you can break it up a bit – for example only the front portion of the flagship. But the way it was shot – and you see all of it going into the distance – we couldn’t do only the front part of it. We ran the simulation for the whole ship, which added complexity to the background in addition to the stuff in the foreground. We had to strategize. Everything interacts with everything else. It took our supervisors a lot of work just to distribute it everywhere and to give us a way to iterate in a reasonable amount of time. We had that running on the side the whole time, but we knew that going in and planned for it.
What was also quite challenging was that we had different formats. For example, in Imax you see the whole frame, in 2.39:1 you miss some of the top and the bottom.

Paul Lambert: We framed for Imax on the set. We discussed 2.39, because there will always be something missing from the frame. But some shots couldn’t be done that way. So if you see it in Imax, there are a fair amount of shots which are different. One in particular: when Paul is standing in front of the worm, and it fills the frame with worm texture. We had to redesign that shot, because it fills the whole frame, even in Imax, from Paul at the bottom to the towering top of the worms mouth.
And on about 30 shots, we couldn’t go from Imax to 2.39. Usually you animate that visible area, and that’s that. But it didn’t work with the narrative, so we extended the Imax frame to the ratio of 2.39.
Funnily enough, when I saw the finished movie for the first time in Imax, I did not remember it like that. „Did we really shoot it like that?“ So, I encourage everybody to see the film twice – once in Imax and once in the usual theatrical aspect ratio.

Brian Connor: We called it the „mega­frame“ – the resolution is just massive. It’s around 7K, because you’re widening the Imax frame, which already is large. You could just cheat and buffer on the side and not have it in Imax-height, but then the quality and the fidelity would have suffered. We got the 7K frames to DI, so they could shrink them into the format.

DP: Couldn’t you have scaled it up?

Paul Lambert: Not yet. AI enhancements are getting scary good, but not quite good enough for this vital scene. I have been keeping an eye on these technologies, and it could influence every aspect of VFX. It feels almost like we are pre-Newtonian. One example is that, while you are doing onset capture of textures and so on, one could do AI-passes to train the AI on actual footage and help with production, for example capturing actors to do certain things as reference to train a machine to do extractions.
In some ways we are already doing that. On “Dune” I always had a couple of witness cameras on set. You might not always use them, but it is so beneficial to have the data. And in the near future, we can do all kinds of things with that.
Another thing I would highly recommend to everybody: Attach a GoPro to your main camera. When your DOP fuses a shallow depth of field, you basically cannot do background extractions. But with the GoPro’s sensor and lens, you can get the camera movement and reference for the backgrounds.

Brian Connor: Another thing that is coming is the saving of props. On a recent production we scanned a few period cars with a smartphone. The prop was just rented for the day, so we got as much of it as we could, and it worked surprisingly well, even with the reflections. And if it is not going to be up close, but seen from an aerial perspective or to populate the background, that guerrilla style of capturing data and assets can really help.
At DNEG, we have a pipeline for that, and you can get many things so much faster than building it from scratch and with lower hurdles in preparation. There can always be somebody with a phone taking pictures, you don’t need scheduling for that.
Paul Lambert: I did the spinner in “Blade Runner 2049” like that. Since the sensors are so small, everything is always in focus and you get a decent solution for photogrammetry. You have the full range of depth.

DP: If we switch the Direction of scanning: Did you use Lidar scanners on “Dune”?
Paul Lambert: Yes, we had a small one running whenever we were shooting, and sometimes during scenes we captured particular setups. Obviously, scenes move around and props are all over the place. Of course, we had scheduled a proper capture of every room before it got taken down, but we had a special person doing scans and Lidar on the go for anything that we requested and whatever came up. Yes, it produces a massive amount of data, but that is easier to handle than wasting a lot of time in post trying to figure stuff out.

DP: With a movie made from a book: Did you read the novel (or novels) in preparation?

Paul Lambert: I had read the book when I was about 14 and had seen the David Lynch movie first. At that point I was fascinated, but in preparation for “Dune” I was torn whether I should read it again or stick with the script. I knew the story, but I stuck with the script and Denis’ vision of it. I was afraid it might create tension. In hindsight: It wouldn’t have.

Tristan Myles: I read the book at a similar age. My dad got me into it, and I read it again in my twenties. And when the script came, it was closer to the book than the movie Paul mentioned, so I stuck with that.

DP: So, with “Dune: Part One” finished, what sticks with you? Which scenes will you put in your showreel?
Paul Lambert: It’s been a while since I updated my showreel (laughs). But what stuck to memory is that I am really happy with what we achieved and the experience of making the movie. It will be hard to replicate the collaboration, having the guys come out to the set and experiencing this whole world. Sometimes it doesn’t happen this way. You want things to be shot for VFX in a certain way, and you don’t get it. This time, having this collaboration was fundamental to getting the movie onto the screen. And having had this experience, I know what I want for future movies (laughs).

And particular scenes? There are ones which were a challenging shoot for me, like Salusa Secundus, where we see the Sardaukar Legions. That was challenging, because suddenly we had rain and sunlight at the same time. The team did a fantastic job – we were worried that it wouldn’t be believable because we had to adapt to the weather at the morning of the shoot. It was a challenge, but it came out really well.
The one thing I liked about “Dune” was that we had time. Originally, the movie was planned to come out October 2020, and that got extended to December, and then Covid hit. And then we finished January after that and went into DI.
Also, we did additional shooting. Denis felt we needed more connection between Paul and his parents, so we did some additional scenes. And we had a really quick turnaround to put that together. But it worked out well. For example, at the spaceport in Arrakeen, where Duncan lands and comes out and hugs Paul, that was a backlot in Budapest, and we had a sand screen going all the way around the backlot. Brian put in the spaceport.

DP: Doesn’t that make it harder?

Paul Lambert: I’m a huge believer in having a harder composite (laughs) – rather than breaking things down in layers and shooting those to be put together, I prefer to get everything in one go. That way then you have a harder composite. Which meant, that a lot of our scenes on Arrakis had us blowing sand and throwing dust from the ornithopters. You see that in the historical scene with the Fremen fighting the Harkonnen soldiers. We were throwing sand like crazy, and it was just texture and swirls. The guys did an amazing job at replacing the background – again, the idea of not doing full extractions, but to blend. The compositors might say it’s really hard, but the result looks more believable. I’m super proud of that approach and the way the artists brought it to life. The same with the city of Arrakeen: We had a helicopter and flew around Jordan, and basically Denis was like: „I want Arrakeen to be there“, and we did Lidar scans of that whole area and imported them. So even when there are full CG scenes, the environment is real, and that adds a lot of believability.

Brian Connor: The scale of that movie looks really good – the massiveness. When I came to the set, Paul took me on a tour, and standing in all of these massive sets – we pretty much took over the whole studio, and the backlot itself is just gigantic. The sets – especially for the interiors – were amazingly detailed. Walking around in them felt like being completely surrounded by the world. That was a luxury to have – a lot of it is really there. And that was something that I’ll take away from this: We never settled and didn’t go for good enough. We didn’t cut corners, but went straight through – even if that meant a lot of work, even if the servers went down. The Tech department is probably not our friend anymore, but we came through with a result that we could be proud of.

The post Dune first appeared on DIGITAL PRODUCTION and was written by Bela Beier.