The challenge for the visual effects design was to achieve a photorealistic visual aesthetic that blends seamlessly into a real film with real actors. A Portuguese courtyard, built in the HFF studio in the summer of 2022, served as the central set for the dystopian narrative of a family drama.
by Franziska Bayer, Valentin Dittlmann, Alexander Hupp, Ines Timmich and Hannes Werner
The plot unfolds on an isolated island in the middle of an endless ocean and thematises the conflict between the spirit of discovery of youth and the entrenched convictions of the older generation. Although officially listed as a “VFX film 02” by the visual effects students, “Wind Up” is a joint project. For example, screenplay student Tamaki Richter wrote the script, while the production was handled by WennDann Film GmbH, which was founded at the HFF, and students from all departments took on additional roles – wenndann-film.de.
The professional expertise of the film industry complemented the work of the HFF students, with Matthias Zentner(velvet.de) acting as director and Moritz Rautenberg(moritzrautenberg.com) as director of photography. The synergy between various educational institutions in Munich should also be noted, with contributions from the Academy of Music, the Academy of Fine Arts for the set design and the August Everding Drama School, which used some of its students for the make-up design. The studio building, which provided the basic setting, was also created as part of the “Entwerfen und Gestalten – Architectural Design and Conception” programme at the Technical University of Munich. The cooperation within the departments of the HFF and with other art academies not only marks an impressive short film, but also emphasises the power of interdisciplinary exchange. “Wind Up” stands as a testament to the creative fusion of talents from different disciplines at the HFF and its partner institutions in Munich.
See for yourself:
- Munich Film and Television University hff-muenchen.de
- VFX Breakdown is.gd/vimeo_windup_vfx
- Making of is.gd/wind_up_making_of
Brainstorming and script development
What is hidden behind the boundaries of a studio courtyard? A lonely family on an island, in the sea, surrounded by monsters. That was the basic idea submitted by a screenwriting student that would eventually become “Wind Up”. The process of developing the idea was a creative collaboration between several departments at the HFF, which included intensive discussions about the relationship dynamics of the characters, the special significance of a lamp and how a hot air balloon works.
The core, the family drama, remained an important guideline throughout the script versions. The result is a story of a family that grows beyond its limits in the fight against its shadows. Lino (16) spends his everyday life building up the crumbling walls of his home island to protect himself and his family from the darkness that lurks outside the island and swallowed up his mother 10 years ago. Until he learns that his sister Benedita (18) and his uncle Afonso (52) want to flee the island to find his mother: She’s alive? And the darkness is just a fairy tale told to him by his grandmother Madalena (83) to keep the family on the island. Lino now has to find his way between mysterious lamps, home-made hot air balloons and family lies and make a decision: Will he stay on a crumbling island with his family? Or will he set off in search of his mother – into a world of shadows?
Animatic & Resolution
When planning the VFX shots, it was important to break down the entire film into individual shots in advance. That’s why the VFX team, together with director Matthias Zentner and cinematographer Moritz Rautenberg, spent several weeks developing shot plans for the film. The set plans were used to create animatics that could be quickly adapted. In contrast to traditional planning with storyboards, it was an advantage to work directly on the previs, as the space of the set could be taken into account. The characters were generated in readyplayer.me, placed in a scanned 3D model of the set and animated in Blender. Different focal lengths and camera movements were created in this way, with many of the non-VFX shots later being made more dynamic or combined as one shot during the shoot. As the dialogue was also set to music in the animatics, screenwriter Tamaki Richter was able to quickly determine whether her story was working the way she wanted and make changes accordingly.
Concept Art
The realisation of the short film “Wind Up” required a well thought-out, consistent visual look that went beyond the set of the Portuguese courtyard to include the entire island. The layout of the courtyard was the starting point for the design of the exterior buildings and the island. An extensive search for references of remote places, coasts and islands laid the foundation for the development of concept art for the location. The aim was to create a doomed island, with buildings slowly decaying and being maintained by the last four inhabitants in an endless battle against decay. The result is a gloomy look of an island that seems to be drowning in fog, illuminated only by a single light source. The VFX students were supported in designing the concepts by concept artist Luis Guggenberger(luisguggenberger.de).
Another crucial aspect was the design of the “shadows”, which embody the fears of 15-year-old Linos in the film. Dark, abstract illusions that gnaw at the walls of the island and ensure its decay were designed as a visual representation. To realise this concept, ink spreads were shot on paper in various combinations with water, alcohol or glycerine on the first day of shooting. In post-production, these black and white shots were used as masks in Nuke to make the shadows move across the walls of the island. In addition to the design of the island and the shadows, the design of the so-called “barkonaut” was also developed. It is a mixture of a small rowing boat and a hot air balloon, as escaping across the ocean is not possible with a boat alone due to stormy waves.
The composite appearance of the vehicle, made from washed-up components, old planks, nails, fabric patches for the balloon and improvised assemblies in the workshop, gave the “Barkonaut” an authentic character. Artificial intelligence was used to generate construction plans for the barkonaut and maps of the remote island. The images were plotted onto semi-transparent paper and patinated with tea to give the maps an aged look. The door to a secret workshop, a painting of the island, was made especially for the film by painting student Elisaveta Bogushevskaya.
Sculpting the island
As the final island model was used as a reference for the above-mentioned painting in the film and some other parts of the set design, the island had to be ready several weeks before shooting began. The sculpting of the island was done entirely in Zbrush. Depending on the concepts, a rough silhouette was first defined, then smaller details were worked out. With the exception of the details in the rocks, the entire island was sculpted completely by hand and without procedural aids.
As the island model in Zbrush ultimately had a very high resolution of over 4 million active points, a lower-resolution duplicate of the base mesh was created, which contained fewer details and would speed up the further texturing process. This mesh required less computational work in programmes such as Houdini or Substance Painter and thus facilitated the entire work process. In Blender, the low-resolution mesh was UV unwrapped. The high-resolution mesh was then projected onto this reduced mesh in Substance Painter in order to retain all the details of the original model in the maps generated in Substance Painter.
Modelling & Texturing
The story takes place on an island threatened by decay. Wind and water have reduced the piece of land, which was once richly populated, to a minimum. A sparse amount of dilapidated houses are the last thing left on the island, but even these will not last much longer. The house assets were divided into five complexes before filming and arranged based on the set scan. An attempt was made to create a recognisable silhouette by varying the heights of the buildings and a tower whose top had been broken off. After the shoot, the models were refined in Blender and customised with Kitbash assets.
As the houses were almost only shown in supertotal shots, it was possible to keep the models low-resolution with only around 100,000 polygons. The roof tiles, for example, were not modelled individually, but were just flat roofs onto which roof tile textures were projected. However, for two shots in which one of the house walls was used as a set extension, a detailed asset with 338,394 polygons had to be modelled. When texturing in Adobe Substance 3D Painter, the wall colours, as found in the set, were adopted and digitally patinated using various layers. Stains and elements that contributed to the worn, dirty look were painted by hand.
The digital colour and structure of the barkonaut’s balloon also had to match that of the real equipment. The basket could be designed more freely, as the real basket is hardly ever seen in the film, but it also had to look worn and dirty. To achieve this, the layers were first projected using smart masks and then adjusted by hand. For the most part, the software’s own PBR textures were used, but textures from Textures.com were used for the roof tiles.
Shooting preparations
Careful planning is extremely important for a production with around 60 people involved. Especially for the smooth realisation of the VFX shots, it should be as clear as possible in advance what will be seen and what you should pay particular attention to during the shoot. Basically, the filming preparation can be divided into two sections: firstly, the internal VFX coordination of the various tasks and secondly, the communication of the VFX-relevant information with the other departments.
The internal preparation ran in parallel and in close collaboration with the director and the DOP. Once the script had been developed to such an extent that it was foreseeable that there would be no major changes, an initial calculation of the VFX shots could be made. Thanks to the existing Previz, the calculation was quite accurate and there were few surprises, as it was easy to see in the virtual set whether the shots could be realised as planned. Based on the calculation, all assets and shots were created in Shotgrid, divided into individual tasks and distributed to the various people. This resulted in an extremely precise schedule, which enabled a largely smooth realisation. Working backwards from the deadline, it was easy to recognise when which tasks had to be completed in order to have enough time for subsequent tasks and still meet the deadline. The calculation, assets, shots and tasks were regularly revised when changes were made to the resolution or the script.
Communication with the other departments involved was essential to ensure that the shots could actually be realised as planned. The above-mentioned painting of the island was also created as part of the set on the basis of an early rendering of the 3D island, which ensured a consistent and coherent depiction of the island.
Plan B’s production designers were able to make targeted additions to the existing set construction using the virtual model, which meant that set extensions could be largely avoided and the real and virtual sets could be made to match. The special effects department, which was responsible for the destruction of the lamp as well as the inflation of the balloon, also had a virtual simulation to fall back on. Based on the simulation, which clearly visualised the inflation process, the set could be built in such a way that the balloon could actually be inflated with the help of a wind machine and the actors could enter the basket of the balloon.
Set supervision and DIT
The combination of CGI and live-action film presents a few more difficulties than the purely animated film that the VFX students produced in their first year. In addition to thorough planning, realisation is one of the most important steps. The role of the VFX supervisor is responsible for the interface between the director, camera and the final post-production.
The aim is to plan and prepare the VFX shots as accurately as possible so that the post-production schedule can be adhered to. The focus is also on recognising and preventing potential problems that would cost a lot of time and money later on. Another important task of the supervisor is data acquisition on set.
There are many tools that provide helpful information during the shoot and make realisation easier later on. Camera and lens data are extremely important. This information must be known so that a seamless transition between VFX and the original material shot is possible. For Wind Up, a LiDAR scan was also used to improve the tracking of the planned set extension.
This was created directly after the shot was filmed. The key is to capture as much important data as possible at the right moment without holding up the entire shoot. The supervisor is also responsible for finding the best compromise between the valuable time on set and the avoidable additional work in post-production. The students were supported by Prof Jürgen Schopper, 3D mentor Berter Orpak and Pipeline TD Jonas Kluger throughout the filming period.
The different “roles” were rotated daily between the 5 VFX students so that everyone could gain an insight into the different activities. They also took on the tasks of the DIT (Digital Imaging Technician). Thanks to a mobile workstation, not only can the backups be made, but the dailies can also be rendered directly. Another advantage was the ability to process 3D scans on site and create slap comps to identify potential problems.
Simulation
An island in the water and a balloon that inflates and flies away were the two simulation tasks. Several full CG shots were planned, in which the island was to be seen surrounded by water, partly with an expanding and flying balloon. The real balloon was measured, photographed and recreated from these references in Blender.
The basket of the balloon was only created on the side facing the interior based on the real one, the rest was modelled inspired by our own concepts. The idea was a flying lifeboat, a so-called “Barkonaut”. This model was revised based on the advice of simulation specialist Felix Hörlein in order to optimise the resolution and distribution of the topology for the subsequent simulation in Houdini. The size of the opening through which the balloon was to expand, as measured on set, was recreated virtually as a collision geometry to ensure that the CG settings matched the real filmed settings well. After several versions and adjustments of various parameters in Houdini, the fabric finally behaved as desired and the simulation looked convincing.
The loneliness is mainly told through the huge and empty ocean in the background. The interaction between the water and the island was one of the biggest challenges in achieving a realistic end product. Like the balloon, the water in Houdini was also simulated and shaded. The latest FLIP solver was used to calculate the base water.
For high efficiency over the entire process, the simulation area was limited to a small part around the island. This allows a fast workflow even with a high scaling of several million particles. In this area, the input parameters were extracted from the original ocean in order to use them as parameters for the simulation. The level of detail of the water is largely represented by the whitewater, which is then calculated as a function of the water simulation. Not only the water, but also the wind influence the dynamics of the spray. To make the behaviour even more realistic, an airfield simulation was simulated around the island so that the turbulence in the air also works.
Lighting, shading & rendering
The aim was to use lighting to create an atmosphere and mood on set that emphasised a tight and oppressive feeling. To ensure that this mood is not interrupted by the full CG shots, the lighting has to be adapted to the mood. Whilst the lighting contributes a lot to the overall mood, it also plays a crucial role in terms of the realism we are aiming for. To give the impression that computer-generated elements are part of the real world, light and shadow must fall on them correctly. Careful lighting ensures that the island is seamlessly integrated into the scenes and matches the real lighting conditions, some of which are provided by the stock footage.
The professional help of CG supervisor Frank Dürschinger, who supported the students with the basic principles of lighting through to the individual shots, was also important here. However, light alone is not enough to create a photorealistic image. The interaction with the light is influenced by the shaders. As the entire process of lighting, shading and rendering took place in Houdini, the students used MaterialX shaders to guarantee a high degree of flexibility between the programmes. To increase the level of realism, the texture of the island was combined with several PBR materials.
The resulting improved level of detail creates a real look even with closer settings. A modified version of the Houdini shader was used for the water and spray. To maximise the scope for compositing, the image was split into several layers and rendered individually. This makes it possible to adjust areas such as whitewater, the island or the balloon afterwards. Individual AOVs were also calculated in the respective layers, for example to make the light in the balloon flicker or to re-insert the reflection in the water. The use of the Karma render engine enables a very efficient and fast workflow overall. This was also important for the lighting due to the real-time factor.
Compositing
Lino’s anxiety, which is symbolised by shadows, was visually represented using real rotated ink spreads. With the help of Nuke, the recorded elements were used as masks to darken specific areas. To break up the fluid look and give the “shadows” a creepy, organic aspect, the ink shots were also distorted with noise.
To keep objects and actors in the foreground, they were rotoscoped in the scenes where the shadow spread was located. For shadow shots with tracking shots, the movement was tracked to allow for the integration of the shadows; for static shots, minimal camera movement was added in post-production to increase authenticity.
The students integrated the rendered island model in the long shots of the island with real ocean footage to create a more appropriate atmosphere. The overly friendly sky of the original footage was replaced with more dramatic matte paintings to emphasise the sombre mood.
The black and white levels of the digital image were then adjusted to achieve a seamless integration with the original shot. The digital island blended more realistically with the real ocean through simulated white water effects and supporting VFX elements such as fog. Additional effects such as lens distortion and chromatic aberration contributed to the fusion of CGI and real footage.
To add scenes with a visible exterior wall of a building on the island, the students extended the physical set. With the help of 3D equaliser expert Ando Avila, the camera movements of the crane ride were tracked and reconstructed in digital space. This enabled a correct representation of the digital set in conjunction with the original shot, supported by additional VFX elements such as fog and particles for seamless integration.
In the film sequence where Lino’s fear reaches its climax and he goes into a panicked state where he perceives his grandmother as a demon-like being, her altered appearance was also supported with visual effects. The eyes were tracked in Nuke, rotoscoped and coloured black, while retaining the real highlights to preserve the plastic appearance of the eye.
In addition to the main tasks, inattentions that happened during the shoot were also addressed. This included removing the special effects operator including his leaf blower in the background or adding a forgotten oil lamp to the barkonaut’s burner. The finished VFX settings were delivered by the HFF students to the post-production company Pharos. There, senior colourist Andreas Lautil not only gave the entire film the finishing touches with his cinematic colour grading, but also by taking over the VFX shots. For the compositing tasks, the students received support from Nuke expert Martin Tallosy.
Soundtrack & sound mixing
The musical and tonal layer of the film was extremely important, as it not only emphasised the moods of the characters in all the scenes, but also provided a way to make the location and supernatural events such as the shadows more believable and real. The film’s music was composed by film composer Victor Ardelean. As part of his final thesis at the Munich University of Music and Theatre, it was even possible to record parts of the final composition with the Munich Symphony Orchestra.
Rehearsing and recording with such a large and renowned orchestra was a unique and unforgettable opportunity, not only for the composer but also for the rest of the team. The final piece “Ballonflucht” in particular has an epic orchestral sound that emphasises the final scene and its hopeful mood.
The tonal layer had two main tasks: the acoustic unification of the scenes shot entirely in the studio with the narrative location on an island in the sea and the visualisation of the living shadows. Since the entire film had been shot in a studio building, the soundscape of an island surrounded by the roaring ocean had to be added later. Artistic collaborator Dr Rodolfo Anes Silveira took over the sound mixing here and added subconscious sound elements in addition to the obvious sounds. In addition to the sound of waves and the occasional screech of a seagull, you can practically feel the breaking of the waves on the rocks of the island in the form of a deep bass rhythm in your own chest. The acoustic design of a short film is often a creative challenge, especially when the question arises: How can shadows sound at all?
The answer to this proved to be subtle and yet effective. Whispering noises, crackles and pops were mainly used to shape the acoustic identity of the shadows. The audience should not only see the shadows, but also literally feel their presence. The quiet but haunting sounds meant that the shadows were no longer just an embodiment of fears and evil, but also a reminder of actual dangers such as real cracks in the walls.
