Table of Contents Show
Archive notice: This article was originally published on March 20, 2019. Links and embedded videos are preserved as part of the historical record.
Inspired by the fantastic “Fortnite” trailer by Epic Games and the accompanying making-of white paper, the group started their endeavour. The group’s aim was to be able to map as many work steps as possible directly in the Unreal Engine in order to utilise the advantages of a real-time engine and the parallel work that this makes possible. The group agreed on the following pipeline for the realisation of the production: Autodesk Maya, Houdini, Marvelous Designer, Unreal Engine, Premiere Pro.
Before the technical pipeline could be used, the group had to work out the idea in the concept phase and take care of the design of the characters and locations.
Story
In just under two minutes, the story tells how the young homosexual man Cozmo overcomes his fear of showing his feelings for his partner in public. It was important to the group’s director, Ilhan Soydemir, to raise the viewer’s awareness of the issues surrounding this topic, which are still relevant today.
Screenplay / Storyboard / Animatic
As usual, the script went through several iterations until the final version was finally on the table. Parallel to the script development, Hanna Gärtner started working as a storyboarder and concept artist on the first storyboard thumbnails. As the name thumbnail suggests, the storyboard images at this stage are actually only 3 x 1.5 cm in size. This helps the artist to focus on the image composition and the visual direction. All details are omitted in this phase. (see illustration: Storyboard). The next storyboard images were then each drawn individually in postcard size and hung on a pinboard. This enabled the team to get a quick overview of the story and to try out different editing variations by simply rearranging the images. Once an initial flow of the film was recognisable, Tanja Wax created a first animatic to test the shot lengths and initial camera movements.
Character design and style development
When designing the characters, concept artist Hanna Gärtner always had to bear in mind that the figures would later have to work in 3D. The entire team was able to agree on a reduced 3D style with simple textures.
Image composition / lighting and colour concept
In addition to the image composition, which is already taken into account during storyboarding, the overall impression of an image only becomes clear through the associated lighting and colour concept. Hanna Gärtner has taken inspiration here from the book “Framed Ink: Drawing and Composition for Visual Storytellers” by Marcos Mateu-Mestre and constructed several shots based on his methods.
If the space between the characters is free of obstacles and they are illuminated in the same way, the impression is given that their communication is positive and fluid (cf. Mateu-Mestre, 2010). If the space is separated by an appropriate room layout and positioning of the room elements, the communication seems disturbed.
In the concept art and in the final film, this is represented by the pillar that stands between the two figures. The disrupted communication is further emphasised by the lighting, with Ben’s face in the light and Cozmo’s in the shadows.
The colour concept
Hanna Gärtner describes the colour concept as follows: “The concept for the general colour scheme of the film was to use colours of a sunset and sunrise over the course of the film. The colours range from reddish and yellow tones to light blue in the vase room, the midnight blue planet room and the warm yellow-orange illuminated final room. This colour concept is intended to visually support Cozmo’s inner transformation by clearly separating his individual inner states through the colours in the surroundings. If you place the two characters next to each other, you can see that their shades are similar and the colours of their tops and trousers are mirrored.
This was chosen so that they harmonise in colour and can be clearly separated regardless of the lighting, and also plays on the fact that their personalities are almost mirrored.”
Modelling
When modelling locations, you can save a lot of memory in the engine by setting up your location in such a way that certain parts can be replicated. For a corridor with many columns, for example, it is sufficient to build just one column in Maya and then duplicate it in the engine. For this reason, the dome of the planetarium in the museum was created from a single segment that was duplicated in the engine. The figures were modelled by Christof Reisenegger, Ilhan Soydemir and Matthias Krammer. The only difference between the workflow and a production that is rendered normally is that you have to try not to exceed certain polygon numbers. For example, the character Cozmo had 135,000 triangles in the final version, which was still easy for the Unreal Engine to process. Before exporting the meshes, you should make sure that all geometries that are still displayed as smooth mesh previews in Maya are converted to polygons and then delete the non-deformer history. Without deleting the history, the mesh will be displayed in the engine as a low-poly version.
Rigging
Matthias Krammer was responsible in the group for the entire import of Maya data into Unreal. In order to ensure a smooth import of the Maya rigs into Unreal, Matthias Krammer discovered the following points during his tests.
- The entire skeleton with all joints of a character must have a single superordinate root joint, because the engine requires all joints in a hierarchy.
- Geometry that has already been skinned must not be split again (e.g. Extract). The engine can then no longer find the skeletal mesh.
- Objects in a large hierarchy must all have a unique name. While objects in Maya can have the same name as long as they are in other hierarchies, this leads to serious errors in Unreal.
The rigs of the main characters Cozmo and Ben were created by Christof Reisenegger in Autodesk Maya. Once it was clear from the animatics what the characters needed to be able to do, the rig for the Cozmo character was created first. The Maya rigging tools were mainly used here. If you create the rig yourself, you have full control over it and you know exactly what is possible and what is not. As this manual method unfortunately also takes a lot of time, the second character Ben was rigged with the addition of some plug-ins to save time. The mGear plug-in from Miquel Campos in particular saved Christof Reisenegger a lot of time when rigging the second character. The ability to export the skin weights to a separate file and apply them to the mesh again later after making changes to the rig saved a lot of time. The ngSkinTools plug-in, which can be used to paint skin weights in layers, was also very helpful. Working with layers that can be edited and deleted individually gives the artist the opportunity to try out different variations very quickly. So there is no need to worry about overwriting skin weights that have already worked.
Exporting the figures from Maya to Unreal
The export to Unreal was regulated via FBX files and the Game Exporter in Maya. If objects with skin weights are exported, as with the two characters Cozmo and Ben, the skeletons are automatically exported as well. When imported into Unreal, a so-called skeletal asset is then created (see top left). The animations of a skeletal asset are carried out in the Game Exporter via the Animation Clips tab. To do this, the root joint of the hierarchy is selected and the Game Exporter is set to “Export Selection”. The export of the animation also works via the FBX format, which recognises the corresponding skeletal mesh when importing into Unreal or offers a selection of skeletal meshes. When exporting rigs, it is important to note that Unreal can only export skin weights (skin deformers) and no other deformers. However, as the faces and arms of Ben and Cozmo were simulated using other deformers and elaborate rigs with twist joints, elbow pins and the upper part of Ben in Marvelous Designer by Tanja Wax, these elements had to be exchanged between the programmes using Alembic caches.
ART is fully automated and therefore offers little control when creating the rig. A major advantage is the low labour input, the user-friendly structure and the possibility of making at least minor modifications to the rig. A purely joint-based rig without additional deformers should be used for the museum keeper’s face rig so that it can be imported into Unreal as a skeletal asset and not as alembic films
The third character in the film, the museum guard, only required very simple movements due to his role in the film.
Matthias Krammer used the Maya plug-in Animation and Rigging Toolkit (ART), which is provided by Epic Games, for the rigging. With just a few settings, you can create a complete rig for characters in ART.
ART is fully automated and therefore offers little control when creating the rig. A major advantage is the low labour input, the user-friendly structure and the option to make at least minor modifications to the rig. A purely joint-based rig without additional deformers should be used for the museum attendant’s face rig so that it can be imported into Unreal as a skeletal asset and not as alembic films.
Hair setup
Based on the character design, Fabian Scheuringer’s task was to develop a workflow for realising the hair. Simply modelling the hair as a mesh did not seem realistic enough to the group. Simulating the hair using hair curves looks great, but is too complex for real-time rendering in the Unreal Engine. The solution was therefore somewhere in between. Using a plug-in from Gary Hanna, so-called haircards were generated from the hair curves created in Houdini, which can be further processed in the engine.
Particle effect in Houdini
Tanja Wax, who organised the entire team very competently as project manager, was also responsible for the particle effect in the planetary machine in addition to the clothing simulation in Marvelous Designer. As this effect accompanies the mood of the film and underlines the climax of the film, the effect had to be particularly good.
After the first tests looked promising and the team was satisfied with the simulation, a way had to be found to bring the simulation of Houdini into Unreal.
Instead of importing the particle animation with over 2 million particles as sprites into Unreal, which would have made real-time playback impossible, the simulation was filmed using a 360-degree camera in Houdini, which was placed in the centre of the particle effect. This film, which was rendered in a resolution of 2,048 x 1,024 pixels, served as a so-called “Img Media Source” in Unreal. This was played as an image sequence on a sphere in Unreal. A second, slightly larger sphere with a glass material was placed over the sphere with the simulation projected onto it, and the effect was complete.
Conclusion
The entire team faced a number of challenges during the production time of 9 weeks and produced a great film with “COZMO”.
The workflow developed by the group with rendering in the Unreal Engine will certainly be used in other student projects.
The group also developed a special highlight for the showcase, where all Bachelor projects were presented. As the entire film was available in the engine, the group prepared the locations for the showcase so that the HTC VIVE could be used to enjoy a virtual tour of the museum.
Anyone who has not yet seen the film can do so on the MD.H YouTube channel.
The group of students delivered a great project with the film “COZMO”. On behalf of the Mediadesign Hochschule (MD.H), we wish everyone involved a great start to their careers after graduating.
