No fewer than eleven studios produced effects for the fifth Harry Potter film, including heavyweights such as Double Negative, ILM, Framestore CFC and MPC. They brought creatures to life, created digital set extensions, gave shape to spells – and saved Daniel Radcliffe from being strangled.
While Heyday Films and Warner Bros. have already worn out four directors, visual effects supervisor Tim Burke has been a piece of continuity in the past three Harry Potter films; he also worked on another as VFX supervisor for the studio “The Mill”. Burke received an Oscar in 2001 for the visual effects in “Gladiator”; in 2005 he was nominated for “Harry Potter and the Prisoner of Azkaban”.
Of all the films he has worked on, Burke believes the last one offers the greatest variety of effects. “Although the book is thick and we had to focus the story on the essentials, we didn’t leave out any of the exciting new discoveries,” he explains, referring to the new digital environments. “I encouraged director David Yates and production designer Stuart Craig at an early stage of production to use digital environments and make them bigger and more magical than they initially thought.”
Glass and water vapour
In the “Hall of Prophecies”, for example, glass shelves stretch into infinity. Millions of crystal spheres rest on them, in which prophecies float in swirling vapour. Double Negative created the set in 3D and inserted actors filmed in front of a green screen into the environment during compositing.
“We see this environment several times in the film,” explains Paul Franklin, VFX supervisor at Double Negative. “In the first shot, in Harry’s nightmares that lure the children into this environment, the shelves are only a few centimetres away from the camera.” The swirling vapour in the crystal balls reacts to the approaching children and in the end they glow strongly enough to light up the environment.
To ensure that the lighting of the digital set matched the lighting of the real-life actors, Double Negative set the lighting first. “We tested the lighting effects before we shot the actors to create an environment with the right amount of lighting,” says Burke. “I think in the end a lot of people won’t realise it’s a digital set, which is obviously amusing.”
Destruction with DN Dynamite
Trina Roy led the initial research and development work at Double Negative. She worked with the studio’s own DN asset tool, which had already taken over the complex geometry of Gotham City in “Batman Begins”. She also used DN-Dynamite, a plugin for Maya that can perform rigid-body simulations for a large number of objects. “When the battle in the hall reaches its climax,” Franklin says, “the giant rows of glass shelves topple like dominoes, creating a crystal waterfall of destruction.”
When it became clear that the render times for the 170-shot sequence were threatening to get out of hand, Roy and CG supervisor David Vickery sat down with overall CG supervisor Justin Martin and shading supervisor Philippe LePrince to work out a combination of solutions. “We used RenderMan tricks to render reflections and inter-reflections under the glass shelves with a ray-traced pass,” says Franklin. “For each surface, we created hold-out mattes.”
For the swirling vapour in the crystal spheres, they initially thought of Maya Fluids, but when it turned out they would have up to 30,000 visible spheres in one shot, Technical Director Chris Mangall created a pseudo-volumetric shader in RenderMan instead. “It has nested layers that have a 3D noise texture running through them and are animated and spaced out like the layers of an onion,” explains Franklin.
Creatures under black smoke
In the Hall of Prophecies, Harry and his friends battle Lucius Malfoy and the Death Eaters, who are made up of swirling clouds of black smoke. Double Negative had previously designed the Death Eaters for “Harry Potter and the Goblet of Fire”, but this time the director wanted the smoke to look like it was swirling over the creatures’ bodies. “We used a combination of Maya Fluids for the smoke with our volume renderer and Soft Bodies in Houdini,” says Franklin. “And we used Soft Bodies to make ribbons and clothes flutter off the Death Eaters’ bodies. It took a lot of tweaking to make them not look like the Dementors.”
As one of eleven VFX studios involved in the production, Double Negative undertook 950 shots, including 2D correction and paint-outs. As well as the Hall of Prophecies, the studio also created a five-metre high CG giant, a digital set for the ‘Veil Room’, a sequence where children practise conjuring a ‘Patronus’ effect, as well as views of Hogwarts, Privet Drive, Harry’s home when he’s not at boarding school and other environments.
Of all these effects, the giant Grawp was undoubtedly the most difficult. “He’s twice the size of Hagrid, but of course he had to look completely convincing,” explains Franklin. “He plays side by side with live actors in the shots.” To create the giant’s facial expressions, the studio was able to utilise the facial animation tool it had recently developed with Image Metrics. With this system, they capture an actor’s play with digital video cameras and then adapt data extrapolated from that play to a digital model that doesn’t have to resemble the actor’s face. “We shot reference footage of actor Tony Maudsley on video in HD quality and recorded the motion capture data, but the final version is keyframed,” Franklin reveals. “David Yates developed the CG actors away from what we had done with Tony.”
To make the giant’s body look realistic, the studio developed new skin and cloth shaders and used complex muscle simulation. “We wanted him to look human, not like a creature,” explains Franklin. To achieve the desired level of detail in the textures, painters covered the body with texture maps in 8K resolution and the face with resolution maps in 16K. The hands and feet had different sets of 8K resolution maps. “You can see the pores on his skin,” says Franklin happily. “As the render times were considerable, we were able to pick a body part at random and increase the texture resolution.”
It turned out that they had rendered Grawp’s skin with so much detail that the compositors had to soften the result. The shots with Grawp are set in a large forest set that Double Negative digitally enhanced. The studio grew the tallest trees from eight and a half to 92 metres.
Patronus magic
The studio’s Patronus effects take place in an indoor set, the “Room of Wishes”. The spell appears for the first time in the third film. It appears as a cone of shimmering white light from which emerges a being that reflects an aspect of the magician’s personality. In this film, the students practise the spell in the Room of Wishes. One creates a horse, others an otter, a rabbit and a dog.
“We animated the animals in Maya and then exported them to Houdini. We only recently added Houdini to our pipeline because of its ability to handle complex setups,” says Franklin. They used soft-body veils and bands that get lost behind the creatures. They then sent the result to Maya, where they shaded and lit the creatures.
“The results you can get with particles and fluids are limited,” says Franklin. “They tend to look familiar; you have this predictable look. That’s why we used old optical effects from films like ‘Andromeda – Deadly Dust from Space’ (1971), ‘Cocoon’ (1985) and ‘Close Encounters of the Third Kind’ (1977) as a reference. Also old Star Trek films, where they broke laser beams into prisms and beamed them into motion control cameras. We recreated these effects digitally.”
Soft body animation
The biggest advantage of using soft bodies was speed. “Simulations are always about rules and dynamic forces,” says Franklin. “Even with fast computers, it’s slow and it’s also hard to manage and get the level of detail we wanted. We wanted something that was more stylised, like Cloth underwater and a swirling effect. The soft-body animation gave us a direct approach so we could create a stylised feel.” To give the Cloth a light-soaked look, they used customised shaders in RenderMan to create glittering light trails.
Industrial Light & Magic, who had already created the Patronus effect in the previous film, also contributed their magic to a Patronus shot in which Harry attacks two Dementors with the spell at the beginning of the film.
For Harry’s Patronus creature, a stag, the studio attached Cloth strips to the animation of the large, running animal, then rendered the Cloth with transparency and finally added lighting effects in compositing using the studio’s own Saber system, which is based on Inferno.
The shape of evil
The studio also used Cloth for the Dementors. “We had created the Dementors for the third film,” recalls VFX supervisor Tim Alexander, “I think that was the reason the team approached us again. However, this time they wanted them to feel more physical.”
The creatures that looked like windswept ghosts in the third film have heads in this film and arms emerge from the swirling darkness. They chase Harry and Dudley Dursley into a tunnel. One throws Harry against the wall and chokes him until he can use his spell, the other attacks Dudley and sucks all the joy out of his soul until Harry chases it away too.
As with Harry’s Patronus stag, the VFX team also used Cloth strips for the Dementors; twelve for each of the creatures. In order for the animators to be able to control the Cloth strips, the TDs developed an attraction system. The animators used a ball to move each Cloth strip; each ball in turn attracted a Cloth strip. A digital model of Harry’s neck was needed to create the illusion of the Dementor pushing Harry against the wall. The neck model was then “matchimated”: The artists recreated the movement of a bluescreen element of actor Daniel Radcliffe image by image. Shapes were used here to make it look as if the Dementor’s claw was wrapped around Harry’s neck. Instead of painting texture maps, rendering the shapes and swapping the live elements with the digital ones, they rendered occlusion passes. These created shadows from the digital hand and the bulges in the digital flesh. The compositors generated greyscale mattes from the shadow passes to lighten or darken the neck in the composite.
Half horse, half dragon
ILM also created the thestrals, a cross between a dragon and an emaciated horse. “They have a horse body and a wingspan of around eight metres,” reports Alexander, “and a long bony tail.” The modellers worked from concept art and 3D scans of a full-size model to create the surface and underlying bones of these creatures. The skin was the hardest part. “It had to suck into the ribs,” says Alexander. That might not have been so difficult, but the skin had to slide over the surface elsewhere. “We fixed certain areas by using a texture map to determine where it shouldn’t slide.”
ILM used its PhysBam engine to simulate the skin, the same engine they used to create the maelstrom in Pirates of the Caribbean: At World’s End [see DP 04:07]. This time the engine moved soft bodies – the skin – over hard bodies – the ribs. The engine was also used to create water and waves breaking on rocks.
Centaurs and elves
A third studio, Framestore CFC, created two types of creatures: The centaurs and a variant of the house elf Dobby called Kreacher. The development department programmed Maya plugins based on prototypes developed in Houdini to merge the skin and muscle play of the hybrid creatures. For Kreacher’s facial expressions, Framestore CFC used reference material from actor Timothy Bateson, but keyframed the sequence at the end. “He’s a miserable old house-elf who hates everything good,” Burke describes. “We developed his look in the concept department and then finalised it at Framestore CFC.”
Serviceable natural elements
In the end, The Moving Picture Company (MPC) also created two big sequences, one with fireworks, the other a big fight between Dumbledore and Voldemort. In the fireworks sequence, the twins fly into the examination room on broomsticks and let off hundreds of fireworks.
“They did a great job using a rock concert set-up to create interactive lighting effects on set,” Burke praises. “They programmed it using pre-visualisation, hit the smoke with the pre-programmed lights and then added CG fireworks.” The twins integrated MPC via a composite with bluescreen elements shot with motion control cameras. Digital doubles were also used.
The battle between Dumbledore and Voldemort takes place in a huge room. “We built the biggest green screen set we’ve ever had,” says Burke. During the battle, the good and evil wizards fight each other with spells that take the form of natural elements. Voldemort sends fire, Dumbledore counters with a water vortex. Voldemort shatters the windows and fires the shards at Dumbledore. He turns the glass into sand. “We used a combination of animation, effects animation and Scanline’s simulation engine for the CG water,” says Burke. The effects build up until Voldemort turns into the sand from the shattering glass and takes possession of Harry.
The next production is already underway
“The work of the other studios was less animation-based,” concludes Burke, who is already working on the next film, “Harry Potter and the Half-Blood Prince”, which is due to be released in cinemas in November 2008. The schedule envisages six months of preparation, six months of filming and six months for post-production. That’s tight, but shouldn’t be a problem: “The Harry Potter production is running like a well-oiled machine,” says Franklin happily. “It’s been a great experience.”
