Table of Contents Show
At this year’s Adobe MAX sneak session, it was clear to see where Adobe’s headquarters are located in Germany. The gigantic background projection presented itself in the white and blue Oktoberfest look and the co-host Co-host, actor and comedian Adam DeVine, was presented with a beer mug and pretzel in true Oktoberfest style. As always, numerous innovative, experimental technologies were presented by Adobe researchers and engineers. Surprisingly, the focus was on artificial intelligence. You can watch the complete show on Youtube(is.gd/youtube_adobe_sneaks). Here is a short summary of the most interesting innovations!
Project Fast Fill
What is called Generative Fill in Photoshop is now also finding its way into the world of video. The removal and addition of objects in videos. In the examples shown, this worked amazingly well and without any noticeable artefacts. We are curious to see when
the tool will be available in After Effects and Premiere.
Project Draw & Delight
There are now many text-to-image generators. Whether DALL-E, Stable Diffusion, Midjourney or Adobe Firefly, they all have the same problem: the results are unpredictable and the image structure can only be roughly influenced. With Draw & Delight, you can influence this not only with prompts, but also with rough doodles. In the example shown, a comparatively high-quality vector graphic illustration was generated from a very amateurish sketch in just a few steps. It starts with an outline drawing, which can then be filled with any colours. Similar to a colouring book. Backgrounds can also be created. And the whole thing can then be edited as a vector graphic in Illustrator.
Project Neo
This online tool is used to generate 3D shapes from simple basic shapes. The workflow is somewhat reminiscent of Substance 3D Modeler. The focus here is on ease of use when creating 3D elements using Boolean operations and object blending. The finished 3D scene can be virtually illuminated, with the “shadow vectors” adapting to the direction of light. At the end, it can be exported as an Illustrator file. The two-dimensional vector elements after import can be further modified in the graphics software.
Project Glyph Ease
Developing your own fonts for special designs used to be a lot of work. One of the difficulties was to ensure a uniform style. In Project Glyph Ease, only three reference letters need to be designed. The AI then uses these to generate the complete font including textures. This even works with hand-drawn designs that have been imported into Illustrator.
Project Scene Change
Changing the background of shots taken with a green screen is old hat. If you include enough tracking marks, this is also possible with a moving camera. But cutting out actors from normal shots without a green screen also works quite well now. Be it with the new version of the now also AI-supported Rotobrush in After Effects or the AI clipping tool from Runway.ml. Scene Change now combines and automates both techniques by converting the background scene into a 3D environment using photogrammetry. The camera data tracked from the foreground is then transferred to the background scene so that the actor can be credibly integrated into the scene. The positions of objects in the Z-axis and the actor are also analysed so that he disappears behind the foreground objects and casts shadows.
Project Poseable
This software facilitates storyboarding and illustration by combining Mixamo with a text-to-image generator. A 3D puppet is brought into the desired pose using the joints of its skeleton
into the desired pose or placed in an armchair using automatic adjustment. The IK system ensures that only natural positions are possible. The camera perspective is then aligned and the prompt for the desired result is entered. However, the pose does not have to be set up manually, but can also be taken from a loaded image. The 3D mannequin models can also be imported from Mixamo into Poseable.
Project Res Up
High-quality upscaling of videos is already familiar from Topaz Video AI (see this issue, page 52) or the AI upscaling of DaVinci Resolve. While DaVinci scores particularly well when upscaling good low-res material, the Topaz tool is also able to correct image errors such as colour noise, blurring and interlace problems to a certain extent. Adobe is now developing its own tool. In the examples shown, Res Up also appears to use comparison images during upscaling and recreate the structures from them. Whether and with which material it can really do this better and/or faster than the other tools remains to be seen. However, the examples shown were impressive.
Project Dub Dub
In this case too, various AI tools such as Descript already exist. It is about lip-synchronised text translation. Dub Dub Dub uses generative AI to automatically synchronise videos or audio clips in more than 70 languages and over 140 dialects. The tone of voice, intonation and acoustics of the original clip are also taken into account. And with the voices from the original video. The intonations in the last example shown were somewhat strange and difficult to understand in the German version of a very fast-paced dialogue with Cary Grant. But impressive for a 1-click automatic solution.
Project See Through
How often have photographers been annoyed by the fact that a pane of glass between them and the object of their desire has unfortunately obscured their view and they have mostly seen their reflection in the photo. Until now, retouching such photos has been difficult or even impossible. With this tool, glass reflections can now be removed and even extracted. In this way, the reflection can even be blended into the image. What the AI achieves in the examples shown borders on witchcraft. To dispel my doubts, I’ll probably have to try it out for myself when the beta phase arrives.
After Effects
Victoria Nece showed us Roto Brush 3 at the IBC, which now works much faster and more effectively with AI support. However, to ensure compatibility with old projects, versions 1 and 2 are still available. The new 3D workspace now also has physically based rendering and environment lights. The workflow with the Advanced 3D Renderer is much smoother than with the Cinema 4D Renderer. However, it is still somewhat difficult to set the lighting conditions so that the render results look really good. This is still easier with the Element 3D plugin. The Glb format works best in the new 3D workflow. Here the textures are imported 1 to 1.
Firefly / Adobe Express
Version 2 of Firefly was also released for Max. In addition to the improved image generation and the higher resolution, it is now also possible to enter negative prompts. You can also influence the style using a library of reference images or even upload your own reference images. And in the photo settings, you can now set the aperture, shutter speed and field of view.
With the Edit menu you can now also use the generative fill and the removal of unwanted elements as in Photoshop, or crop the foreground object. Although the file can then be downloaded as a PNG, the mask that has just been created is not taken into account, but the complete image with background is saved. You can transfer the image directly to the browser version of Adobe Express.
There, however, you must first set the background colour to transparent and then download the cut-out object, this time with the alpha channel.
This is unnecessarily cumbersome and I hope that Adobe will simplify this workflow by the time this article is published. It’s still in the beta stage. Firefly and Express continue to grow together. The web version of Express also features AI image generation, generative fill and background removal. With simple animations in several individually animatable layers and the integration of videos and music, short motion graphics trailers can be created without having to open After Effects. The “Text to template” module is also new. This allows editable templates to be created using text prompts, which can then be easily customised to your own requirements. Design projects can now also be synchronised with Photoshop and Illustrator. For example, Photoshop and Illustrator files can be thrown directly into the browser window. They are then converted by Express into editable templates with layers. With Illustrator files, all vector layers are retained – and the text also remains editable. I have just received the beta update of the mobile version of Express and it is hardly any different from the web version. You can now open files from the mobile version in the web version again and even work in parallel. As you can see, the development of AI integration at Adobe is making huge strides – this helps with brainstorming and makes some time-consuming edits easier.
Illustrator
In the current beta, text to vector graphics is now also possible. This means that even “vector legastics” can generate interesting templates. There is a choice of motif, scene, symbol and pattern in the settings. The generated graphics can also be coloured using the AI-controlled colour selection. the new modelling function also makes it easier to create interactive mock-ups. An image is analysed for its geometric properties and the vector object can then be placed correctly on the surface in terms of perspective.
The image generated with the prompt “SciFi Spaceship Interior with large Monitor” was analysed via the Object>Model>Create menu. The star logo could then be placed on any surface. The preview on the right shows the logo on various mock-ups from the Adobe Stock Library. Live Text Retype Beta can make texts that have already been converted into vectors editable again. The appropriate font is searched for and a suitable one can be selected from a list. In principle, this even works with texts on pixel images. An attempt is also made to reconstruct the background using Generative Fill. Depending on the material, this works more or less well. But it’s still beta. Another new feature is the Smooth tool, which smoothes vector paths automatically or with a slider. In contrast to the simplify tool, peaks are also rounded here.
