Talent. A relic? Creativity. An algorithm? 3D artists who have agonised through thousands of hours of tutorials, often accompanied by the miserable soundscape of distant Indian intersections, are now looking at upcoming tools like “Sora” with interest and intimidation. Is what my brilliant YouTube mentor from the other side of the world taught me in his flat really becoming superfluous? No! Image Generative AI is powerful, no question. But as anyone who has dipped their toes into the world of LUTs and ready-made scripts knows, the magic word is control, and anyone who believes that an art director can now fulfil their own correction requests with text-to-image command input (“prompten” for short) is wrong… Because gaining control over the AI-generated image requires more than a weekend course in “prompt engineering” at an adult education centre.

We gain control over Computer Generated Imagery (CGI), just as we CGI artists have always done, not through prompts and magic words, but through dedicated software. The one we will be talking about is ComfyUI. And if you’re afraid of the art director using it to carry out the next change loop himself, you’re probably also afraid of the “creative” from the advertising agency stealing your Houdini files and opening an FX department with them tomorrow.

ComfyUI?

ComfyUI is a node-based interface for Stable Diffusion – and offers maximum control over Image Generative AI. It’s about more than the simple mid-journey principle: I write a prompt and it generates an image – that would be pure surface scratching. With the help of Stable Diffusion and ComfyUI, I can not only create images on command, but also extract a good depth path from a real photo – and much more. Beyond the gamble of prompting, I can gain control over an image in a way that Midjourney cannot. I can generate a mountain during the day and just change the time of day or the position of the sun. I can mask out the main character from footage at the touch of a button, faster than the After Effects police allow. And I can create 3D models from images and text input. And that’s what we’re going to talk about today.

What can you actually do?

If you have experience with nodes and a solid understanding of compositing, Houdini, Maya or Blender, then you are well equipped to benefit from ComfyUI. If you’re just looking for a 1-click solution to create deepfake porn, ComfyUI is definitely not for you. Unlike Midjourney, for example, where you just need to be able to type a few words (and if you don’t know how to do that, you could dictate it to the AI you trust), the bare “text to image” image generation wouldn’t do justice to the tool’s capabilities. It’s very easy to set up, but understanding it and being able to use it in a controlled manner requires the skillset of an experienced mid- to senior-level 3D generalist. Not only to get what you want in a controlled manner, but also to understand where the potential lies. The average user would probably be put off by the node-based interface, but not the 3D artist who feels at home in Maya’s Hypershade, Nuke, Houdini and the Geometry Node Editor.

Installing and setting up ComfyUI

Installing ComfyUI is simple and can be done in just a few steps. Here is your guide to getting everything up and running quickly. First you need a programme for unpacking files. 7Zip or WinRAR are top choices here. (A little tip: the trial version of WinRar is also sufficient)

Download ComfyUI

Then go to the ComfyUI page on GitHub github.com/comfyanonymous/ComfyUI Under ‘Installing’ you will find a direct link to the download. Download the 1.4 gigabyte 7-Zip file. Unzip the downloaded file into a folder of your choice. And ComfyUI is installed!

Indispensable: The ComfyUI Manager

Not just a recommendation from me, but an absolute must-have is the ComfyUI Manager, which you can find here.

github.com/ltdrdata/ComfyUI-Manager. I’ll explain why it’s so important in a moment. First, navigate to the custom nodes directory: Open the Windows Command Prompt by clicking “Search” in the taskbar, typing “cmd” and pressing Enter. Then navigate to the custom nodes directory in your ComfyUI folder by copying the path and entering, for example, “cd D:\ComfyUI\ComfyUI_windows_portable\ComfyUI\custom_nodes”. (Attention! This is my path, it may vary for you of course.) Press Enter. Clone the ComfyUI Manager: Copy the following command into the command line: git clone https://github.com/ltdrdata/ComfyUI-Manager.git and press Enter. This will clone the ComfyUI Manager into the directory. This may take a few seconds. Restart ComfyUI: If you now restart ComfyUI, you will find the menu item “Manager” in the sidebar on the right-hand side. The manager is an extremely powerful tool that allows you to do the Gitclone fiddling you just did to install the manager only once. Without the Manager, you would have to install every other plug-in in the same cumbersome way. The Manager, on the other hand, offers you a database in which all ComfyUI plug-ins are stored, which you can then install at the touch of a button. Not only that, you will later learn that ComfyUI node set-ups, also known as workflows, can be shared from user to user using drag & drop. If a user shares a workflow with you for which you have not installed the necessary custom nodes, the Manager recognises this immediately and automatically installs them for you.

Selecting and activating the stable diffusion variant

You have now installed ComfyUI. What you still need is a suitable AI image generation model. Modifications of Stable Diffusion 1.5 and Stable Diffusion XL are currently the means of choice in the open source world – and you can find a huge selection of all kinds of Stable Diffusion models at civitai.com.

Installing the model

Once you have decided on a model, download it from www.civitai.com and move the model file to the models/checkpoint folder within your ComfyUI directory. You have now installed ComfyUI and the Manager and set up a ComfyUI model. Now you are ready to go!

First steps: Starting ComfyUI

To start ComfyUI, all you need to do is double-click on NvidiaGPU.bat in the ComfyUI Windows Portable folder. If you do not have an Nvidia Gpu, select runcpu.bat – but be prepared for a considerable loss of performance without a Gpu. After starting, the console loads all the necessary data. This may take a few minutes, especially when starting for the first time after restarting the PC. Once opened, ComfyUI presents itself in a grey window. On the right-hand side you will find a self-explanatory menu bar. Double-click in the grey area to open the search, which you can use to navigate for nodes.

Hardware requirements

In order to use ComfyUI optimally, a powerful workstation is required. If you are not sure what the difference is between a workstation and a conventional PC, then ComfyUI might be above your requirements (psychologically I mean). However, for 3D generalists, your own PC should already be sufficient. Recommended: A RAM of at least 32 gigabytes, ideally 64 gigabytes. Plus a graphics card: an RTX3070 or similar, but as is so often the case, bigger is better. And last but not least, a processor: a modern I7 or similarly powerful. With this equipment, you are well equipped to use ComfyUI in a relaxed manner.

Off to the full! Moodboard, layout and happy art directors

In large studios, a google-savvy intern is hired to find as much input and reference material as possible on the internet. But what if you can’t afford a google-savvy intern and are too busy to google yourself?

ComfyUI: Your unpaid intern

This is where ComfyUI comes into play. If I’m looking for inspiration for a very specific topic, I can of course search Pinterest and Google Images. But I can also get inspiration in parallel by writing a prompt that describes exactly what kind of inspiration I’m looking for.

Inspiration instead of plagiarism

And while, realistically speaking, I have often found just 20 useful inspirations in Google images after an hour, ComfyUI creates 100 images for me within 20 seconds. if these are all too strict and single-layered, I can play with different variables to change the randomness and prompt fidelity so that the programme adds unexpected combinations to my initial request.

Blocking

The blocking. The first animatics traditionally consist of grey and dark grey and slightly greyer cubes and spheres, which are representative of cars, boats and houses. Sometimes a difficult image to decipher, not only for the art director. And this is where ComfyUI can help. How about using proxy assets instead of cubes in greyscale? So that the people who have to continue working with the layout at the end and have to create real 3D models can visualise much better whether they are dealing with the Venus de Milo or a stone in this cube.

Installed, what now?

Let’s create a workflow together. As a “demonstration task”, which has nothing at all to do with what films we watched last night, lay out a village in anime style. I’m imagining bunches of houses standing on rock pillars in the water. Boats sail between them. I’m sure you’ve heard these or similar details from your art director before. Can you visualise anything like that? Me neither. That’s the well-known art director problem, when you talk first and then try to listen to yourself in order to understand what you’re saying. What we need here is a communication bridge between us and the confused thoughts of an art director. Maybe the art director will even give you a scribble on the back of his last payslip. Which will look something like this.

We’re building a Ghibli-style village

Here we are with the art director’s doodle and his idea. Instead of having a long back and forth discussion with the art director to understand what he even means, we can let ComfyUI do the talking, because we’ve already got it set up and ready for this exact moment. You haven’t? Then it’s about time. Let’s get started with our first node tree.

A first node tree

We start ComfyUI and when it has loaded, we double-click on the image and enter “KSampler” in the search field. We create our first node. The KSampler is the centrepiece of every workflow. The KSampler combines our stable diffusion models with our prompts and our idea of what the dimensions and the image should be and generates an image for us. We’ll go into the setting options in more detail in a moment, but first let’s set up the node tree. ComfyUI offers us context-related suggestions as to which node we can plug into which input.

Let’s now create a checkpoint loader node by left-clicking on the “model” input and dragging the mouse into the grey area and releasing it. A small menu opens in which we get suggestions as to what we could put in here. We take the checkpoint loader Simple. Here you can now load the model of your choice under CKPT name, provided you have saved it correctly as described above.

Next, we want to create our text boxes for the positive and negative prompts. To do this, we drag the mouse from the Clip item in the checkpoint loader into the empty space.

Here we select clip “textencode”. We do this twice. We connect the “conditioning” point once with the “positive” point in the KSampler and the other “conditioning” point with the “negative” point in the KSampler. Next, we need a latent_image.

What is important for us is that the latent image determines the resolution of the image we want to generate. (Remember: The native resolution of 1.5 models is 512×512, the native resolution of XL models is 1024×1024.) We set this resolution here. We now move the mouse back to the “latentimage” item in the KSampler, hold down the left mouse button, drag the mouse to empty space and release it again. Here we select empty latentimage. Now we access the “latent” item in the KSampler, drag it into the empty space and select “VAE Decode”.

The VAE decoder converts the KSampler data back into visible pixels. We connect the VAE point from the VAE decoder with the VAE point in the checkpoint loader. Next, we extract another new node from the “image” point, this time the safeimage node. We have now created our first functioning node tree. Let’s now look at the most important settings.

The most important settings – positive and negative prompts

First the prompts: The text box in the “positive” item is filled with the information you want to have in your image. If you have loaded an XL model, it is much easier to fill this box because you can describe what you want in a natural English language. In the negative prompt you write in what you don’t want. If a generated image is too ugly and you have not written a negative prompt, this could be the reason. So you write “Beatrix von Storch” in the negative prompt, for example. ComfyUI now also knows that it should avoid anything associated with this term.

Seed

The “Seed” setting is used to define a starting point for the random generator, which enables the reproducibility of results. Using the same seed number should theoretically generate the same image. For variations on each run, this value should be changed or set to “randomise”.

The “control_after_generate” parameter determines what happens after an image is generated. If set to “randomise”, a new random value is taken after each generation, which ensures diversity in the results. This is useful for creating a wide range of variations.

Steps

Defines the number of steps the sampling process should go through. More steps can lead to more detailed results, but can also increase the generation time. A medium number of steps is often a good compromise between richness of detail and speed.

Classifier-free guidance scale

The cfg value influences how closely the model follows the text descriptions. A higher value can lead to more accurate but possibly less creative results. Experiment with different values to find the best compromise between accuracy and creativity.

Sampler Name

Sampler_Name specifies the sampling algorithm to be used. Different samplers can emphasise different properties in the generated image. “Euler” could be an option that offers a good balance between speed and quality.

Scheduler

This controls the schedule according to which the sampling steps are performed. “Normal” could stand for a standard run, but other options could influence the quality or speed.

Denoise

Leave this value at 1 if you are only working with text input but no input images. Yes, images can also be input, we’ll come to that in a moment! The denoise value becomes relevant if we want to control what is generated for us with an image in addition to our prompt. The closer the value is to 1, the more dissimilar the image generation looks compared to the input image.

If you are not sure what to do, leave the settings as they are in the picture. You may want to play with the sampler and switch between Euler and other samplers. Now we have our first node tree and know how to use it. So we are prepared for the conversation with the art director.

Visual ideas through ComfyUI

Let’s type into the positive prompt: “anime water village houses on rock pillars.” Then we press “Queue Prompt” in the sidebar on the right and the programme starts to generate an image for us. When the Stable Diffusion Model is activated for the first time, loading the model can take one to two minutes – but it remains loaded until the PC is restarted and then works “instantly”. In the menu bar, we can also switch on Autoqueue and click on Change. This means that if we now press Queue prompt, Stable Diffusion will generate one image after the other. With these settings and a halfway decent graphics card, the generation of an image should not take longer than 2 seconds. The programme will now spit out a large number of images very quickly – on the basis of which you can evaluate with your art director what he actually wants. (Here I show a folder with 200 images that were generated within 2 minutes)

Introduction to the colourful 3D scene

So, let’s assume that we have now agreed on a rough idea with the art director. Next, we want to block the scene, but in colour. To do this, we need 3D models of houses, cliffs with houses on them, boats and maybe a few people. I have provided you with the node tree for this on the DP page. Download it and simply drag it into your ComfyUI interface. (is.gd/comfyUIsetup_plugin) It will automatically recognise it and offer you exactly my workflow – if you want to participate or need a starting point. Your ComfyUI will recognise that plug-ins are required, which you probably haven’t installed yet, and offer to install them. You agree to this and after a few minutes you are ready to go. And if we now go through the various tools, we have a common basis.

From doodling to 3D model

This is what the nodetree looks like. So that ComfyUI knows that we want to have a house, we promptly write as a positive: “beautiful studio Ghibli Hero House with a lot of details”. Now we doodle a very rudimentary house of St Nicholas so that ComfyUI knows what kind of perspective and what kind of format we want to generate our house in. The image then goes into a VAE node and now serves as a latent image. The Denoise value is very important here. If you set the Denoise value to 0, about 0.1, then ComfyUI will spit out the image pretty much exactly as you have given it. I recommend a value of around 0.8 here so that your 3D world doesn’t just consist of 2-dimensional stick houses.

The path to the detailed 3D object

The nodetree goes further and scales up our image so that we get a better resolution than 512×512. To create a 3D object from this, we use the TripoSR plug-in. To do this, we first use the AI to mask the background of the image using the RemoveBackground Node (the AI is quite clever, so it often knows exactly what we want without us having to change anything in the settings) and then we enter the mask and image into the TripoSR sampler, which then spits out a 3D model. You will then find this in the ComfyUI output folder. Remember, the model is not textured, but the colour representation is displayed via a vertex colour attribute, i.e. the level of detail of the texture or the colours depends on the geometry resolution. You can of course also use this workflow for all other objects. I did all the assets for my rendering this way.

Off to Blender! Selecting and creating low-poly models

You start by selecting the variants you want to load into Blender. I recommend that you first create a few dozen low-poly models with a very low geometry resolution of 128 using the auto queue function. This will result in models that work very well and represent what you want on the one hand, but also some that only work poorly on the other. But that doesn’t matter. Because we generate one model per second with this technique – with hundreds of objects, it’s not difficult to delete what we don’t want.

Importing into Blender

As soon as you have a few models that you like and would like to import into Blender or another 3D programme, you can search for the corresponding image in the output folder. Then drag this image into your ComfyUI interface. The nodetree used to create the image is stored in the metadata of each image. In the ComfyUI interface, you can then generate a geometry resolution of 512 for the selected model. You can then load the corresponding model into Blender as an OBJ file. In the Shader Editor you create a material for it and use the “Color Attribute Node” as colour input to give it back its colour in Blender. This is completely sufficient for a quick colour layout.

Colour resolution and texturing

For those of you who want a texture resolution that corresponds to the input image, I have written a small Blender plug-in (link in the description). With this plug-in you can nicely project the texture from the high-resolution input image onto your 3D model with one click, which in some cases is far more impressive than the vertex colour.

Model factory and distribution

Now to the Blender model factory: Let’s generate 100 models of a house via auto queue. Select all models at the same time and load them into Blender. With my tool you can distribute them with one click and give them the vertex colour at the touch of a button. You can then quickly render these fixed houses in the viewport and send them to your art director. He can then paint circles around the little houses that he likes. You can use the same procedure for cliffs, boats, trees etc.. This will give you some atmospheric images of how the scene should be put together, as well as models that you can place in this scene. Using your imagination, you can then quickly lay out the scene according to your wishes and those of the art director.

Conclusion: Art Director to Artist

ComfyUI can help you overcome your communication barriers within a pipeline. We are moving away from our grey-graded bricks towards an initial layout that is not only more colourful than simply working with grey blocks, but the Proxy3D models could be a real inspiration for the final look of a product. Especially in the initial phase, you can offer the customer a palette of possibilities so that they already know more clearly in which direction the look of their product should go. Nasty surprises in the second half of a project, where the customer wants to completely overturn the look, can be avoided in this way. Well, most of the time.

And as you can see, even if we are dealing with a powerful tool here, it is necessary to have a real CG artist at the keys when using it, as an intern is in a bad position. And if someone thinks that the customer will throw everything into a prompt themselves in future, they forget that they first need to know what they actually want. So our jobs are safe in the medium term.

What happens next?

Well, this was the first article in a series that will continue over the next few issues – there are simply too many ways in which Comfy UI can help an artist in everyday life. In this series, we can expect future articles on PBR materials at the touch of a button, quartered render times, stylised renderers and one or two other delicacies. There’s a lot to look forward to!

Arne Palluck is a 3D generalist and designs 3D animations for German TV programmes such as “TerraX” on ZDF or “PM Wissen” on Servus TV. He is also a 3D layout artist for large VFX studios, where he takes care of the camera and layout for feature films. He brags to his daughter’s friends that he has already worked on Star Wars.

The post Comfy UI – AI for artists! first appeared on DIGITAL PRODUCTION and was written by Arne Palluck.

Many tasks in Houdini are carried out in the network editor and not in the viewport; every single step, whether bevel or extrude, for example, creates a new node. So it’s all purely procedural, isn’t it? The overview goes to zero in parallel with the working speed and the user goes to other DCCs that can be modelled directly.

Modelling directly in Houdini has some clear advantages. In addition to the obvious saving of export/import loops, Houdini offers wonderful opportunities for automation, experimentation and endless iteration (or quick non-destructive customer feedback rounds) with its basic, but not exclusively, procedural approach.

Add to this a bit of VEX coding at the right time and the conversion of complex modelling setups into simple shareable or sellable HDAs, in which the artist simply has to adjust given parameters to create countless variants of the model, and you have a gigantic arsenal of creative possibilities. Unfortunately, Houdini is not always particularly intuitive or user-friendly and some tools are a little awkward to use.

SideFX has recently been trying to address this with exciting new features. There are more interactive tools, many nodes have been given a tidier UI, more Quality of Life nodes from the Labs or the Tech Art Challenges, where users can submit their own HDAs of all kinds (some of which are permanently integrated into Houdini), plus Quicktips and, of course, more tutorials. Basically, modelling in Houdini is better and more interactive than expected. Nevertheless, there are some areas where we would like to see a better flow and easier ways to achieve the goal.

Direct Modelling!

And this is exactly where Alexey Vanzhula’s solution for direct modelling with the plausible name “Modeler for Houdini” (version 2023) comes in. The talented developer’s inexpensive tool aims to model quickly, intuitively and above all interactively, primarily in the viewport, to compensate for missing tools and generally to simplify and streamline the workflow. Alexey defines his work with the following words: “Modeler is designed to quickly create and edit polygonal\subd meshes. This plugin allows to concentrate only on editing polygons without unnecessary manipulation of Houdini nodes. It has a full set of mesh editing tools, like in other classic 3D programs.”

The centrepiece of the current version 2023 is the launcher, a global menu that opens at the mouse position and from which the artist can quickly and clearly select the desired tool, even faster of course via hotkey.

A bit like Maya, but with its own new ideas and – this is important – seamlessly linked to all the powerful on-board tools of Houdini itself. Modeler is not a black box, but is wonderfully integrated and can be controlled in many places – if required – with attributes or outputs useful groups such as the extrudeFront known from the vanilla Extrude and thus builds the urgently needed bridge between direct and procedural modelling.

It is important to understand that Modeler still creates nodes in the background for many tools, but you can safely ignore them in the interactive flow (tip: hide Network Editor) or send them to nirvana. With the latter, however, the procedurality also disappears. A typical example of remixing the Houdini and Modeler workflows is shown in the following project, in which the various plants were created as individual assets with Modeler and its companion Z (more on this later), partially refined in ZBrush per Z or with Houdini tools such as the Voxelmesh and then scattered onto the basemesh with various procedural tools. Or the complex elements of a bicycle, which can easily be created using Modeler and then welded together using Houdini’s on-board tools. Or the furnishing of a graphically stylised room, created with Modeler and then brought to life with Houdini’s RBD plus classic CHOPS animation (with the exception of characters).

The tools, workflow and launcher

Modeler is activated via the shelf tool of the same name. A yellow line in the viewport shows that Houdini is now running in Modeler mode (and some vanilla shortcuts will now be used by Modeler elsewhere). Let’s take a closer look at the launcher and the most important tools. Viewport with Alt-B in fullscreen mode and off you go!

The heart and control centre of the entire tool is opened in the viewport via a tab (hotkeys are of course customisable). Categorised according to logical use cases, all available commands can be quickly selected here or the associated shortcuts can be learned. A tooltip for the current tool can be found at the bottom.

PolyPen

With this probably most innovative and versatile tool from Modeler, you can interactively move, slide, cut, remove and add any components (points and edges) with a single node (!). You can add any number of loops (including interactive subdivision), adjust the loop flow, slide the entire loop and, of course, delete it again. An inset face and bridge function rounds off the package. All these commands are – and this will blow the mind of any Houdini user – easy to implement interactively in a single node (a subnet is generated in the background, but you won’t notice this).

Brush Tool

Fancy a bit of basic sculpting? Interactively move geometry/points and then smooth them? You can do this without ZBrush and without any edit node cramping using the tool of the same name. Points can be moved freely or along their normal with a freely adjustable radius. The counterpart to ZBrush’s Move Brush and therefore a much-needed tool with which you can even work with axis symmetry..

Symmetry

Oja, Modeler allows interactive axisymmetric work for everything based on the edit node, i.e. moving, scaling, rotating and the brush tool including helpful tools such as beaconing, instantiating or clipping the symmetry. In addition, there is also the Simple and Vanilla Tool-like Mirror Node.

From Extrude to BevelPro

Extrude and Inset actually work in exactly the same way as in Vanilla Houdini, only more interactively and faster via shortcut or “Repeat last” command. The addition of the Extrude-Pro tool, which works a bit like ZBrush’s qMesh and allows you to “punch through” geometry, is exciting.
Another “Pro” tool is the addition to the classic bevel (also interactive and wonderfully fluid), which allows interactive “magnet-based” changes to the respective bevel radius at point level – without the use of attributes. Bevelling with interactive falloff, which can also be fine-tuned with a ramp.

Knife Tool

Unfortunately, Houdini’s own Knife Tool is not exactly flexible, which is why the Modeler version, which allows any number of cuts through the selected faces in a node, is incredibly helpful. The cutoff function, which immediately removes cut polys, is particularly time-saving.

Other useful tools

• Falloff transforms selected elements, but with a falloff defined by ramp or freely settable.
• The array tool works very similarly to ZBrush and is a quick alternative to copy2Points or offers a little more than Copy&Transform.
• Lattice is a simplified version of the Houdini tool of the same name.
• To Circle transforms selected elements into a circle. Straighten aligns selected elements in a line.
• Flatten aligns selected elements on a plane or free axis defined by a construction plane.
• The Crease tool is characterised by its own subdiv preview.
• Quickpivot is a quick and interactive way to position the transform handles of each tool faster than before.
• The CurveTools thank you for FixCurves, which prepares the input for further processing (aka conversion to clean polygonal curves) and avoids a few problem areas of the native workflow, as well as the HoseTool, which lets you quickly create all kinds of cables and hoses.

Parm Slider

This is not a tool as such, but rather the ability to adjust various parameters such as loop subdivisions or extrude divisions at the touch of a button and mouse or pen drag. Information on this can be found in the tooltips of each supported tool.

Align View

Also not a tool in itself, but very time-saving and welcome: Modeler simplifies viewport navigation by aligning the camera rotation to the next global view plane when the Shift key is pressed. Pressing Space-1 to -4 thus becomes obsolete.

Display Options & Material Workflow

For even faster work, Modeler comes with a few desirable functions on the subject of “What do I actually see and what don’t I see?”. For example, selected faces can be easily hidden or all non-selected faces can be hidden. Assigning and editing or copying and pasting materials on a polygonal level is also accelerated or made possible with a few useful tools.

Further utilities from the launcher

If the nodetree does become too large, a bold click on “DelHistory” packs the entire geometry without attributes into a single node and deletes the now superfluous nodetree.
The DelAttr and DelGroups commands help you to get rid of unnecessary data clutter. Of course, you can also remove groups and attributes as before, but this makes things a bit faster again – and that is the basic idea behind the entire tool. Speed up what already exists, add what is missing.

Companion Z

Alexey’s dedication to simplifying modelling in Houdini can also be seen in the two smaller siblings of Modeler, which add even more functionality as separately available tools.

“Z for Houdini” connects ZBrush with our beloved Houdini and allows the artist to quickly and easily exchange and process geometry between the two programmes. Simply send the basemesh from H to ZBrush via Shelftool (unbelievably good: even groups from Houdini are transferred as polygroups), sculpt and otherwise modify it as required and retrieve it again with a click of the Shelftool. Incidentally, fibre meshes are also interpreted as curves and blend shapes and layers are supported. What’s more – and particularly tasty – the powerful zRemesher can be started and adjusted with a node from Houdini.

Soft Boolean

“Softboolean” does exactly this task with flying colours and saves a lot of work steps required in vanilla Houdini. A few useful tools are included to make the workflow as interactive as possible, such as quickly extracting the seamcurve or subdividing the meshes.

Community, resources and development

Modeler benefits from an active community centred around the Discord channel. In addition to official how-tos and highly helpful users who can answer almost any question, there is also the opportunity to report bugs or directly express new feature requests and discuss the best ways to implement new tools with the developer. Modeler is also regularly updated with new functions and bug fixes. Sneak peaks of new functions and helpful, detailed tutorials can be found on the official YouTube channel.

Setup and purchase

Modeler is sold for 99 US dollars via the Gumroad Store and is very easy to install: “Copy the modeler and packages folders to the Houdini user preferences folder.” A preconfigured *.json file is used.

Conclusion

Of course, almost a hundred euros is not a small sum, but measured against the gain in speed, fun and convenience, the tool is inexpensive and a really good investment. Modeler belongs in every well-stocked Houdini collection and makes the assembled DCC competition look even older..

Links

Gumroad Store alexeyvanzhula.gumroad.com
Discord Channel https://is.gd/discord_hm
Youtube is.gd/houdini_modelers

The post Direct Modeling (R)evolution für Houdini first appeared on DIGITAL PRODUCTION and was written by Manuel Kotulla.

Why transcribe?

With social media content being consumed everywhere, captions are becoming increasingly important – both for viewers and for data-driven edit workflows – text is easily searchable and instead of sifting through hours of footage, you only need to type the one quote into the search box to find the spot in the interview.

Adobe Premiere has just taken a huge step forward here and integrated the transcript tool into the current 2021 version (see article in issue 21:03, or here Speech-to-text Help Site). As part of this, the entire subtitles workflow has been linked to the Essential Graphics Tool and can be used intuitively. We have tested this and fed it with various voice recordings – in German and English, of course, and a little French, Allgäuerisch and Pfälzerisch. You can find videos here – and of course at postproduction-tutorials.net.

Why in Premiere?

There are other tools and tricks to get an automatic transcription, but if you’re already editing in Premiere anyway, it would be “unwise” to add another tool to the pipeline. Other news about the latest update can be found here: Premiere Pro What’s New/ User Guide

And for those who say “But I’ve always done that by hand”: respect and appreciation, but we suspect that there is potential for saving time and effort in your pipeline. Because unlike many tools out there, “Speech to Text” is included for free in the Creative Cloud – so you don’t have to worry about additional costs.

But how?

To put it very quickly, this is how it works: For the transcription, the audio is rendered and uploaded to the Adobe server. After a few minutes, the text is displayed and you can select the subtitle type and one of the text layouts already created in advance in the Essential Graphics Panel. The subtitles are now distributed in blocks on the subtitle track in the timeline. The blocks can be merged or separated as required. You can change the texts both in the subtitle window and directly in the programme window.

If you want to change the text layout, you can simply select one or more text blocks in the timeline and change the parameters in the Essential Graphics Panel or select another previously created track style. The transcript can be exported as text or SRT file. Importing the SRT file into the video uploaded to YouTube also worked straight away. That was the very quick version – but let’s go through it in detail – with screenshots and adjustments.

And now step by step

It’s best to use a concrete example: At the beginning of the year, I recorded a remote interview with Noel Powell, the programmer of these innovative After Effects templates, for my Creation FX article in DP 2021:01. I’ve been wanting to edit and transcribe this for a long time to publish it on my tutorial portal. Preferably also with German subtitles.

Firstly, I transcribed the entire interview, as a transcript is particularly helpful for long interviews when selecting unnecessary parts. As the video follows the text, you can simply cut and mark the relevant parts.

The easiest way to transcribe is to switch to the new subtitle workspace. Then select the timeline to be transcribed and click on the Transcribe tab in the text window.

After selecting the language and starting the transcription, an audio file is automatically rendered and uploaded to the Adobe Cloud.

The transcribed text then appears in the text window after a short wait and can also be edited directly there. The timeline runs in synchronisation with the highlighted text passage.

Different speakers can also be recognised. The names of the speakers can be edited by clicking on the 3 small dots. It is not immediately obvious that there is a small bar on the side of the window that can be used to scroll to the second speaker in order to change it.

There is a search field at the top of the transcript window that you can use to search for words or quotes. This allows you to quickly find text passages in interviews, for example to correct misspelled technical terms or simply to find important text passages. By clicking on the circle-arrow symbol, you get a replace function to effectively correct repetitive errors.

Transcribing clearly spoken texts works very well. The tool also copes quite well with English technical terms in a German text. Once the errors in the transcript have been corrected, the subtitles can be created. They now have their own track and the look is controlled via the Essential Graphics Panel.

Subtitles, nicely done

The best way to do this is to create a template in the Essential Graphics Panel in advance, which you can then select directly when creating subtitles. Simply click with the text tool in the programme monitor and write a representative two-line text. Then select a suitable font, set the colours, border, shadow and background to your liking and save it with “Create style”. The text style then appears immediately in the project window. It can be saved from there by right-clicking on “Export text formats” for later use in the Adobe Creative Cloud, for example. From there, it can later be dragged and dropped into another project.

All text styles available in the project can be selected directly in the dropdown of the “Create subtitles” tool. In this way, the type design can be modified very easily and you can quickly switch between different layouts so that the subtitles can also be adapted to the customer CI. The subtitles are then divided into blocks on the subtitle track at the top of the timeline. The blocks can then be easily merged or separated as required using the arrow icons. They can also be changed in length or moved just like images or videos using the corresponding mouse tools.

If you want to modify some subtitles, you can select individual ones or even several using the Shift key and edit them in the Essential Graphics window, for example to change the colour of the text background for a speaker. Subtitles can also be positioned differently if they are overlaid with a name insert. The easiest way to do this is to use the Align and Transform widget by clicking on one of the nine squares.

However, you can also move and scale the text field in the programme window using the mouse.

Important: This step should be carried out after selecting the track style. If you change it again, all changes will be lost and all subtitles will be positioned in the same place again. This is because, as the name suggests, the track style always affects the entire subtitle track.

It would be advantageous to be able to apply a track style only to individual subtitles in order to work a little faster.

Once the subtitles have been created and corrected, there are various options for outputting them. As text to integrate an interview into an article or as SRT for YouTube subtitles.

Test subtitles before uploading!

The free Pot Player presented in issue 2020:06 is also suitable for checking the SRT files before uploading. It automatically displays the SRT subtitles if they are in the same directory and have the same name as the video file.
You can also see straight away that the display of SRT subtitles does not offer the same options as in the Premiere Essential Graphics Panel due to the format. Only simple formatting such as text attributes, text colour and text position are adopted via HTML-like tags.

There are also formats for various broadcast standards, which are embedded in the corresponding format when the video is rendered out. Of course, this only works with formats such as MXF, which also support this metadata. If you want to retain the graphic design from Premiere, the subtitles can always be “burnt in” to the video.

The SRT files can also be used to initiate translations. For example, you can copy them into online translation tools such as Deepl and have them translated there.

We copy these texts into the text editor and save them with the .srt file extension. Simply drag this into the project and into the corresponding timeline. This automatically generates a new subtitle track and the embedded timecode markers are retained. This ensures that the translated subtitles are in the right places again.

Languages

Well, it wouldn’t be “language to text” if it could only be a slightly Hanoverian German – so far there are 12 languages to choose from: English (US), English (UK), Simplified Chinese (Mandarin), Traditional Chinese (Cantonese), Spanish, German, French, Japanese, Portuguese, Korean, Italian, Russian. Hopefully that will be enough to start with, but it is likely that more will be added over time. Maybe even Allgäu and Palatinate! Or other special dialects – but as in all areas of AI development, that remains to be seen.

And here’s something to look at

In the future, there may even be a possibility for automated translations. But for now, there are various ways to use other tools to help – here’s an example workflow in the video for the variant with Deepl.

But as so often: under Premiere Pro Tutorials you will find practically all the functions of Premiere explained – it’s worth browsing. And if you want to see the whole workflow especially for Youtube again as a video: Here is a short English tutorial with Youtube SRT as an example.

And here is a long video in which Colin Smith explains everything in detail… But without translation.

And you want to test it?

As promised earlier: If you want to test it quickly on a video, you can send me a file with audio, and we’ll run it through the automatic transcription and send it back to you – if you’ve already exceeded the trial period.

Here is the interface to the DP Pipeline – upload the files via Wetransfer, Dropbox or whatever else you like. Enter your name and the email address to which we should send the srt file (or one of the other export formats). Please note: After transcription, the video file will be deleted immediately due to data protection.

And if you want, you can also take part in the raffle for a 1-year licence in the form, until 30 September 2021.

The post Premiere listens – and types! first appeared on DIGITAL PRODUCTION and was written by Bela Beier.

Houdini can read in external data in various ways. The most comprehensive is certainly a self-written Python parser, but that is not necessary here, because Houdini comes with the Table Import Node as standard. This makes it possible to import tables in CSV format. If you look under the bonnet of the table import node, you will of course find a Python script there anyway. This is one of the reasons why I like Houdini so much: almost everything is open and can be studied or adapted to your own needs.

Import

The CSV format contains values that are separated by commas (CSV = comma separated values). These can be words or numbers. Most spreadsheet programmes can save or export tables as CSV. I use Google Sheets here, and it works great: the table import node initially only creates points, which then contain these values in attributes. I use the attributes:

@word, @pscale, @font.

@word gets the values from column A, which is the first column. In Houdini, counting usually starts with zero, hence Column Number 0. The attribute type is a string.

@pscale is a special attribute specified by Houdini. @pscale stands for Point Scale, i.e. the size of a point. This is Column Number 1, and the type in this case is a float, i.e. a floating point number. I could therefore also have entered 1.5 or 2.3 as values in the table.

i have defined @font as type integer, i.e. as an integer. As I want to use three different fonts, I have entered 0, 1 or 2 as values in the table.

The Table Import can also translate (e.g. Lat/Long to Sphere), but the Translator is not needed here.

The second node is Enumerate. This creates an @index attribute with which the points are counted in order to be able to assign them later. The group type must be changed from “Primitives” to “Points”. The next two nodes are not absolutely necessary, but as I wanted to create other sizes and these should be somewhat random, I first used Attribute Expression to convert the @pscale values 1,2,3 into 1.5, 2 and 2.5. To do this, “Custom” must be entered in the node attribute and “pscale” as the name and “Float” for the type. I use the VEXpression for this: “1 self * .5”.
And of course, I could have entered these values in the table straight away, but I didn’t. The zero named here as “OUT_points_with_attributes” is not necessary for the function, but is part of good style.

Loop it!

Now the first loop comes into play. This is a For Each Point loop. It contains 3 font nodes, one for each font. Firstly, I created a font node for this and added the spare input required later. To do this, click on the cogwheel and select “Add Spare Input”.

This font node was then referenced twice using drag-and-drop with the Shift, Ctrl and Alt keys held down. For the copies, I then held down the Shift and Ctrl keys and clicked on Font to remove the reference link. Alternatively, this can also be done with the right mouse button menu and “Delete Channels”.

Fonts

The most important node when it comes to text is of course the font node. It can generate both polygons and Bezier curves or both at the same time. I use polygons here. I have set the level of detail to 4 so that the tessellation is nice and fine.

Working with Bezier curves is also great, but unfortunately not useful here, as the UV layout node used later cannot process them. Now it gets a little more magical or technical. In the “Spare Input0” parameter, the “../foreach_begin1” node is referenced by drag-and-drop or, if you want and like typing, you can do that too. The text to be created is normally simply entered in the text field, but we want to read it in from the CSV table, which has already been converted into points. Backticks can be used here to execute code.

The command required here is “points”, whereby it is not about several points, but about string values from points. The “s” at the end stands for “string”. The first spare input is addressed with “-1”. The zero after it is the first point, here it is always just one point anyway, as we are looping over the points. Then “word”, because we want to read the @word attribute. Brackets around it and a backtick and that’s it.

As an aside and for the sake of completeness, “Add Text Attributes” should also be mentioned here. It is not needed here, but is very cool for other text gimmicks. If the box is ticked, the attributes @textindex, which indicates the number of letters in the word, and @textsymbol, which indicates which letter it is, are created for each letter or primitive. However, this is an ASCII value, i.e. a number. “f”, for example, is then 102.

The three font nodes are then connected with a switch that reads the @font attribute. I have also created a spare input for this. This time it is the “point” command and not “points”, as only a number is read out here and not a string. So “point(-1,0, “font”,0)”. In this case, “-1” is again the spare input, “0” is the first point, “font” is the attribute and the last “0” is a necessary argument, which doesn’t really make sense here. However, if we wanted to read out a vector, “0” would be X, “1” would be Y and “2” would be Z. This is also only mentioned here for the sake of completeness.

Copy to points

Okay, Houdini! Now we come to the copy to points node. First of all, some information in passing: This node will change a lot in H18 and make actions like this easier. No matter. Pack and Instance is not needed here. Transform Using Point Orientation is also not required here, but can also be on. Copy Points Attributes is used because we still need @index. So enter “index” under Set/To Primitive. Delete everything else.

Now you should be able to see a bunch of fonts on top of each other. Before, there was nothing to see in the viewport except a dot in the centre, at most. Tip: In the “foreach_end1” node you can set to “Single Pass” for debugging, then only one iteration of the loop is executed. And always keep an eye on the geometry spreadsheet, where you can see the values of attributes.
Now we come to the UV layout node, which is used here to distribute the words in a shape. In the picture, I have coloured the shape green. I have left space for the two logos of DP and DIGITAL PRODUCTION. Connecting this shape to the second input of the UV layout node should not be too complex. Of course, the words with the @index attribute go into the first one.

UVs

The UV layout node normally uses the @island attribute to delimit the UV parts from each other and the @uv attribute for the positions of the vertices on the UV map. Since there are no UV coordinates here at all, we replace “uv” twice with “P”.

in Houdini, @P is the position of the vertices. For Axis Alignment we select “none” and for Island Rotation Step we set “No Rotations”. If you want to create a Word Cloud with rotated words, they should either be rotated beforehand or you should select 90°, for example, so that no words are upside down.

Padding is the distance between the shapes to be distributed, in this case words, and Scale should be set to “Largest Packable”. The “Advan…” settings are also interesting. This must stand for Advanced. There are iterations here. Here the node randomly tries different layouts and then uses the layout with the best space utilisation. However, each iteration takes time, and 4 is sufficient for this purpose. Different layout versions can be generated with “Random Seed”.
That’s it as far as the layout is concerned, but now we want it in 3D. To switch to a so-called Packed Workflow, we need an @name attribute with a string, i.e. text content. At the moment, we only have an @index attribute, and these are numbers. We do this with an attribute wrangle node.

Select “Primitives” under Run Over and enter the following VEXpression: “s@name = sprintf(“%d”, i@index);”. This creates an @name attribute, which then contains the number as a string. For more details on this VEX function, please refer to the Houdini documentation, which would go beyond the scope of this article.

Now the whole thing is packed, i.e. each word is now represented by a point. This is practical because it is now easier to select the words, for example, in order to make manual corrections to the positions. Simply use an Edit Node to do this.

Before or after the edit node, I used an attribute copy node to write the attributes from the beginning @font, @pscale, @word back to the points, which now contain the packed geometry. Again, @index is used to assign the source and target points (Attribute to Match). Only @pscale is actually needed, but never mind. If you want, you could customise this even more variably and select words more easily or, for example, extrude them differently depending on the font.
Now rotate another loop and extrude the words. To do this, create a For-Each Primtive Loop and place an Unpack followed by a Pack Node. A polyextrude is placed in between.

Then either drag and drop the “foreach_begin3” into the spare input created or enter “../foreach_begin3”. I have set the parameter “distance_multiplier” to 0.025.
The magic is in the Distance parameter; enter “point(-1,0, “pscale”,0)*ch(“distance_multiplier”)” enter. Now @pscale is used to control the thickness of the letters. Do not forget to tick the “Output Back” box, otherwise the letter will be open at the back.

Finally, a file node to save the geometry. It can then be integrated into the rendering scene. The outline plate behind the words was also created procedurally, unfortunately no longer part of this article. But this much can be revealed: I converted a geo into a volume, more precisely into an SDF/VDB. Then enlarged it as an SDF or made it thicker. Then “vdbsmooth” and converted it into a mesh. An Intersection Aanalysis followed by Polypath, then Resample, then Fit and Resample again, and the outline is ready.

Outlook

I would also have liked to have written something about splines, i.e. curve modelling and editing. This works very well in Houdini, even if it takes some getting used to. I would also have liked to discuss rounded edges. But I ran out of pages – we’ll just do that in one of the next issues.

The post Blergh! Word Art! In Houdini? first appeared on DIGITAL PRODUCTION and was written by Olaf Finkbeiner.

After the recently released Affinity Photo and Designer 1.7 updates, the next major release is now coming. With Affinity Publisher, the development studio Serif is offering a third subscription-free alternative for the market. According to the keynote (below), the number of customers has already doubled to 2 million since last year. As the name suggests, Publisher is geared towards desktop publishing and therefore competes with tools such as Adobe Indesign. What does Publisher have to offer on release?

Affinity Publisher feature overview

With its tidy interface, Publisher has all the usual functions that no software of this type should be without. In addition to document setup, the most important basics are covered with typography, design and layout tools, raster and vector graphics integration, master pages, baseline grid, as well as proper export and import formats. But there are also some exciting additional workflow features.

• Design tools with drawing pen, node tools, Boolean operators, bitmap fill methods, stroke panel, layer effects, automatic alignment,
• Layout tools including double-page layouts, (nested) master pages, customised text frames, text wrapping and wrapping, table options
• Typography features with live preview, path support, styles, OpenType, initials, graphic text, flow options, glyph browser, baseline grid, placeholders and more
• Graphics import for linking, embedding and editing PSD, AI, PDF, JPG, TIFF, PNG, GIF, EXR, DNG, EXR and Affinity file formats
• Colour management with support for RGB, LAB, CMYK and grayscale colour spaces, Pantone, ICC colour management profiles, Smart Color Picker and more
• Adjustment layers for adjustments such as blur, shadows, gradation curves, glow and much more
• Performance with up to 60 fps, real-time preview for object transformations, gradients, effects, font adjustments, aperture modes and more
• Export to PDF, PDF/X, PNG, JPG, GIF, TIFF, PSD, SVG, EPS, EXR and HDR
• Additional features such as a history slider, transparency tool, asset management, smart colour picker and more.

A detailed overview of all functions can be found in the technical data sheet.

For all those who are concerned about compatibility with old templates: It looks like Publisher can’t convert or output InDesign documents yet, but AI files can be imported with PDF Stream and rebuilt accordingly.

StudioLink – Workflow Integration

One of the most exciting workflow functions is“StudioLinK“. With Affinity Publisher, you can directly access the features of the other two software tools in order to use photo editing from Affinity Photo and the vector tools from Affinity Designer. You can see exactly what this looks like in the keynote video above.

Availability

Affinity Publisher is available for Windows and Mac – an iPad version is also planned. The software currently costs €43.99 (20% discount), the regular price is €55. For an introduction to the range of functions, you can watch a selection of over 50 tutorials on the official website.

Further information: To the Serif website and the technical data sheet

The post Affinity Publisher released first appeared on DIGITAL PRODUCTION and was written by Bela Beier.