So that’s over and Solaris has grown up together with Karma XPU, the new hybrid renderer. Time to take a look at it!

Although I had experimented with Karma and Solaris (LOPS) time and again, I didn’t want to make the switch until it had fewer teething problems. That was the case with Houdini 20. I also wanted to build up my scenes in Houdini now and then open them up in Omniverse and Marmoset and see if that works. But more on that later…

Karma

There are a few new features in the Karma renderer. The most important one is that the XPU of the graphics card renderer is now out of beta. Many other things – for example dispersion, absorption, nested transparent materials (dielectrics), material blending, geometry lights (not for volumes), rounded corners, hextiling etc. etc. don’t really need to be explained in detail as they are not really innovative – but they are also very useful and complete Karma. An important function in Karma CPU is to be able to set the samples per geo and thus shorten the render times easily and sometimes massively. Unfortunately, it is not yet available in XPU and you have to make do “old skool style” with compositing and tricks.

Solaris (LOPs)

Anyone who has ever worked with a client/art director or director on the lighting of a scene knows what this means for the 3D artist – stress and a lot of organisation. The respective change requests have to be documented somehow, e.g. with screenshots of the parameters or with saved scenes or with copies of lights or their positions. A great deal of discipline is required to ensure that the overview is not lost. But there is now a remedy for this in Solaris. The significantly improved Render Gallery in Solaris is worth its weight in gold, because the status of the scene, e.g. the settings in the LightMixer, is saved with every snapshot and the images in the gallery are saved together with the Houdini scene. They can of course be named and tagged (keywords) and then filtered according to the tags. If a photo agency or the customer works with image numbers, these can be assigned to the snapshots as names and/or tags. A dream come true. This is a real workflow enhancement and normally only part of a pipeline in a larger studio.

Solaris – UI

Solaris is there to provide all the functionalities that are needed when a scene is assembled from geometry, materials, lighting and this should be separated from the geometry creation. It is therefore primarily about a user interface to USD. It is therefore only logical that a lot of improvements have been made to the user interface and a special Solaris LookDev desktop has been added. This then displays the Render Gallery, for example.

Viewport

In addition to the viewport, there are new lighting functions: “Disable Lighting” – switches off the light, e.g. to inspect self-luminous materials. “Headlight Only” – switches off all lights and only displays a direct light. “Dome Light Only” – switches off all lights except for the dome light in the viewport. “Normal Lighting” – displays the current lighting situation The interaction with the light mixer and the very convenient options for positioning lights makes illuminating the scene with Solaris very comfortable.

Render statistics

If you want to render, you want to know why it takes so long or where there is potential for optimisation. The RenderStats, which provide information on memory consumption, render time and much more, help with this. Pretty cool! But unfortunately, some of the information can only be viewed with an HTML viewer, which is cumbersome and annoying. There is also a heat map that shows where a lot of rendering time was used in the image. The metadata is saved in the EXR as JSON and can therefore also be used elsewhere. The UI is still in need of improvement with regard to RenderStats, but as this is the first incarnation of Render Stats, this can be forgiven and we look forward to the next versions.

From SOPS to LOPs

Working in Solaris has generally become much easier and more accessible for freelancers over the years. However, there are also areas where a lot of new things need to be learnt in order to create typical Houdini scenes with many objects or many instances in Solaris. There are two new nodes in H20: The New Merge Point Instancer LOP. In a point instancer prim, each point in a geometry is replaced at view or render time by an instance of the geometry of one of the prims with a “prototype” relationship to the instancer. This LOP allows you to efficiently merge multiple point instances. In this way, only the rest positions of the mesh and the points representing the animated transformations of the parts need to be saved. This means that less space is required on the hard disc. A new geometry clip sequence simplifies and speeds up the saving of valuable clips. With Value Clips, you can split large amounts of data across multiple files. This node should be your first choice when working with value clips.

The USD Scene Graph and Component Builder

For the scene graph to look so nice, the geometry must have attributes that are then used to sort the geo in the graph. The “@Name” attribute is the basic version and the minimum, so to speak. It is better to work with “@path” attributes which, if available, also overwrite “@Name”. So not like in other 3D programmes with drag and drop or something like that. However, it is even better and recommended to use the new “component builder” which takes care of all USD attributes and also the linking with materials. If you do this properly for all assets, you can export the scene as a USD scene and open it in Nvidia Omnisphere, for example.

Rendering

The Clone Control Panel looks almost like the Render Gallery described above. However, it is “only” the first step towards a “Multi-Shot and Multi-Asset Management System”. Nevertheless, it is already impressive and shows where the journey will take us. Anyone who has ever had to create a lighting set-up that looks good from several camera positions at the same time or during a tracking shot will really appreciate this feature, as it allows several render jobs to be running at the same time. The number of cores to be used in each case must be specified beforehand. The network and XPU – i.e. the graphics card – can also be selected. However, this is not limited to different cameras or frame numbers. Different render settings, visibility of objects, lights or whatever else is required can be compared and assessed interactively. This is still a little cumbersome, as the LOPs network has to be built accordingly. BUT so cool and really innovative.

Talking with SideFX

Besides playing with the new tools, DP hopped on a video call with the Karma (and Mantra) developers at SideFX, to get an idea where it is going…

Mark Elendt is a senior mathematician at SideFX. Mark has been with SideFX for over 32 years and worked on a lot of different parts of Houdini. He actually started working on Prisms, which was the product before Houdini. Marks passion is rendering – he wrote the Mantra render engine for prisms and rewrote Mantra for Houdini. And now he is working on Karma, the new flagship renderer from SideFX.

Brian Sharpe is a senior rendering deve­loper at SideFX. He has been in the graphics industry for 25 years, and spent a long time in computer games. He is at SideFX now for six years and working on the Karma XPU renderer.

DP: Karma XPU is marketed as a hybrid renderer. What does that actually mean? What is hybrid about Karma?

Mark Elendt: There are a lot of different purposes for rendering. You‘ve got the film production which has to deal with hundreds of millions of polygons and very complicated shading networks. But you have to provide all the way down to motion graphics, which need a really, really fast turnaround on renderings, or you‘ve got the various different things interior design, architectural design, even scientific visualization. Every renderer has to deal with different types of scenes. We came from Mantra, which was a real workhorse of a renderer – not really fast, but it was very flexible. So we have worked on a new renderer called Karma. Karma actually has two different engines in it. A CPU engine, which deals with very large scenes, but it also has the XPU engine, which can also deal with large scenes and it is geared for faster turnaround and more flexible use of hardware. The more you want to harness the GPU for speed and power, the more restrictive your renderer has to become. Mantra was very flexible. People could write shaders that could reach over to other pieces of geometry and do heaps of wacky stuff. Reach all the nooks and crannies of Houdini and do really powerful stuff.

Brian Sharpe: But then, as you start moving to a more efficient renderer, such as Karma CPU and then Karma XPU which harnesses the GPU so much more, the rendering architecture becomes a lot more rigid and people find they don‘t have so much flexibi­lity, but yet so much more speed. There‘s a trade-off. But we knew we needed to harness the GPU cores and get this really fast performance from them. Karma XPU can harness that, but it‘s a little bit more rigid and a little bit less flexible. But then we have Karma CPU which can still reach into all the nooks and crannies of Houdini to do powerful stuff and access the VEX language, but it‘s CPU only. XPU is a hybrid renderer that just views any sort of CPU or hardware on the machine as potentially executable, So it looks at any kind of GPU and says right, I can use that for rendering. And it also looks at the CPU says right, can I use that for rendering as well. And then it uses both, all the power on the machine to do rendering. It means that you‘re really maximizing performance, but it comes with a lot of benefits as well. For example, if you‘re saying it‘s too big to fit on the GPU memory that you can still keep rendering using only your CPU. So that‘s a very powerful thing.

DP: For me a hybrid renderer would be renderer that is using GPU and CPU at the same time and not using the CPU as a fallback? Maybe sharing the RAM or sharing the memory and doing some clever stuff there – and it would not create different results, even if the GPU is running out of VRAM Is this the case with Karma?

Mark Elendt: Karma’s code is written for the CPU and for the GPU. We find that when you work collaboratively, when you‘ve got both devices working on the same image, the CPU may only do 20 percent of the work, the GPU does 80 percent of the work. In some cases, the CPU only does 10 percent of the work while the GPU does 90percent, but in some cases the CPU does more. Right? It‘s a balance.

DP: And it balances itself and it takes care of that?

Mark Elendt: Yeah. You know, one device will do more work than the other.

DP: Depending on the shaders and what it does?

Brian Sharpe: Certain features like subsurface scattering doesn‘t run as efficiently on the GPUs…

Mark Elendt: If the CPU is only doing 20 percent of the image. And your image takes a minute to render and all of a sudden the GPU can‘t work anymore. Mm hmm. Well, the CPU seems really slow because the GPU was doing 80 percent of the work, so now it‘s only doing 20 percent of the work. The render is going to take five times as long. But we don‘t actually decide what‘s better. We just make every device work at full speed. Both devices can do subsurface scattering. But the GPU is not as efficient during subsurface rendering, so the CPU will take up more of the rendering time. It‘ll contribute maybe 30 or 40 percent of the render instead of only 20 percent. So both devices are working full out to generate the image. But some devices are better at some things.

DP: What about distributed rendering, as in many machines working on one image. Is that something that Karma CPU and XPU can do?

Mark Elendt: XPU is built to have multiple devices. If you‘ve got three GPUs, three graphics cards or four graphics cards in your box and one CPU, you can actually have five XPU devices working collaboratively on the image. It‘s not just one GPU and the CPU, it‘s as many GPUs as you have and CPU, but we can extend that in the future. We can then have other devices that are not necessarily just GPU or CPU devices. Currently it‘s not able to do distributed rendering, but the engine is prepared for it. XPU works by using USD underneath for the scene description. So XPU will take the USD buffers and send the required data to the device. So, if you‘ve got a teapot geometry, it will send that geometry to the GPU and to the CPU device. It might send it over the network to the device and then all the devices have the copies of the data and work together collaboratively to build the final image.

DP: Currently it is a Cuda implemen­tation?

Mark Elendt: Cuda has a lot of the production features that we‘re looking for. High quality programmable shading with C shaders. They‘ve got a very good development toolkit, but that doesn‘t hinder us from developing other devices. Whether we develop a Vulkan device or a metal device, we have the architecture that we can build these new devices using different technologies, and it will all work together on different platforms.

DP: But still also currently not executed?

Brian Sharpe: Currently its CUDA and Optix, whereby Optix is the library toolkit and CUDA is the language you want to plug into.

DP: Why not OpenCL?

Mark Elendt: We‘re leveraging a lot of the technology that comes with Optix and the Nvidida drivers. So it makes it a lot easier for us to have some stepping stones, which make it easier for us to implement.

Brian Sharpe: We can gain access to the RTX hardware. So that‘s the ray tracing hardware on Nvidia GPUs via Cuda OptiX. But they haven‘t exposed that to open CL yet. So if we wrote an openCL device, all the ray tracing would be in software, so it wouldn‘t be as performant. For now on video cards, it looks like we‘ll be sticking with Optix Cuda, but for other GPUs we will see.

Mark Elendt: We are trying to get to the stage where people are already using Karma XPU for production, but we‘d like to get to the stage where everyone feels comfortable and safe using Karma XPU for real production. Once we get there, then we can start expanding into other devices, whether it‘s distributed rendering or Metal or Vulkan or Opencl. So a lot of ways we can go in the future.

DP: What about USD?

Mark Elendt: There is a lot going on in USD development. At SideFX, we started on a look dev project called Solaris. When we started to work on Solaris, we evaluated the possibility of having Mantra ported to import USD and we realized that Mantra was getting a little bit long in the tooth. The architecture was not as flexible as we wanted. At that point we decided it was time for a new renderer and lets call it Karma. That will work straight natively with USD. Karma has got a lot of the rendering heri­tage that mantra has, but it‘s also streamlined and architected for more modern technologies.

Brian Sharpe: The architecture of XPU lives and with the same design principles as Karma CPU, it just chooses to execute the stuff on GPU. But there are things that KARMA XPU can‘t do, such as running things like VEX, so I decided just to leave it on the shelf for now and just concentrate on getting the MaterialX version going.

Mark Elendt: A lot of the technologies that KARMA is building on are open source and part of the Academy Software Foundation, the ACLU group. USD is not part of ASF yet, though they do have a working group. But again, it‘s a relatively new project. It‘s been used in ILM for four years and getting larger exposure recently. Autodesk is doing a lot of work on MaterialX, AMD has a big MaterialX gallery that you can download directly in Houdini20 and USD has a built in MaterialX support. So all of these software libraries work well together. Leveraging that means that users will be more familiar with the concepts. Well, what we‘re finding is MaterialX has a lot of the features that we need and is very general and flexible, but there are certain things that it doesn‘t have support for yet. We‘re working with the MaterialX team, we‘re pushing nodes up to them and working with them on building and extending and making MaterialX more flexible and more accessible to everybody.

Brian Sharpe: It‘s I‘ll add one thing to that, what we‘re finding and one really good reason and one really good benefit that we‘ve got Karma XPU. And also our CPU engine under the one umbrella, which is Karma is if someone‘s working within KARMA XPU and they‘re doing all their work and their are using MaterialX, they really do find this a certain feature that they just cannot exist, such as casting an arbitrary ray from a shader or something there is always a fallback to our Karma CPU render and do all that stuff in VEX that we really have to do. So that could be in the middle of production and they‘re not going to at that moment look out for another renderer. They keep going because there‘s a big safety net in Karma CPU.

Mark Elendt: And that‘s always been a philosophy of SideFX. Which is not to make users hit any brick walls. You want to be flexible enough that you can do anything, with Karma. We‘ve sort of pulled the reins in a bit, but we still have those back doors that if you are really savvy where you really are stuck, you can get out of the problem and solve your solution.

DP: Any plans for a sort of real time incarnation of Karma?

Mark Elendt: No. Karma is always going to be a path tracing renderer. There are path tracers out there that are real time, but they are specifically written for real time. So, they will send very few rays and rely on denoise. Karma is really intended to be a more general-purpose offline renderer. You get really fast feedback and sometimes images can take just seconds to render, but that‘s not real time. It‘s interactive.I think you might be able to look for something coming in the Houdini viewport that the real time rendering in the viewport using OpenGL or Vulkan the Raytracing support. The two types of rendering are getting closer together. We might end up with raytraced reflections in the viewport or some raytraced, soft shadows or something like that. But you may not get the subsurface scattering and the high-fidelity kind of lighting that you would get with Karma.

DP: What about denoising?

Mark Elendt: I don‘t think that we‘re going to spend a lot of research time on denoise theories at SideFX, but there are a lot of public denoisers and even proprietary denoisers which we can leverage inside of the architecture. We have integrated the OptiX denoiser and as well as the Intel Open Denoiser into Karma and we are also working with Nvidia. We feed back data to them and discuss denoising issues, but we let them do the lion‘s share of the heavy lifting.

Brian Sharpe: What we have in Karma CPU already is the automatic convergence mode, where Karma is smart about where it sends the rays. Where an area of noise is Karma hammers it with more samples.

DP: Is there anything that you think you want to tell the readers about the future of XPU?

Brian Sharpe: One thing needs a bit more explanation, the CPU running out of m­emory. And in extreme cases it can. But we‘re currently working on out-of-core rendering for you, so it doesn’t mean it‘s going to magically work on a one gigabyte scene or something, but it‘s going to get a lot better with memory going forward!

The post USD, Solaris and Karma in Houdini 20 (and elsewhere…) first appeared on DIGITAL PRODUCTION and was written by Olaf Finkbeiner.

And here’s the beta disclaimer: This article was written before the release of H19.5, so something may have changed..

Rigid Body Dynamics – RBD

It’s getting down to the nitty gritty! RBD is what Houdini is best known for – rigid body simulations. Usually, but not always, to break something. Many of the RBD nodes have been given a tidy new user interface. The parameters are now organised in logical tabs and panels. Rarely used parameters have been “hidden” so that they are not in the way and distracting.

Even more important, however, is the new workflow. Parameters should no longer be set in the solver but in separate “Configure” nodes (e.g. “RBD Configure”). Import and export nodes have been have been adapted to each other and are structured in the same way as the I/O nodes of other solvers, for example the Pyrosolver. The other I/O improvements are not so obvious, as they are more about speed improvements under the bonnet and better handling of inputs and outputs.

RBD I/O 2.0

In Houdini, data, be it geometry or animation, is always outsourced wherever possible. Of course, this also applies to simulation data, proxies and constraints. Geo, proxy geo and constraints can now also be written together in a single file. “RBD I/O” makes it easier to name this data, select the path and folder and create version numbers. In the past, Houdini users had to build this functionality themselves or the Studio pipeline developed tools for it. The file name and path are therefore made up of parts and these can also be variables. In Houdini, $HIP is the path where the Houdini Scene *.hip is saved. All this already exists here and there outside of RBD in Houdini, now this naming workflow is also implemented for RBD. Furthermore, the fourth input of the node is now exclusively for Simulation Points (instead of Geometry and Points in previous versions).

DOP I/O – DOP Import 2.0

The new 2.0 version is of course faster and has a tidier UI. The improvement in speed is particularly noticeable in large scenes, such as crowd animations. And here, too, there is now the Quick Setup drop-down menu in the UI with 2 selection options: “Transform High-Res Geometry” and “Cache Simulation”.

RBD Material Fracture: Performance and memory improvements here too. If, for example, two pieces of wood that are in a group (selection), but also 100 units that are far apart, were to be transformed into wood splinters, this took ages. Now such a scenario is much quicker.

RBD Transform: “RBD Transform” was added in response to numerous user requests and makes it easier to place RBDs. It is similar to the normal Transform Node, but has the three inputs and outputs required for RBD (High-Res Geometry, Constraints, and Proxy Geometry). RBD Match Transforms: With RBD Match Transforms, manual adjustments to individual fragments of an RBD simulation can be included in the stream again. UX improvements: RBD Constraints from Curves and RBD Constraints from Lines
now have a nice HUD.

Pyro & FLIP Fluids

FLIP solvers are hybrid solvers that use both particles and volumes for simulation. These FLIP fluids are not new per se – but they are now also available in the SOP context. That’s awesome!
SOP-based FLIP fluids enable fast, simple setups that no longer require sending back and forth between SOP and DOP. In the past, this has made many things technically difficult to understand and cumbersome. This is now a thing of the past and – I repeat – very cool!
In addition, new methods have been introduced that simplify the addition and removal (source/sink) of particles. The simulation container, i.e. the area in which the simulation is to take place, can not only be a box, but also a cylinder or a sphere. Even deforming animated shapes are possible. There are also presets for temperature, viscosity, beach waves, oceans and melting rubber animals.

Pyro

The “Pyro Sources SOP” can generate pyro instances which are distributed to scattered points – for example for very large explosions, dusty storms or forest fires. Compared to before, these can now be different sources at the same time. The timing of the ignition and the transformation can be adjusted. The whole thing can also be realised with a special packed workflow, the Pyro Source Pack. Pyro Spawn Sources and Pyro Source Instance Operators enable an even more flexible workflow. The new Pyro Spawn Sources and The Pyro Source Instance SOPs create and instantiate spawn points. And Houdini Pyro Simulations have GPU-accelerated noise.
KineFX and Channels (CHOPs)

KineFX, the rigging and animation framework and toolset freshly introduced in Houdini 19, mainly gets many detail improvements. But also new functions in the area of UI and user-friendliness. I/O is also a big topic here again.

Animation

In the area of character animation and rigged animation with skeletons, H19.5 contains a huge number of detailed improvements to the MotionClip SOP, the MotionClip Extract SOP, the MotionClip Pose Delete SOP, the Rig Pose SOP, Rig Match Pose SOP, etc. Some of the improvements here with a little more detail.

MotionClip SOP Viewport State

SOP nodes that output motion clips can now visualise these clips to display movements or poses. A freely selectable colour gradient can be chosen to represent time. The current frame is also highlighted (or “highlighted”? What a word!). This makes it easier to edit rigged animations, be it motion capture data or animations created elsewhere. The whole thing is a so-called ViewPortState, which shows the direction in which Houdini is heading in terms of UI and viewport. It is becoming more and more user-friendly and the basis for this is the development of viewport states.

Frame rate change and key position handling

Frame rates and animation keyframes are closely related. It is annoying if the frame rate has to be changed later for whatever reason. Should the existing keyframes remain at the keyframe number or at the same time? And what happens if there is no frame at all? Should it then be snapped to the next one or should the keyframe be on a subframe? All of this is now easier to set. Any changes to the timings must now be confirmed explicitly and are not carried out automatically as before.

Animation Channels – CHOP

The good old CHOPs also get something new: The Pass Filter CHOP, which filters audio as a control signal for animations, now also gets a Butterworth filter, which is particularly steep and thus enables a more precise frequency separation.

Procedural Rigging

The new Joint Capture Paint SOP replaces the old Capture Layer Paint SOP in the KineFX Capture workflow. There is a Rig Tree in the UI, and some new viewport states. A new Point Capture SOP, Character Blend Shape Channels SOP. Since it would go beyond the scope of this article, here is a list of the Procedural Rigging SOPS that have been improved: Attach Joint Geo SOP, Joint Capture Biharmonic SOP, Character Blend Shapes Add SOP, Character Blend Shapes Extract SOP, MotionClip SOP, MotionClip Extract SOP, MotionClip Pose Delete SOP, Rig Pose SOP, Rig Match Pose SOP, Blend Shapes SOP, Rig Doctor SOP, Orient Joints SOP, Visualise Rig SOP and if I’ve missed any I apologise. Procedural rigging is obviously an important topic. Unfortunately, it’s a bit neglected in this article, but we’ll make up for it!

Input/Output

A lot is really happening with regard to I/O in the H19.5 release. I/O is getting the attention it deserves. There is a special character I/O SOP with motion clip support. FBX support for characters has been improved. “Dem Bones Skinning Converter” is not a band name, but a converter to turn animations with geometries that have no topology changes into bones animations. The USD Skin Import SOP, Mocap Stream SOP have been improved. And something with Qualisys.

Still water is new! The Shallow Water Solver

The new Shallow Water Solver SOP is another small highlight in H19.5. What was that? “Still waters run deep” is not true here. Still or little moving water, i.e. without breaking waves or splashes, can now be displayed in Houdini as a 2.5D heightfield (like the terrains). In the example, the water flows down a mountain. Sources and outflows are defined by masks. And moving objects can also displace the water. The “Shallow Water Solver SOP” is not suitable for complex water, but that’s what Vellum and Flip are for.

Oceans

On most computers, Houdini uses the FT implementation from Intel’s MKL library in the “VolumeFFT SOP” and Ocean_sample VEX function. This is much faster at 4K Ocean Spectrum resolutions. And Ocean Rendering in Karma CPU is made possible. Yeah..

Solaris

Solaris is Houdini’s scene layout, lighting and rendering tools. Solaris is based on the Universal Scene Description (USD) framework from Pixar. Several interface improvements, for example in the render settings and some new nodes, ensure that Solaris, which was introduced in H18 (end of 2019), is slowly emerging from the early adopter/ beta feeling. Look development, material library, rendering, caching and import and export are also being optimised. The new Houdini Procedurals Hair LOP generates rendertime hair from guide curves and skin prims. And the Houdini Preview Procedurals LOP enables interactive previews of the procedures.

Other new Solaris Nodes are:
Reference 2.0 LOP, Edit Prototypes LOP, Isolate Scene LOP for Solaris stuff
Light Filter Library LOP for Karma Light Filter Shaders and MaterialX nodes.
Motion Blur LOP, File Cache LOP, Match Size LOP – Matching and centring of prims and models.
Cache 2.0 LOP – efficient caching of subframe samples.
Coordinate System LOP – coordinate systems.
Karma Cryptomatte LOP – Cryptomatte AOVs for Karma.
Karma Ocean LOP – Rendered Ocean Spectra and foam with Karma CPU.
Scene assembly and layout

Again some detail improvements: The Layout LOP now also has a stack and a paint brush. Houdini is on the way to providing interactive tools for set dressing and scene layout. Houdini wants to be not only procedural but also art-directible and artist-friendly. It will be!

Karma

Karma is Houdini’s own renderer, which works together with Solaris and will probably eventually replace Mantra. Some of the new features have therefore already been mentioned in the Solaris section. As you would expect, Karma has become faster. The highlight with regard to Karma is a first version of the “Karma User Guide”, with tips and tricks for optimising rendering with Karma. As with all new help and user guides, I’m really impressed by the detail and illustrations. This is how documentation works!

Shading

The new features in the shading toolset are quickly listed: Principled Shader VOP and Karma Nested Dielectrics VOP now support dielectric materials in MaterialX/Karma. The Karma OCIO Colour Transform VOP transforms colour manually, and Karma Ray Import VOP imports ray variables for MaterialX shading graphs. Karma Volume VOP combines VDF and EDF volumes Karma-friendly, and last but not least Karma Rounded Edge VOP render-time edge rounding and Karma Curvature Surface surface curvature – but both only work with
Karma CPU.

MaterialX

MaterialX is the material counterpart to USD. Not much new and Lama (Material Layer) is unfortunately not yet supported. MtlX Color Correct is a new wrapper to make the MaterialX Colour Correction Nodes more user-friendly. MtlX Facing Ratio is the new node to create a nice soft falloff. The version numbers are now: UsdMtlX update USD 22.05, MaterialX library version 1.38.4.

What’s new PDG/TOPs

PDG (Procedural Dependency Graph) can be used to distribute and process tasks. This can be, for example: rendering, simulation, geometry and landscape generation, Python scripts, connections to other applications (via API) such as render manager, project manager etc. The whole thing works both locally and on other computers and in other apps such as Photoshop or Maya. Python always works anyway, but many things are already well connected.

The Procedural Dependency Graph or TOPs (Task Operators) is certainly the most “exotic” and technically challenging part of Houdini. TOPs was introduced in H17.5 (March 2019). In the meantime, it has become an indispensable part of larger and smaller pipelines. The PDG options for working with USD data have been improved. Error messages for attribute evaluation and the PDG API have been improved. I won’t go into details for reasons (DP is not a programmer magazine and I only understand this to a certain extent).

Global attributes and two nodes are also new: Attribute Promote TOP and Attribute Reduce TOP. TOP Schedulers: The connection from PDG to Deadline, Tractor or even Houdini’s own render manager HQueue, get several “enhancements” in H19.5. TOP Partitioners have been improved in terms of the use of arrays and the partitions can now be sorted by name. The JSON input TOP has been extended.

PDG Services

Crowds

There are now new versions of Agent Look At DOP and Agent Look At Apply DOP. The agents now also track point clouds or other agents. The attention span of the agents can be defined – how interesting a target is can depend on the distance, the relative speed or whether it is in the field of view of the agent. The rigs can now be controlled by the agent.

Agents

Agents can now have a “Rest Pose” that is used when animations do not move every joint. The Crowd Solver DOP can now handle animation layers. And other detail improvements.

Vellum

Vellum shape matching simulations are another small highlight. The shape is first filled with spheres of different sizes. The constraints of the spheres are set very hard. These sphere objects are then used to simulate. This is ultra-fast, as they are only spheres. At the end, the sphere objects are then exchanged for the geo again, hence “shape matching”. In order to run these “shape matching” simulations on the GPU, Nvidia Ampere graphics cards (from 30 series) are probably required. Otherwise it has to run on the CPU..

Vellum I/O 2.0

The Vellum I/O 2.0 node is now like the Pyro, FLIP fluid, and RBD I/O nodes in terms of UI structure, parameters, and basic functionality.

Static topology

In the Vellum Constraints SOP there is the new Static Topology parameter. This makes them static, i.e. they will not slip, break, be removed or welded. This prevents large and dense objects, such as high-resolution tetrahedra, from having to be recalculated (recoloured), even though this would not be necessary.

Vellum Attach Constraints SOP

The new Vellum Attach Constraints is an interactive tool for editing geometry and stitch constraints. Lines can be attached interactively between fabric and target geometry. Very similar to the RBD Constraints from Lines node.

Resolution Independent Tetrahedron

Then there is the fact that the tetrahedrons have rehearsed a rebellion and can now change their resolution without changing the simulation much.

Modelling

I had already announced it in the last Houdini release article. Houdini is becoming more interactive. Both TopoTransfer and Curve, the curve tool, are getting significant improvements. For example, better operation with a stylus/pen input as far as Topotransfer is concerned. The Curve tool can now add Orient attributes to points and has better interaction in Bezier with two curves. In addition, rounded edges are much easier to set, and keyboard shortcuts and radial menu are optimised. “Convert Line” can now convert edges into curves, to be found in the “Curve from Edges” shelf.
And there is now a primitive for 2D spirals and 3D helixes that leaves nothing to be desired. Both archimedean and logarithmic spirals are created, left or right turning, with all necessary attributes.

Other news

Python 3.9! And Houdini now supports native fullscreen mode for the poor macOS users. Volumes can be illuminated by environment lights in the viewport. And 15 nodes are now verbified i.e. multithreaded, e.g.: dopimport::2.0, fit, groupsfromname, lidarimport, polywire, pointcapturecore and some kinefx nodes.
And last but not least, there is something completely new: the “Dashbox – An interface for finding things”. This new, nerdy search is to be integrated into other existing search functions in the future (i.e. not yet)… At the moment it is self-sufficient, but a quick keyboard control à la Everything/Autohotkey will certainly be helpful when it is in full use.

The post Houdini 19.5: I/O – UI – RBD -FLIP – PYRO – UX .. first appeared on DIGITAL PRODUCTION and was written by Olaf Finkbeiner.

In addition, Houdini, which is of course still mainly procedural, is becoming more and more interactive. The topic of interactivity will be a recurring theme throughout this article. In addition to interactivity, more and more high-level tools are being added. What I mean by this is best explained at this point, as it will feature frequently in the article. One of the Labs tools will serve as an example: The new Tree Generator or Tree Toolset, which was released in mid-December, well after the 18.5 update. A high-level tool is a special tool for a very limited purpose, such as creating trees, albeit somewhat less flexible, but without a great deal of learning effort and ready for immediate use. Previously, the willing Houdini user could also create trees, but they first had to build their own tool or get started with the existing low-level tools. So the philosophy of only providing a construction kit with low-level functions from which users first have to create tools is gradually changing. But don’t get me wrong, this is of course still possible, and the new high-level tools are generally open source and can therefore be customised.

Another term that requires explanation are the so-called shelf tools in Houdini. Shelf tools are, for example, the pyro setups, the modelling tools and the top margin bar. Next term: Python Viewer State Handles. These were introduced in Houdini a while ago, but 18.5 is the first release in which they are increasingly used. The technical development seems to have been completed so far. What is still missing, however, are high-level tools to make it easier for users to create user interfaces in the viewport with the Python handles, because that is what they are there for.

Solaris

The Solaris network is also able to display shots. This means that different versions of the scene can be created depending on the frame number. What I find really impressive is the Solaris Frame Buffer: when saving previews, not only is an image saved, but also the settings of light setups, so that you can restore different versions of a light setup. The Solaris Edit SOP has also been completely redesigned and has a history of the individual edit steps. The edits are displayed as a list and can be undone individually. So much for Solaris, or perhaps this: Solaris is very complex, requires some familiarisation and is still anything but intuitive despite the new tools.

Solaris works with the new Karma Renderer (still in beta) and other USD-compatible renderers, such as Pixar’s Renderman. Mantra, the old Houdini renderer, does not work like this. Karma is still in beta status, as there is still some work to be done before Mantra can be fully replaced. In 18.5-Karma, hair is supported for the first time, but probably not yet satisfactorily. A Curved Edges Shader and Physical Lens have also been added. In addition, a tech preview of Karma on the GPU, a double beta so to speak.

KineFX in SOPS

KineFX is a completely new procedural rigging and retargeting toolset. Character rigging was an outdated feature of Houdini and not exactly intuitive. An update, or rather a complete overhaul, was long overdue.

The tool and workflow includes over 60 new nodes. The bones and joints are now normal Houdini Geometry. This has the great advantage that all other tools can now modify the skeleton, and conversely the KineFX tools can also be used for other modelling or animation tasks. It’s not just about characters here, machines or other things like plants can also be edited with the tools. The whole thing is already very extensive, but is only the beginning of a full character animation pipeline that SideFX has announced.
Here are a few keywords about KineFX: motion clips, FBX import, foot placement, less Python, new Auto Rig tool, editing of motion capture data (also in real time), agents for crowd animations ..

Geometry

There are also some new features in the modelling area. Topo Transfer makes it possible to adapt a template mesh to a scan mesh. To do this, some source and target points are connected. The rest is done by the Topo Transfer Node. The cool thing is that the meshes do not have to have the same pose if they are characters, for example, and even additional features in the scan are warped. Poly Bevel is the next tool to receive a major update. The fact that something so basic is still being updated (not only in Houdini) shows how much innovation is still possible and necessary in the field of 3D software. Poly Bevel is now, as expected, much more interactive and intuitive. Adding and removing faces now requires far fewer clicks and no more constantly closing the selection to see updates.

The third tool is a high-level tool, and SideFX has named it Chain Operator. Here you should not only think of chains, but more of sequences of geometries, e.g. different stones. As an example, SideFX has taken chairs in a cinema, one with the seat folded down, the next slightly damaged, and so on. In this way, the entire cinema can be plausibly seated in no time at all, because rules such as the first and last chair backs can also be mapped with the Chain SOP. Path Deform and Bend are further examples of Houdini’s interactive use of Python Handles. See the magically bent spoons in the screenshot ..

Environment Building

Houdini Conduits & Services

The UE4 plug-in can now handle Unreal Blueprints and is significantly faster. Large terrains can now be shared better with UE4 using Houdini PDGs – world building has been significantly improved. A two-way live link is now also possible. It is even possible to send live MoCap data to Unreal via Houdini. As far as Houdini’s connections to the outside world (conduits) are concerned, the new Houdini Engine Python API and the Houdini web server should also be mentioned, which enable “Houdini as a Service”. I’m curious to see what will be done with it.

Dynamics

In the Dynamics area, the new Attach and Sliding Constrains for FEM are worth mentioning. There is also a new and 25% faster viscosity solver for honey, for example. This is mentioned in one of my favourite channels: Two Minute Papers, in the video “Simulating Honey and Hot Showers For Bunnies!” bit.ly/2_minute_papers_honey_bunny – in the video you can see visually what is possible.

Pyro

Pyro and explosions have been expanded to include a new high-level workflow. Pyro Burst and Pyro Trails generate basic geometry, which is then converted into volumes for the pyro simulation. I was previously only familiar with this as a particle simulation. Here, however, a geometry is generated on an SOP basis, which can then be customised interactively. Pyro Trails generates trajectories of parts ejected by explosions. These trajectories are even deflected by collision geometry. Pyro Burst generates a point cloud with the attributes required for Pyro. Both tools are HDAs and can be extended or customised. They are tools to be taken seriously, and the so-called shelf tools (Fireball, Aerial Explosion, Ground Explosion, Bonfire, Spreadingfire etc.) are fully developed and in many cases sufficient for a production.

The whole thing is garnished by the GPU solver, which enables real-time explosion previews at a rate of 1 to 2 frames per second. Here, too, the focus is on interactive work. Depending on the intended use, these already look very cool and can be completely sufficient under certain circumstances. The included shader for Viewport and Mantra completes the new Pyro workflow.

Labs

Labs tools are developed by SideFX Tech Artists in Houdini and are therefore often very practical. I’ve made extensive use of the now simplified ability to version HDAs or change their namespace to tidy up my HDA mess. The new FBX importer is also very helpful and simplifies everyday FBX imports. For “crazy-pipeline-special” FBX imports, you can always create a customised Python script. Other Labs innovations are: a Road Generator, Mesh Sharpener, Triplanar Displacement, Box Cutter for fast Boolean Modelling, Pick and Place Tool, Compositing: Blackbody, Grid Texture, Demosaic and a MoCap Importer for Axis Neuron and Rokoko (which we will test in detail in a future issue).

Conclusion

And now I still haven’t mentioned everything important! So here it is in a nutshell: Many SOPs are now compilable and can therefore be used in parallel. Hair grooming is significantly faster. Python 3.7 support has been enabled and VEX has a new data type: Dictionaries. 2020 was also a special and strange year for SideFX and Houdini. SideFX employees were unable to attend trade fairs or travel and had to come up with something else. They took advantage of these special circumstances and created a lot of educational material and sample scenes. So much that I will never manage to watch it all, let alone try it out.

The post Houdini becomes hyperactive first appeared on DIGITAL PRODUCTION and was written by Olaf Finkbeiner.

Houdini can read in external data in various ways. The most comprehensive is certainly a self-written Python parser, but that is not necessary here, because Houdini comes with the Table Import Node as standard. This makes it possible to import tables in CSV format. If you look under the bonnet of the table import node, you will of course find a Python script there anyway. This is one of the reasons why I like Houdini so much: almost everything is open and can be studied or adapted to your own needs.

Import

The CSV format contains values that are separated by commas (CSV = comma separated values). These can be words or numbers. Most spreadsheet programmes can save or export tables as CSV. I use Google Sheets here, and it works great: the table import node initially only creates points, which then contain these values in attributes. I use the attributes:

@word, @pscale, @font.

@word gets the values from column A, which is the first column. In Houdini, counting usually starts with zero, hence Column Number 0. The attribute type is a string.

@pscale is a special attribute specified by Houdini. @pscale stands for Point Scale, i.e. the size of a point. This is Column Number 1, and the type in this case is a float, i.e. a floating point number. I could therefore also have entered 1.5 or 2.3 as values in the table.

i have defined @font as type integer, i.e. as an integer. As I want to use three different fonts, I have entered 0, 1 or 2 as values in the table.

The Table Import can also translate (e.g. Lat/Long to Sphere), but the Translator is not needed here.

The second node is Enumerate. This creates an @index attribute with which the points are counted in order to be able to assign them later. The group type must be changed from “Primitives” to “Points”. The next two nodes are not absolutely necessary, but as I wanted to create other sizes and these should be somewhat random, I first used Attribute Expression to convert the @pscale values 1,2,3 into 1.5, 2 and 2.5. To do this, “Custom” must be entered in the node attribute and “pscale” as the name and “Float” for the type. I use the VEXpression for this: “1 self * .5”.
And of course, I could have entered these values in the table straight away, but I didn’t. The zero named here as “OUT_points_with_attributes” is not necessary for the function, but is part of good style.

Loop it!

Now the first loop comes into play. This is a For Each Point loop. It contains 3 font nodes, one for each font. Firstly, I created a font node for this and added the spare input required later. To do this, click on the cogwheel and select “Add Spare Input”.

This font node was then referenced twice using drag-and-drop with the Shift, Ctrl and Alt keys held down. For the copies, I then held down the Shift and Ctrl keys and clicked on Font to remove the reference link. Alternatively, this can also be done with the right mouse button menu and “Delete Channels”.

Fonts

The most important node when it comes to text is of course the font node. It can generate both polygons and Bezier curves or both at the same time. I use polygons here. I have set the level of detail to 4 so that the tessellation is nice and fine.

Working with Bezier curves is also great, but unfortunately not useful here, as the UV layout node used later cannot process them. Now it gets a little more magical or technical. In the “Spare Input0” parameter, the “../foreach_begin1” node is referenced by drag-and-drop or, if you want and like typing, you can do that too. The text to be created is normally simply entered in the text field, but we want to read it in from the CSV table, which has already been converted into points. Backticks can be used here to execute code.

The command required here is “points”, whereby it is not about several points, but about string values from points. The “s” at the end stands for “string”. The first spare input is addressed with “-1”. The zero after it is the first point, here it is always just one point anyway, as we are looping over the points. Then “word”, because we want to read the @word attribute. Brackets around it and a backtick and that’s it.

As an aside and for the sake of completeness, “Add Text Attributes” should also be mentioned here. It is not needed here, but is very cool for other text gimmicks. If the box is ticked, the attributes @textindex, which indicates the number of letters in the word, and @textsymbol, which indicates which letter it is, are created for each letter or primitive. However, this is an ASCII value, i.e. a number. “f”, for example, is then 102.

The three font nodes are then connected with a switch that reads the @font attribute. I have also created a spare input for this. This time it is the “point” command and not “points”, as only a number is read out here and not a string. So “point(-1,0, “font”,0)”. In this case, “-1” is again the spare input, “0” is the first point, “font” is the attribute and the last “0” is a necessary argument, which doesn’t really make sense here. However, if we wanted to read out a vector, “0” would be X, “1” would be Y and “2” would be Z. This is also only mentioned here for the sake of completeness.

Copy to points

Okay, Houdini! Now we come to the copy to points node. First of all, some information in passing: This node will change a lot in H18 and make actions like this easier. No matter. Pack and Instance is not needed here. Transform Using Point Orientation is also not required here, but can also be on. Copy Points Attributes is used because we still need @index. So enter “index” under Set/To Primitive. Delete everything else.

Now you should be able to see a bunch of fonts on top of each other. Before, there was nothing to see in the viewport except a dot in the centre, at most. Tip: In the “foreach_end1” node you can set to “Single Pass” for debugging, then only one iteration of the loop is executed. And always keep an eye on the geometry spreadsheet, where you can see the values of attributes.
Now we come to the UV layout node, which is used here to distribute the words in a shape. In the picture, I have coloured the shape green. I have left space for the two logos of DP and DIGITAL PRODUCTION. Connecting this shape to the second input of the UV layout node should not be too complex. Of course, the words with the @index attribute go into the first one.

UVs

The UV layout node normally uses the @island attribute to delimit the UV parts from each other and the @uv attribute for the positions of the vertices on the UV map. Since there are no UV coordinates here at all, we replace “uv” twice with “P”.

in Houdini, @P is the position of the vertices. For Axis Alignment we select “none” and for Island Rotation Step we set “No Rotations”. If you want to create a Word Cloud with rotated words, they should either be rotated beforehand or you should select 90°, for example, so that no words are upside down.

Padding is the distance between the shapes to be distributed, in this case words, and Scale should be set to “Largest Packable”. The “Advan…” settings are also interesting. This must stand for Advanced. There are iterations here. Here the node randomly tries different layouts and then uses the layout with the best space utilisation. However, each iteration takes time, and 4 is sufficient for this purpose. Different layout versions can be generated with “Random Seed”.
That’s it as far as the layout is concerned, but now we want it in 3D. To switch to a so-called Packed Workflow, we need an @name attribute with a string, i.e. text content. At the moment, we only have an @index attribute, and these are numbers. We do this with an attribute wrangle node.

Select “Primitives” under Run Over and enter the following VEXpression: “s@name = sprintf(“%d”, i@index);”. This creates an @name attribute, which then contains the number as a string. For more details on this VEX function, please refer to the Houdini documentation, which would go beyond the scope of this article.

Now the whole thing is packed, i.e. each word is now represented by a point. This is practical because it is now easier to select the words, for example, in order to make manual corrections to the positions. Simply use an Edit Node to do this.

Before or after the edit node, I used an attribute copy node to write the attributes from the beginning @font, @pscale, @word back to the points, which now contain the packed geometry. Again, @index is used to assign the source and target points (Attribute to Match). Only @pscale is actually needed, but never mind. If you want, you could customise this even more variably and select words more easily or, for example, extrude them differently depending on the font.
Now rotate another loop and extrude the words. To do this, create a For-Each Primtive Loop and place an Unpack followed by a Pack Node. A polyextrude is placed in between.

Then either drag and drop the “foreach_begin3” into the spare input created or enter “../foreach_begin3”. I have set the parameter “distance_multiplier” to 0.025.
The magic is in the Distance parameter; enter “point(-1,0, “pscale”,0)*ch(“distance_multiplier”)” enter. Now @pscale is used to control the thickness of the letters. Do not forget to tick the “Output Back” box, otherwise the letter will be open at the back.

Finally, a file node to save the geometry. It can then be integrated into the rendering scene. The outline plate behind the words was also created procedurally, unfortunately no longer part of this article. But this much can be revealed: I converted a geo into a volume, more precisely into an SDF/VDB. Then enlarged it as an SDF or made it thicker. Then “vdbsmooth” and converted it into a mesh. An Intersection Aanalysis followed by Polypath, then Resample, then Fit and Resample again, and the outline is ready.

Outlook

I would also have liked to have written something about splines, i.e. curve modelling and editing. This works very well in Houdini, even if it takes some getting used to. I would also have liked to discuss rounded edges. But I ran out of pages – we’ll just do that in one of the next issues.

The post Blergh! Word Art! In Houdini? first appeared on DIGITAL PRODUCTION and was written by Olaf Finkbeiner.

With Color Grab, colours can be collected without having to resort to photos. You point your mobile phone camera at the colour area and with two clicks it is added to the palette. Of course, colours can also be collected from photos or a photo can be analysed automatically. It is a pity that the analysis always analyses the entire photo and not the previously selected area.

If you want to do this, you must first crop the photo in another app. The analysed colour map then shows in percentages which colours are present in the photo. You can then select which of the colours should be transferred to the palette.

Certainly a mobile phone cannot replace an X-Rite colour scanner, but Color Grab at least offers a white balance function, colours can be mixed and palettes can be generated with a number of colour harmony rules. In addition, a number of colour models such as RGB, HEX, HSV, LAB and many more are supported. Unfortunately, only RAL is available as a colour reference, as Pantone (X-Rite) is certainly not freely available like this app.

The highlight of Color Grab, however, is the export function. The following are supported: Photoshop and Illustrator, CorelDraw and Paint, Gimp, Inkscape, AutoCAD, Krita and Cinepaint. In addition, CSV, text and a preview image as PNG are also supported.

The post Color Grab – Everything is so colourful here first appeared on DIGITAL PRODUCTION and was written by Olaf Finkbeiner.