Assets need to be quickly accessible and follow a consistent pattern so that we can complete our creative tasks efficiently. In this article, we take a closer look at how we can achieve this and what problems need to be solved along the way.

What are assets?

In the context of visual effects, an asset refers to any digital element or resource used in the production process. Assets can include a wide range of digital files and components that contribute to the overall visual enhancement of a project. There are different types of assets that can come from different sources, e.g. from previous projects, supplied by clients or downloaded from the internet by third-party providers.

It makes sense to reuse existing assets so that we can focus on the creative part of our work. If we bring these into an organised structure now, we are well prepared and can react flexibly to future changes.
For example, some VFX companies are working on ongoing long-term projects. Perhaps a second season will be shot in the future, so it is advisable to archive previous work. A well-managed asset library can be reused across projects, saving time and money.

Another example is that more and more tools are coming onto the market that use “AI” to generate images. These models are trained using large amounts of data. The better the training data, the better the result. However, this data only becomes useful when it is put into a format that can be used to learn the machine learning models.

The types of assets range from elements such as film footage or HDR images to animation rigs and 3D models with textures, shaders and simulation caches. Each of these elements needs to be treated a little differently in terms of proxy generation. Let’s take a look at what needs to be considered and how the organisation of our asset library can be achieved.

Naming convention and folder structure

Every studio has different needs and workflows. However, the basic idea is always the same. We have a number of files that we want to store in a clear folder structure that follows a commonly agreed scheme for folder and file naming, also known as a “naming convention”.

If you work on VFX projects, this should sound very familiar. The same concept can be applied to an asset library. Certain details in the file path, such as the resolution of an element, can be helpful to get some basic information. However, it is advisable to store additional information such as tags and metadata in a separate location rather than on the file system. Similar to project management software, the asset management tool also stores most of its data in a database. As an artist, however, it shouldn’t matter where the files are located. Artists just want to interact with a user interface that contains all the information they need.

For example, by separating the metadata of files on the server, we can assign multiple categories to an item. If we tried to map this with a folder structure, we would quickly run into problems. Take, for example, a film file of a burning fireplace with lots of smoke. Do we put it in the “Fire” folder or in the “Smoke” folder? If we abstract tags and categories from the file system, we can easily deal with such cases. It is also important that files are not renamed once they have been added to the library. This prevents the media from no longer being found when older project files are opened.

Single file vs. sequences

Another issue related to the naming convention is the way the file is named on the server. We are often dealing with file sequences that contain the frame counter in the file name. On the other hand, there are film files that do not need this frame counter. Images taken with a DSLR camera also have sequential naming, but the individual images may not be related and must be considered as individual files. In other cases, however, they represent a sequence, such as bracketing images of an HDR or a stop-motion animation.

Unfortunately, it is not always clear which scenario we are dealing with. Therefore, it is complicated to automate this task. At this point, we still need the user’s input to tell the asset library how to handle each of these cases. If we add an asset (e.g. a matte painting) to the library that contains a framecounter in the filename, we may want to remove the framecounter as we no longer need it and to indicate to the artist that it is a single frame. In other cases, we may want to keep the frame counter, even if it is a single frame. Unfortunately, there is no clear
Answer to the question of what is the best solution in these cases, as it very much depends on the company’s pipeline and workflows.

Example for single files:
movie.mov/img_1001.jpg
Example for a sequence:
img.1001.jpg/img_1001.jpg

What is the exact file name of an element? There are different names for files, which can vary depending on the operating system and software application. We suggest the following name:

img_1001.exr

Base name: img
Stem Name: img_1001
File Name: img_1001.exr

If the assets come from our internal VFX pipeline, in most cases we want to remove the frame counter and any subsequent characters such as dots, hyphens or underscores to get the actual base name. If the images come from a DSLR camera, we expect the file name to contain the frame number of the image (img_1234). If this were not the case, the files would all have the same name (img) and we would no longer be able to tell them apart.

In most cases, we want to recognise and display a warning if images are missing in a file sequence. With UDIMs (U-Dimension – more on this at FXGuide under is.gd/fxguide_udim), however, “missing frames” are permissible, as the neighbouring tiles are not named consecutively (texture.1001.exr, texture.1011.exr, etc.). Here too, the user must tell the software how to handle this scenario.

Category vs. tags

We want to have different options for a detailed search for elements. This is where tags and categories come into play. These are the keywords that artists will use to search for assets.
This process is similar in every studio. The reason we want to differentiate between all these categories, tags and synonyms is so that we don’t have to reinvent the wheel every time a studio wants to improve their asset management. Also, by using pre-defined terms, we can avoid duplicate tags and misspellings to keep our systems and databases clean. There is no need for two tags if they both describe the same thing. For example, singular or plural terms are often one and the same, so we can avoid unnecessary duplication.

Category: Categories are fixed classes that are constant and do not change. A category is a class or subdivision of things that have certain common characteristics, such as fire, smoke or explosions.

Tag: We can use as many tags as we want to describe an asset. A tag (sometimes called a label) is attached to a person or thing to identify it or to provide additional information. They can be considered dispensable, i.e. “the more the merrier”, to describe an asset in as much detail as possible. Tags can indicate what you see, e.g. smoke and fire, but also descriptive information such as the amount, direction and speed (“fast approaching the camera”).

Synonyms – same same but different

The beauty of the VFX industry is that we come together from all parts of the world. From this connection, we want to develop a common vocabulary that unites all our languages. The asset library software “das element”, for example, uses descriptions and synonyms for each category from a publicly accessible knowledge database.

Wikidata ID

Behind every Wikipedia page is a specific Wikidata entry with a unique Wikidata ID. The advantages of using data from this publicly accessible knowledge database are additional information such as descriptive texts, synonyms and translations into different languages.
Because each category can be linked to a unique Wikidata page, we can create a common understanding and vocabulary of what a particular category is called. No matter where you’re from or what language you speak, you’ll have no trouble finding the perfect item.
Wikidata (wikidata.org) is a centralised data repository for the various wiki projects. The Wikidata ID for each category looks something like this: Q3196 – This text is an identifier (ID for short) that can be used as a link to a human-readable text in the Wikidata database.

Tree diagrams of the category hierarchy

Since most category names such as smoke, fire and explosions are known to all artists around the are known to all artists around the world, the asset library software “das element” has developed a system that emphasises the importance of this terminology. On the one hand, the hierarchy tree is structured according to the actual order of the categories. on the other hand, it is optimised for the vocabulary commonly used in the VFX industry. One concept is to organise everything into super-categories, which makes the most sense for grouping certain types of elements such as fire, smoke or explosions.

As each category is linked to another, we can visualise the relationships between the categories to create an ontology and a knowledge graph. This can help us in the future not only to find similar assets that are currently available, but also to develop future solutions based on machine learning.

Assets

Let’s now take a look at the assets themselves. The source files can be in different file formats and file types. The spectrum ranges from old film Old film scans such as Cineon or DPX, to modern file formats that are not yet supported by most software applications, to anything that can be found on the internet Internet. In an ideal world, we get everything in the same file format and with the same colour space. We don’t want artists to have to deal with these things. Unfortunately, that’s not the case, and achieving this is one of the biggest challenges to overcome. It’s best to save everything in the same file format and colour space. This will make it easier for us to batch process all files in the future when newer systems like USD and MaterialX are integrated into our pipelines.

Today, a good solution is OpenEXR with the ACES colour space. Converting video files into image sequences makes it easier for artists to use, as each sequence can start with the same frame (e.g. 1001). But be careful! This increases the amount of memory required. This must be taken into account for larger libraries or if we want to store everything in the cloud.

For a quick preview of assets, we also want to create proxies or, for example, a rendering of a turntable. We want to get an idea and preview of what is happening in a large film or project file without having to open it. This speeds up the work process without having to wait for a large file to be loaded with the appropriate software.

In some cases, a project file is an older version of software that a studio may no longer have a licence for. Or maybe the file is stored in an archive on a LOT of tapes and you’re not sure if it’s even usable. A good preview can save money, time and a lot of headaches.

The preview can have a lower resolution, for example an 8-bit image like a plain JPEG (or a newer version like HEIC) with Rec709 colour space. For video files it was common to use the H264 codec in a Quicktime container (.mov). There are some new codecs that require less file size with better quality. E.g. HVV (also known as h266) or AV1. Please note that newer codecs may not yet be supported by all applications and operating systems.

Version & Iterations

You should consider how to deal with different versions and variants. An example of different versions would be a recording of a green screen. Version 0 (version zero) could be the original material, version 1 would be the first key. In some cases, you might want to rework the previous key due to new keying techniques or something similar. Now another artist will render a new sequence, which is called version 2. Similar to versioning renderings of VFX shots, this concept can also be applied to an asset library.

Variations can be different resolutions for a 360° HDR image that is available in 4k, 8k or 16k resolution. They all have the same content, but in order to work more efficiently, we want to switch between the different resolutions. However, variations should not be confused with proxies. A proxy for an HDR image can be an 8-bit jpg file, while the variations are still an .exr or .hdr file. The same applies to different LODs (Level of Detail) for 3D assets.

Metadata

Often we want to store any additional information about an asset, e.g. the lens an element was shot with. If we are archiving it for current projects, we may want to store data from our project management software, e.g. the name of the show, the artist or the location of the studio that created this asset.

When purchasing assets from the internet, it is also important to store the relevant licence information with the asset. Years later, no one may remember if you are allowed to use this asset for another project or if there are other restrictions you need to be aware of. Depending on the file type, the amount of metadata that can be linked to a file is limited. By storing the metadata in a database, we can store as much information as we want.

UUID & ULID

When managing assets from multiple studios within an organisation, we want to keep track of when, where and by whom an asset was added to the library. Using the asset name can lead to problems, as names such as “fire_0001” can occur multiple times in different studios.
We can therefore use so-called universally unique identifiers (UUID – is.gd/wiki_uuid) to differentiate between the various elements. These randomly generated IDs are temporally and spatially unique (depending on the version used) and we can trace back when and from where an element was entered. If we store these IDs in the database, it is also advisable to take a look at ULID (Universally Unique Lexicographically Sortable Identifier – more on this at github.com/ulid/spec). The big difference to UUID is that the timestamp is the front part of the ID, which makes the IDs alphabetically sortable. This can increase the performance of database queries.

Automation

Automation is great. However, there are some specifics that need to be considered. One of the biggest problems with automation is human error and the naming of source files. If a file is misspelled, it is difficult to find a solution with a Python script to catch certain errors. Example: T_Grass_Sppecular.png with the word “Sppecular” with two “p” instead of “Specular”. Or the spelling error “Rougness.png” instead of “Roughness”.

Another problem is that the frame counter can change in a continuous image sequence. For example, from 3 (999) to 4 (1000). This often happens with sequences that are rendered by Premiere Pro or After Effects. In most cases, these frames belong together, but this is not always the case.

Name.998.exr
Name.999.exr
Name.1000.exr
Name.1001.exr

It is also important to recognise corrupted files before adding them to the asset library. When buying assets from the internet that come in ZIP files (or copying a ZIP file), the extraction may fail without anyone noticing and we end up with file sizes of 0kb. A simple file size check should help to detect such cases. Some helpful tools are node-based systems that can be used to create automation pipelines. For example: Gaffer(gafferhq.org), NXT(is.gd/git_nxt) or iograft(iograft.com).

3D assets

Creating an asset library for 3D assets is a complex undertaking with many hurdles, and let’s face it, it’s a lot more work than organising 2D elements. There are a few software solutions for 3D asset libraries, such as Quixel Bridge(quixel.com/bridge) and Kitbash Cargo(is.gd/kitbash_cargo). These tools work great because they use an already organised structure in the background from the vendors and their variety of assets. Using these systems is only possible as long as you stay within their ecosystem and don’t want to worry about where and how the files are stored on your hard drive.

If you need more control or want to organise assets from a different source, these systems no longer work as you have no control over the database. Again, we need to find an alternative and generic solution that gives us the flexibility to add any asset type.

Technology

If you have already gone to the trouble of organising everything, you should also consider doing this just once and being well prepared for the years to come. It’s worth researching what options USD and MaterialX offer.

The 3D models come with different types of textures. These are often named differently, even though they represent similar things. Here too, it is recommended to structure and standardise everything so that it follows the same pattern. This makes it easier for artists to use the asset.
A 3D scene purchased from the internet can either contain a single object with associated textures or multiple assets with variations, e.g. different shapes of pipes that can be used for kitbashing. The supplied folders contain multiple textures, often with cryptic names and different resolutions such as 2k, 4k, 8k.

Organising all these files into a clear structure can be a tedious task, but will save artists time in the long run. Splitting assets into individual files can help us speed up the process. However, the initial time required to set it up needs to be considered. To be honest, most of the elements may never be used again in production. On the other hand, valuable time is saved in the crunch time if the assets are well organised
are well organised. If you have the time and resources, it is advisable to take care of the assets and organise them properly.
Experience has shown that the only way to automate this step is to use (Python) scripts to extract certain search terms and assign the texture files to the corresponding texture channels. In the script you can create some rule sets to process the files in a batch process.

Additional files and previews

It is worth creating previews of 3D assets. This way the artists can find them faster and get a simple preview. Otherwise you spend too much time starting the 3D software only to realise it was the wrong file. A simple and less elegant way is to take a screenshot of the viewport to get an idea of what a particular 3D file contains.
A more sophisticated approach would be to create 3D models with low resolution and textures. Sometimes it is necessary to look at a model from a different angle to check whether it is useful for the current task.

Useful file formats are USDZ (is.gd/usd_usdz) and GLB (is.gd/gltf_glb). USDZ is a variant of USD, with the major difference that all textures are merged into a single file, which makes it easier to manage the files on the server. An alternative is GLB (as a variant of glTF), which is already widely supported by web browsers and other applications.

The big challenge could be to create a low-resolution version of the 3D model or scene. For large scenes, it is often not possible to automatically reduce the polygons of the 3D files as well as the textures/shaders without artists having to edit the scene manually. Again, it’s a question of time and resources and what effort you want to put into the library. There are some tools like InstaLOD(instalod.com) that can help with exactly this task.

Scale of the scene

When adding new 3D assets, it makes sense to standardise the scaling for all assets. It doesn’t really matter which scale you choose, at least it should be the same for all files. Some studios choose a scale of 1:10 (1 unit = 10 cm).

Textures

Similar to OpenEXR sequences of 2D elements, using proxies for texture files allows you to quickly display a specific layer. The texture files can be small jpg or png files, but can also be several gigabytes in size if you are working on high-resolution textures for feature films. It is therefore advisable to familiarise yourself with the open standard MaterialX (materialx.org). The aim is to solve the problem of assigning shading and textures to the corresponding geometry. This approach can be useful if you want to be optimally equipped for 3D in the future.

Caches

There are other additional files such as cache or VDB files that need to be saved along with the assets. In some cases, you may be tied to a specific proprietary software. However, studio workflows change over time and certain software packages may become obsolete. Again, it’s worth looking at open source file formats that aren’t dependent on a specific software to build a sustainable asset library for years to come.

About das Element

The asset library software “das element” was first released in August 2021 by a software company based in Munich. Rather than using one of the existing software solutions that are broadly tailored to as many needs and industries as possible, “das element” is customised to the requirements of VFX studios.
Years of experience in building and maintaining asset libraries have gone into the development of this software.

The focus is on combining the above concepts and taking care of the problems you will encounter when building an asset library. Developed to the highest industry standards, the software works completely offline and is therefore ideally suited to the stringent security measures of feature films and television series. Due to its flexibility and configurability, it can be used in a wide range of applications. From high-end visual effects work to fast-paced commercial productions and even non-VFX related projects such as games or architectural visualisation. If you want to start managing your asset library, you can find more information at: www.das-element.com

The post Hitchhikers Guide to … asset libraries? first appeared on DIGITAL PRODUCTION and was written by Jonas Kluger.

We talk to Jonas Kluger, the developer who, after studying design with a focus on film and animation at the Technical University in Nuremberg, first completed an internship at Trixter. His career then began as a VFX editor at Mackevision in Stuttgart. Over the years, his career has taken him to various VFX studios, including Marvel’s “The Avengers”, “Rogue One: A Star Wars Story”, “Transformers: The Last Knight” and several seasons of “Game of Thrones”. His team even won an Emmy for the latter! He is also a board member of the Visual Effects Society (VES) of the German Section.

DP: Let’s start from the beginning: As a compositor and VFXer, what happened when you developed software just like that?
Jonas Kluger: I discovered the need for VFX asset library software early on. It was often one of my tasks to tag new elements and sort them in a meaningful way. With a focus on the technical side and a passion for the creative, I have always been a fan of smart and simple workflows supported by useful tools.

It took 3 years, a short career change into software development and countless hours of perfecting the software with many beta testers around the world to finally present my own VFX asset library software: Das Element.

DP: What is Das Element anyway?
Jonas Kluger: In a nutshell, Das Element is an asset management software. It allows you to organize and sort videos, images and image sequences based on a VFX pipeline. Organize all your assets in a logical order and quickly find what you are looking for.
You can imagine it like this: You have a lot of images, video clips or other media files. You want to organize them all in a folder structure and then be able to find them easily and clearly using keywords and categories. Proxy formats are created for this purpose for a quick preview.

Related elements are collected in so-called libraries. You can create as many of these as you like. This allows you to create a global library for the entire studio or manage elements within specific projects. Many different types of VFX elements can be sorted. These can be Practical Elements, textures, reference images, PDFs, clips from the Internet or images from the Matte Painting Department. Whatever the user wants.

The software consists of three parts: the Gallery View, the Ingest View and the Configuration. The Gallery View allows you to search for and find assets. This is where artists will spend most of their time.
The Ingest View is used to select new elements to be added to the library. The software supports the user enormously in tagging the elements quickly and easily and adding categories. The better the elements are tagged, the faster you can find what you are looking for.
The third part consists of the configuration. With the default setting, you get a great structured folder structure, naming convention and the appropriate transcoding processes. You don’t have to worry about this, it happens automatically. However, if you have special requirements and want more freedom in this process, you can make your own settings. You don’t have to use the default. The software is cross-platform compatible and therefore runs on Windows, Linux and Mac OS operating systems. For example, it is no problem if you use Linux for compositing, while the matte painting department prefers Windows and Production Mac OS.

An important feature of the software is that it works completely offline, bucking the trend towards cloud solutions. A security feature of many studios is that only selected computers are allowed to use internet access. Artists rarely have the opportunity to upload or download data to servers. Das Element therefore operates offline, which has a very positive effect on performance. Another argument in favour of this is the reduced Internet traffic. This saves a lot of CO2 emissions and is good for the environment. Many people do not realize how much energy is consumed every time a website is accessed.

DP: So it manages assets – but in theory I can also do that with folders and a file manager, can’t I?
Jonas Kluger: The big advantage that is immediately apparent when using the software is that the user immediately gets a visual overview of all available elements without having to click through folder structures on the file system. Preview formats (proxies) are generated for each element entered, allowing the content to be analysed much more quickly without having to open individual files, which are often several gigabytes in size.

It is certainly not uncommon for individuals to use a folder structure and, through habit, find their way around it over time. However, as soon as a second person joins in, it becomes difficult. A lot of time is wasted on familiarization and searching. Not an ideal situation. With a folder-only system, there is a great risk that assets will not be sorted, but simply downloaded and unzipped. Elements that would actually belong in the same category can be found in different folders. Freelancers who are new to a studio then have to ask where to find which element. This means that at least two people are involved in a search. When ingesting new elements, Das Element indicates if an element already exists. This avoids duplication.

DP: And which tools in a VFX pipeline does it talk to?
Jonas Kluger: The nice thing about element is that you can actually use it with any software. It is a stand-alone software. No plug-ins are required to work with other applications.

On the one hand, you want to search your library and import the elements found into other software. To do this, you mark the selected elements in Das Element and can now drag and drop them into the desired application. The path to the main element is always used in the background. It does not really matter whether the path is an image sequence or perhaps a project file. The software with which you want to use Das Element will recognize this.

Of course, every software has different requirements, but this has also been taken into account. Let me give you an example using image sequences: In Nuke, I can drag and drop in the selected element and the image sequence will be imported. In Flame, however, a very special formatting of this path is necessary: (filename.[1001-1002].exr). This can be set in Das Element so that it fulfils the correct requirements in this case.

The second situation is the process of entering the elements into the library software and depends heavily on the respective needs of the user. These can include anything. A simple copy from A to B, a “leave my files where they are” to complex processes such as transcoding processes on the render farm to convert everything into an OpenEXR sequence with ACEScg Colourspace using Nuke, to then sync this with another location and its server and then perhaps create an entry in FTrack or Autodesk Shotgrid (formerly Shotgun).
If you need an even more extensive configuration, you can hook into certain places in the software at any time with your own scripts (so-called hooks). This makes it possible to use the respective API of another product. This allows you to create very powerful setups.

DP: What was it about the usual asset managers that bothered you so much that you developed your own?
Jonas Kluger: Anyone who has ever looked for library software for VFX will have realised that unfortunately there is no perfect solution for the requirements of the industry. Most of them can’t handle image sequences or the OpenEXR file format.

I have often seen in various VFX studios that software developed for another industry has been adapted to the VFX workflow by hook or by crook. Countless compromises were made in the process. Alternatively, larger studios have started to build their own solutions. These are very specifically adapted to their own needs and could possibly be optimized, because in my experience they often lack the resources to employ someone full-time.
I also didn’t quite like the license model of some asset managers. Either everything is in the cloud, which is not feasible in film and advertising production for data protection reasons, or you had to install a licence manually (node-locked) on each computer. This can be super annoying if you constantly have to worry about which version is installed on which computer. That’s why it was very important to me to use a similar license model to other VFX applications. You use a license server and only floating licenses, so it doesn’t matter which computer I’m on. I can open the element anywhere. This reduces the workload for IT enormously. An RLM server is used in the background, which has now become the industry standard.

It is also important to have an open system. You should have access to all data at all times. There are no proprietary formats or encrypted databases. All assets are available at all times, and the information in the database and configuration files is also created in common formats. This ensures that you still have access to the data, even if the license expires. As a technically experienced person, you can easily open the config files with a text editor, make changes and integrate them into a version management system such as Git or Perforce.

I have gone to great lengths to make the configuration of the folder structures and the transcoding task simple and visually appealing. On the one hand, it’s really fun to create your paths with Das Element’s Path Builder, and on the other, it takes a lot of work out of using the predefined structure and only having to overwrite values for the proxy formats, for example, in individual places. Other applications often make this process too technical and time-consuming. My approach is simple and intuitive with a fun factor.

DP: And all of this is already available?
Jonas Kluger: Yes, exactly. With the help of the dynamic and visually appealing user interface and the gallery view, you can search for tags, categories or their synonyms and then simply drag and drop the elements found into the compositing software.
All elements are first added to the library using the ingest view. The auto-tagging function can be used here to automatically analyse image content and set tags.
With the out-of-the-box setting of the folder structure and proxy formats, you can set up a new library with just a few clicks. Many small details have been taken into consideration, e.g. the fact that preview images are always generated with a meaningful thumbnail or that the complete data can be moved to a new server at a later date.

You can configure the paths and transcoding processes at any time to customise everything to your own requirements. This also makes it possible to outsource the computationally intensive processes to a render farm. Any number of libraries can be created, either to organise a specific project or to share elements across different locations. The software works as a cross-platform and can be used on Linux, Windows and Mac OS operating systems.

There are a series of video tutorials on YouTube to help you get started with element and quickly learn and find your way around the software. They can also provide an insight into previously unused features.

DP: What can it do so far?
Jonas Kluger: With many filter options such as resolution or media format, you can quickly find what you are looking for. However, the software goes one step further. For each element there is information such as the original colour space and also the original path where the file was once located when it was added to the library.

I would like to mention a few less obvious features that have proved to be important over many years of working with asset libraries. A lot of emphasis is placed on productive day-to-day work. For each library, a field called “Library Root” is used to define where the actual data is located. The big advantage of this is that you can move the entire folder, e.g. to a new network storage location if you purchase a new server or want to copy the elements to another company location. A simple adjustment of this root directory is sufficient and all data is directly re-linked and found. This scenario happens more often than you might think, which is why element can handle it. In addition, a library can be used with different operating systems, as only the root path needs to be adapted.
The predefined naming convention and folder structure is a compromise between readability for the user on the file system and a long-lasting structure that allows you to change the category of an element at a later date without causing problems. There is nothing more annoying than opening old project files only to realize that a file no longer exists and you first have to find out where this file is now located or whether it has been renamed. With Das Element, a lot is solved via the database. This avoids such problems.

In addition, when new elements are added, it is also directly documented when which files with which tags were added. This is a simple CSV file that is saved with each ingest. On the one hand, this makes it possible to restore the status of the list at ingest with all tags and categories at any time. On the other hand, this is a good way to directly perform a batch import of many files. This history of imports is important for larger productions such as Fox Studios or Disney, which require such documentation. This kills several birds with one stone.

The software makes it possible to work with Python hook files. This means that tags are generated directly from the file name, metadata or other sources when importing elements. These can be specific keywords in the folder structure or file names, which are then generated directly during ingest.

DP: Can it be placed as a layer over existing structures?
Jonas Kluger: Absolutely! If there is an existing structure, it can be used without any problems. The naming convention, folder structure and all transcoding tasks are freely configurable. Scripts or other pipeline connections can also be adopted and reused. Existing tags and categories can also be imported.

For example, a VFX studio from Canada had already created an existing library including a category hierarchy tree in the XNView software. With a small script, all tags and the hierarchy tree could be customised directly. This made the move to element very smooth. The batch import was made possible with the aforementioned CSV files.

The minimum basic requirement of the software is that there are thumbnails for each element. All information is stored in a database so that you can work more effectively. For videos and image sequences, it is recommended to render a filmstrip. The entire clip is reduced to a length of around 24 frames. If you move the mouse over the preview, you can get a quick impression of the content of the sequence. This scrubbing feature is very helpful in the gallery view. That’s all you need.

DP: What exactly happens when new elements are added?
Jonas Kluger: A central function of the software is the ingesting of new elements. With the ingest view, you can easily add new elements to the already explained folder structure. With its extensive yet user-friendly user interface, element offers maximum control for tagging files.

Firstly, all elements are simply dragged and dropped into the ingest view of the software. A preview is created in each case so that you are visually supported during the subsequent tagging process. You assign a category to each element and can then assign tags as required.
Each new element goes through a series of transcoding steps that generate different preview formats. The software offers useful presets for thumbnail and proxy videos as well as film strips.

Existing in-house scripts can be used by running a custom command as a transcoding task. This means that you can also send everything to the render farm as a nuke job. The software offers various example scripts and you can download the latest version from a Git repository.

DP: What can be imported? And how much of it?
Jonas Kluger: The current main focus of Das Element on VFX and animation elements. This also covers a large proportion of the elements that can be obtained from various resources, whether from the customer or from a web shop.

The software processes various image formats and codecs. Whether videos from a DJI drone, clips from YouTube, Photoshop files, PDFs, HDRs or JPEG, TIFF, EXR or DPX sequences. All this and much more is possible. If you have older files such as Cineon (CIN), you can also manage them. The handling of camera raw formats such as ArriRaw and REDRaw is still limited at the moment. I would generally recommend developing and converting the files using the camera manufacturer’s software before adding them to a library. As soon as an EXR sequence has been rendered, for example, Das Element can then handle it perfectly.

How many elements can be imported depends entirely on the storage space on the company’s server. For the productive operation of larger libraries (I am talking about a few thousand elements here), it is recommended to use the PostGres database. This allows you to make several thousand requests per second. The database should therefore not be a bottleneck.

Due to the way the software is optimized in the background, a request is made to the database once at startup to get the information, whereupon everything is cached locally by the user as long as the software is open. This increases performance enormously when working, as information does not have to be constantly queried via the network. All that is loaded from the server are the thumbnails and any other proxy files. These are very small and allow you to work smoothly. If additional files are to be copied, this can be automated with a separate script.

DP: Can Das Element handle other sources (Studio Three location via network or a Dropbox folder in the middle of nowhere)?
Jonas Kluger: The software can handle this perfectly. A distinction is made internally between the high-res data and the preview files and metadata stored in the database. The database must be accessible for all participants. Where the preview files are located is not important for the software as long as the path is stored in the database. Whether the preview files are located on the server, in a shared folder that is automatically synchronized via the cloud (Dropbox, Google Drive or Microsoft OneDrive) or on a web server is simply a question of personal configuration.
One use case for the current home office situation is that an artist has a compressed library at home with only proxy files and a local database file or a database within the company that can be accessed via VPN.

Using the preview files, you can search through the elements and even try out proxy videos in the compositing software. The high-resolution data can then be requested from the company using the name of an element.

Alternatively, Das Elements can be stored on the company’s internal web server and users can access the thumbnails. In this way, the information can also be shared with multiple locations. If the high-res data is not available at one location, you can create a transcoding task that synchronizes everything to that location.

DP: And where do I define tags?
Jonas Kluger: Tags can be created at will, either when ingesting or editing an existing element. As soon as you start typing in the tags input field, all similar existing tags appear. There is an option to take an existing value or create a new tag. This prevents duplication of tags and unnecessary cluttering of the system.
Categories can also be added or customized as required, as can synonyms. For tags that are likely to occur frequently, it is worth creating a separate category or using it as a synonym for an existing category.

How the tags are used can be determined very individually by a company. For example, it can be a project name or manufacturer of elements, but also a descriptive text such as “original plate” or “keyed element”. The design is completely free and left to the company or user. The only important thing is that an artist ultimately finds what they are looking for.

DP: To what extent are the tags customizable? Or can I customize the tags from other sources (spelling lists, metadata, etc.)?
Jonas Kluger: It is possible to have tags filled with a Python hook file directly when loading files into the ingest list. The file receives all paths that have been dragged and dropped onto the software, for example. You have the option of extracting information from the metadata of a file and, for example, generating a new tag with the focal length for this element. If a tag does not yet exist in the database, it is created automatically.

A fully functioning proof of concept was even created for a film team. The VFX team had the requirement to sort all elements for VFX directly on the film set after the shooting day. They proceeded as follows: There were different photo cameras for HDR shots, reference material or turntables of assets. The respective memory cards were copied to a predefined folder structure on a local network storage device on set. This was simply quicker and could be done by many people at the same time. The whole folder from one tag was then dragged onto Das Element, and the tags were all automatically created from the folder names. Now you just had to click on “Ingest” to create the thumbnails and enter all the information into the database.

The big advantage is that you can search for an asset name at any time and instantly find what was ever shot for that asset, regardless of the day of the shoot. If it is important which day the material was shot, this could be easily filtered out by adding the date.

DP: But how can I make sure that the artists find what they are looking for and don’t have to spend a long time searching for the right day?
Jonas Kluger: The software comes with a well thought-out arrangement of categories in a hierarchical structure. A lot of time, energy and brainpower has gone into this. Users benefit from this well thought-out system.

Let’s take a look at an example of a torch element. As everyone probably has a rough idea of what element they are looking for, you can simply select the main category. In our case, “Fire”. The search can now be further refined using the hierarchy view. You might next select the “Flames” category, where you can directly see which elements are available in the respective subcategories. The search then continues to the “Torch” category, and here you will find what you are looking for without even knowing which tags were used during the search process.

The beauty of the VFX industry is that we come together from all parts of the world. With that in mind, Das Element focuses on finding a common vocabulary that unites all of our languages. The software uses descriptions and synonyms for each category from publicly available sources.

Wikidata IDs play an important role here. Behind every Wikipedia page is a specific Wikidata entry. The advantages of using data from this publicly accessible knowledge base are descriptive texts, synonyms and translations in different languages.
Since each category can be linked to an exact Wikidata page, we can standardize a common understanding and vocabulary of what a particular category is called. No matter where you come from or what language you speak, this approach should make it easier to find the perfect element.

For example, the categories “fume” (smoke) and “wisp of smoke” (cigarette smoke) are often confused. This is where the machine learning model helps, because it can analys exactly how these two categories differ visually.

DP: But I can then use the tags in the search, or can you find more uses for them?
Jonas Kluger: You have to differentiate between categories, tags and a third type, the wildcards. The idea of tags is that you can use them in numerous ways to describe an element as precisely as possible. Categories, on the other hand, are there to give a name to a specific grouping of elements. A category is therefore something more permanent.

For example, you can include the category name or a category ID in the file or folder name in order to recognize exactly what it is on the file system. Whether an element can be seen on the left or right of the image is more likely to be clarified via a tag. If the image is mirrored or rotated in the software, this parameter would otherwise no longer be correct if it had been written in the file name.
The third type, wildcards, are placeholders for parameters that are dependent on the individual elements. The software analyses each element carefully before it is entered. Values such as the resolution or file format, the first and last frame for image sequences and the codec or frame rate for videos are entered. These are also values that can be used to describe the element.

They can be filtered in the gallery view, for example to search for the minimum length of an element. These element-dependent parameters can be used in the settings and the naming convention to act as a kind of placeholder. They are also called wildcards.
As we only know the resolution of an element when we ingest it, we need these placeholders to define our folder name structure in advance. The actual parameters only become known during ingestion, the wildcard is resolved and replaced with the correct values. This is how the final file path is created.

As a user, I have access to a wide variety of values here, which can be helpful. If you want to execute a user-defined command line command, these wildcards are extremely helpful and can be used to pass on various values to other scripts or programs.

DP: So Das Element sorts all my Studio assets and makes them searchable with tags. But how does it decide which keyword is suitable for which file?
Jonas Kluger: The data can be analyzed by users using auto tagging before it is ingested in order to find the best possible tags and tag them automatically. This works with a machine learning model from Das Element that has been specially trained with thousands of VFX elements.
Most services on the market that offer AI use a model type in the background that is based on a very specific data set of images. This is usually the ImageNet Challenge; a collection of countless images divided into 1,000 categories. This is a benchmark training set that can be used to compare how well a type of neural network performs. The advantage lies in the large number of categories with everyday objects, animals or similar. However, if you search for a specific word, e.g. “fire”, you only get a few suitable categories, such as fire grate for a fireplace or campfire, a fire boat or fire engine and the fire salamander. Of course, this is not particularly helpful for professional work with visual effects.

That’s why I decided to create my own training data set. I have been continuously expanding this collection for 3 years now. I have bought all kinds of element libraries that can be found on the Internet and feed them into the training set. One major difficulty, however, is finding enough data points, i.e. images or videos. For the search terms “fire”, “flame” etc. you get countless material. For footage of nuclear explosions on a black background or VFX portal effects, it is more difficult to find enough different image material. For classification with machine learning, you need a different and varied selection for a category so that you can easily differentiate it from an obvious other category.
If I can’t find enough elements for a category on the Internet, then I take a few photos of glowing coals at a family barbecue, or if I notice a container on the street with rubble and debris that are nicely arranged from the perspective of a VFX artist, I take out my mobile phone to constantly expand the data set. All images and videos are sorted and tagged by hand to achieve the best possible result. A machine learning model, a convolutional neural network, is then trained and passed to the users of the software.

As soon as elements are added to the library, a second machine learning model is trained in the background. This model is based on existing elements in the library. The more elements are added, the better it gets. In the ingest view, you receive suggestions as to which category it could probably be. The idea is to develop an independent model that can be trained on any category as long as there are enough training samples. As the library grows, the model becomes more accurate.

DP: And if I have assets that are not recognizable in your sorting?
Jonas Kluger: The auto-tagging feature can be a great help with many elements. However, I still recommend a manual check to add further keywords that the model does not yet recognize, for example, and to add further characteristics such as speed or size. These characteristics can be rather subjective under certain circumstances.

The technology used for machine learning models is comparatively new, although the theory behind it has been around for a very long time. It can happen that the model either does not recognize a category and therefore does not enter a value, or it mistakenly thinks that it is some random other category. In this case, however, the user quickly realises that the tag supplied makes no sense and removes it. Of course, you can always edit the existing library in the gallery view if you notice something that is incorrectly tagged during a search.
It is recommended that you always briefly double-check each element, regardless of auto-tagging. Every element should be tagged correctly at least once before it is added to the library. After all, that’s what the input process is for.

DP: Where will Das Element go from here?
Jonas Kluger: In order to make the software even more accessible from the outside, a Python API is to be added. This has been put on the back burner for various reasons. I am curious to see how the whole topic of Python 3 will develop in the VFX industry.

There are also plans to set up a marketplace where you can search not only your own elements, but also all libraries from element providers on the Internet. This makes it possible to buy individual elements, add them automatically and thus expand your own library with elements that are needed for the current task.

A web view of the software is also conceivable for the future, so that you can log in via a browser. Perhaps even a cloud solution for freelancers so that they always have online access to all the references they have ever collected.

DP: What other features are available?
Jonas Kluger: We are planning some tidying up in the background, a kind of spring-cleaning for the software. Stability is much more important to me than lots of new features that don’t really work. A library is a long-lasting thing that should exist and function for years. When I talk to users, they often still have files from the last millennium in PAL that are still in use.
One particular innovation that is on the immediate to-do list is an improvement to the rating and voting system for elements, which analyses the most popular elements in the background. In addition, the option is to be created to rate elements of poor quality so that they slide further into the background. A few users have requested this change and I would like to fulfil this request.

A playlist system will be introduced to strengthen team collaboration. This means that a team lead can put together a collection of suitable elements in advance and distribute them to all artists in a kind of playlist. This means that not everyone has to search through various libraries individually, but already has, for example, the top 20 fire elements that are suitable for a sequence or setting. The playlist system can save an enormous amount of time and promotes collaboration within the team.
Other file formats such as project files or setups for Nuke or After Effects will also be supported. These can also be very helpful if you want to reuse a rain or noise setup several times. The project file is ingested into the library and a preview image can then be added so that you can see at a glance what is happening in this script.

DP: What other new features are planned in the long term?
Jonas Kluger: A lot is planned and conceivable for the future. However, the community is an important decision-maker. Nobody can say which requirements are particularly important better than those who use the software on a daily basis and thus recognize and pass on useful suggestions for improvement in the course of their work. After all, the software was (or is) programmed for specific users and not for someone who has thought about something in a quiet room but can’t relate it to reality. I welcome creative suggestions and an open dialogue. OCIO (OpenColorIO) is to be integrated for better colour management.

DP: Where can you get Das Element if you want to try it out?
Jonas Kluger: If you want to find out more about Das Element and see it in action, you can make an appointment for a live demo at any time via the website. To do this, you select an available date and time from a calendar. You will then receive a confirmation email and a Zoom link.
I personally take a lot of time to present the software during the appointment and discuss all questions and use cases. If you want to try out the software straight away, you will find an e-mail address on the website where you can request a 30-day trial version. The trial version offers the full scope, you can test the software extensively and learn to love it. If you have any other questions or suggestions, please do not hesitate to contact me. I am very interested in optimizing Das Element for everyone.

The post Element-ary Asset Management first appeared on DIGITAL PRODUCTION and was written by Jonas Kluger.