Baby’s first pipeline

The term “pipeline” is made up of “pipe” and “line” and describes a process in which something goes in at one end and comes out at the other. It is often used in the fields of visual effects, animation and games, but “workflow” is also a good synonym. Pipelines can be very different. They can range from a 3D software rendering pipeline to a production pipeline. A production pipeline is there to manage the workflow and data output at each stage.
Pipelines are the secret to collaboration. They connect different departments together. Often artists don’t understand how these magical tubes work or how to connect them. But don’t worry, we’re here to shed some light and unlock the secrets of the pipeline!
Teamwork made easy

In every new project, artists face the challenge of teamwork. Even if we are masters in our art, that doesn’t mean we are automatically team players. This is where the pipeline comes into play! It creates standards and workflows so that every artist can work freely within clear expectations. A well-functioning pipeline reduces friction and allows artists to focus on their creative work, while the transition between departments is (ideally) buttery smooth. By minimising redundant tasks, we can focus on the really important things. This means fewer mistakes because we’re not constantly repeating the same work clicks and less stress because everything runs like clockwork. A well-planned pipeline ensures that everyone knows their role and everything goes hand in hand. This leaves more time for creative explosions and less time for organisational headaches.

Everyday life in the pipeline

There are many moments in everyday production when we have to focus on uncreative tasks. Imagine doing this over and over again: opening certain menus, navigating to folders, executing commands or waiting for processes to be completed. It may sound boring, but these routines are our safety net. They give us security and stability so that we can realise our creative ideas without crashing.
With a pipeline behind us, we can focus on what really matters – our art. So, let’s celebrate the magic of the pipeline and make collaboration a smooth and stress-free dance! To create our own pipeline, we need to take a closer look at the following four elements:

Folder Structure

Every pipeline starts with the folder structure, the files are the essential glue between and representation of each department’s work. The first question we ask ourselves when working with our project files is “Where do I store my files?”
The folder structure defines the folder names, their hierarchy and the content of each folder. Depending on the production and studio, this can be customised as required.
Task-based: The task-based approach is best suited to specialised teams where tasks are strictly divided into departments so that artists have a simple “everything in one place” structure. Assets and shots are organised under tasks. (ASSETS/RIG/Shots)
Asset-based (most common): The asset-based approach allows for easier switching when each artist is working on multiple tasks at once. A great generalist approach for smaller projects. (ASSETS/Mike/RIG)
Further planning folders provide space for production data such as asset/shot spreadsheets and budgets, while Preproduction is where we store artwork and mood boards as well as footage, where we find general project-independent resources to help with asset and shot production.

A project folder structure can be divided into the following categories:

Naming Convention

Now that we have organised our files into different project folders, we need to answer another question: How do we name the working files, texture files, assets, shots and nodes in the scene? These questions are answered by the naming convention, which defines general and specific rules for certain types and phases of the project. These rules must answer all naming questions to avoid confusion and help in the later stages of script development. General naming conventions
could be:

• English
• camelCase (ragingBull)
• no spaces “” instead underscore “_” in the file name
• Tasks are in CAPITAL LETTERS (RIG)
• Versions are a “v” combined with 3 digits (v001)
• no names and comments in the file name (are saved as metadata)

Tasks can be shortened for use in asset and shot files:

Here is the final result for our project files using the naming conventions:

Working file: name_TASK_version.extension
mother_RIG_v001.mb
Render file: shotNr_AOVs_version.####.extension
s010_diffuse_v001.1001.exr

Software pipeline

Now that we have structured our project and naming conventions inside and outside the scene, it’s time to look at the software and plug-ins. The software pipeline defines the software, its versions, the files and how they relate to each department. The purpose of the software pipeline can be summarised quite simply by defining what goes in, what comes out and what software is used in the middle of the process. The result is that every artist uses the same tools and plug-ins, while the data flows clearly to and from each department.

Software

Software version (Maya 2025, Houdini 20)
File format per department (input & output)

Extras

Start Frame: 1001 (pre-run time for simulations)

Frame handles: 5 (pre- and postrun time for the edit)

Script Pipeline

We have now reached the final stage of our pipeline: the script pipeline. This is what most people think of as a pipeline. In reality, this is only partially true. The previous sections are much more important and essential to creating a pipeline than the scripts that support it. The purpose of the script pipeline is really just to automate what has been defined by the folder structure, naming convention and software pipeline. The Script Pipeline helps our “Studio Pipeline” with the following tasks:
Folder Structure: Creating folders and organising the working files.
Naming Convention: Save, load, import and export files using a standardised naming convention within and outside the scene.
Software Pipeline: Opens the appropriate project software and loads the required plug-ins, scripts and files via import and publishing applications.

The script pipeline automates the previous steps and realises implementation smoothly. Even the best folder structure, naming convention and software pipeline can be ignored by a single artist. Automating these processes with scripts eliminates manual errors and additional training while saving time, allowing the artist to focus on their craft and not on data processing.

Successful pipelines

The key to a successful script pipeline is automating these 3 elements:

• Software setup at startup
• Save & Load
• Export & Import

Launching the right software with the required plug-ins, settings and scripts ensures that all team members are working with the same applications. Saving, loading, exporting and importing enforce the pipeline structure, simplifying work and publishing to other departments.

“One of the biggest mistakes we made was trying to automate things that are super easy for a person to do, but super hard for a robot to do.”
– Elon Musk

On the other hand, we tend to see certain automations as the only way to get things done, without realising that we can achieve the same results with little preparation and a simple manual workflow. Studios tend to be plagued by the same issues, but they remedy this with pipelines that automate as much “idle time” as possible to increase artist productivity.
The best time to create a script pipeline is after a thorough review of the previous three phases to determine which part will benefit from automation, as automation is time-consuming and comes with its own challenges.

(is.gd/xkcd_efficency)

Extras

Finally, a few points that may be extremely important when looking at the pipeline of a project and that you should definitely keep in mind.
Onboarding: The introduction of new employees to a project should always be a priority at the beginning. In addition to the familiarisation period, it is important to show them how the pipeline works specifically for their work and at the same time give them the opportunity to read about this in the pipeline documentation. (For example in the Open Source Pipeline Plex Wiki – is.gd/plex_wiki)

Backup: It is always advisable to make several backup copies of our project in case we lose it or need to revert to a certain state. Every operating system offers backups with timestamps, or you can simply use Google Drive or Dropbox to save a copy in a remote location. For Script Pipeline, git (www.git-scm.com/) is the best backup system as it allows a tremendous amount of control over backups, changes and rollbacks.

Simple automation: A variety of free and paid external tools can take over certain tasks and create a simple script pipeline. For example, tools such as Total Commander, Directory Opus, the Element or Adobe Bridge can be used to rename or move a large number of files according to a specific logic.

Conclusion

Pipelines are a key element in any production of visual effects, animations and games. A good pipeline guides the artist through the complexity of a multi-stage project and can mean the difference between meeting or missing a deadline. A bad pipeline adds complexity, confusion and unnecessary steps that slow down creativity, teamwork and progress.
To create a good pipeline, it’s important to be clear about the project and have enough experience from previous projects to plan a workflow in advance that guides everyone involved from script to final cut. It’s not just about automation and scripts, because a smart and clear workflow can achieve similar results without relying too much on programming. Ultimately, it’s about achieving the result as smoothly as possible.