PDG
PROCEDURAL DEPENDENCY GRAPH
PDG is a procedural architecture designed to distribute tasks and manage dependencies to better scale, automate, and analyze content pipelines for Film, TV, Games, Advertising and VR.
PDG is designed to describe these dependencies visually using nodes that generate sets of actionable tasks then distributes them to multiple cores on the same machine, the compute farm, or even the cloud. PDG provides a rich set of stock nodes to enhance productivity, and manages the dependencies smartly to minimize recompute upon changes.
TOPS
Introducing Task Operators
TOPs is PDG technology being applied to Houdini. TOP nodes integrate seamlessly with the rest of Houdini. This brings the power of PDG to FX workflows, but is not limited only to FX. Games, Motion Graphics and other workflows can also benefit, as can technologies such as Houdini Engine.
Intro to PDG
VFX TOP Workflow
PIPELINE TOOLS
Accessing PDG Technology
Houdini and its node-based workflow is often referred to as a pipeline-in-a-box. With PDG, you can break this box wide open to scale up to a bigger procedural pipeline. PDG lets you organize and schedule tasks then distribute them intelligently to your compute farm to work in parallel.
MANAGE
Pipeline Dependencies
PDG can describe large parts of your pipeline to ensure that assets are being loaded, processed and saved out efficiently. The fine-grained dependencies can ensure a minimum of compute upon a given change. The TOP nodes can play a number of different roles that will replace manual labour and scripts that can bog down a complex studio pipeline. You can even set up TOP networks that fetch other TOP networks to create multiple levels of abstraction within your pipeline.
Pipeline Workflows
AUTOMATE
Iterations & Variations
A key workflow in PDG is the wedging workflow that lets you quickly create multiple iterations of a design. This lets you output a contact sheet that can be used to evaluate the iterations to make the best choice or choose many prop variations to richly populate your scene.
Wedging
Prop Variation
ENHANCE
Houdini Workflows
Houdini artists can take their work to the next level with TOP networks enhancing existing workflows, to take advantage of the compute farm to parallel-process geometry networks and dynamic simulations.
SOP Integration
Crowd FX
INTERACT
with Game Engines
For Gamedev, Houdini Digital Assets can already be created to work inside Game Engines such as Unity or UE4. Now with PDG, you can set up visual procedural systems that capture the relationships between various assets, rather than treating each asset as independent from the others. When a change is made to part of a level, PDG can figure out which terrain tiles, vegetation assets, roads and buildings are affected, and make sure that only those are updated in parallel without re-cooking the whole level.
GameDev Workflow
MACHINE LEARNING
Data Synthesis, Augmentation, and Hyper-Parameter Space Search
Machine learning involves lots of trial and error with large datasets and correspondingly large compute to train with those datasets. With PDG, you can synthesize or augment lots of data to help train Deep Learning systems, and try multiple Hyper-parameters on ML models simultaneously.
Terrain Erosion
Terrain Sketching
BUILD
Extensive PDG Toolsets
TOP Nodes
FAQ
Q | Is PDG accessible for Artists?
A | Yes. PDG can bring benefits to FX artists, technical artists, and even CG generalists. It can help scale and automate common workflows such as wedging, and help automate mundane pipeline tasks like data conversion. It uses a node-based interactive workflow that is very artist-friendly. You may need a little help setting up the farm but once you start using PDG, the workflows are very accessible.
Q | Can I use PDG for non-Houdini Tasks?
A | Yes. Although PDG is hugely beneficial for Houdini workflows, there are many workflows involving common pipeline processing and data transformation that can be implemented in a very simple and straightforward manner. You only need either Houdini Core or Houdini FX to set up the graph but after that the processes can be designed to work with other applications. The data-augmentation workflow for ML or a photogrammetry pipeline are examples of this. Pipelines like this are easy to implement and will benefit from PDG’s built-in scale, automation, and performance-monitoring capabilities.
Q | Can I use PDG in Houdini Indie and Houdini Apprentice?
A | Yes. Houdini Indie can access PDG using Task Operators to distribute tasks to a small compute farm using up to three Houdini Engine Indie licenses. PDG is also available in Houdini Apprentice for learning purposes but only works with the local scheduler to process tasks on the computer running Apprentice.
Q | How do I use PDG if I don't have access to a farm or the cloud?
A | PDG can used to fully utilize the cores on your local machine. This requires no setup at all and will work out of the box.
Q | How does PDG compare to a scheduler such as Deadline?
A | PDG makes use of existing schedulers, such as Tractor, Deadline, and HQueue, but is not a scheduler itself. PDG merely tells the scheduler(s) which tasks are ready to be worked on, and leaves it to the scheduler to match task requirements to available HW resources.
Q | Can we use PDG if our studio doesn't use Tractor, Deadline, or HQueue?
A | Yes. It’s easy to bind your custom scheduler to PDG - it only takes a few hundred lines of Python code. We have tutorials to help guide you through the process, and the existing scheduler bindings to Tractor, Deadline and HQueue can be used as reference.
Q | How are licenses used on the farm when a PDG graph is cooked?
A | When you cook a PDG graph, tasks are distributed to the farm based on the rules set up in your chosen scheduler. If the TOP node is referencing either Houdini networks or Houdini Digital Assets then a Houdini Engine license will be used. If the task involves another application such as Autodesk Maya or Adobe Photoshop then a license will be needed for that application.
Q | How can I use PDG for Gamedev beyond big scale environment creation?
A | Although PDG is a great solution for environments creation, it also excels at pipelines in general. There are plenty of data processing / transformation tasks in a games pipeline that can benefit from PDG outside of environment creation.
Q | Does PDG integrate with Shotgun's project management system?
A | Yes. There are a number of stock nodes available to download, find, update and upload files to and from Shotgun.
LEARN PDG
Learn how to leverage PDG's node-based workflow to build robust pipeline workflows using tutorials created by industry professionals and SideFX.
PDG IS AVAILABLE IN:
Houdini CORE | Houdini FX | Houdini Indie | Houdini Education | Houdini Apprentice