Visual Dataprep Overview

How the Visual Dataprep system works, and how to use its UI

Windows
MacOS
Linux

This page provides an overview of the Visual Dataprep system, and how to get started creating your own custom import recipes to prepare your 3D data for real-time visualization.

Enabling Visual Dataprep

Enable the following plugins for your Project:

  • DataPrep Editor

  • Datasmith Importer

  • If you want to import from other kinds of source files that Datasmith supports, you may need to enable other importer plugins for those file types. For details, see Importing Datasmith Content into Unreal Engine 4.

If you started your Unreal Project from a Template in the Architecture, Engineering, and Construction or Automotive, Product Design, and Manufacturing categories, these Plugins may already be enabled.

Visual Dataprep Assets

The Visual Dataprep system is based around a new kind of Asset, called a Dataprep Asset. This Asset is similar to a Blueprint in that it lays out a series of steps in a visual representation. However, the purpose of this Dataprep graph is to transform scene data read from one or more input files that you specify.

To create a new Dataprep Asset, right-click in the Content Browser and choose Dataprep > Dataprep Asset from the context menu.

Create a Dataprep Asset

Dataprep Editor UI

Like Blueprint classes, Dataprep Assets have their own specialized editor window. You can open this window by double-clicking any Dataprep Asset in the Content Browser.

The Dataprep Editor UI is divided into several panels, which you'll use at different points in the Dataprep workflow.

Number

Name

Description

1

Toolbar

Shortcuts for the most important user actions in the Dataprep workflow, including the Import, Execute, and Commit actions.

2

Content Browser Preview

Lists all Assets you have imported from your input files. This is a preview of the Assets that the Visual Dataprep system will create in your Content Browser when you Commit your graph.

3

Viewport Preview

Displays a visual preview of the 3D scenes you have imported from your input files.

4

World Outliner Preview

Lists all Actors in the 3D scenes you have imported from your input files. This is a preview of the Actors the Visual Dataprep system will create in your Level, and the scene hierarchy that will be created in your Level's World Outliner.

5

Settings Panel

Use this panel to specify the input files you want to import your 3D scenes from, and where you want the Assets to be created in your Project's Content Browser.

6

Palette

Offers building blocks that you can drag into the Dataprep Graph Editor to build up your import recipe.

7

Dataprep Graph Editor

Area where you can prepare Dataprep recipes, which are left-to-right steps that you want the Dataprep system to take to prepare your input content for real-time visualization in Unreal Engine.

8

Details Panel

Similar to the Details panel in the main Level Editor, shows information about any Actor that you have selected in the World Outliner Preview. Note that these settings are read-only. They are provided only for your information so you can evaluate the effects of your Dataprep Graph on your Actors.

Dataprep Workflow

The typical workflow for setting up a Visual Dataprep asset follows these steps:

  1. Specify input files: In the Settings Panel at the top right, create a new input producer for each file you want to import, or each folder of input files. You can also set the path to where your Assets will be created in your Project's Content Browser.

  2. Preliminary import: Press the Import button in the Toolbar to import your source files into a temporary world.

    You'll see the 3D viewport and other preview panels update with the content read from your file. This content has not yet been added to your Project; it's in a temporary world that only exists in the Dataprep Editor. This gives you an opportunity to modify the Assets and scene hierarchy read from your input files before you save the final results to your Project.

    This step is technically optional, but it makes it easier for you to evaluate the effects of your Dataprep Graph as you build it.

  3. Build the Dataprep Graph: Drag Select By and Operations nodes from the Palette into the Dataprep Graph Editor, and connect your Action nodes in the order you want them to be carried out. See Dataprep Graphs below.

  4. Test the Graph: Press the Execute button in the Toolbar to run the data you've imported from your source files through the Dataprep Graph you've built. You'll see the preview panels update to show the results.

  5. Commit: When you're happy with the effects of your graph on the 3D data imported from your input files, press the Commit button in the Toolbar to finalize the import process.

    When you commit the results, the Visual Dataprep system saves the Assets in the Content Browser Preview into Assets in your Project. It also modifies the Level that you currently have open to add the hierarchy of Actors shown in the World Outliner Preview.

    If you prefer to import your Actor hierarchy into a new Level instead, set the name of the Level in the Sub-Level setting in the Settings panel. The Visual Dataprep system will create a new Level with this name (if it does not already exist), add your Actors to that Level, then add that Level as a sublevel within the Level that you currently have open in the main window of the Unreal Editor.

Dataprep Graphs

At the heart of every Dataprep Asset is the Dataprep Graph—the set of operations that the system will carry out on whatever set of input files you specify.

Every Dataprep Graph is made up of building blocks called Action nodes: vertical stacks of blocks. For example, the graph shown above contains three Action nodes. When you execute the Dataprep graph, it starts at the Start node on the left, then performs each Action node in order, from left to right.

Unlike Blueprint graphs, where conditional branching is allowed, Dataprep graphs always progress from left to right, along a single linear execution path. In addition, there is no data flow between successive Action nodes. All Actions use the same context—a set of Assets and Actors that are imported from input files you provide.

Each Action node is made up of a stack of one or more blocks. When the Dataprep graph performs an Action, it treats each block in the Action node from top to bottom.

Operations and Filters

There are two basic types of blocks you can use in a Dataprep Action node:

  • Operations carry out some kind of modification on Assets or Actors. For example, the Set Material, Compact Scene Graph, and Set Simple Collision blocks shown above are all different types of operations.

    For details on all the different operations you can use in your Dataprep graphs, see the Visual Dataprep Operation Reference.

  • Filters, also called Select By blocks, determine what Assets and Actors the operations under them in the current Action block should modify. By default, every Action step performs its operations on all Assets and Actors contained in the temporary world constructed from your input files. You can use these Filter blocks to define a subset of those Assets and Actors, thereby controlling the objects your Action node will modify.

    For details on all the different filters you can use in your Dataprep graphs, see the Visual Dataprep Selection Reference.

Action Example

The Action pictured below sets a new Material on the parts of a CAD assembly. This node performs a sequence of steps, processing data through its stack of filters and operations, from top to bottom.

Sample Action node

Step Number

Name

Block Type

Description

1

Input Pin

N/A

The Action gathers all data from the temporary world, including all Assets and Actors, and passes those objects to the block at the top of the stack.

2

Filter by Tag

Filter

This filter only keeps Actors that have the "Metal" tag, passing that list of Actors to the next block down.

3

Filter by Actor Label

Filter

This filter only keeps Actors whose names contain the word "Mechanical", passing them on to the next block down.

4

Set Simple Collision

Operator

This operator sets up a Box collision shape on all Static Mesh Assets that are referred to by the set of Actors identified by the filters above. It then passes the list of Actors on to the next block down.

5

Set Material

Operator

The last operator finds all Materials on all Static Mesh Components owned by the set of Actors identified by the filters above, and replaces those Materials with the substitute specified in the Material setting in the block.

Select Skin
Light
Dark

Welcome to the new Unreal Engine 4 Documentation site!

We're working on lots of new features including a feedback system so you can tell us how we are doing. It's not quite ready for use in the wild yet, so head over to the Documentation Feedback forum to tell us about this page or call out any issues you are encountering in the meantime.

We'll be sure to let you know when the new system is up and running.

Post Feedback