---

Introduction to FaceFX

---

Overview

FaceFX is a facial animation solution developed by OC3 Entertainment and integrated with Unreal Engine 3.0. All Unreal Engine 3.0 licensees have a license to FaceFX. This tutorial describes how to get FaceFX data from Max and Maya into UE3. It also covers the basics of generating new animations in FaceFX and working with the application. More extensive documentation of the FaceFX standalone product can be found in the attachment at the bottom along with an overview of the differences between the standalone version of FaceFX and the UE3 integration.

Installing and Launching the Plug-ins

The first thing you need to do is install the Max and Maya plug-ins. Even if you have used the standalone version of FaceFX before, you will need to install the UE3-specific FaceFX plugins included in the Binaries\FaceFXPlugins folder of the main UE3 distribution (Note: as of version 1.7.3.1 of FaceFX, the UE3 plugins are identical to standalone native-coordinate plugins except that UE3 plugins autoconvert names with spaces to names with underscores). The plugins should be copied into the appropriate 3D Studio Max, Maya, XSI, or MotionBuilder plugins folder which will look something like: C:\Program Files\Autodesk\3ds Max 2010\plugins or C:\Program Files\Autodesk\Maya2009\bin\plug-ins. If you are using Maya, be sure to copy the Binaries\FaceFX\FxTools\FxMaya\MEL Script\FaceFX.mel file into your default scripts director for easy access. You can also copy the bitmap in the res folder into the plug-in folder. You will need to download these files from the ActorX Wiki page (or use the ones in the Binaries folder) and copy them into your Max or Maya executable directories. Be aware that the FaceFX plugins require the Microsoft Visual Studio 2008 SP1 redistributable to be installed.

To launch the Max plugin, go to the Utilities panel on the right, click on the "More" button and select FaceFX from the list. To launch the Maya plug-in, make sure the plug-in is loaded in the Window->Settings/Preferences->Plug-in Manager dialog, then open the MEL script in the script editor and execute the script.

Deleting FaceFX Assets

Although this tutorial teaches you how to create FaceFX assets, it uses the OC3_Slade FaceFX asset that comes with UE3 in \UTGame\Content\TestPackages\MatineeTest_Resources.upk. OC3_Slade is already hooked up to use FaceFX and can be used as a sample to get you started. In this tutorial however, we are going to delete the FaceFX Asset so we can start from scratch. To do this, right-click on the Slade skeletal mesh and select Edit using AnimSet Viewer.

From the AnimSet Editor, go to the Mesh Tab and find the FaceFX Asset. Hit the "Clear All Text" Button.

If there are any FaceFX animation sets that referece the FaceFX asset, you must also dereference them by right-clicking on them, selecting properties, and deleting all text from the DefaultFaceFXAsset field.

Now you should be able to delete the Slade FaceFX Asset. Make sure FaceFX Studio is closed before doing this or UnrealEd will crash. From the Generic Browser in the OC3_Slade package, right-click on the OC3_Slade_AMesh_FaceFX FaceFX Asset and select Delete. If you get an error saying that the FaceFX Asset is still in use and referenced by the TransBuffer, you may need to save everything and restart the editor before you can delete the FaceFX Asset. Save the package when you have successfully deleted the FaceFX Asset.

Creating an FXA File from the Plug-ins

Before diving into FaceFX, you will need to create an FXA file with the FaceFX plug-ins. You can read the standalone FaceFX documentation for screenshots and more detailed information about this, but following the below steps will get you started quickly. To follow this tutorial exactly, you will need to use Maya. All of the functionality is identical in Max, but this tutorial uses the skeletal mesh present in the OC3_Slade package, which was created in Maya. If you are using Max, you will need to use the Slade-Max.max source file attached below to export as a FBX file and then import it into Unreal Editor.

1. Open the Slade sample Max or Maya file found in the below attachment and launch the FaceFX plugin.
2. Go to the frame that contains your FaceFX "Reference Pose." In most cases, this will be your skeletons bind pose. In Slade, go to frame 0.
3. Select all of the bones you want FaceFX to control. For Slade, select all of the bones. Export your reference pose by clicking on the "Export" button in the Reference Pose section of the plugin.
4. Batch export all of Slade's poses by clicking on the "Batch Export" button.
5. Select the Slade-Batch-Export.txt file included in the attachment below.
6. Save your FXA file.

Useful Links

• Exporting Skeletal Meshes using FBX
• Importing Skeletal Meshes using FBX

UnrealEd Setup

Creating a FaceFX asset in UnrealEd

Now you need to create a new FaceFX Asset for the Slade skeletal mesh in the OC3_Slade package. Right-click on Slade's skeletal mesh, and select "Create New FaceFX Asset". If you are using Max, you will not be able to use the skeletal mesh that exists in the OC3_Slade package and you will need to create your own. The textures are identical.

Importing an FXA file into UnrealEd

Next import the FXA file you created from the Max or Maya Plugin. Right-click on the newly generated FaceFX asset and select "Import from .FXA".

Launching FaceFX from UnrealEd

Now you can launch FaceFX Studio from inside of UnrealEd. Right-click on the FaceFX Asset and select "Edit Using FaceFX Studio".

Working with FaceFX

Create Mapping

Go to the FaceFX Mapping tab. Slade uses the default mapping, which can be created by clicking on the "New Default Mapping" button at the top of the Mapping tab. The Mapping tells FaceFX what curves to create when it recognizes a particular phoneme. You can create any sort of Mapping you want, but the default mapping is a good example of a target-based mapping that gives you enough poses to realistically move the lips and tongue for a character.

Face Graph Setup

If you look at the Face Graph tab, you can see the list of bone poses exported from Max or Maya. Some of the bone poses exported from Max and Maya have names that correspond to the names of the targets in the default mapping. When you analyze an audio file with FaceFX, curves will be created for these targets and Slade will lip-synch to the WAV file. The Face Graph is capable of very complex relationships however, and creating new combiner nodes and links in the Face Graph will allow you to create very complex behaviors that are easy to animate. To set up Slade's sample Face Graph setup, you can run the Slade-Face-Graph-Setup.fxl batch script included in the attachment below. It will issue all of the commands necessary to setup the Face Graph. Simply copy this file to your C:\ directory and issue the following command from the FaceFX command line:

exec -f "C:\Slade-Face-Graph-Setup.fxl"

If you go to the Console tab, you can see that all of the commands in the batch script were executed and logged to the Console tab. Only the most recent commands can be seen in the Console tab, so for a full log of your session, check the Binaries\FaceFX\Logs\FaceFxStudioLog.html file. This file can be very helpful for debugging problems, so if you experience any issues using FaceFX, please copy and rename the log file before starting FaceFX again so the log is not deleted.

Another way to get Slade's Face Graph set up is to sync to a template file (*.FXT). Templates allow you to copy the Face Graph, node groups, mapping, camera (not supported yet) and workspace information from one actor to another. From FaceFX you can export and import a template file from the Actor menu.

Generating Animations

Now you are ready to generate some animations for Slade. The Face Graph will not only receive values for the targets in your mapping, but it will also rotate Slade's head to emphasize certain words and create blink and eyebrow curves. Much of the complexity of Slade's Face Graph is designed to realistically move Slade's eyelids and eyes as he moves his head about. For a simpler face graph, check out the Slade-Simple example included in the attached zip file below.

To generate an animation, make sure you have first imported the sound asset and sound cue you want to analyze into UnrealEd. The welcome sound file is already present in the OC3_Slade package, so we will analyze that file as an example. Select the welcomeCue asset in the OC3_Slade package so you can access it in the steps below. Then from the FaceFX Browser, click on the Animation Manager button at the top of the application.

From the Animation Manager, click on the "Create Animation..." button at the bottom. The steps to the wizard are explained below:

1. Assuming you want FaceFX to automatically generate curves for your animation, leave the default selected in the first page of the wizard. To hand animate from scratch, you can select "Do not generate curves" and optionally import an audio file. For silent animations, leave the "Use Sound Cue" checkbox unchecked.
2. The welcomeCue sound asset should already be selected because it was selected before you started generating the animation. Click Next.
3. The welcome sound file is the only one present in the welcomCue and it should be selected. Click Next.
4. Next you can specify an English text file to help FaceFX analyze the audio correctly. Text files can use "text tags" to generate curves in sync with specific words to drive other nodes your Face Graph. Use the following text to create a curve for "Gaze Left and Right" in addition to the speech target, blink, eyebrow, and head movement curves:
   • Welcome to Face Ef Ex Studio. Change the animation group from ["Gaze Left and Right" v2= -1 v3=-1 easein=.5 easeout=.5] the actor panel [/"Gaze Left and Right"] on your left for additional sample animations.
5. The next dialog allows you to select the language you want to analyze with along with the co-articulation and gesture configuration files you want to use. These configuration files can be found in the Binaries\FaceFX\Configs directory. Use the default setup and click Next.
6. Finally, name your animation and click Finish.

Previewing Animations

Now that you have an animation loaded, you can watch it from the Preview tab. The Preview tab contains a UE3 viewport along with sliders on the right that can be in Animation, Combiner or Mapping mode. To watch your animation, select it from the "Animations" dropdown in the Actor Panel on the left. Then click the play button at the bottom left of the application. Notice that Slade moves his head to look at the Actor Panel because of the text tag that was added.

Curve Properties

Notice in the Animation dropdown of the Actor Panel that all of the curves in the animation have a weaveform icon except for the "Gaze_Left_and_Right" curve. The curves with the waveform icon are "owned by analysis" and they can not be modified from the curve editor or the sliders until you make them "owned by the user" by right-clicking on them and changing their properties. Curves "owned by analysis" will be automatically regenerated every time you move a phoneme in the phoneme bar.

Managing/Moving Animations

So any anims that are directly inside the FaceFXAsset will be loaded all the time. Even if they are for a different level. So the best way is to make a bunch of FaceFX AnimSets (e.g. one for each matinee, one for each level's dialog). Then you can mount them via the Kismet action.

How to move Animations around to new AnimSets:

1. Right click the FaceFX Asset and choose: "Create new FaceFX AnimSet" (This links that AnimSet to that FaceFXAsset
2. Open the New FaceFX AnimSet
3. Actor->Animation Manager
4. Now you should see the group you made in the drop downs
5. Put the new group in the Secondary Pane
6. Put the group you want to move animations from in the Primary pane
7. Select the animations and then click on Move button to move them
8. Prepare to wait if you are moving lots of anims (e.g. 1-2 thousand will take some time :-) )

Getting More Information

For additional information, check the CHM file posted below. In most cases, the FaceFX Studio application in Unreal is identical to the standalone FaceFX application and the documentation will let you quickly learn the nuances of the application. Some of the main differences between the version of FaceFX in UE3 and the standalone versions are listed below:

• The UE3 version of FaceFX uses a UE3 viewport and therefore does not use the FaceFX Sample rendering engine or the FXR file format
• The UE3 version does not output FXS files to store audio, phoneme lists, etc. All of this data is stored in the FaceFX Asset which is integrated with the UE3 package system.
• The UE3 version of FaceFX uses custom Unreal node types in addition to the standard FxCombinerNode, FxBonePoseNode, FxDeltaNode, and FxCurrentTimeNode. The custom Unreal node types are FUnrealFaceFXMorphNode and FUnrealFaceFXMaterialParameterNode. Check the more advanced FaceFX Tutorials for how to use these nodes.

Downloads

• Download the sample content used in this document.
• Download the FaceFX_Documentation (1.7.3.1) documentation as a zip.
• Download the FaceFX Documentation (1.7.3.1) documentation as a CHM.
• Download a video tutorial demonstrating the Face Graph.