unreal.NeuralMorphModel

class unreal.NeuralMorphModel(outer: Object | None = None, name: Name | str = 'None')

Bases: MLDeformerMorphModel

The neural morph model. This generates a set of highly compressed morph targets to approximate a target deformation based on bone rotations and/or curve inputs. During inference, the neural network inside this model runs on the CPU and outputs the morph target weights for the morph targets it generated during training. Groups of bones and curves can also be defined. Those groups will generate a set of morph targets together as well. This can help when shapes depend on multiple inputs. An external morph target set is generated during training and serialized inside the ML Deformer asset that contains this model. When the ML Deformer component initializes the morph target set is registered. Most of the heavy lifting is done by the UMLDeformerMorphModel class. The neural morph model has two modes: local and global. See the ENeuralMorphMode for a description of the two. See: ENeuralMorphMode

C++ Source:

  • Plugin: NeuralMorphModel

  • Module: NeuralMorphModel

  • File: NeuralMorphModel.h

Editor Properties: (see get_editor_property/set_editor_property)

  • alignment_transform (Transform): [Read-Write] The transform that aligns the Geometry Cache to the SkeletalMesh. This will mostly apply some scale and a rotation, but no translation.

  • anim_sequence (AnimSequence): [Read-Write] The animation sequence to apply to the base mesh. This has to match the animation of the target mesh’s geometry cache. Internally we force the Interpolation property for this motion to be “Step”.

  • batch_size (int32): [Read-Write] The number of frames per batch when training the model.

  • bone_groups (Array[NeuralMorphBoneGroup]): [Read-Write] The set of bones that are grouped together and generate morph targets together as a whole. This can be used in case multiple bones are correlated to each other and work together to produce given shapes. Groups are only used when the model is in Local mode.

  • bone_include_list (Array[BoneReference]): [Read-Write] The bones to include during training. When none are provided, all bones of the Skeleton will be included.

  • curve_groups (Array[NeuralMorphCurveGroup]): [Read-Write] The set of curves that are grouped together and generate morph targets together as a whole. This can be used in case multiple curves are correlated to each other and work together to produce given shapes. Groups are only used when the model is in Local mode.

  • curve_include_list (Array[MLDeformerCurveReference]): [Read-Write] The curves to include during training. When none are provided, all curves of the Skeleton will be included.

  • delta_cutoff_length (float): [Read-Write] Sometimes there can be some vertices that cause some issues that cause deltas to be very long. We can ignore these deltas by setting a cutoff value. Deltas that are longer than the cutoff value (in units), will be ignored and set to zero length.

  • geometry_cache (GeometryCache): [Read-Write] The geometry cache that represents the target deformations.

  • global_num_hidden_layers (int32): [Read-Write] The number of hidden layers that the neural network model will have.nHigher numbers will slow down performance but can deal with more complex deformations.

  • global_num_morph_targets (int32): [Read-Write] The number of morph targets to generate in total. Higher numbers result in better approximation of the target deformation, but also result in a higher memory footprint and slower performance.

  • global_num_neurons_per_layer (int32): [Read-Write] The number of units/neurons per hidden layer. Higher numbers will slow down performance but allow for more complex mesh deformations.

  • include_bones (bool): [Read-Write] Specifies whether bone transformations should be included as inputs during the training process.

  • include_curves (bool): [Read-Write] Specifies whether curve values (a float per curve) should be included as inputs during the training process.

  • include_normals (bool): [Read-Write] Include vertex normals in the morph targets? The advantage of this can be that it is higher performance than recomputing the normals. The disadvantage is it can result in lower quality and uses more memory for the stored morph targets.

  • invert_mask_channel (bool): [Read-Write] Enable this if you want to invert the mask channel values. For example if you painted the neck seam vertices in red, and you wish the vertices that got painted to NOT move, you have to invert the mask. On default you paint areas where the deformer should be active. If you enable the invert option, you paint areas where the deformer will not be active.

  • learning_rate (float): [Read-Write] The learning rate used during the model training.

  • learning_rate_decay (float): [Read-Write] The learning rate decay rate. If this is set to 1, the learning rate will not change. This is a multiplication factor. Every 1000 iterations, the new learning rate will be equal to CurrentLearningRate * LearningRateDecay. This allows us to take slightly larger steps in the first iterations, and slowly take smaller steps. Generally you want this to be between 0.9 and 1.

  • local_num_hidden_layers (int32): [Read-Write] The number of hidden layers that the neural network model will have. Higher numbers will slow down performance but can deal with more complex deformations. For the local model you most likely want to stick with a value of one or two.

  • local_num_morph_targets_per_bone (int32): [Read-Write] The number of morph targets to generate per bone, curve or group. Higher numbers result in better approximation of the target deformation, but also result in a higher memory footprint and slower performance.

  • local_num_neurons_per_layer (int32): [Read-Write] The number of units/neurons per hidden layer. Higher numbers will slow down performance but allow for more complex mesh deformations. For the local mode you probably want to keep this around the same value as the number of morph targets per bone.

  • mask_channel (MLDeformerMaskChannel): [Read-Write] The channel data that represents the delta mask multipliers. You can use this feather out influence of the ML Deformer in specific areas, such as neck line seams, where the head mesh connects with the body. The painted vertex color values will be like a weight multiplier on the ML deformer deltas applied to that vertex. You can invert the mask as well.

  • max_training_frames (int32): [Read-Write] The maximum numer of training frames (samples) to train on. Use this to train on a sub-section of your full training data.

  • mode (NeuralMorphMode): [Read-Write] The mode that the neural network will operate in. Local mode means there is one tiny network per bone, while global mode has one network for all bones together. The advantage of local mode is that it has higher performance, while global mode might result in better deformations.

  • morph_compression_level (float): [Read-Write] The morph target compression level. Higher values result in larger compression, but could result in visual artifacts. Most of the times this is a value between 20 and 200.

  • morph_delta_zero_threshold (float): [Read-Write] Morph target delta values that are smaller than or equal to this threshold will be zeroed out. This essentially removes small deltas from morph targets, which will lower the memory usage at runtime, however when set too high it can also introduce visual artifacts. A value of 0 will result in the highest quality morph targets, at the cost of higher runtime memory usage.

  • num_iterations (int32): [Read-Write] The number of iterations to train the model for. If you are quickly iterating then around 1000 to 3000 iterations should be enough. If you want to generate final assets you might want to use a higher number of iterations, like 10k to 100k. Once the loss doesn’t go down anymore, you know that more iterations most likely won’t help much.

  • quality_levels (Array[MLDeformerMorphModelQualityLevel]): [Read-Write] The list of quality levels, where the first item represents the highest quality and the last element the lowest quality level. The number in each quality level represents the number of active morph targets for that quality level. These numbers will be clamped internally to be within valid ranges in case they go beyond the amount of morphs that exist.

  • regularization_factor (float): [Read-Write] The regularization factor. Higher values can help generate more sparse morph targets, but can also lead to visual artifacts. A value of 0 disables the regularization, and gives the highest quality, at the cost of higher runtime memory usage.

  • skeletal_mesh (SkeletalMesh): [Read-Write] The skeletal mesh that represents the linear skinned mesh.

property batch_size: int

[Read-Write] The number of frames per batch when training the model.

Type:

(int32)

property global_num_hidden_layers: int

[Read-Write] The number of hidden layers that the neural network model will have.nHigher numbers will slow down performance but can deal with more complex deformations.

Type:

(int32)

property global_num_morph_targets: int

[Read-Write] The number of morph targets to generate in total. Higher numbers result in better approximation of the target deformation, but also result in a higher memory footprint and slower performance.

Type:

(int32)

property global_num_neurons_per_layer: int

[Read-Write] The number of units/neurons per hidden layer. Higher numbers will slow down performance but allow for more complex mesh deformations.

Type:

(int32)

property learning_rate: float

[Read-Write] The learning rate used during the model training.

Type:

(float)

property learning_rate_decay: float

[Read-Write] The learning rate decay rate. If this is set to 1, the learning rate will not change. This is a multiplication factor. Every 1000 iterations, the new learning rate will be equal to CurrentLearningRate * LearningRateDecay. This allows us to take slightly larger steps in the first iterations, and slowly take smaller steps. Generally you want this to be between 0.9 and 1.

Type:

(float)

property local_num_hidden_layers: int

[Read-Write] The number of hidden layers that the neural network model will have. Higher numbers will slow down performance but can deal with more complex deformations. For the local model you most likely want to stick with a value of one or two.

Type:

(int32)

property local_num_morph_targets_per_bone: int

[Read-Write] The number of morph targets to generate per bone, curve or group. Higher numbers result in better approximation of the target deformation, but also result in a higher memory footprint and slower performance.

Type:

(int32)

property local_num_neurons_per_layer: int

[Read-Write] The number of units/neurons per hidden layer. Higher numbers will slow down performance but allow for more complex mesh deformations. For the local mode you probably want to keep this around the same value as the number of morph targets per bone.

Type:

(int32)

property mode: NeuralMorphMode

[Read-Write] The mode that the neural network will operate in. Local mode means there is one tiny network per bone, while global mode has one network for all bones together. The advantage of local mode is that it has higher performance, while global mode might result in better deformations.

Type:

(NeuralMorphMode)

property num_iterations: int

[Read-Write] The number of iterations to train the model for. If you are quickly iterating then around 1000 to 3000 iterations should be enough. If you want to generate final assets you might want to use a higher number of iterations, like 10k to 100k. Once the loss doesn’t go down anymore, you know that more iterations most likely won’t help much.

Type:

(int32)

property regularization_factor: float

[Read-Write] The regularization factor. Higher values can help generate more sparse morph targets, but can also lead to visual artifacts. A value of 0 disables the regularization, and gives the highest quality, at the cost of higher runtime memory usage.

Type:

(float)