unreal.LearningAgentsCritic

class unreal.LearningAgentsCritic(outer: Object | None = None, name: Name | str = 'None')

Bases: LearningAgentsManagerComponent

A critic used by some algorithms for training the managed agents.

C++ Source:

  • Plugin: LearningAgents

  • Module: LearningAgents

  • File: LearningAgentsCritic.h

Editor Properties: (see get_editor_property/set_editor_property)

  • asset_user_data (Array[AssetUserData]): [Read-Write] Array of user data stored with the component

  • auto_activate (bool): [Read-Write] Whether the component is activated at creation or must be explicitly activated.

  • can_ever_affect_navigation (bool): [Read-Write] Whether this component can potentially influence navigation

  • component_tags (Array[Name]): [Read-Write] Array of tags that can be used for grouping and categorizing. Can also be accessed from scripting.

  • editable_when_inherited (bool): [Read-Write] True if this component can be modified when it was inherited from a parent actor class

  • helper_objects (Array[LearningAgentsHelper]): [Read-Only] The list of current helper objects.

  • interactor (LearningAgentsInteractor): [Read-Only] The agent interactor this critic is associated with.

  • is_editor_only (bool): [Read-Write] If true, the component will be excluded from non-editor builds

  • is_setup (bool): [Read-Only] True if this component has been setup. Otherwise, false.

  • manager (LearningAgentsManager): [Read-Only] The associated manager this component is attached to.

  • network (LearningAgentsNeuralNetwork): [Read-Only] The underlying neural network.

  • on_component_activated (ActorComponentActivatedSignature): [Read-Write] Called when the component has been activated, with parameter indicating if it was from a reset

  • on_component_deactivated (ActorComponentDeactivateSignature): [Read-Write] Called when the component has been deactivated

  • primary_component_tick (ActorComponentTickFunction): [Read-Write] Main tick function for the Component

  • replicate_using_registered_sub_object_list (bool): [Read-Write] When true the replication system will only replicate the registered subobjects list When false the replication system will instead call the virtual ReplicateSubObjects() function where the subobjects need to be manually replicated.

  • replicates (bool): [Read-Write] Is this component currently replicating? Should the network code consider it for replication? Owning Actor must be replicating first!

evaluate_critic() → None

Calling this function will run the underlying neural network on the previously buffered observations to populate the output value buffer. This should be called after the corresponding agent interactor’s EncodeObservations.

get_estimated_discounted_return(agent_id) → float

Gets an estimate of the average discounted return expected by an agent according to the critic. I.E. the total sum of future rewards, scaled by the discount factor that was used during training. This value can be useful if you want to make some decision based on how well the agent thinks they are doing at achieving their task. This should be called only after EvaluateCritic.

Parameters:

agent_id (int32) – The AgentId to look-up the estimated discounted return for

Returns:

The estimated average discounted return according to the critic

Return type:

float

load_critic_from_asset(neural_network_asset) → None

Load a ULearningAgentsNeuralNetwork asset’s weights into this critic.

Parameters:

neural_network_asset (LearningAgentsNeuralNetwork) – The asset to load from.

load_critic_from_snapshot(file) → None

Load a snapshot’s weights into this critic.

Parameters:

file (FilePath) – The snapshot file.

save_critic_to_asset(neural_network_asset) → None

Save this critic’s weights to a ULearningAgentsNeuralNetwork asset.

Parameters:

neural_network_asset (LearningAgentsNeuralNetwork) – The asset to save to.

save_critic_to_snapshot(file) → None

Save this critic’s weights into a snapshot.

Parameters:

file (FilePath) – The snapshot file.

setup_critic(interactor, critic_settings=[], neural_network_asset=None) → None

Initializes this object to be used with the given agent interactor and critic settings.

Parameters:

  • interactor (LearningAgentsInteractor) – The input Interactor component

  • critic_settings (LearningAgentsCriticSettings) – The critic settings to use

  • neural_network_asset (LearningAgentsNeuralNetwork) – Optional Network Asset to use. If provided must match the given CriticSettings. If not provided or asset is empty then a new neural network object will be created according to the given CriticSettings and used.

use_critic_from_asset(neural_network_asset) → None

Use a ULearningAgentsNeuralNetwork asset directly for this critic rather than making a copy. If used during training then this asset’s weights will be updated as training progresses.

Parameters:

neural_network_asset (LearningAgentsNeuralNetwork) – The asset to use.