Gauntlet Automation Framework Overview

Framework to run sessions of projects in Unreal Engine that perform tests and validate results.

Choose your operating system:

Windows

macOS

Linux

Gauntlet is a framework to run things and validate results. It is specifically designed for, but not limited to, running Unreal sessions on a variety of platforms. An Unreal session is all of the processes needed to execute a game with the Unreal engine. For example, a multiplayer game may require four clients and a server.

Gauntlet does not require any specific game-side automation code or test framework — how your game performs tests is entirely up to you. There is however a Gauntlet Plugin that provides a useful TestController class to assist with puppeteering and monitoring a game instance. It is well suited to smoke tests that require several steps to execute, but it is entirely optional.

Gauntlet Does

  • Platform independent implementations of builds, devices, and tests as low-level units that provide the functional operations necessary to be configured and combined.

  • Provide high-level classes that can launch complicated configurations (e.g. 5v5 clients and a server) for games.

  • Provide utility functions for parsing log files, crashes, and accessing /saved data from devices.

Gauntlet Doesn't

  • Require specific Unreal-side code. The intention is that the execution within Unreal uses whatever makes the most sense. It could be something extensive such as the Unreal Automation Framework, or just some command-line parameters that the game interprets.

  • Create builds. You need to provide Gauntlet with a network or locally cooked build.

Architecture

Gauntlet offers three tiers of functionality. Earlier tiers may be (and are!) used in isolation for things like scripting device interactions, non-Unreal unit testing, etc whereas the latter tiers abstract a great amount of complexity behind simple user-configurable options.

Level 1

  • Interfaces (devices, builds, apps) that abstract the concept of devices, builds, installations, and processes.

    • ITargetDevice

    • IBuildSource

    • IBuild

    • IAppInstall

    • IAppInstance

  • A framework that supports sequential, parallel, and dependent execution of user-created tests.

    • TestExecutor

    • ITestNode

Level 2

  • Device and AppInstance implementations for all platforms Unreal supports (PC, Mac, PS4, Xbox One, Switch, iOS, Android).

    • TargetDeviceWindows

    • TargetDeviceMac

    • TargetDevicePS4

    • TargetDeviceXboxOne

    • TargetDeviceSwitch

    • TargetDeviceIOS

    • TargetDeviceAndroid

  • Classes that support the configuration and launch of one or more "roles" that exist in an Unreal session (e.g. four clients, one server)

    • UnrealAppConfig

    • UnrealSessionRole

    • UnrealSession

    • UnrealSessionInstance

Level 3

  • Classes that implement ITestNode and IBuildSource in the context of an Unreal Session

    • UnrealBuildSource

    • UnrealTestConfiguration

    • UnrealTestContext

    • UnrealTestNode

  • RunUnreal - a class that inherits from UAT's BuildCommand and configures a number of parameters before constructing a list of tests and then executing them.

Gauntlet Core Interfaces

gauntlet_overview_interfaces.png

Core Classes

TestExecutor

The TestExecutor class handles creation, queuing, and monitoring of a set of tests. Multiple tests can be executed in parallel if the TestNode supports such a thing. An example of where care is needed is port usage for Unreal Servers. For the first test a default port is fine, but when tests are executing in parallel both the server and client must use unique ports.

ITestNode

ITestNode is an interface class that describes the API all test nodes must implement.

Unreal Test Node

Unreal Test node implements ITestNode, largely by configuring an UnrealSession object and calling its relevant functions.

언리얼 엔진 문서의 미래를 함께 만들어주세요! 더 나은 서비스를 제공할 수 있도록 문서 사용에 대한 피드백을 주세요.
설문조사에 참여해 주세요
취소