Learn to use this Early Access feature, but use caution when shipping with it.
This feature is Early Access.
Features available in Early Access give you the opportunity to learn how they work, plan your pipeline, and create test content. You should use caution when using these features in production, as we are still working to get to shipping-quality performance, stability, and platform support. We support backward compatibility for assets, and the APIs for these features are stable.

Pixel Streaming Overview

People typically experience your Unreal Engine application on the same device that runs the gameplay logic and renders the game world to the screen, regardless of whether you build for a desktop platform, mobile OS, or console. Multiplayer networked games may distribute parts of the gameplay logic among multiple instances of the application, but each individual instance still does the work of rendering the game world locally for its own player. Even when you use the HTML5 deployment option to create a version of your Project that can run inside a Web browser, the game logic and rendering still happens locally within each user's Web browser.

With Pixel Streaming, however, you run your Unreal Engine application remotely, on a computer that users probably never see. For example, this could be a physical desktop somewhere inside your organization, or a virtual machine provided by a cloud hosting service. The Unreal Engine uses the resources available to that computer — CPU, GPU, memory, and so on — to run the game logic and render every frame. It continuously encodes this rendered output into a media stream, which passes through a lightweight stack of Web services. Users can then view that broadcast stream in standard Web browsers running on other computers and mobile devices.

Rendering in the cloud with Pixel Streaming

The result for the user is just like watching a video stream from a service like YouTube or Netflix, except for two things:

  • Instead of playing back a pre-recorded video clip, the stream is playing back the rendered frames and audio generated by the Unreal Engine in real time.
  • Users can control the experience from their browsers, sending keyboard, mouse, and touch events, plus custom events emitted from the player Web page, back to the Unreal Engine.

Benefits

Using the Pixel Streaming system offers several possible benefits:

  • It makes mobile devices and lightweight Web browsers capable of displaying better quality graphics than otherwise possible. They can show complex scenes at high resolutions, using rendering features that are only possible when rendering in a native desktop application with a powerful GPU.
  • The user doesn't need to download large executables or content files in advance, and doesn't need to install anything. The only thing the user needs to download is the media stream as it plays.
  • You can support multiple platforms without creating and distributing multiple separate packages. You package your application once for Windows, and people can use any platform to experience your content. Users can view the stream in any modern browser that supports the WebRTC connection model, which includes Google Chrome and Mozilla Firefox on desktop, iOS, and Android platforms. (See the Pixel Streaming Reference.)
  • The Pixel Streaming system contains a minimal number of components that are relatively easy for anyone to set up inside a local network. However, it's powerful enough for teams with experience in deploying web services to use as a basis for creating custom cloud-hosted platforms.
  • Pixel Streaming uses the WebRTC peer-to-peer communication framework for the lowest possible latency between the user and the Unreal Engine application.

Architecture

The following image summarizes the components of a simple Pixel Streaming setup:

Pixel Streaming architecture

Components

  1. Pixel Streaming Plugin - This Plugin runs inside the Unreal Engine. It encodes the final results of every rendered frame using H.264 video compression, packs those video frames along with the game audio into a media stream, and sends that stream to the WebRTC Proxy Server.
  2. WebRTC Proxy Server - The WebRTC Proxy Server is responsible for relaying the media stream produced by the Pixel Streaming Plugin to one or more connected browsers, over direct peer-to-peer connections.
  3. Signaling and Web Server - The Signaling and Web Server is responsible for negotiating connections between browsers and the WebRTC Proxy Server, and for providing browsers with the HTML and JavaScript environment that plays back the media stream.

Connection Process

  1. When you start up all the Pixel Streaming components, the Pixel Streaming Plugin running inside the Unreal Engine first establishes a connection to the WebRTC Proxy Server, and begins sending it the media stream.
  2. The WebRTC Proxy Server then connects to the Signaling and Web Server, to say that it is ready to accept new connections.
  3. A client connects to the signaling server, which serves it an HTML page that contains a player widget and control code written in JavaScript.
  4. When the user starts the stream, the signaling server negotiates the establishment of a direct connection between the client browser and the WebRTC Proxy Server.
    In order for this connection to work, the browser and the WebRTC Proxy Server need to know each other's IP address. If both are running on the same network, they are typically visible to each other directly at their own IP addresses. However, network address translation (NAT) services running between the two endpoints may change the externally visible IP address of either party. Resolving this usually involves the use of a STUN or TURN server, which tells each component what its externally visible IP address is. For details, see the Hosting and Networking Guide.
  5. As soon as the connection between the client and the WebRTC Proxy is established, the WebRTC Proxy starts streaming media directly to the browser. Input from the client is sent by the player page's JavaScript environment directly to the WebRTC proxy, to be relayed back to the Unreal Engine.
  6. The Signaling and Web Server maintains its connection to the browser and the WebRTC Proxy Server even after the media stream starts playing, so that it has the ability to kick users out of the stream if needed, and to handle disconnections initiated by the browser.
For step-by-step instructions on setting up the architecture shown above, see Getting Started with Pixel Streaming.