Pixel Streaming Sample Project

The Pixel Streaming Demo showcase demonstrates how you can design Unreal Engine content for viewers to experience as a live stream in a Web browser or mobile device.

Google has informed developers of a vulnerability in versions (earlier than M102) of WebRTC. Impacts, workarounds, and updates can be found here.

The Pixel Streaming Demo showcase demonstrates how you can design Unreal Engine 5 (UE5) content for people to experience as a live stream, using a Web browser on either a desktop or on a mobile device. It includes:

  • An HTML player page that plays back a media stream generated by your UE application, and that offers custom UI elements for controlling the Engine remotely.

  • An UE5 Project that is already set up to generate a media stream using the Pixel Streaming Plugin, and to respond to the custom commands issued from the HTML player page.

You can use this sample as a model for building your own custom HTML5 player that interacts with your UE5 content.

Prerequisites:

  • Make sure that you understand the basics of the Pixel Streaming system.

  • Follow all the instructions in the Getting Started guide at least once to make sure that you have everything that you need installed and working with the default player page.

Getting Started

  1. Find the Pixel Streaming showcase on the Learn tab of the Epic Games Launcher, and use it to start a new Unreal Engine Project.

  2. Ensure you have the relevant branch of the Pixel Streaming Infrastructure

    It's vital that you have the version of the Infrastructure that matches your version of Unreal Engine, e.g 5.2 to 5.2. For more information on the Infrastructure, head to this page here

  3. Open the PixelStreamingDemo.uproject file in the Unreal Editor.

  4. Follow the process described in the Getting Started with Pixel Streaming page to:

    • Package the Project or start it from the Unreal Editor as Standalone Game.

    • Start the Signaling and Web Server (Using the Infrastructure you have above)

    • Alternative: Use the Pixel Streaming Toolbar to start a signalling server.

  5. Open a Web browser, and navigate to the showcase.html player page being hosted by your Signaling and Web Server. For example: http://<your-server-ip-address>/showcase.html

Interacting with the Showcase HTML

The custom frontend for the showcase allows you to control a variety of elements in the scene. The example drop down in the left panel contains a variety of different controls, each showcasing different elements of the scene and Pixel Streaming itself.

GettingStartedDropdown.JPG

  • Send Data to UE: This section allows you to change the character and character skin in your running application.

    SendData.JPG

  • Send Commands to UE: This section contains commands you can send to UE5 to change the resolution the application is running at, as well as toggle extra stat information onscreen.

    SendCommands.JPG

Refer to the details panel at the bottom of the screen for information relevant to the "Send Commands" and "Send Data" categories respectively.

Controlling the Player Page

When you have the Pixel Streaming system set up correctly, and you use a supported Web browser to access the custom showcase.html player page, you can use the controls described in the following sections to interact with the running Unreal Engine application.

Pixel Streaming showcase player page

  1. Use the drop down menu on the left hand side of the page to switch between different examples, such as sending commands and sending data to UE

  2. This details panel provides information about your current selected examples.

  3. The viewer widget itself offers direct mouse and touch control over the Unreal Engine application, as well as information about the current Pixel Streaming feature:

    Control

    Effect

    Click and drag, or touch and drag

    Rotates the camera around the current character.

    Mouse wheel

    Zooms the camera in and out.

  4. Click these buttons to switch to full screen, open stream settings and open stream information. These are the same buttons present in the default interface :

    Control

    Effect

    Enter fullscreen mode

    Switches the viewer to full-screen mode. Press the Esc key to exit.

    Open settings

    Opens the stream settings panel. This panel is provided by default with the Pixel Streaming Infrastructure and allows extensive configuration of a running stream.

    Open stream information panel

    Opens the stream information panel. This panel contains the real time WebRTC session stats of the stream such as bitrate, packet loss and framerate.

Understanding the Custom UI Events

The custom HTML player page, showcase.html, uses the showcase.ts file found in the PixelStreamingInfrastructure/Frontend/implementations/EpicGames/src/directory to control its various commands

For the capture of mouse, keyboard, and touch events relayed to the Unreal Engine application, navigate to the PixelStreamingInfrastructure/Frontend/library/src/Inputs/ directory as seen below: InputsFrontEnd.JPG

Most of the UI elements in the player page are implemented by calling the emitUIInteraction() function to pass different information back to the Unreal Engine application. To understand how any piece of the UI works under the hood:

  1. First, find the UI item you want to understand in the showcase.ts file, and see what JavaScript function is set to trigger when that button is pressed. For example, in this code block we set up a button to change to the first character skin.

        const skin1Btn = document.createElement("button");
        skin1Btn.onclick = () => { this._onSkinClicked(0); }
  2. Look at the implementation of the bound function in the showcase.ts file. For example, the _onSkinClicked(); function takes a parameter to set which of the character skins we wish to use. It then passes the following JSON object to the emitUIInteraction() function, like so:

        private _onSkinClicked(skinIndex : number) {
        this._pixelStreaming.emitUIInteraction({ Skin: skinIndex });
  3. In the Unreal Engine Project, these events are responded to by the Blueprints/Pawn/Orbit_Controller class.

    Orbit_Controller Double-click this class to open up its Event Graph.

  4. In the Bind JSON Events area, you'll see how the Bind Event to OnPixelStreamingInputEvent node is used to register the OnJSONEvent event as a handler each time emitUIInteraction() is called in a connected browser.

    Bind Event to OnPixelStreamingInputEvent

  5. Each time the OnJSONEvent event triggers, the Blueprint calls Get Json String Value to check the keys and values stored in the JavaScript object tht was passed by the player page to the emitUIInteraction() function. It uses those values to decide what other events it should trigger. For example, when the event receives an input that contains a Skin field, it calls the Change Skin event.

    Get JSON String Value

Pixel Streaming Widget

The Pixel Streaming features widget in the top left of the game window is designed to demonstrate a few useful elements of Pixel Streaming. These functions can be easily adapted into your own purposes and are included in Pixel Streaming with minimal effort.

FeaturesWidget.JPG

By pressing the "F" key during play, the widget will cycle through each option and explain how to test each feature. The features are as follows:

  • Freeze Frame: Freeze or unfreezes the active Pixel Stream on a single frame. Note that the application is still running in the background

    FreezeFrame.JPG

  • Audio Component: Allows you to stream your microphone audio input through the stream via WebRTC and play it back through UE5

    AudioComponent.JPG

  • Pixel Streaming Stats: Toggles the display of the stat PixelStreaming and stat PixelStreamingGraphs information via console.

    PSStats.JPG

Tags
Help shape the future of Unreal Engine documentation! Tell us how we're doing so we can serve you better.
Take our survey
Cancel