Creating 3D Widget Interaction

Traditional UI consists in 2D space on top of a 2D or 3D world (for example menus, health bars or score displays). However there may be times when you want a piece of UI to exist within your 3D environment that players can interact with (a keypad, a virtual menu, an inventory or other interactive UI element). In VR projects, this is even more prevalent and UMG supports these types of interactions with the Widget Interaction component.

In this how-to we create a widget containing a button that can be placed in the level and clicked on by a player.

Steps

For this how-to, we are using the Blueprint First Person Template with Starter Content enabled.

  1. Inside the Content/FirstPersonBP/Blueprints folder, open the FirstPersonCharacter Blueprint.

    WidgetInteraction_01.png

  2. Click the Add Component button and search for and add the Widget Interaction component.

    WidgetInteraction_02.png

  3. Left-click drag and drop the Widget Interaction component on top of the Sphere component attached to FP_Gun.

    WidgetInteraction_03.png

    This will allow us to point the Widget Interaction component in the direction the gun is pointing.

  4. In the Details panel for the Widget Interaction component, zero out the Location and check the Show Debug option.

    WidgetInteraction_04.png

  5. Inside the Event Graph, add a Right Mouse Button Mouse Event node.

  6. Drag the Widget Interaction component into the graph, then drag off it and add a Press Pointer Key node.

    WidgetInteraction_05.png

  7. Connect to the Pressed pin of the Right Mouse Button node and set the Key to Left Mouse Button. By default UMG registers click events as a result of a Left Mouse Button key press, using this node will allow us to simulate that when the Right Mouse Button is pressed. You could use any Key Input event to call this function (such as a trigger press or button press to simulate a Left Mouse Button click).

    WidgetInteraction_06.png

  8. Add a Release Pointer Key node (set to Left Mouse Button) and connect it to the Released pin of the Right Mouse Button node.

    WidgetInteraction_07.png

  9. In the Content Browser, create a Widget Blueprint called InteractiveWidget.

    WidgetInteraction_08.png

  10. In the Visual Designer, remove the Canvas Panel then add a Button with a Text widget on top of the button. For this example we are purely adding a button that can be clicked on. The text on the button we will indicate the number of times the button has been clicked by the player. You can create any type of interactive widget and interact with it using the Widget Interaction component (things like sliders, combo boxes, check boxes, etc. can be interacted with in world space).

    Click for full view.

  11. In the Details panel for the Button, under Style/Hovered change the Tint to a different color.

    WidgetInteraction_10.png

  12. In the Details panel for the Text, change the Text content to 0 and increase the Font Size to 48.

    WidgetInteraction_11.png

  13. In the Graph, create a Text Variable called Value then Compile and set Value to 0. This is the variable that we will update when the player clicks on the button which we will bind to our Text widget.

    WidgetInteraction_12.png

  14. On the Designer tab for the Text widget, click the Bind option next to Text and select the Value property.

    WidgetInteraction_13.png

  15. In the Details panel for the Button widget under Events, click the + next to On Clicked. This will fire whenever the button is clicked on which is called from inside our FirstPersonCharacter Blueprint.

    WidgetInteraction_14.png

  16. On the Graph tab, hold Ctrl and drag in the Value variable then off it use To String followed by String To Int. We are converting the Text to a String, then from a String to an Int so that we can increase the value when the button is clicked.

    WidgetInteraction_15.png

  17. Off the String To Int pin, use an Integer + Integer node set to + 1.

  18. Hold Alt and drag in the Value variable to set it, then connect the remaining pins to the OnClicked event as shown. The ToText(int) node will be automatically created when you connect the out pin of Integer + Integer to the text input pin on Value.

    WidgetInteraction_16.png

  19. In the Content Browser, create a new Blueprint based on Actor called ExampleWidget and add a Widget component to it. This will represent the 3D version of our InteractiveWidget Widget Blueprint.

    WidgetInteraction_18.png

  20. In the Details panel of the Widget component under User Interface, change the Widget Class to InteractiveWidget.

    WidgetInteraction_19.png

  21. In the Content Browser, drag the ExampleWidget Blueprint into the level and scale, rotate and position as desired.

    WidgetInteraction_20.png

  22. Click Play to play in the editor.

End Result

When you play in the editor, you should have something similar to below. You can use the Left Mouse Button to fire your weapon and when pointing at the button, you enter the Hovered state (causing it to change its style). When you Right Mouse Button click on the button, the text value will increase each time it is pressed.

As noted during the steps, we are using the Right Mouse Button to register click events however you can use gamepad button clicks to register Pointer Key Presses/Releases or (as shown below) Motion Controller Trigger presses. Here we point our Right Motion Controller (with an attached Widget Interaction component) at the button and press the Right Trigger to register a button click.

WidgetInteraction_21.png