Creating 3D Widget Interaction

Choose your OS:

Traditional UI consists of 2D space on top of a 2D or 3D world (for example menus, health bars or score displays). However, there may be times when you want a piece of UI existing within your 3D environment that players can interact with (a keypad, a virtual menu, an inventory or other interactive UI element). In VR projects, this is even more prevalent and UMG supports these types of interactions with the Widget Interaction component.

In this how-to, we create a widget containing a button that can be placed in the level and clicked on by a player similar to the video below.

Steps

For this how-to, we are using the Blueprint First Person Template with Starter Content enabled.

  1. Inside the Content/FirstPersonBP/Blueprints folder, open the FirstPersonCharacter Blueprint then add a component of the Widget Interaction type.

    WidgetInteraction_02.png

  2. Left-click drag and drop the Widget Interaction component on top of the Sphere component attached to FP_Gun.

    WidgetInteraction_03.png

    This will allow us to point the Widget Interaction component in the direction the gun is pointing.

  3. In the Details panel for the Widget Interaction component, zero out the Location and check the Show Debug option.

    WidgetInteraction_04.png

  4. Inside the Event Graph, add a Right Mouse Button Mouse Event, drag in the Widget Interaction component and add a Press Pointer Key node.

    WidgetInteraction_05.png

  5. Connect to the Pressed pin of the Right Mouse Button node and set the Key to Left Mouse Button.

    WidgetInteraction_06.png

    By default UMG registers click events as a result of a Left Mouse Button key press, using this node will allow us to simulate that when the Right Mouse Button is pressed. You could use any Key Input event to call this function (such as a trigger press or button press to simulate a Left Mouse Button click).

  6. Add a Release Pointer Key node (set to Left Mouse Button) and connect it to the Released pin of the Right Mouse Button node.

    WidgetInteraction_07.png

  7. In the Content Browser, create a Widget Blueprint called InteractiveWidget.

    WidgetInteraction_08.png

  8. In the Visual Designer, remove the Canvas Panel then add a Button with a Text widget on top of the button.

    Click for full view.

    For this example, we are purely adding a button that can be clicked on. The text on the button we will indicate the number of times the button has been clicked by the player. You can create any type of interactive widget and interact with it using the Widget Interaction component (things like sliders, combo boxes, check boxes, etc. can be interacted with in world space).

  9. In the Details panel for the Button, under Style/Hovered change the Tint to a different color.

    WidgetInteraction_10.png

  10. In the Details panel for the Text, change the Text content to 0 and increase the Font Size to 48.

    WidgetInteraction_11.png

  11. In the Graph, create a Text Variable called Value then Compile and set Value to 0.

    WidgetInteraction_12.png

    This is the variable that we will update when the player clicks on the button which we will bind to our Text widget.

  12. On the Designer tab for the Text widget, click the Bind option next to Text and select the Value property.

    WidgetInteraction_13.png

  13. In the Details panel for the Button widget under Events, click the + next to On Clicked.

    WidgetInteraction_14.png

    This will fire whenever the button is clicked on which is called from inside our FirstPersonCharacter Blueprint.

  14. On the Graph tab, hold Ctrl and drag in the Value variable then off it use To String followed by String To Int.

    WidgetInteraction_15.png

    We are converting the Text to a String, then from a String to an Int so that we can increase the value when the button is clicked.

  15. Off the String To Int pin, use an Integer + Integer node set to + 1.

  16. Hold Alt and drag in the Value variable to set it, then connect the remaining pins to the OnClicked event as shown.

    WidgetInteraction_16.png

    The ToText(int) node will be automatically created when you connect the out pin of Integer + Integer to the text input pin on Value.

  17. In the Content Browser, create a new Blueprint based on Actor called ExampleWidget and add a Widget component to it.

    WidgetInteraction_18.png

    This will represent the 3D version of our InteractiveWidget Widget Blueprint.

  18. In the Details panel of the Widget component under User Interface, change the Widget Class to InteractiveWidget.

    WidgetInteraction_19.png

  19. In the Content Browser, drag the ExampleWidget Blueprint into the level and scale, rotate and position as desired.

    WidgetInteraction_20.png

  20. Click Play to play in the editor.

End Result

When you play in the editor, you should have something similar to below. You can use the Left Mouse Button to fire your weapon and when pointing at the button, you enter the Hovered state (causing it to change its style). When you Right Mouse Button click on the button, the text value will increase each time it is pressed.

As noted during the steps, we are using the Right Mouse Button to register click events however you can use gamepad button clicks to register Pointer Key Presses/Releases or (as shown below) Motion Controller Trigger presses. Here we point our Right Motion Controller (with an attached Widget Interaction component) at the button and press the Right Trigger to register a button click.

WidgetInteraction_21.png