Camera Lens Calibration Quick Start Guide

Quick Start guide for the Camera Calibration plugin.

Choose your operating system:

Windows

macOS

Linux

Goals

In this Quick Start guide, you will go through the steps of connecting and calibrating a video feed using the Camera Calibration plugin.

Objectives

  • Connect your camera to Unreal Engine so it provides a live feed.

  • Calibrate the camera by using the Camera Calibration plugin.

1 - Required Setup

For this guide we will use a Blackmagic Ultra HD camera, a Panasonic Lumix lens and an HTC Vive Tracker 3 to control the CineCamera Actor in the scene.

Unreal Engine supports only a limited range of ultra-wide camera lenses at this time.

  1. Create a new Unreal Engine project. Select the Film, Television, and Lived Events category and click the Next button.

    Select the Film, Television, Live Events category

  2. Select the Virtual Production template and click the Next button.

    Select Virtual Production template

  3. Enter the file location and project name and click the Create Project button.

    Click the Create Project button

  4. Once the editor is loaded, click Settings > Plugins to open the Plugins Menu.

    Open the Plugins Menu

  5. Select the Virtual Production category and Enable the Camera Calibration and LiveLinkXR plugins. Select Yes on the popup box and click the Restart Now button to restart the editor.

    Enable the Camera Calibration plugin

    Enable the LiveLinkXR plugin

    Click the Restart Now button

Section Results

You enabled the Camera Calibration and LiveLinkXR plugins and restarted the editor. You are now ready to calibrate your camera.

2 - Setting your Scene

  1. Go to the Place Actors panel and search for CineCamera Actor. Drag the Actor into your Level.

    Place a CineCamera actor in your level

  2. In the Place Actors panel search for Camera Calibration Checkerboard. Drag the Actor into your Level.

    Place a Camera Calibration Checkerboard actor in your level

  3. In the Content Browser, click the View Options button and select the Show Engine Content and Show Plugin Content options.

    Show plugin content

  4. In the Content Browser, navigate to CameraCalibration Content > Devices > Tracker. Drag the BP_UE_Tracker3 Blueprint into your Level.

    Navigate to the tracker

  5. Select the CineCamera Actor in your Level and navigate to the Details panel. Scroll down to the Filmback section and enter the matching Sensor Width and Sensor Height values for your physical camera.

    Enter the sensor width and height

  6. With the CineCamera Actor selected, go to the Details panel and click the Add Component button. Search for and select the Live Link Controller.

    Add the Live Link Controller component

  7. Go to Window > Live Link to open the Live Link window.

    Open the Live Link window

  8. Go to Source > LiveLinkXR Source and click the Add button to add your connected Vive trackers. You should now see the trackers connected using Live Link.

    You can learn about developing for HTC Vive and SteamVR devices by reading the Developing for SteamVR documentation.

    Add the trackers as a source

    Receiving tracking information

  9. Since we aren't using an actual FIZ source, we need to create a virtual source using LiveLink Blueprint Virtual Subjects. Right-click in the Content Browser and select LiveLink > Blueprint Virtual Subject. Click the dropdown, select LiveLinkCameraRole and click the OK button. Name the Blueprint VirtualPrestonFIZ.

    Create a new virtual subject

    Select the LikeLink Camera Role

  10. Double-click the VirtualPrestonFIZ Blueprint to open it. Click the + Variable button to add a new variable. Name the variable Focus. Go to the Details panel and set the Variable Type to Float. Enable the Instance Editable checkbox.

    Add a new variable

    Name it Focus and set it as Float

  11. Repeat the above step and create two additional Float variables named Zoom and Iris.

    Create Zoom and Iris

  12. Right-click in the Event Graph then search for and select Update Virtual Subject Static Data. Connect the Update Virtual Subject Static Data node to the Event On Initialize node. Right-click the Static Data pin in the Update Virtual Subject Static Data node and select Split Struct Pin.

    Add Update Virtual Subject Frame Data node

    Right click the Frame Data pin and click Split Struct

  13. Enable the Focal Length, Aperture and Focus Distance checkboxes, as seen below.

    Enable Focal Length, Aperture, and Focus Distance

  14. Right-click in the Event Graph then search for and select Update Virtual Subject Frame Data. Connect the Update Virtual Subject Frame Data node to the Event On Update node. Right-click the Frame Data pin in the Update Virtual Subject Frame Data node and select Split Struct Pin. This event is triggered on every tick and will be used to update the FIZ data available for each frame.

    Add Update Virtual Subject Frame Data node

    Right click the Frame Data pin and click Split Struct

  15. Connect the Zoom variable to the Focal Length pin. Connect the Iris variable to the Aperture pin. Connect the Focus variable to the Focus Distance pin. Compile and Save the Blueprint.

    Add Update Virtual Subject Frame Data node

  16. Go to Source > Add Virtual Subject to open the Virtual Subject window. Select the VirtualPrestonFIZ subject and click the Add button.

    Add the trackers as a source

    Select the Virtual Preston FIZ subject

    Virtual subject is now set up

  17. Right-click within the Content Browser and select Miscellaneous > Lens File to create a Lens File asset. Name the asset LumixLens.

    Create a Lens File asset

  18. Select your CineCamera Actor and go to the Details panel. Scroll down to the Live Link section and click on the dropdown menu next to Subject Representation. Select your tracker from the list.

    Add your tracker to the CineCamera

  19. With the CineCamera Actor selected, go to the Details panel and click the Add Component button. Search for and select the Live Link Controller to add another component. Scroll down to the Live Link section and click on the dropdown next to Subject Representation. Select your virtual subject from the list.

    Add your virtual subject to the CineCamera

  20. In the Details panel, scroll down to the Camera Role section and click the dropdown menu next to Lens File. Search for and select LumixLens.

    Add your Lens File to the CineCamera

  21. Select the BP_UE_Tracker3 Blueprint and go to the Details panel. Select the LiveLink Component Controller and scroll down to the Live Link section. Click the dropdown menu next to Subject Representation and select your tracker.

    Add your subject to the Tracker

  22. Select the CameraCalibrationCheckerboard Actor and go to the Details panel. Scroll down to the Calibration section and enter the number of rows and columns, as well as the Square Side Length. For this example, we are using the image below for the measurements.

    Checkerboard example

    Enter the calibration information

  23. For better visibility, add materials to the Odd Cube Material and Even Cube Material slots.

    Add materials

    Resulting checkerboard after adding materials

Section Results

In this section you added a CineCamera Actor, a Camera Calibration Checkerboard Actor, and a BP_UE_Tracker3 Blueprint to your Level. You connected your tracker using Live Link and configured your actors correctly. You are now ready to calibrate your lens.

3 - Calibrating your Lens

  1. In the Content Browser, double-click the LumixLens asset to open it. Click the Calibration Steps panel and select the Lens Information tab. Go to the Lens Info section, enter your lens information and click the Save Lens Information button.

    Select the Lens Information tab

    Enter the lens information

  2. Click the Lens File Panel and select Focus.

    Select Focus

  3. Click the + button and enter a value of 0 for Input Focus. Enter a value of 100 for Encoder Mapping. Click the Add button to add this data point to the graph. Repeat this step and enter a value of 1000 for Encoder Mapping and a value of 1 for Input Focus.

    Enter the focus information for a value of 0

    Enter the focus information for a value of 1

  4. Select Iris and enter a value of 1.8 for Encoder Mapping and a value of 0 for Input Focus. Click the Add button to add the data point to the graph. Repeat this step and enter a value of 4.5 for Encoder Mapping and a value of 1 for Input Iris.

    Enter the iris information for a value of 0

    Enter the iris information for a value of 1

  5. Go back to the Calibration Steps panel and select the Lens Distortion tab. Open the checkerboard image in full screen and point your camera towards it. With the checkerboard in full view, click the viewport to capture an image. Repeat this step multiple times to capture several images.

    Repeat this process at multiple zoom levels and focus values for the best possible calibration.

    Point the camera to the checkerboard

  6. Once you have taken several images from different angles, click the Add To Lens Distortion Calibration button. Click OK on the popup window to accept the calibration data.

    Add the images for calibration

  7. Select the Nodal Offset tab and under the Nodal Offset section click the Nodal Offset Algo dropdown menu. Select Nodal Offset Checkerboard.

    Select the Nodal Offset Checkerboard algorithm

  8. Point the camera toward the checkerboard on the screen again and click the viewport. This will detect the corners of the image. Click the Apply to Calibrator button to align the Camera Calibration Checkerboard Actor with the onscreen checkerboard .

    Click on the Apply to Calibrator button

    The checkerboards are now aligned on the screen

  9. Now you can calibrate the Nodal Offset of the virtual camera. Set the Nodal Offset Algo to Nodal Offset Points Method, and the Calibrator to your tracker. Set the Calibration Point to PointLed.

    Enter the Nodal Offset information

  10. Hold the tracker's light towards the camera and click the light in the viewport. Repeat this step multiple times to create several data points. Click on the Add To Nodal Offset Calibration button.

    Click on the light several times to create points.

    If the tracking is not as accurate as expected, you can repeat this process using additional points.

    The trackers are now aligned

  11. Close the Lens File window and verify that the CineCamera Actor is now moving correctly as you move your tracker.' The tracker is setup correctly

Section Results

In this section you used the Lens File to enter your lens information, calibrate lens distortion, and add the correct nodal offset. Your CineCamera Actor is now simulating your physical camera's position, rotation, and lens distortion.

Help shape the future of Unreal Engine documentation! Tell us how we're doing so we can serve you better.
Take our survey
Dismiss