2. Lens Calibration
Different camera lenses bend and distort the picture in different ways (think of a fisheye lens). You don’t want that distortion to be carried over into the captured scene, so use this lens calibration step to figure out how to undistort the image.
During this step, you will need a printout of the checkerboard image that you downloaded along with the calibration tool. In this step, the calibration tool continuously takes screenshots using the checkerboard pattern as a baseline.
Hold the printout at different locations in front of the camera. Use different angles and depths, especially around the edges of your frame, so that the calibration tool gets a wide collection of samples. The text at the top of the screen will change to let you know when the calibration tool has collected enough samples.
If your camera is set to auto-focus, you might find it helpful to disable auto-focus during this step.
After the calibration tool has collected enough samples, the tool will show you a preview of the undistorted feed. If the feed looks acceptable, press Enter to continue. If needed, you can add more samples, or press R to reset and start over.
A good way to tell if the undistortion process is working is to find elements in frame that should be straight (wall corners, doorways, etc.). Lens distortion will often warp straight edges (especially towards the edge of the frame).
The reported “reprojection error” lets you know how accurate the process thinks it is. The lower the reprojection error value, the better. You’re in great shape if the value is under one!
Angle of View
Probably the most important use of this step is to approximate the camera’s angle-of-view (or FOV, ‘field-of-view’).
The FOV determines how much of the world the camera can see, and it is important that the FOV of our virtual camera matches this as closely as possible.
If you already know the FOV of your camera (in degrees), you can use the mrc.FovOverride console variable to set it.
Most USB camera manufacturers list the diagonal-FOV (DFOV) for their devices. However, we are interested in the horizontal-FOV (in degrees). You can compute the horizontal-FOV from the diagonal-FOV with the below formula, using the height (h) & width (w) of the resolution you selected in the previous step.
There are also ways to calculate the horizontal-FOV for DSLR lenses.
If you do not know your camera’s specific FOV offhand, don’t worry, this step will approximate it for you.
If you have a camera with an adjustable lens, you will want to be very careful not to adjust the zoom after you’ve completed calibration. Adjusting the zoom will change the physical camera’s field-of-view (FOV), but not the virtual one. The virtual camera will use the FOV that was used during calibration. If you adjust the camera's zoom after calibration, you will need to repeat the calibration process.