Virtual Shadow Maps

Overview of Unreal Engine 5's high resolution shadowing designed with film-quality assets and large dynamically lit open worlds in mind.

Windows
MacOS
Linux

Virtual Shadow Maps (VSMs) is the new shadow mapping method used to deliver consistent, high-resolution shadowing that works with film-quality assets and large, dynamically lit open worlds using Unreal Engine 5's Nanite Virtualized Geometry, Lumen Global Illumination and Reflections, and World Partition features.

Goals of Virtual Shadow Maps

Virtual Shadow Maps have been developed with the following goals:

  • Significantly increase shadow resolution to match highly detailed Nanite geometry

  • Plausible soft shadows with reasonable, controllable performance costs

  • Provide a simple solution that works by default with limited amounts of adjustment needed

  • Replace the many Stationary Light shadowing techniques with a single, unified path

Conceptually, virtual shadow maps are just very high-resolution shadow maps. In their current implementation, they have a virtual resolution of 16k x 16k pixels. Clipmaps are used to increase resolution further for Directional Lights. To keep performance high at reasonable memory cost, VSMs split the shadow map into tiles (or Pages) that are 128x128 each. Pages are allocated and rendered only as needed to shade on-screen pixels based on an analysis of the depth buffer. The pages are cached between frames unless they are invalidated by moving objects or light, which further improves performance.

Enabling Virtual Shadow Maps

In the Project Settings under Engine > Rendering in the Shadows section, you can set what Shadow Map Method your project supports, whether Virtual Shadow Maps or the conventional Shadow Maps that have been used in previous Unreal Engine releases.

Existing projects will need to opt-in using the project setting, or console variable r.Shadow.Virtual.Enable. New projects use Virtual Shadow Maps by default.

vsm-projectsetting.png

What happens with existing shadow methods when VSMs are enabled?

When VSMs are enabled, they replace a variety of existing shadow methods in Unreal Engine:

  • Stationary precomputed shadows, including 2D Distance Field and shadow factor "shadow maps"

  • Pre-shadows

  • Per-object / inset shadows

  • Cascaded Shadow Maps (CSMs)

  • Distance Field Soft Shadows from Mesh Distance Fields

  • Movable dynamic shadows for local lights

Fully baked shadows from Static Lights will still work as before (when not using Lumen). Their contributions are entirely represented in the baked lightmaps and there is no runtime lighting evaluated at all. Stationary lights will use the indirect diffuse contribution from any baked lightmaps, but their direct lighting and shadows are evaluated dynamically (the same as Movable lights) when VSMs are enabled.

Due to VSM's high resolution and accuracy, the screen space Contact Shadow feature controlled with the Contact Shadow Length property is no longer necessary to achieve sharp contact shadows. It still may have value when used to pick up cheaper shadows from objects set to not render into shadow maps but is not recommended otherwise as it will be less accurate than the shadows VSMs will create.

Ray-traced shadows still take precedence over VSMs as they generally provide the highest quality solution.

Soft Shadows with Shadow Map Ray Tracing

Shadow Map Ray Tracing (SMRT) is a sampling algorithm used with virtual shadow maps to produce more plausible soft shadows and contact hardening. Objects that cast shadows farther will have softer shadows than objects casting shadows closer to a shadow-receiving surface.

For example, the mesh pictured below is tall and casts its shadow over a long distance. Shadows near the base are sharper than those farther away.

vsm-softshadowscontacthardening.png

Example with a Point Light casting soft shadows and contact hardened shadows using Shadow Map Ray Tracing.

Past methods based on Percentage-Closer Filtering (PCF) would over-blur and reduce the visual impact of high-resolution geometry and shadows.

Percentage-Closer Filtering blurs uniformly, | removing important detail

Shadow Map Ray Tracing produces plausible | soft shadows with contact hardening

The SMRT algorithm works by shooting a number of rays towards the light source, but instead of evaluating intersections with geometry — like conventional raytracing does — a number of samples along the ray are projected and tested against the virtual shadow map to achieve soft shadowing and contact hardening.

Shadow rays are distributed based on the light using Source Radius on local lights or Source Angle on Directional Lights.

vsm-details-locallight-sourceradius.png

vsm-details-directionallight-sourceangle.png

Local Light Source Radius

Directional Light Source Angle

Local lights have no Source Radius by default, compared to Directional Lights which start with a low Source Angle. When either are set with an appropriate value, SMRT produces real-time soft shadowing with contact hardening, like the example below using a Point Light with a Source Radius of 10.

Point Light with Source Radius 0

Point Light with Source Radius 10

Controlling Penumbra Shadow Quality

The penumbra shadow's softness and quality is set by console variables for both local and Directional Lights. They include their own defaults that are generally good for most scenes.

The softness quality ois set by the number of rays being used, and both local and Directional Lights uses seven rays by default. Use the console variables r.Shadow.Virtual.SMRT.RayCountLocal and r.Shadow.Virtual.SMRT.RayCountDirectional to adjust the number of rays. Fewer rays show visible samples in the penumbra shadow, and having zero rays disables SMRT and reverts to single-sample hard shadows.

Directional Light Source Angle: 5.0 | SMRT Ray Count: 1

Directional Light Source Angle: 5.0 | SMRT Ray Count: 7 (Default)

Furthermore, the number of shadow samples taken along each ray's path can be set for both local and Directional Lights to control the maximum softness. Fewer shadow map samples are cheaper to render, but limit the amount of penumbra softness that can be achieved by the light's shadows.

Use the console variables r.Shadow.Virtual.SMRT.SamplesPerRayLocal and r.Shadow.Virtual.SMRT.SamplesPerRayDirectional to adjust the number of samples used. Using values between four and eight (default) shadow map samples works well.

Dragging the slider will show what happens when a Directional Light with a Source Angle of 5.0 uses Shadow Map Samples that number 0, 2, and 8 (default).

Limitations of Shadow Map Ray Tracing

Quality of SMRT is generally good with the default setting, but there are limitations inherent to using the data from a single shadow map projection rather than testing against true geometry.

Penumbra Size Limits

The shadow's penumbra is clamped for local and Directional Lights to avoid sample divergence from the ray origin that can become increasingly "bent" compared to the ideal test along the ray itself. Using reasonable values for Source Radius on local lights and Source Angle on Directional Lights will avoid results that are too extreme, limiting the extent that the ray can diverge in various ways. Values that are too large can both impact performance and cause shadow penumbrae to visually warp as the camera gets near to them.

vsm-penumbrashrinkage.gif

Local and Directional Lights can use the console variables r.Shadow.Virtual.SMRT.MaxRayAngleFromLight and r.Shadow.Virtual.SMRT.RayLengthScaleDirectional to loosen or tighten the clamped extents.

Inconsistent Penumbra

Since the Virtual Shadow Map only stores the first depth layer where naive iteration misses intersections with any occluders behind the first one, a variety of light leaking artifacts can happen where the occluders overlap. These types of light leaks are resolved using a gap-filling heuristic that extrapolates depths behind the first occluder based on depths seen before the point of occlusion.

This works well in resolving lighting leaks, but causes penumbrae on surfaces parallel to the light to shrink in size. There is currently no direct way to counteract this effect other than to keep penumbra sizes reasonable.

vsm-smrt-inconsistent_penumbra.png

Examples of inconsistent penumbra shadows.

Penumbra Artifacts

By default, Virtual Shadow Maps only guarantees that the samples around the ray origin (the receiver pixel) will be present. As the algorithm traverses the ray, it can encounter unmapped pages where no shadow data exists. A variety of techniques are used to reduce the impact of this, including dilating the page mappings slightly and having various fallbacks in place during iteration. Even still, occasional artifacts stemming from missing pages can happen, especially around edges of the screen when zoomed in on soft penumbra. These artifacts will manifest as noisy light leaks in shadowed areas.

Like other limitations with VSMs, issues can mostly be avoided by keeping shadow penumbra sizes reasonable and avoiding zooming in to the point where they cover large portions of the screen.

Clipmaps for Directional Light

A single virtual shadow map does not provide enough resolution to cover large areas. Directional Lights use a clipmap structure of expanding ranges around the camera, with each clipmap level having its own 16K VSM. Each clipmap level is the same resolution but covers twice the radius of the previous one.

Directional Light Clipmap Visualization

Virtual Shadow Map Pages Visualization

By default, clipmap levels 6 through 22 are allocated virtual shadow map page tables. This means that the default settings have the most detailed clipmap covering 64 cm (2^6 cm) from the camera position, and the broadest clipmap covering about 40 kilometers (2^22 cm). The cost of virtual clipmap levels is insignificant if nothing is present in them, so these defaults work well to cover very large scenes with fairly high resolution near the camera.

The first and last levels can be adjusted using the console variables r.Shadow.Virtual.Clipmap.FirstLevel and r.Shadow.Virtual.Clipmap.LastLevel.

The resolution allocated to a given pixel is a function of the distance away from the clipmap origin — the camera. It is possible to bias the resolution of a Directional Light's virtual shadow maps using the console command r.Shadow.Virtual.ResolutionLodBiasDirectional. A value of 0 picks the resolution required based on the perspective projection of the camera.

Projective aliasing — when a shadow is cast on a surface almost parallel to the light direction — is still possible, even at high resolutions, but it can be reduced partly by biasing the resolution. Like Mip biasing in textures, lowering the value by -1 doubles the resolution of shadows with the associated performance tradeoffs. The default value of -0.5 provides a reasonable balance for many scenes, but scenes with high projective aliasing (due to low lighting angles or many flat, parallel surfaces) or high contrast shadows may need to decrease it further.

Local Lights

Spot Lights use a single 16k VSM with a mip chain rather than clipmaps to handle shadow level of detail. Similarly, Point Lights use a cube map of 16k VSMs, one for each face.

Local lights provide a significant increase in resolution compared to traditional shadow maps. It is possible to run out of virtual resolution with very large local lights, and care should be taken to use Directional Lights where possible in these cases.

Scene lit by a Spot Light

Virtual Shadow Map Pages for a Spot Light

Appropriate mip levels are picked by projecting the size of the screen pixels into shadow map space. Like Directional Lights, it's possible to bias the resolution for local lights using the console variable r.Shadow.Virtual.ResolutionLodBiasLocal.

For Early Access, per-light resolution controls are not available but may be added in a future release.

Caching

Reusing shadow map pages from previous frames is key to maintaining high performance with Virtual Shadow Maps, especially in complex scenes. Caching is enabled by default, but can be turned off for debugging purposes using the console variable r.Shadow.Virtual.Cache.

Currently, the cache invalidation is very conservative: if the light moves, all cached pages for the light are invalidated. In instances where you have a slowly moving light, or a shifting Directional Light changing the time of day, VSM pages will effectively not be cached at all.

In situations like time-of-day changes, it's recommended to quantize the changes by some small amount to allow cached pages to live for some number of frames since the tiny differences in direction will not be noticeable.

Similarly, any objects moving within the projected area of a cached page will invalidate that page, which includes rigid motion, skeletal animation, and any materials using World Position Offset (WPO). A good way to find objects that may cause shadow invalidations using the visualization Draw only Geometry Causing VSM Invalidation found in the Level viewport under Show > Visualize.

vsm-vis-drawonlygeocausinginvalidation.png

Since pages are only rendered for on-screen pixels, changing camera visibility due to movement or disocclusion can reveal new pages that need to be rendered. Generally, as long as camera movement is relatively smooth, it is not a major source of new pages. On the other hand, things to watch out for are very fast movement that is very close to objects, large disocclusions, and camera cuts.

Beyond Early Access, caching should improve in the following ways:

  • Caching static geometry in pages separately from dynamic geometry so that a single moving object does not require re-rendering all of the static geometry in the pages that it covers.

  • Adding a priority system and per-frame update budgets to allow finer control over the cost of rendering shadows, such as allowing shadow resolution to temporarily lower in cases where too many pages need updates.

Coarse Pages

Depth buffer analysis is used as the primary method of marking pages that are needed to render. There are some systems that need to sample shadows at more arbitrary locations though, such as volumetric fog and forward-rendered translucency. Most systems only need low-resolution shadow data that gets filtered and blurred through other data structures.

To accommodate shadowing in these situations, Coarse Pages are marked to ensure that at least low-resolution shadow data is available across the whole domain for sampling. Local lights simply mark the root mip page while Directional Lights can mark a series of pages at various low detail levels to form some coarse clipmaps.

Depending on the scene, coarse pages can create performance issues. This is particularly true for non-Nanite dynamic geometry since they are effectively rendering low-resolution shadows over large regions of space, which can result in draw-call bottlenecks.

Local light coarse pages can be toggled off with r.Shadow.Virtual.MarkCoarsePagesLocal, if not needed.

Directional Light coarse pages can be toggled off with r.Shadow.Virtual.MarkCoarsePagesDirectional, or the range of clipmap levels that coarse pages are marked in can be modified with r.Shadow.Virtual.FirstCoarseLevel and r.Shadow.Virtual.LastCoarseLevel

In a future release, a more elegant solution would be where some of these effects can be marked for localized pages directly in advance rather than the current, overly conservative coarse pages being used.

GPU Profiling and Optimization

Unreal Engine provides tools that help you check performance in your projects, and the GPU Profiler (or platform-specific tool) is a good starting place to troubleshoot and debug performance issues.

There are two main performance buckets where VSM costs will show up: Shadow Depths and Shadow Projection. The tradeoffs in each of their categories are relatively independent of one another.

Be aware that commands used to list statistics, such as stat gpu and associated counters, can provide unreliable timings, especially if your project's performance is CPU-bound.

Shadow Depths

The Shadow Depths category refers to the cost of rendering geometry into shadow maps.

vsm-gpuprofile.png

The RenderVirtualShadowMaps category contains all rendering of Nanite geometry into VSMs. All Directional Lights are rendered in a single Nanite pass, and all local lights are done in a second pass.

RenderVirtualShadowMapsHw handles the rendering of non-Nanite geometry. Each visible light has a separate pass with individual draw calls for various objects and instances, same as conventional shadow maps rendering.

vsm-gpuprofile-shadowdepths.png

Atlas and Cubemap, along with other similar passes, following the VSM passes are rendering conventional shadow maps. There are still a small number of geometry types that are not yet supported in the Virtual Shadow Maps path, and those follow the old path.

For geometry types that are not yet supported by VSMs, their passes should be cheap with little or no geometry being rendered. These passes — along with the associated memory allocations — will disappear as soon as they are supported by VSMs.

The cost of the Shadow Depths pass with VSMs is directly related to how many shadow pages need to be rendered, and how much geometry needs to be rendered into them. Non-Nanite geometry is more expensive to render into VSMs than Nanite geometry. For this reason, it's recommended to enable Nanite on all supported geometry, including low-poly meshes. The only exception is for features not yet supported by Nanite Virtualized Geometry.

Understanding the Number of Pages Being Rendered

The on-screen stats for VSM pages used can give an idea of how many are being used and where to look to resolve potential problems.

Use the following console variables in succession to enable stats:

r.ShaderPrintEnable 1 r.Shadow.Virtual.ShowStats 1

vsm-stats.png

Stat Name

Description

MAX PAGES

The maximum number of physical pages that virtual shadow maps can use.

PAGES

The total number of requested shadow map pages by the current view. It should always be less than MAX PAGES, otherwise corruption can occur. (See the Known Issues(#knownissuesforearlyaccess) section below)

CACHED PAGES

The number of pages that are already in the virtual shadow map page cache, and that do not need to be rendered in the current frame. Cached pages are very low cost and will have almost no impact on performance.

CACHED PERCT

The percentage of CACHE PAGES that are in the virtual shadow map PAGES.

DYN INVAL

The number of otherwise-cached pages that were invalidated by dynamic geometry in the previous frame. These pages need to be rendered again because something moved that covers them.

CACHED FB PCT

The percentage of DYN INVAL pages that are invalidated.

Poor performance in shadow depths is typically associated with a high number of pages being used and lots of dynamic invalidation happening, which leads to poor caching of VSMs. The total page counts are a function of the average number of lights that affect each pixel on screen. They can be lowered by reducing screen resolution, shadow resolution (using the console variables for resolution LOD bias), light extents, or the number of shadow casting lights.

Decreasing cache invalidations is best done by first visualizing what is causing the invalidations (see Caching), then finding shadow casters that are moving that do not need to be. Having a material that uses World Position Offset is enough to trigger dynamic invalidation. Because of this, it's recommended to use LODs to switch to materials without WPO at distances where the effect is no longer obvious.

The Shadow Projection category is the cost of sampling shadow maps using Shadow Map Ray Tracing. These passes are located under ShadowedLights and there will typically be one VSM projection pass per associated light. The most expensive pass is usually the main SMRT loop in VirtualShadowMapProjection. The rest should be relatively low cost.

The VSM projection pass described in this section is different from the experimental One Pass Projection described in the next section.

vsm-shadowprojection.png

Shadow projection is purely a function of the total number of shadow map samples that are taken across the screen, and performance doesn't depend on the number of pages or caching.

When SMRT is being used, it comes down to:

  • Average lights per pixel

    • The more lights touching large portions of the screen, the more expensive rendering will be. Lights covering small numbers of on-screen pixels are cheaper, although there are still some fixed costs per light.

    • Care should be taken to avoid having the majority of pixels on screen taken up by several large lights.

  • Rays per pixel

    • Visible softness of shadows affects performance due to the adaptive ray count. Try reducing the Source Radius of local lights or Source Angle of Directional Lights before lowing any of the ray and sample counts.

    • See the Shadow Map Ray Tracing section of this page for console variables to adjust max ray counts for local and Directional Lights.

    • If the profiling category for VirtualShadowMapProjection says RayCount: Static, the slow path is always being used. This could also be a sign that DX11 is being used, which is not recommended.

  • Samples per ray

    • See the Shadow Map Ray Tracing section of this page for console variables to adjust max ray sample counts for local and Directional Lights.

In general, shadow projection costs are much easier to control than caching costs at this time. Shadow projections have a number of console variables and parameters to adjust the tradeoffs between shadow softness, noise, and performance.

One Pass Projection

This feature is considered experimental.

Even though smaller lights have a lower cost, they still have some fixed pass overheads. This is being addressed by developing a single-pass shadow projection solution where the majority of local lights in a scene can efficiently evaluate their shadowing in one pass. The shadowed contribution from these lights can then be applied all at once using clustered shading.

This experimental path is enabled using the console variables r.UseClusteredDeferredShading 1 and r.Shadow.Virtual.OnePassProjection 1. For scenes with a lot of small local lights, this can result in significant improvements to performance.

In the captures below, the left shows a shadow projection pass happening per light compared to the right side's one pass projection.

vsm-onepassprojection.png

Currently One Pass Projection shows up under the NonShadowedLights category due to an issue in Early Access, but the output image is identical in both cases.

This pass is currently still in development and not enabled by default for several reasons:

  • Various light features are not yet supported and are ignored in this pass, such as Light Functions and IES profiles. For features that are feasible, they will be supported by the new path. In cases where a feature isn't able to be supported, it will fall back on the current pass per light path being used.

  • There is a limit of 10 shadowed lights per screen tile. Any additional lights will be treated as unshadowed. This will improve and be more flexible in future releases.

  • Currently, three bits are used per pixel per light to encode attenuation. So, only up to seven SMRT rays can be represented in the One Pass Projection path. Beyond Early Access, this may remain a tradeoff, but encoding is likely to be adjusted based on the SMRT ray count settings directly.

In Early Access, this path is primarily a proof of concept for evaluating performance in scenes with many local lights. Once the current limitations have been addressed, it will become the new default path.

Supported Platforms

Virtual Shadow Maps are currently supported on PlayStation 5, Xbox Series S|X, and PCs with graphics cards meeting these specifications, using the latest drivers with DirectX 11 or 12:

  • NVIDIA: Maxwell-generation cards or newer

  • AMD: GCN-generation cards or newer

Virtual Shadow Maps work with DirectX 11 but with significantly reduced performance. It is highly recommended to use them with DirectX 12 for the best experience.

Known Issues for Early Access

Virtual Shadow Maps in Early Access are still actively being developed. There are a number of known issues and limitations to using them currently, and they are currently targeted at high-end desktops and next-generation consoles. Additional work to make them more scalable will come in future releases of the engine.

Overflow of Dynamic GPU Resources

The majority of the VSM pipeline runs on the GPU, which presents challenges for dynamic allocation of temporary resources. As such, there are currently a few areas that can cause overflow, like complex scenes or complex lighting situations, which has potential to cause shadow corruption. Caching makes it possible to cache corrupt shadow pages, making these types of issues more visible.

Increase the Max Number of Physical Pages

The first resource that can overflow is the total number of physical pages allocated to VSMs. Using the console variables r.ShaderPrintEnable 1 and r.Shadow.Virtual.ShowStats 1, in succession, is the best way to identify when this happens.

vsm-stats.png

If the number of Pages exceeds Max Pages corruption is likely to occur somewhere. Sometimes this manifests visually as a checkerboard pattern, or corrupt or missing shadows. The solution is to increase the maximum number of physical pages beyond the default 2048. Do this using the console variable r.Shadow.Virtual.MaxPhysicalPages. Increasing the number of physical pages will increase VRAM usage, but may be necessary in scenes with many lights or high shadow map resolutions.

Incorrectly Cached Shadows

More subtly, certain Nanite intermediate buffers can overflow while rendering shadow pages, resulting in missing geometry. These types of issues are often more subtle because they are related to the number of new pages that must be rendered each frame, and how much geometry is rendered into them.

Rendering may be working at a steady rate with issues, but a camera cut or dynamic geometry might suddenly result in a lot of pages being rendered causing overflow and subsequently incorrect cached shadows.

If this issues is suspected, it can be checked by:

  1. Disable VSM caching with r.Shadow.Virtual.Cache 0.

  2. Enter into the console NaniteStats VSM_Directional for Directional Lights or NaniteStats VSM_Perspective for local lights. This displays Nanite and VSM stats in the active level viewport.

vsm-nanitevsmdirectionalstats.png

If the Candidates or visible Clusters values exceed the values set by r.Nanite.MaxCandidateClusters (default value is 8,388,608) or r.Nanite.MaxVisibleClusters (default is 2,097,152), corruption is possible. See Nanite Virtualized Geometry for more information.

In many cases, the frame-to-frame values will not exceed the limits when caching is enabled. But if they ever exceed it with caching disabled, then it is possible for a frame that invalidates most, or all, of the cache to render corrupt cached data.

In this situation, if VRAM allows, MaxCandidateCluster and MaxVisibleClusters console values should be increased.

Virtual Shadow Map Quality

The quality of VSMs is good in most situations, but there are a few remaining limitations.

Materials

For materials, subsurface and transmissions are not yet implemented. If a material is using them, that material will be shadowed as though it is opaque.

Shadow Resolution

VSMs provide significantly increased resolution compared to conventional shadow maps, but shallow light angles (or projective aliasing) and very large local lights can exhaust the available virtual resolution. This can present itself as box-like shadows and bias issues depending on the geometry's surface. Directional Light clipmaps are much less susceptible to running out of resolution, but very narrow camera fields-of-view can eventually exhaust those as well.

There is no simple solution to solving projective aliasing with shadow maps. Even with VSMs, some care must be taken to avoid the worst cases.

Directional Lights with Low Light Angles

There is a rare issue that can occur with Directional Lights at very low light angles, such as being just above the horizon. Due to the depths in cached pages being currently rescaled each frame, it is possible to lose precision when moving quickly close to a surface that is almost parallel to the light direction.

If you suspect this type of artifact, try disabling VSM caching with the console variable r.Shadow.Virtual.Cache 0 to see if the issue disappears. You may also need to rule out GPU resource overflows.

This particular issue with Directional Lights at low light angles will be fixed in a future release of the engine.

Light Parameters User Interface

One goal of VSMs is to have minimal parameters, and thus many of the shadow parameters on lights are ignored when using virtual shadow maps. For instance, parameters related to shadow bias, cascades, inset shadows, and Stationary Light are ignored. In Early Access these parameters are all still visible, but in future releases there are considerations to hide these when the project uses Virtual Shadow Maps over conventional shadow mapping.

Map Check Warnings

Virtual Shadow Maps cause some inaccurate map check warnings to happen:

  • The Lighting needs to be rebuilt message does not appear when VSMs are enabled, even though lighting may in fact need to be rebuilt when not using Lumen dynamic Global Illumination and Reflections features. While Stationary direct lighting is dynamic with VSMs enabled, Stationary indirect lighting is still baked.

  • Warnings regarding preshadows can be ignored as they are not relevant when using VSMs.

Draw Only Geometry Causing VSM Invalidation Visualization

The visualization that draws only geometry that invalidates VSMs may include some effects and particles that do not actually invalidate anything. Content that doesn't invalidate but appears will be cleaned up and removed in future releases of the engine.