---

SoftMasked Blend Mode

---

Overview

This page explains the SoftMasked material blending mode. It can be used instead of Masked to limit aliasing that is caused by the hard comparison done there (A pixel is either fully opaque or translucent). It solves the anti-aliasing problem of masked content is many cases. In some cases it falls back to a result similar to Masked. When used with care it can greatly improve the rendering quality of vegetation.

Transparent rendering allows to render the same content much softer but has other limitations (Doesn't affect depth of field, fog and no correct z occlusion). SoftMasked tries to combine the two methods keeping most of the benefits of both of them. The difference is less noticeable in screen-shots but more in motion as aliasing of Masked content appears to change depending on the viewer position and with animated content.

Here is how SoftMasked looks like when used on vegetation:

Notice the soft grass and how the leafs appear anti-aliased where the texture defines the border. Additionally ambient occlusion appears to be nicer as further grass contributes less to the depth buffer and therefore has less noisy SSAO artifacts.

This is the same content with the classic Masked material blend mode:

How to activate and tweak

The method can be activated simply by setting the Blend Mode in the Material Editor to Blend_SoftMasked. For good results, it is best to change the Opacity Mask Clip Value to a value near 0.7 (useful range is from 0.4 to 0.9). Higher values allow to use more of the soft transition but it also reduces the amount that is rendered in the classic Masked way (with the correct pixel depth for fog and Depth of Field).

Good content

Magnification of the texture means the texture resolution is lower than the pixel size on the screen. This should be avoided as the border of the object becomes bigger and sorting issues and fogging problems might become bigger than a pixel on the screen and therefore more noticeable.

The texture of the leafs on the left are magnified and the right ones not. Here Mipmapping together with a good black and white mask results in good anti-aliasing on the borders (This image doesn't show the problems).

Even without magnification the border can be big if the texture content has bigger areas with gray-scale content. This can often be avoided. Masked content anyway was resulting in a binary mask so the content can be altered to avoid big transition areas without affecting the result much (compared to Masked). This modification can be done in Photoshop. We use the following example of a bad texture mask:

In Masked this texture would be ok as all brighter than the threshold is opaque. In SoftMasked the grey parts would partly shine through and some might suffer from wrong sorting and fogging. The background of the content isn't even black which means all triangles appear to be partially occluding even in areas where they shouldn't. A good mask would look like that (Here the grey areas are minimized and the results are as good as possible):

To repair a texture quickly a few steps in Photoshop can be performed:

• Make sure the mask is visible (RGB or alpha channel)
• Change Image Size to 400% (horizontal and vertical). Make sure Resample image is set to bilinear.
• Optionally blur the image a bit (e.g. Gaussian blur 2.5 pixel)
• Apply Adjustments Threshold, tweak depending on content (In Masked this was tweaked with the Opacity Clip Mask value.)
• Change Image Size to 25% (horizontal and vertical). Make sure Resample image is set to bilinear.

This is how the resulting image looks like:

Performance

The method has good performance but is slower in CPU and GPU performance compared to Masked. It's also slower than Transparent because if required two passes.

GPU

Very little extra pixel shader instructions are required. The geometry needs to be rendered twice (EarlyZ and Base pass, possibly shadowmaps). The blending operation with the frame buffer should be neglectable on Xbox360 but might be noticeable on other hardware when using short shaders. For the base pass the objects needs to be rendered sorted from back to front.

CPU

The required sorting adds additional overhead to the CPU rendering cost (sorting function, more state changes). This is view dependent and is worse when multiple different materials needs to be rendered interleaved.

Limitations

• The current implementation is at an early stage to evaluable the potential and to be able to use it on Xbox360. Further improvements can solve some of the limitations but the additional cost might be a noticeable performance drop. A Xbox360 like preview for PC would be useful and could be one of next improvements.
• The current implementation only works well on Xbox360 and there only for ambient lighting and one dominant light (sun)!
• On PC it only works for the ambient lighting. Overlapping soft masked borders affected by the dominant light appear to be too bright. This is very noticeable on grass and less on less occluding objects.
• It's currently considered unsupported for Playstation 3 and shader model 4.
• SoftMasked rendering doesn't support more lights than ambient and one directional light
  (All SoftMasked objects need to be rendered in single pass otherwise the border appear to be too bright, as it can be seen on PC).
• Shadows are only supported on XBox360 and only from the single dominant light.
• Fog
  SoftMasked only affects the depth buffer only in areas of maximum opacity. As the fog rendering is based on the depth buffer some pixels become fogged too much. Tweaking the Opacity Mask Clip value to be smaller helps but then the result appears to be less soft. This problem can be seen in the following image (first is SoftMasked, second is Masked).