Techniques for Creating Normal Maps

The process used varies slightly, but for the most part, I dump down the saturation (in hue/saturation) to greyscale and run a highpass filter on it in photoshop.

This neutralizes the image a bit, and evens out the tones out somewhat. Play with the settings to see what works best.

After I've got that to where I want it, I convert it to 16 bit. This will create a cleaner normal map than just running the nvidia filter on an 8 bit image.

Then just run the nvidia normal map filter on the image. If it is too noisy undo and use a Gaussian blur on it to soften it a bit then run the nvidia filter again.

I usually set the nvidia filter from 20-60 and get a nice deep normal map. All of our detail bump-maps are done this way.

Then you can just overlay them, or use hard light on top of your processed normal map and adjust the opacity of the layer in photoshop to get the desired effect.

On using Max to generate normal maps...

Often it can also be the cage itself that does not lend itself to a proper process. Specifically, if you are optimizing your hi poly and low poly elements by removing all the areas of the lowpoly that are mirrored on the UV's, like if there was a rectangle and the long sides were mirrored, you had deleted the additional side so you have an open face. Sometimes this or comparable setup's to this can cause the cage to be skewed and changing the size of it will only cause it to become more skewed.

I often rely on not using the cage so that max's way of grabbing information is based more on firing off in a number of directions based on the normals of each of the lowpoly faces which is more similar to what is done in SHtools or more recent versions of 3D Studio Max.

It's good to do your test process's without any Supersampler options enabled and at a smaller size until you have ironed out as many of the kinks in your setup's grab distance (offset max/min height) as possible, as these can be done more speedily.

On correcting problems with normal maps in Photoshop...

Once you have found what you can and can't fix you can set the colour of your missed parts to be the base normal map colour (127,127,255) so it doesn't show up as an annoying un antialiased blob of colour and then you can just massage it out by hand in photoshop if there is no 2d workaround.

On errors that occur when rendering normal maps in Max...

The target slot message is unimportant, it exists only because max expects that you are doing a normal map process in order to create a max material and so is confused that you are not assigning the outputted images to form a max material.

System exception during render means you screwed something small and annoying up in your hi poly objects that shouldn't really be an issue if not for the fact that max is REALLY REALLY picky!!!

You're going to need to trawl through all your hi poly assets to narrow down where the error lies. Below are some of the common errors that cause this:

• An empty layer/named selection or object;
• A 3d part that has undergone too many operations and now needs its Xform reset;
• An Edit poly model that has some weird voodoo thing happening that can only be cured by converting to edit mesh and then back to edit poly;
• Or more voodoo that can only be cured by exporting the selected model section out as an obj and then reimporting it back into the scene.

The best way to track down these things is to incrementally check each part and add it to the process list of your low poly and try processing again. Sometimes it will crash the moment you add it to the list, so save often but this is the easiest way to track them down.

Sometimes it is simply because you have too many polys or lights or materials in the scene and need to split the scene into multiple scenes for separate process and reassembling later on.

Instead using Nvidias Normalmaptool you could use Crazybump http://www.crazybump.com/ which generates way better normalmaps and also diffuse, specular and displacement maps from simple photosource.