For very diffuse Materials, you may be inclined to set this to zero. Resist! All Materials
have specular, see this post for examples . What you really want to do for very
diffuse Materials is make them rough.
Commonly, if we modify Specular, we do so to add micro occlusion or small scale shadowing, say from cracks represented in the normal map. These are sometimes referred to as cavities.
Small scale geometry, especially details only present in the high poly and baked into the normal map, will not be picked up by the renderer's real-time shadows. To capture this shadowing, we generate a cavity map,
which is typically an AO map with very short trace distance. This is multiplied by the final BaseColor before output and multiplied with 0.5 (Specular default) as the Specular output.
To be clear this is BaseColor = CavityOldBaseColor, Specular = Cavity0.5.
For advanced use, this can be used to control the index of refraction (IOR). We have not found this to be necessary for 99% of Materials. Below are Specular values based off of measured IOR.
Measured Specular Values: