Reworking the fog

[illwieckz]

This thread is dedicated to discussing technical (re)implementations for the fog.

As other transparent things, fog works differently in naive blending and linear blending.

The currently implementation is to use in GLSL values from the alpha channel of a “fog” image that is generated by the renderer.

I noticed that linearizing it seem to bring interesting results in linear blending, at least for the atcshd map.

Then @slipher said:

The fog image is pretty pointless, all it really does it calculate sqrt(x). We should just do that in GLSL. I think it only exists because someone ported the code from GL1 in an overly mechanical way.

So, disregarding to naive and linear blending, we may drop that image and do the computation in GLSL.

Now, about that computation, @slipher said:

Square root doesn't make much sense as a model for the fog; probably they just tried random functions and picked something that looked OK. Instead of sqrt(a*x), 1 - exp(b*x) would be an obvious model to use (as if the fog is formed from layered alpha blending). a/b is a constant based on the fog density.

So maybe that square root computation for naive blending is already a workaround for the naive blending being broken by design, and linearizing is like another random guess to get something that pleases, a bit like the known fact that using quadratic light attenuation with naive blending is somewhat canceling the mistake, while both light attenuation and blending should be linear to be done properly.

So, we may need two computations:

• one for naive blending (we have to keep sqrt() for compatibility I guess)
• one for linear blending

Once those two computations are defined, we have to define how to do them. We can do them in a precomputed image the GLSL code picks value from, or we may do the computations in GLSL directly.

Doing the computations in GLSL directly would void the image sampling (and then, binding, I guess), but switching the computation would require to compile the two different codes, while using images can use the same compiled GLSL while we just switch the image.

Please share any thought you may have on the topic!

[VReaperV]

• one for naive blending (we have to keep sqrt() for compatibility I guess)
• one for linear blending

I don't see why you'd need 2 different ones in the shader when shaders work in the linear space regardless.

[slipher]

Here are some things I want to do with fog. Some of them I have done on a local branch.

• Fix global fog being drawn twice with material system (I already fixed that with the default renderer)
• Fix fog distance to use the real distance, instead of only counting distance along the view axis. When you look directly at something, it is more fogged than if you look at it out of the corner of your eye. Both Quake 3 and global fog are programmed this way.
• Don't use the model coordinate system because this is wrong for scaled models. This bug is just barely noticeable in our assets, for example with the Adv Dragoon which is scaled by 2x.
• NUKE fog image and replace it with a simple computation in the GLSL.
• For "Quake 3" fog, instead of drawing a corresponding fog surface for every opaque surface inside the fog, draw the whole fog with a single draw call, using the depth buffer to find the depth in fog like the global fog shader does. The 6 outer faces of the fog brush will be rendered. For each fragment calculate the intersection with an inner face. The length in fog will be the minimum of the 2 lengths from the intersection and the depth buffer.

So, we may need two computations:

* one for naive blending (we have to keep `sqrt()` for compatibility I guess)

* one for linear blending

Once those two computations are defined, we have to define how to do them. We can do them in a precomputed image the GLSL code picks value from, or we may do the computations in GLSL directly.

Doing the computations in GLSL directly would void the image sampling (and then, binding, I guess), but switching the computation would require to compile the two different codes, while using images can use the same compiled GLSL while we just switch the image.

Well, for the other shaders that needed this so far you used a uniform u_SRGB rather than having a compile variant. We could do that here too for consistency. Though with the new lazy shaders implementation it doesn't hurt us so much anymore to proliferate compile-time variants.

[illwieckz]

• one for naive blending (we have to keep sqrt() for compatibility I guess)
• one for linear blending

I don't see why you'd need 2 different ones in the shader when shaders work in the linear space regardless.

One of the linear computations is delinearized before display, so we may have to compensate for this transformation.

[slipher]

I just uncovered a big bug affecting fog in this line:

Daemon/src/engine/renderer/tr_shade.cpp

Line 1634 in ed4ffa8

fogDistanceVector[ 3 ] += 1.0 / 512;

This is seemingly intended as an epsilon intended to make sure the distance from the viewer to a vertex doesn't go below 0. But it's in a vector scaled by 1 / (8 * distanceToOpaque), which means it can actually be a big distance! For example, if the distanceToOpaque is 4096, this artificially adds 64 units to the distance. I believe this is the major reason why fogs have hard edges instead of fading out gradually.

If we just remove that, it changes the appearance by a lot. So we may have to keep it for compatibility. Omitting the fake distance boost is something we could change when defining new fog semantics for sRGB maps.

[illwieckz]

Do we know if that bug was there in Tremulous or Quake 3?

[VReaperV]

Quake 3

[VReaperV]

Don't know if that's global fog or not though.

[illwieckz]

It's not global fog because it doesn't fill the whole map / screen, also global fog is from Wolf:ET, while Q3 has… Q3 fog.

[slipher]

Well, the fogDistanceVector[3] += 1.0/512; line is there in ioq3.

@VReaperV What's the name of the map?

[slipher]

By the way if testing with ioq3, make sure to use the GL1 renderer for a fair comparison. I actually tried ioq3 with fog the other day and the GL2 one, besides possibly using different code, was buggy: the fog would visibly change in brightness as you moved toward/away from it.

[illwieckz]

Yes, I was going to suggest that. 😁️

I expect ioq3 renderer2 to not be as tested as the original one and there may be remaining bugs on some stuff (also, it's now less tested than ours).

[slipher]

Found those screenshots. This is one with GL1 on an older version of atcshd. Edges look pretty hard...

[VReaperV]

@VReaperV What's the name of the map?

q3dm4