texture lookups in mental mill

One thing that confused me in the early beginning when I started writing shaders was this: Sometimes texture lookups that caused artifacts that looked somewhat like aliasing when previewing them in the realtime preview. Later I learned why and how that happens and how to avoid it. So I will present the problem and its solution:

Cutting texture coordinates

Whenever you 'cut' texture coordinates, this creates discontinuities. In this example I take a fraction of the texture and loop it over across my preview geometry. This is accomplished by using an fmod() operation. After performing the texture lookup and previewing the shader, you can see some jaggy artifacts that are especially noticeable and disturbing when moving the geometry. In the magnified region you see that between

the tiles some dark pixels show up.

Wrong Mip Map lookups

So what is going on here? You might have guessed already that the fmod() operation did something bad to our coordinates. Well, one could say so, but the operation itself is innocent. The reason for this behavior is that during rasterization the GPU tries to determine the MipMap level automatically by comparing the texture current coordinates with the values of the adjacent fragment or pixel. As long as the coordinates are continuous it can correctly determine the right MipMap level.

But when the coordinates were 'sliced', this created discontinuities. When such a discontinuity is reached, the difference between two neighboring pixels or fragments is so big, that the GPU chooses a very small MapMap which causes the artifacts in the realtime preview.

Fixing incorrect lookups by using custom derivatives

How can we remedy this situation? We need the texture lookup to choose the appropriate MipMap level, even though there is a discontinuity. That is why there is an overloaded version of the tex2D texture lookup in mental mill (which also exists in the Cg language, by the way).

tex2D(texture2D texture, float2 coordinates, float2 coord_dx, float2 coord_dy)

The second two arguments are the derivatives to use. First we need to know: What are derivatives? The derivatives indicate the changes from one pixel or fragment to the next. If the user performs a texture lookup without providing custom derivatives, the derivatives of the texture coordinates are automatically calculated and used.

We just saw that this operation failed. We also know why, so let's calculate our own derivatives and pass them to the texture lookup function above.

How to determine the correct derivatives

First we need to learn about the functions that calculate derivatives for us. These are:

ddx() and ddy()

These are overloaded functions which take a float, float2, float3, float4 or Color. The functions return a variable of the same type which contains the derivative. The derivative returns the difference of a variable between two fragments. ddx() calculates the difference in the x-direction while ddy() calculates the difference in the y-direction.

The derivatives that we really want to use are the derivatives of the original texture coordinates before we sliced them up. Why is that?

We didn't change the scaling of the sliced texture coordinates. We just set a different tiling. This means that we can use the derivatives of the original texture coordinates for the texture lookup.

So we can write:

float2 x_deriv = ddx(texture_coordinate[0].xy);

float2 y_deriv = ddy(texture_coordinate[0].xy);

result = tex2D(tex, coord, x_deriv, y_deriv);

Now, lets see how that affects the shader:

As you can see now, the aliasing artifacts are gone.

So whenever you see such artifacts, check if you are causing discontinuities in your texture coordinates. These are created by using the fmod operation for example, but if-else clauses that assemble texture coordinates based on a condition can cause this.

If that is the case, try to resolve these issues by using the overloaded texture lookup that uses derivatives. This is especially relevant if you intend to use your mental mill shader in a realtime application.

I hope next time you run into this problem you will remember this post and quickly resolve these issues.