In the game industry, the work on visual quality of effects is done at the intersection of art and technology. This is especially true for mobile applications.
In this article we will tell you how to use floating point numbers when debugging shaders for mobile devices on PC. We are sure that experience of Krasnodar studio Plarium will be useful for you no matter what engine you use.
How we identified the problem
In the process of creating effects, we have constantly encountered the fact that some of them crumble into pixels over time.
We investigated this problem and determined the causes of its occurrence: it turned out that not all gadgets are prone to artifacts. Mostly it concerns old mobile devices or models with weak characteristics. And some seemingly identical effects worked properly, then lost the smoothness of movements and acquired pixelation.
While analyzing the technical documentation of video processors for devices at risk, we found out that not all gadgets have full support for 32-bit floating-point numbers. This is what causes artifact and image problems.
How is it in binary?
Recall what floating point numbers look like in binary.
You can think of such a number as a container for storing a fixed number of significant digits and a comma anywhere in that number. Obviously, large numbers are less accurate.
For example, we have a significant number – 123456789. If we put a comma after the first character (1.23456789), we can change this number to within 0.00000001. But if we put a comma before the last character (123456789.9), the accuracy will be 0.1.
If we use 32-bit numbers, any such number accurately transmits 7 characters. This allows the application to run for about 70 hours without visible problems. We are talking about the time the user sees the game running, not paused or minimized. Quite a big margin, isn’t it?
But the thing is that many mobile devices only support 16-bit floating-point numbers in the pixel (fragment) shader. Such numbers have only 4 exact digits, which leads to a noticeable loss of accuracy when animating textures after 10-15 minutes.
In search of a solution :
- We have divided the causes of loss of accuracy into three groups :
- When transmitting – the value is converted from 32 bits to 16 bits.
- When calculating – attempting operations with numbers of different order.
- In overflow – any floating point number has its maximum and minimum value.
It’s not realistic to solve the problem on every single device, so we focused on visualizing the error on the PC when creating the effect and taking action to fix it in advance.
There is no way to directly emulate the 16-bit number mode on a PC. The graphics card driver automatically converts all 16-bit instructions to 32 bits, and it is only possible to visually track the loss of precision after a long time. Therefore, we wrote our own time manager and shader extensions to visualize the artifacts from the loss of precision.
Our "debugging" time starts several hours ahead of time. This way the artist can immediately see problems with the effect. The manager also automatically resets the time every time the application is paused or minimized. It is at this point that the jump in effects is least noticeable to the player.
With the introduction of the Scriptable Rendering Pipeline in the new version of Unity, we were able to do away with the need for a separate time management script, and do everything directly when rendering the frame.
Extensions for shaders
The idea was to overload a 32-bit floating point number the same way a 16-bit number is overloaded. You need to shift the significant part by some number of bits. But bitwise operations are not available for Shader Model 2.0, and on top of that, using a higher shader model causes the engine to stop warning effect artists about overcomputing for mobile devices, and the effect budget is exceeded.
We decided to make as accurate a simulation algorithm as possible to get the desired result.
- We add some big number to the original number. This big number itself is determined by the number of digits of the prime number.
- Subtract it. We need to trick the compiler, which will automatically reduce the operations. To do this we make a slight error in the number after the increase.
This way, by changing the shader a bit, we get a visually identical image for weak devices. This allows our artists to control the effects development process and reduces the number of errors in the work.
Hint : help
When working on effects, make sure that all the numbers responsible for the smoothness of the effect and requiring precision are in the range 0 to 1. Use the frac(x) operation to do this.
Basically, frac is the conversion of a floating point number to an integer and then subtracting that integer from the original number. That is, frac = x – floor(x).
Since this operation uses the original number in its original form, the loss of accuracy can also occur during the transfer stage of this number within the shader. Therefore, we recommend using a time that does not increase to large values. A good practice would be to force time zeroing at load times or at times when the game interface overlaps the game level.
The work done allowed us to significantly reduce the number of visual errors. Our artists were able to control the process of creating the effects without a long and complicated check on mobile devices.