Jason Ronald from Xbox
Lightning tracking has always been the "holy grail" of computer graphics, says Jason Ronald, head of program management for the Xbox console.
The technique simulates a three-dimensional image by calculating every ray of light and promises impressive lighting effects with realistic shadows and reflections.
The method finds where it bounces, collects information about what these objects are, and then uses it to establish a pixel and compose a scene.
Although the techniques have been around for a long time, "we just didn't have the processing power to deliver all of this in real time," says Ronald.
In Hollywood, special effects have been using ray tracing for a decade. For an important sequence, computers can produce overnight a single frame.
To do this in real time games, you need to condense it into 1/60 of a second. Processing speed has now achieved the task.
Lightning strokes create more realistic shadows and light
Technology company Nvidia announced last year that its latest graphics processing units (GPUs) will handle real-time lightning tracking.
Working with Nvidia, Microsoft has developed a lightning tracking update for Windows 10.
Microsoft and Sony have announced that their next consoles, the Xbox Series X and PlayStation 5, will have lightning tracking capabilities. Both systems are built on Advanced Micro Devices (AMD) hardware.
And now the technology is being incorporated into some of the most popular games in the world.
Minecraft, which first appeared in 2009, allows players to build vast complex structures. Developed by Swedish gaming studio Mojang, it is currently the best-selling video game in history.
The creators of Minecraft released an initial version of the game on April 16. A general release will follow towards the end of the year.
"It looks very different from the traditional rendering mode and it looks better," says Jarred Walton, hardware editor at PC Gamer.
The big problem, he says, will be price barriers for now. "The only way to play is with a PC that has a graphics card that costs at least $ 300 (£ 240)," he says.
Lightning tracing requires expensive processor chips
Until now, developers used another technique called rasterization.
It first appeared in the mid-90s, is extremely fast and represents 3D shapes in triangles and polygons. The closest to the viewer determines the pixel.
Next, programmers need to employ tricks to simulate the appearance of lighting. This includes light maps, which calculate the brightness of surfaces ahead of time, says Ronald.
But these hacks have limitations. They are static, so fall apart when you move. For example, you can zoom in on a mirror and find that your reflection has disappeared.
Programmable shaders started to appear around 2001. They did a much better job of 3D lighting tasks, but they demanded much more computational power.
"If we put it all in one game, the most amazing processor in the world would just not be, it's too much," says Ben Archard, senior rendering programmer at 4A Games Malta, developers behind a post-apocalyptic 2019 game called Metro Exodus.
There were ways around this. If a programmer wanted to simulate the foggy light from the fog, instead of calculating all the points, he could just calculate a sample of them. (These are called stochastic, statistical, or Monte Carlo approaches.)
Underwater scenes are Kasia Swica's favorite use of ray tracing
But with these workarounds, "you quickly lose that realism in one scene," notes Kasia Swica, senior program manager for Minecraft, based in Seattle.
Lightning strokes do better with realistic shadows in real time or reflections lurking in water or glass.
"My favorite thing with ray tracing is to dive into the water," says Miss Swica.
"You get really realistic reflections and refractions, and pure rays of light also appear," she says.
With blockages worldwide due to the coronavirus pandemic, the need for people to feel close while isolated "will accelerate" the progress of technology, says Rev Lebaredian, vice president of simulation technology at Nvidia in San Francisco.
"With virtual and augmented reality, we are beginning to feel that we are in the same place together," he says.
Therefore, coronavirus will drive demand and progress, agrees Frank Azor, AMD's chief game solutions architect.
Impressive lighting is possible, even without the trace of rays
A "diabolical problem" with lightning tracking involved how shaders can call other shaders if two rays interact, says Andrew Goossen, a Microsoft technician who works on the Xbox Series X.
GPUs work on problems like lightning in parallel: making parallel processes talk to each other is complex.
Working with technical problems to improve ray tracing will be the main task "in the next five to seven years of computer graphics, at least", says Ronald.
Meanwhile, game companies will use other techniques to make the game appear smoother.
Earlier this month, Epic Games, creator of Fortnite, released its latest game architecture, Unreal Engine 5.
It uses a combination of techniques, including a library of objects that can be imported into games such as hundreds of millions of polygons and a hierarchy of details treating large and small objects differently to save on your processor resource demands.
For most game makers, these "tricks and tricks" will be good enough, says Walton.