At GTC 2016 this week, NVIDIA’s Vice President of Graphics Research demonstrated a novel prototype display running at an incredibly high refresh rate, all but eliminating perceptible latency.
Update (4/6/16, 11:08AM PT): An earlier version of this story transposed ‘17,000Hz’ in place of the correct 1,700Hz.
When it comes to latency in virtual reality, every part of the pipeline from input to display is a factor. You’ll often hear the phrase ‘motion to photons latency’ which describes the lag between the instant you move your head to the moment that the display responds to that movement. Between those two points are several sources of latency, from the detection of the input itself, to the rendering, to the time it takes for the display to illuminate its pixels.
For desktop-class VR, current state of the art VR headsets have displays running at 90Hz, which means that they’re capable of showing 90 images per second. And while we’ve seen that 90Hz is more than sufficient for a comfortable VR experience, NVIDIA Vice President of Research David Luebke says that ever higher refresh rates could improve the VR experience by further reducing latency.
At GTC 2016 this week, Luebke demonstrated an experimental display with a refresh rate that’s almost 20 times faster what we see in current consumer head mounted displays. Running at a whopping 1,7000Hz, the display was mounted on a rail system which allowed it to be rapidly moved back and forth. When shaken vigorously, the image on the display stayed locked in place to an impressive degree. Even when magnified closely, the image on the screen seemed entirely fixed in place.
A 90Hz display shows an image every 11 milliseconds, while this 1,700Hz display shows an image every 0.58 milliseconds.
“…if you can apply this to a VR display, that kind of ultra-low latency would help things stay rock-solid in the environment, to the point that the display would no longer be a source of latency. So this is effectively a zero latency display,” said Luebke.
One thing I find particularly interesting about this (and all VR displays in general) is that while the object on the screen appeared to be fixed in space to our eye, in reality, the image is racing back and forth across the display, illuminating many different pixels across the screen as it goes. The illusion that it’s still is actually evidence of how quickly it can move, which is curiously counterintuitive.
“You could put this thing in a paint shaker and it would appear to stay solid… it’s very cool,” Luebke said.
Of course, for this level of tracking, you also need extremely low latency input. Thus a second reason for the rail system is revealed; Luebke told me that wheels on the rails feed the movement of the carriage almost instantaneously into the system. Without such precision and low latency input, even a display this fast as the one demonstrated wouldn’t appear to show such a steady image, highlighting the need for low latency across the entire ‘motion to photons’ pipeline.
While less than 20ms of latency from input to display is generally considered good enough for VR, Luebke said that things get better toward 10ms and there’s even measurable benefits down to as low as 1ms of latency.
Until we can brute-force our way to zero latency with super high refresh rates like Luebke’s demonstration, a technique called low persistence is employed by modern VR headsets to capture some of the benefits of a super fast display, namely blur reduction. Low persistence works by illuminating the display only briefly, then turning it off until the next frame is ready (rather than keeping it illuminated continuously from one frame to the next).
Road to VR is a proud media partner of GTC 2016