DisplayLink has been building out its wireless VR compression technology for the last few years, eventually finding its way into the recently released Vive Wireless Adapter. But VR headsets are inevitably moving toward higher resolutions—as we’ve seen this week at CES 2019—making it increasingly harder to make them wireless. Luckily DisplayLink has a few tricks up its sleeve to boost compression efficiency without impacting latency, one of which they demonstrated exclusively to Road to VR this week at CES 2019.
Many readers of Road to VR will be familiar with the concept of foveated rendering: since our eyes only see in high fidelity within a few degrees of the center of our field of view, it’s possible to achieve higher quality (or more efficiency) by rendering lower quality imagery in the peripheral regions and higher quality in the very center. Eye-tracking then can be used to make sure that the high quality region always stays at the center no matter how you move your eye. Done right (with good eye-tracking and smart rendering algorithms) this can be completely invisible to the end users.
Maintaining smooth visuals is key to preventing freezing and stuttering which would hamper any wireless VR experience. To maintain consistency in the imagery, DisplayLink has designed their compression technology to be able to respond to changes in bandwidth on the fly—even in the middle of a frame—so that if, for instance, a user’s hand briefly blocks an antenna, the system can apply more compression to make sure the image can fit over the reduced bandwidth.
Beyond just maintaining consistent imagery under situations of reduced bandwidth, there’s also the need to be more efficient with compression so that larger frames for future headsets with higher resolution can fit in the same available bandwidth.
Both reasons are why DisplayLink is developing foveated compression which takes advantage of eye-tracking data to understand where to compress the frame the most and where to leave it sharp. In doing so, the company claims some pretty huge gains in compression efficiency.
Using a Vive headset equipped with Tobii eye-tracking, DisplayLink showed me a demo using their wireless adapter reference design. Initially the adapter was set to use the same amount of bandwidth available in the Vive Wireless Adapter to get the image to the headset. Then they turned on foveated compression and cut the available bandwidth down to 1/3.
To my eyes the difference between the full bandwidth image and the 1/3 bandwidth image (with foveated compression) looked effectively identical. Even as I raced my eyes around the scene in an effort to catch the edges of the more highly compressed regions, I was wasn’t able to see anything more than a fleeting glimpse of a slightly blocky region in my peripheral, and this is as I was actively trying to spot any visual artifacts.
Even when I asked them to switch rapidly back and forth between full bandwidth mode and 1/3 bandwidth mode with foveated compression, it was difficult to spot any meaningful differences between the two. Had they handed me the headset from the start with foveated compression enabled (without telling me), I don’t think I’d have any idea it was happening, and that’s exactly how it should work.
There’s a few caveats of course: this was a proof of concept demo and I only got to see one piece of content (just standing around in the SteamVR home area). So I don’t know if this foveated compression approach will be valid for all or even most content. It’s possible that it won’t work so well with more complex colors, contrast, and lots of motion. The demo I saw was also on the original Vive, which has a fairly low resolution compared to what else is out there. Higher resolution (like the Vive Pro Eye, which DisplayLink plans to support, and even better lenses) might make it harder to hide the foveated compression.
However, the foveated compression is fundamentally based on DisplayLink’s existing compression technology, which works pretty darn well for today’s VR headsets, so I wouldn’t be surprised to find the the foveated approach works well too.