DisplayLink has been building out its wireless VR compression technology for the last few years, eventually finding its way into the recently released Vive Wireless Adapter. But VR headsets are inevitably moving toward higher resolutions—as we’ve seen this week at CES 2019—making it increasingly harder to make them wireless. Luckily DisplayLink has a few tricks up its sleeve to boost compression efficiency without impacting latency, one of which they demonstrated exclusively to Road to VR this week at CES 2019.

Many readers of Road to VR will be familiar with the concept of foveated rendering: since our eyes only see in high fidelity within a few degrees of the center of our field of view, it’s possible to achieve higher quality (or more efficiency) by rendering lower quality imagery in the peripheral regions and higher quality in the very center. Eye-tracking then can be used to make sure that the high quality region always stays at the center no matter how you move your eye. Done right (with good eye-tracking and smart rendering algorithms) this can be completely invisible to the end users.

DisplayLink is using this same concept, except for compression instead of rendering. Compression is critical to wireless VR because you need to be able to send high resolution imagery at high framerates over a wireless connection that sometimes experiences sudden drops in bandwidth due to the imperfect nature of wireless connections.

Maintaining smooth visuals is key to preventing freezing and stuttering which would hamper any wireless VR experience. To maintain consistency in the imagery, DisplayLink has designed their compression technology to be able to respond to changes in bandwidth on the fly—even in the middle of a frame—so that if, for instance, a user’s hand briefly blocks an antenna, the system can apply more compression to make sure the image can fit over the reduced bandwidth.

Beyond just maintaining consistent imagery under situations of reduced bandwidth, there’s also the need to be more efficient with compression so that larger frames for future headsets with higher resolution can fit in the same available bandwidth.

Both reasons are why DisplayLink is developing foveated compression which takes advantage of eye-tracking data to understand where to compress the frame the most and where to leave it sharp. In doing so, the company claims some pretty huge gains in compression efficiency.

Photo by Road to VR

Using a Vive headset equipped with Tobii eye-tracking, DisplayLink showed me a demo using their wireless adapter reference design. Initially the adapter was set to use the same amount of bandwidth available in the Vive Wireless Adapter to get the image to the headset. Then they turned on foveated compression and cut the available bandwidth down to 1/3.

To my eyes the difference between the full bandwidth image and the 1/3 bandwidth image (with foveated compression) looked effectively identical. Even as I raced my eyes around the scene in an effort to catch the edges of the more highly compressed regions, I was wasn’t able to see anything more than a fleeting glimpse of a slightly blocky region in my peripheral, and this is as I was actively trying to spot any visual artifacts.

SEE ALSO
Eye-tracking is a Game Changer for XR That Goes Far Beyond Foveated Rendering

Even when I asked them to switch rapidly back and forth between full bandwidth mode and 1/3 bandwidth mode with foveated compression, it was difficult to spot any meaningful differences between the two. Had they handed me the headset from the start with foveated compression enabled (without telling me), I don’t think I’d have any idea it was happening, and that’s exactly how it should work.

Photo by Road to VR

There’s a few caveats of course: this was a proof of concept demo and I only got to see one piece of content (just standing around in the SteamVR home area). So I don’t know if this foveated compression approach will be valid for all or even most content. It’s possible that it won’t work so well with more complex colors, contrast, and lots of motion. The demo I saw was also on the original Vive, which has a fairly low resolution compared to what else is out there. Higher resolution (like the Vive Pro Eye, which DisplayLink plans to support, and even better lenses) might make it harder to hide the foveated compression.

However, the foveated compression is fundamentally based on DisplayLink’s existing compression technology, which works pretty darn well for today’s VR headsets, so I wouldn’t be surprised to find the the foveated approach works well too.

Newsletter graphic

This article may contain affiliate links. If you click an affiliate link and buy a product we may receive a small commission which helps support the publication. More information.


Ben Lang
Ben is the world's most senior professional analyst solely dedicated to the XR industry, having founded Road to VR in 2011—a year before the Oculus Kickstarter sparked a resurgence that led to the modern XR landscape. He has authored more than 3,000 articles chronicling the evolution of the XR industry over more than a decade. With that unique perspective, Ben has been consistently recognized as one of the most influential voices in XR, giving keynotes and joining panel and podcast discussions at key industry events. He is a self-described "journalist and analyst, not evangelist."

  • I really hope this helps drag Foveated Rendering out of the concept stage and into reality. There’s no need to render the full, high-rez image at all before sending. Saves on compression and saves on GPU time.

    • Andrew Jakobs

      Except this isn’t foveated rendering.. this happens AFTER the rendering. So I wonder how it will hold up after actual foveated rendering.

      • You didn’t read my post very well.

        Like I said, it would save on all accounts if actual Foveated Rendering was used BEFORE the transmission compression, GPU and wireless transmission. Compressing an image that has already been downscaled would get an even better compression rate. Large, single-color blocks are easy to compress.

        Foveated Rendering was a hot topic in 2016 and before, but has languished since then. Just the existence of eye tracking, in even concept models, is rare now. HMD makers haven’t pushed eye tracking, instead focusing on low-end headsets, and GPU makers aren’t in a rush to setup Foveated Rendering when no commercial hardware exists to make use of it. The FOVE never made it to market and the VIVE Eye appears like it will be the first.

    • Florence S. Nielson

      watch Vice HD 2018 (*available 1080p*).with the best quality …………….
      click here –> xmovieshd-1.blogspot.com

  • MosBen

    That’s pretty impressive sounding. As someone that’s more skeptical than most about exactly how much greater the resolution of our HMDs need to get before other concerns become more important (I’m not saying that I don’t want higher resolution, just that I’d take, say, wireless 4k over tethered 8k), this sounds like it will be a practical solution for at least a generation or two.

    • Mradr

      No you wouldn’t. Wireless alone adds 250 – 300$ on TOP of the headset. Most people would rather stay with the cord or throw that money back into the screens or other part of the hardware instead. This is 100% a optional addon then it would be for something that was force onto the user.

      • MosBen

        Well first, you can’t really tell me what I’d prefer. I know that adding wireless adds cost to an HMD (though I think that you overstate the cost, which would be different if it were built in rather than an aftermarket product), and the freedom that wireless gives is far preferable to me than a bump in resolution.

        As for most people, if you really mean most general consumers, then most people don’t want a tethered (wirelessly or physically) HMD in the first place. Most people don’t own gaming computers and aren’t interested in buying or building one. But even if we limit “most people” to “most people who own gaming PCs”, I doubt that you are correct. Higher resolution is fine, but as it improves other factors become at least as important, or more important.

        • Mradr

          Yes I can. I just did. Wasn’t hard to do so. First off – Not really – wireless at that level cost more than your basic router as it has to hit speeds less than 1ms plus have a lot of bandwidth to spare. You need also 2 devices to trans/rev the data adding cost. Then you also have to warrant the device in both UK (2 years) and US (1 year). Plus if it is built in – that is just another part they could fail and then you have to send in your whole headset. You also have battery limits that won’t be replaceable, extra weight to the HMD, and a bunch of other issues. Ideally they might go from 200-250 down to 150-200. Still a pretty high extra price tag that doesn’t really add much value to the overall headset. Tether usually doesn’t bother anyone. It’s just having the PC on the other hand that bothers anyone either way. Its more of a nice thing to have in the future feature than it is something that is needed right now.

          That same cost could go into eye tracking instead and as I said above offer a lot more usefulness than any wireless will add for the given value of return. For example, allowing lower end systems (as you pointed out) to be attached to the device and offer reasonable performance to that of a console. Not wireless – but teterless. Wireless isn’t what most people want – its either an all in one to keep cost low to owning VR, or on the other side of the stream, higher visual eye candy for that people that don’t mind spending the extra money for it.

          For the few that wants wireless – there are options for them now and they should look into that instead of they have the extra cash to do so. I mean that sounds fair in all reality in terms of what the market wants and what people may prefer over all.

          • MosBen

            Well, I should have said, “You can’t really tell me what I’d prefer without being an obnoxious twat and a likely being wrong.” But then, you knew that. You’re right that adding wireless, and a battery, would add cost to an HMD. I’m not at all confident in your numbers, as I don’t think that we can extrapolate much from the prices of standalone unites like the TP-Link.

            I just think that you’re wrong on wireless not adding value or people not being bothered by the tether. People that I demo for nearly get tripped up all the time, especially when they’re starting out, and even after they get used to it it’s something that you have to be cognizant of, and it would be great if you didn’t have to worry about it. That said, you’re right that it’s owning a gaming PC that’s the biggest hurdle, and maybe it’s the case that group of people that are willing to own and maintain a gaming PC are also just more likely than the general public to put up with a physical tether. That could certainly be true.

            You’re also right that one of the nicer aspects of eye tracking is lowering the system specs necessary for a quality VR experience. I am going to quibble with you insofar as an HMD connected to an external computing platform (be it a PC, laptop, iPad, or breakout box) is what we should call “tethered” in that it needs to be connected to something external in order to function. A Vive with the TP-Link adapter is wireless, but tethered, while the Quest is wireless and untethered because all of the hardware is inside it.

            I still think that you overestimate the cost of adding wireless and undervalue it as a selling feature, but I suppose we’ll see how things shake out with the next generation of HMDs coming in the next few years. MS in particular seemed to want to hold off on doing VR on the Xbox unless it was wireless, but maybe they’ll change their mind (or their statements were misinterpreted).

          • Mradr

            Maybe but looking back over the wireless tech it’s already causing problems trying to keep up with current generation hardware let alone looking to the future of 8k by 8k screens.

            The other problem with wireless is that it took them what we already have had for a while now for 4k TV streaming just to do what 1st generation 1080p screens can do. Eye tracking FOV over wireless will help a ton I am sure. Maybe as much as 50% – allowing them another generation or two to maybe keep up with. The problem is what happens after that? Having it built in means you will have to keep up with each generation as removing a feature is usually consider a nono in most customer heads. This is just another reason to just let 3rd party companies take on lead to get this working than try to force it into the headset alone.

            Rather – as I said – eye tracking, focal point fova, and a bunch of other features seem like it’ll give you more a bank for a buck compare to wanting or preferring wireless inside the headset and overall – there are many other things HMD needs to focus on their time on let alone customer money to upgrade the HMD that covers more usage.

            At some point – mobile hardware + connecting to a PC will be a thing – so you will have tetherless feature either way for when you don’t want to be connected to a PC while still giving you the option to connect to a more powerful device later on. That will cover many fields of usage. Honestly – I am surprised Vive, Steam, or Oculus hasn’t done that already as it would open the doors for a lot more customers while keeping the number of products you make simple. Just sell the mobile hardware as an addon or upgradeable option.

          • MosBen

            But isn’t that the point of this article? Displaylink has apparently made some advances that allow for much better wireless, conceivably allowing the tech to keep up for at least another generation of display upgrades, which is why I originally said that I’d take wireless at 4k over wired at 8k.

            As for mobile devices that connect to a PC, we’re seeing the early version of that with Qualcomm’s newest reference HMD design, which has an SoC built in but can evidently connect to a PC with a detachable wire. I’m sure that Oculus and HTC haven’t done that yet because it adds cost to the HMD and it’s not clear, to me at least, that it’s a feature that would attract a lot of new customers, at least not yet. Or maybe Qualcomm just figured out how to do it and the Quest and Cosmos were already feature locked and will get it in the next generation.

            As I said in my last comment, it’s possible that the audience that’s wanting to do PC VR is simply willing to put up with wires in a way that a general audience isn’t, so wired VR on PC will just continue, while mobile VR like the Quest will attract a broader audience of people who are less interested in raw horsepower. I still think that many people in these parts underestimate the value of being wireless, but they could be right about it specifically in the PC VR space.

      • PJ

        Not me, I would a take wireless 4K headset, even if it had to restricted at 60hz over anything with a cord

    • Dave

      There is a ton of people in VR who really couldn’t give a monkeys about a wireless solution. Afterall what are the most successful AAA titles for VR at the moment, almost all of them are seated (or can be played as) experiences… I’m thinking fps shooters, space, flight, racing etc. I respect your views and it would be great to be untethered but I just feel right now resolution is king for the PC/Console VR market.

      • MosBen

        I don’t doubt that there are a significant number of people who currently own and use VR HMDs that don’t care about the tether. Indeed, there may be a significant number of people who don’t yet own HMDs but are interested, own gaming PCs already, and are only being held back by a desire for higher resolution screens.

        But if there’s any recurring theme to my posts on RTVR, it’s trying to explain to people that nearly all of us here are early adopting enthusiasts. We’re willing to put up with the rough edges of a product in order to be the people on the leading edge of technology. We’re willing to pay large sums of money for graphics cards, and are willing to find a fairly large amount of space in our living spaces to accommodate VR. But when you look at society as a whole, we’re not typical people. The things that we care about are not the things that most people care about.

        The single biggest problem with the resolution of the displays in VR HMDs is that a pronounced screen door effect contributes to feelings of nausea in users, and a low resolution makes it hard to read typeface or other finely detailed pieces of the environment. But the screen door effect in the consumer Rift and Vive already did a decent job of reducing the SDE, and though I haven’t used a Vive Pro or Odyssey+, my understanding is that SDE is very nearly eliminated entirely there, and is essentially a non-issue for the Pimax HMDs. Regarding details like font, like with SDE currently available tech like in the Vive Pro or Pimax HMDs has already dramatically improved it, and that issue can also be worked around in game design.

        So my point isn’t that we should just stop developing new displays because resolution is as good as we’ll ever want/need it to be. My point is that focusing on increasing resolution won’t broaden the appeal of VR to the broader society at large because it’s not a lack of sufficiently high resolution that’s keeping them from buying an HMD. The likely biggest reason why we haven’t seen mass adoption yet is simply related to content. There are some great experiences out there today, but probably none that are good enough to compel someone to spend the money on a gaming PC, and an HMD and put up with all of the other annoyances (like the tether) that we have in the first generation of products. Tethering on its own may not be the thing holding people back, but it’s almost certainly a factor because it is annoying and also limits how you can use your HMD because it always needs to be near your gaming PC. But there are, of course, lots of other factors, most notably cost, that are important barriers to overcome in order for mass adoption to occur. It’s not that there’s no value in increasing the display resolution. It’s that display tech that’s already in shipping products is likely good enough for now such that it’s other factors that are holding VR back from mass adoption.

        • Dave

          100% agree, I guess I wasn’t looking as far as mass adoption. That is a utopia that definately needs more than higher resolution HMD’s.

          In the near future you are also correct the more we can do to make this more accessible the better. Thats why I think the lighthouses and the Oculus sensors have to go and we have to trust Inside out tracking will be refined to a point it’s not even questioned. I definately think the Quest will move us a step closer here.

          The tethering is more difficult to shake because fidelity and wireless connectivity are at odds with each other which was why I was leaning towards visuals instead of an untethered HMD,

          I just feel in the next 24-36 months a high fidelity (like true 4K or better) wireless headset is completely out of the question – although I hope I’m completely wrong.

          • MosBen

            Yep, it’s hard to break the tether, especially with increasing demands from things like higher resolution displays. Hopefully things like the tech in this article will make it easier, at least in the medium term, and we’ll just have to cross our fingers that compression can keep pace with the increasing data demands of VR software.

            Re: true 4k, you’ll have to remind me if you know, but I remember that there’s been ongoing controversies about Pimax’s “8k” and “5k” headsets not actually meeting those specs, but how do they compare with “true 4k”?

            Thanks for the thoughtful, interesting comment though. It’s always nice to engage meaningfully on a topic!

  • TareX

    If this technology only handles compression and not rendering, then that’s excellent news because we don’t have to wait for NVIDIA and each developer to updated their game, and we can just use this for ALL titles (since video, is video and has nothing to do with the title) till foveated rendering is a thing.

    • benz145

      Yup that’s how it works. The frame is compressed after it’s rendered so I don’t believe applications need to do anything special.

      • Hivemind9000

        The network frame compression only really helps the wireless aspects – which means it makes no difference except to help the wireless unit perform better (higher network fps and/or supporting bigger resolutions without dropping network frames).
        Unless foveated rendering is implemented at the graphics renderer level, there will be no benefits to the rest of the VR hardware/software stack. Game developers shouldn’t have to do anything if foveated rendering implemented at the graphics driver/API level (i.e. inside SteamVR for example).
        Don’t get me wrong, I think this is a great technology, but we really need the graphics foveated rendering to support higher resolution and FOV headsets on existing GPUs. Headsets having eye tracking built in is the key technological fulcrum here.

        • Dave

          Only the wireless aspect… some would say thats important if you are discussing bandwidth. Although I’m with you on graphics foveated rendering. I’d like to see higer resolution in HMD’s sooner rather than later and quite frankly I’m happy to wait for the network technology to play catchup in a few (probably more like 10) years time.

  • Amazing. I had never thought of this application of eye tracking… love it

  • kontis

    Optic nerve has something like 1/10000th the bandwidth of HDMI, so we have a looong road of possible compression optimizations.

    • JustNiz

      That’s not true at all. Apart from anything else, the optic nerve is analog not digital.

      • G-man

        so?

  • psuedonymous

    It’s somewhat telling that the comparison was to a compressed stream rather than to an uncompressed (i.e. direct wired) image. “We didn’t make compression /worse/ is nice, but that’s not the standard to meet.

    • Caven

      At least to my eye, whatever compression DisplayLink is currently using is already indistinguishable from a wired connection. So if their foveated compression looks as good as their standard compression, it will also look as good as a wired connection.

  • oompah

    FOVeation is the correct way to do things
    instead of stupidly painting all pixels the same way
    which entails over heating, low fps , latency etc all problems
    we should learn from nature

    • Fabian

      I’m pretty sure nature is still there when you don’t look at it :)

    • JustNiz

      > FOVeation is the correct way to do things

      No it isn’t. The real world doesn’t change its resolution depending where you look.

      Foveated rendering will be an unavoidably noticable “effect” and will always look/feel wierd compared to fully rendering everything, partly because eye-tracking will always have some latency and can never be 100% accurate.

      • peteo

        Computers will never need more than 640k

  • dk

    how did I miss this …apparently I thought it was news about the wireless adapter
    “Even as I raced my eyes around the scene in an effort to catch the edges of the more highly compressed regions” ….next time try to mess with it by closing your eyes and looking in an opposite corner and than open your eyes….just one possible test

  • Tags I812

    Now i ask why is it that we cant get wireless for the oculus rift yet?

  • jacobpederson

    Do you remember if this thing had active cooling? Was this ever released to the public looking like this or did it just turn into the Vive Wireless adapter? Thanks!