Eye tracking experts SensoMotoric Instruments (SMI) have collaborated with Valve to bring their technology to OpenVR. Eye tracking-enabled HTC Vive units are being shown at GDC 2017, as part of an R&D effort to integrate eye-tracking support into Valve’s OpenVR API.

At this week’s Game Developers Conference, along with LG’s ‘next generation’ VR headset prototype, Valve are also showing some upgraded HTC Vive headsets at their booth, fitted with eye tracking technology by SensoMotoric Instruments (SMI). The German computer vision provider is a world leader in eye tracking technology, and has produced kits for the Oculus DK2, Gear VR and HTC Vive HMDs.

A peek inside one of SMI's Modded Gear VR units, from E3 2016.
A peek inside one of SMI’s Modded Gear VR units, from E3 2016.

Eye tracking is a major technology hurdle that must be conquered in order to take VR to the next level of visual quality. Performance requirements for VR rendering exponentially increase with display resolution and field of view, so eye-tracked foveated rendering is essential – a technique that removes unnecessary fidelity from parts of the image that miss the detailed part of the retina (the fovea). While SMI demonstrated 250Hz eye tracking and foveated rendering over a year ago, Michael Abrash described it as “not a solved problem at all” at his keynote at Oculus Connect in October.

SEE ALSO
ARM and SMI to Showcase New Mobile VR Eye-Tracking Demo at GDC

SMI are showcasing their latest work with ARM at their own booth, to highlight the benefits of foveated rendering on mobile VR devices, and Valve are giving demos of OpenVR eye-tracking features using modified Vive headsets. Integrating eye-tracking into the OpenVR API is an indication that Valve want to move forward swiftly with the technology, although it remains unclear whether the next round of consumer headsets will include eye tracking. Speaking to Tom’s Hardware before the event, Valve developer Yasser Malaika said “Eye tracking opens up several interesting possibilities to both VR developers and customers. Our collaboration with SMI on R&D, as well as on SMI’s efforts to make eye tracking enabled Vive units available to the larger VR community, have been critical to our growing understanding of how HMDs with integrated eye tracking will positively impact the future of VR.”

Newsletter graphic

This article may contain affiliate links. If you click an affiliate link and buy a product we may receive a small commission which helps support the publication. More information.


Dominic Brennan
The trial version of Microsoft’s Monster Truck Madness probably had something to do with it. And certainly the original Super Mario Kart and Gran Turismo. A car nut from an early age, Dominic was always drawn to racing games above all other genres. Now a seasoned driving simulation enthusiast, and former editor of Sim Racer magazine, Dominic has followed virtual reality developments with keen interest, as cockpit-based simulation is a perfect match for the technology. Conditions could hardly be more ideal, a scientist once said. Writing about simulators lead him to Road to VR, whose broad coverage of the industry revealed the bigger picture and limitless potential of the medium. Passionate about technology and a lifelong PC gamer, Dominic suffers from the ‘tweak for days’ PC gaming condition, where he plays the same section over and over at every possible combination of visual settings to find the right balance between fidelity and performance. Based within The Fens of Lincolnshire (it’s very flat), Dominic can sometimes be found marvelling at the real world’s ‘draw distance’, wishing virtual technologies would catch up.
  • Foo

    “Performance requirements for VR rendering exponentially increase with display resolution and field of view”

    That’s completely untrue. The performance requirements increase linearly with resolution. Increasing the field of view doesn’t have any relationship to performance requirements.

    • Xron

      O_O
      Lets see, with fov ~100 you need to work with say 1000pixels, increasing fov to 150 adds more pixels for you to work with, so why do you think perf. needed will be the same? -.-

      • ForceKin Gaming

        Because you can fit 150 FoV in 1000 pixels…you aren’t required to increase resolution to meet FoV. If you’ve ever played a pc game that allows you to adjust your FoV, you’ll have noticed that the resolution doesn’t change when you do. They’re independent of each other.

        • Bryan Ischo

          You’re violating one of the basic assumptions of the discussion, which is that pixel density should remain reasonable at all FoV settings. 1000 pixels spread across a 150 degree FoV would not be acceptable.

          Another basic assumption of the discussion is that frame rate should remain above 60 Hz (or 75 Hz, or 90 Hz, or whatever). If you ignore that also then we can say that it’s OK to render a 150 degree FoV at 16000×16000 resolution using current GPUs – nevermind that it would render at 0.5 Hz.

        • Brent

          That makes it blurry!

    • kontis

      It’s partially true. You forgot planar projection getting exponentially worse after 90 deg fov and that causes exponential differences in distortion correction margin that has to be warped (while preserving more or less 1:1 pixel matching in the “fovea” regions).

      http://i.imgur.com/MR530gR.png

      13x more pixels to render when going from 97 deg to 150 deg while just preserving native-like IQ.

      But this is an issue of current GPU tech and may not be true in the future (it’s already mitigated with workarounds like multi-res shading and lens matched shading, but it’s rarely used in games)

    • Brent

      Why do you think we only have 110 fov?? Raising field of view is incredibly expensive on the GPU!

    • Assuming a typical graphics pipeline, for a given scene increasing the FOV will mean more objects land in the frustum so more triangles to rasterize.

    • Adrian Meredith

      Not really because doubleing the resolution quadruples the number of pixels

  • Sam Illingworth

    Oooh, addon kits being sold for our Vives perhaps?

    • Gerald Terveen

      You can forget that – but it will be worth the HMD update I hope :)

      • Sam Illingworth

        Why do you say that? They have the technology developed already, and HTC have shown they’re more than willing to sell addons.

        • Gerald Terveen

          Because eye tracking is complicated enough as it is and the IR camera needs to be integrated into the case. This wouldn’t be an addon but more of an upgrade kit for tinkerers.

        • Gerald Terveen

          Well … now that I have been proven wrong I just wanted to tip my hat good sir :)

          • Sam Illingworth

            Thank you! I think we can all agree it’s great news too, right? :D

            Except for our wallets. New strap, tp cast, now this!

          • Sam Illingworth

            Mind you, by the time a consumer version of this is ready we might be looking at Vive 2…

          • Gerald Terveen

            Absolutely! I never expected them to find the space inside the Vive and am blown away by this actually turning into a peripheral … but it really makes me happy as it extends the usability of my Vive :)

  • OgreTactics

    As far as I’m concerned given how determine they are to scrap cent economic by not updating VR headsets, mobile or PC, it’s varpoware like Tango, RealSense, Leap, NimbleBit etc…

    SMI are doing the job, Samsung, HTC, Oculus aren’t doing shit.