Sensomotoric Instruments (SMI) is a German-based eye tracking company who has released an eye tracking kit for the Oculus DK2 & Gear VR, and most recently for the HTC Vive. At SIGGRAPH this year, Nvidia was showing a foveated rendering demo where it only renders high resolution to the sections of the scene that you are actually looking at. It’s really an imperceptible difference that would allow mobile technologies to render higher resolution scenes, or potentially help make it more feasible to wirelessly transfer data to a desktop VR HMD.

LISTEN TO THE VOICES OF VR PODCAST

At SIGGRAPH, I had a chance to talk with Walter Nistico, Head of R&D and Lead Architect Computer Vision, as well as Tom Sengelaub, Manager Solution Delivery, about SMI tracking, and some of the applications in foveated rendering, medical applications for autism research and concussion detection, marketing and analytics, and even deception detection with Converus’ EyeDetect.

new-fove-industrial-design-closer-square
See Also: FOVE Debuts Latest Design for Eye Tracking VR Headset

Researcher Hao Li told me that eye tracking is pretty essential in order to take VR social presence to the next level, and so I expect that the second generation of the Oculus Rift and HTC Vive would both include eye tracking technologies. In talking to Walter and Tom at SIGGRAPH, they’re also very confident that we’ll start to see eye tracking technologies in the next generation of VR headsets.

From SMI’s perspective, they’re hoping to be able to license their eye tracking algorithms to the big headset manufacturers. In my interview with Tobii eye tracking, they also told me that they’ve also been in discussions with some of the major VR HMD manufacturers. SMI says that the hardware required for eye tracking is not a huge barrier, and so it will likely be a matter of whether the eye tracking algorithms are going to be developed in-house or licensed from one of the big eye tracking players.

Here’s a video of NVIDIA’s foveated rendering demo shown off at SIGGRAPH:

Here’s a shadertoy fovea visualizer demo that illustrates how your fovea works (be sure to watch it in full screen).

Here’s a recent demo of using SMI eye tracking with the HTC Vive.

Here’s a demo of eye tracking of a spatial search task within VR.

Support Voices of VR

Music: Fatality & Summer Trip

Newsletter graphic

This article may contain affiliate links. If you click an affiliate link and buy a product we may receive a small commission which helps support the publication. More information.


Kent Bye
  • DiGiCT Ltd

    This is for sure needed in future VR.
    It is like culling for camera in games.
    It will just give more performance boost.
    However it will not make it look better but it makes it possible to upgade much higher res HMD screens much faster as that GPU’s can follow up.
    So we might see an HMD havin 2x 2k or even 2x 4k screens be able to run smooth using this tech.
    On the other hand it also will make mobile VR better and also will open VR for Lower spec VGA cards as current requirements.

    Seeing people’s eyes direction in VR is an extra feature but not something most really will use or prioritize. it is the performance boost that is the mayor part.

    • MasterElwood

      Problem is: nobody seems to know HOW much performance we can gain with it! Some people say about “33-50 percent” – some say “up to 500 percent”.

      So – what is it now?

    • Chris

      With this tech reducing the amount of traffic going to the HMD could this open doors for a wireless setup?