Today at the the World Conference on VR Industry in China, HTC announced plans to bring hand & finger tracking to Vive Pro via the headset’s front-facing cameras.

Back in April HTC launched new tools to help developers take advantage of the Vive Pro’s largely unused front-facing cameras. Now it seems the company plans to double down on that by adding hand and finger tracking that works “natively” on the headset.

On the Vive Pro today, the system can understand the position of a user’s hands thanks to a pair of controllers which are tracked with external beacons. This new capability, which HTC demonstrated today at the WCVRI conference, enables hand-tracking without the need of the need of externally tracked controllers. HTC also indicates that the capability can track individual finger movements too.

Other companies like Leap Motion have developed similar technologies, employing computer vision processing to understand the position and movement of a user’s hands without any special gloves or attachments.

Vive China President Alvin Wang Graylin on stage today at WCVRI 2018 in Nanchang, the capital of China’s Jiangxi province. | Image courtesy HTC

While highly interactive VR content benefits from the binary input capabilities and haptic feedback of controllers, hand tracking can sometimes be more immersive and intuitive, and may work well for certain situations where nothing more than ‘casual’ input is necessary.

HTC says that developer tools for the new feature will be made available soon to registered Viveport developers.

Newsletter graphic

This article may contain affiliate links. If you click an affiliate link and buy a product we may receive a small commission which helps support the publication. More information.


Ben Lang
Ben is the world's most senior professional analyst solely dedicated to the XR industry, having founded Road to VR in 2011—a year before the Oculus Kickstarter sparked a resurgence that led to the modern XR landscape. He has authored more than 3,000 articles chronicling the evolution of the XR industry over more than a decade. With that unique perspective, Ben has been consistently recognized as one of the most influential voices in XR, giving keynotes and joining panel and podcast discussions at key industry events. He is a self-described "journalist and analyst, not evangelist."
  • Xron

    Nice, one of the things I want for 2nd gen device.

    • Bob

      Hand gesture tracking is great when it works but it lacks feedback which is a major problem when you’re trying to get yourself properly immersed. Ideal for menu navigation and other menial stuff but it’s not the “go to” method for serious gaming applications and experiences.

      • dk

        nobody cares ….u can use controllers all u want no one will force u to use just your hands …..having your actual real hands in vr “for proper immersion” is a MUST for every new headset from now on ….not having hand tracking in every single new headset when tracking like leap motion(and other) already exists is just pathetic

      • kuhpunkt

        It would be excellent if properly implemented in racing games for example. If the camera recognizes your hands and translates your hands to the steering wheel… that would be great for immersion. Like in Project Cars 2. I see a steering wheel and hands in the cockpit, but those aren’t my hands.

        • Bob

          That would be a great implementation but if you needed to take it to the next level then it would seem the best solution is to wear something over your hands in order to induce force feedback and create the feeling of touching the steering wheel which wouldn’t be possible with simple hand based gesture tracking.

          I think the other problem with gesture tracking in this scenario is that your fingers might tire out at some point after a prolonged session of “holding” a phantom steering wheel which could potentially be mitigated through the use of gloves designed with an exoskeleton or some such.

          • kuhpunkt

            Nooo… I’m not talking about a phantom wheel. You would still need a real one im combination with hand/finger tracking. That would be optimal.

          • MosBen

            As he explains below, it’s not about tracking your hands on a phantom wheel, but about tracking your hands on the physical input that you already are using. It’s neat when using the Touch controllers to move your fingers around and see your virtual hands react, but it would be neat if your hands were 1:1 tracked while still using a controller. Maybe you’d be using a gun controller and gesture with your off hand, or maybe you’d just shift your grip in real life and see it reflected with your virtual hands. Hand tracking isn’t likely to be a great input on its own because VR benefits a lot from using interactions with physical objects in the real word but overlaying the virtual on top. So, for instance, those big virtual reality events like at the recent Oculus Connect, where players are in a big room with actual physical crates, and the virtual world has virtual crates in the same locations.

            Similarly, having your body and feet tracked wouldn’t be great on its own for input, but it would likely contribute a lot to creating a sense of presence.

        • AndroidVageta

          Try using the keys to adjust your seating position (forward, backward, up and down) to make your hands IRL match those in your car. You might have to map them, I know I did (this is for PCars2 btw).

          Not a fix for what you describe but it adds to immersion tenfold. Also, if you can for your wheel, adjust the degrees of rotation to match that of the car you’re driving.

          With your hand position matching the game and the wheel rotation matching the game it really does a lot to make you feel much more connected. Like night and day difference. I love it!

    • nejihiashi88

      nah hand tracking isn’t good because there is no feedback, the best implementation of hand tracking is knuckles vr controlller because you can grab with a feedback from the controller and the interactions with it is far more diverse squeezing throwing etc.

      • kuhpunkt

        Handtracking would be good as an addon for haptic tools.

      • jj

        you obviously aren’t educated ore experienced with vr…. Anybody whos tried the leap motion knows that hand presence without gloves is an amazing thing. You saying that just shows u haven’t tried it. you’re jsut the same as someone saying vr isn’t good enough when you haven’t even tried on a headset

      • daveinpublic

        I agree, controllers are better than hand tracking in games. I’d like the ability to have both of course, but I think controls are the best way. Imagine trying to do a FPS with just your hands. Let’s say you hold an imaginary gun in your hand and pull the trigger to shoot, now your entire hand is devoted to what used to be one button press. And it probably wouldn’t be as accurate or quick as to when you fully pulled the trigger. How can you rapidly go through tons of different options and fire different weapons and strafe and use a jetpack and toss a grenade at the same time in this scenario? I’m sure you can figure out some workarounds but overall, the controller is best, unless it’s some new category of game, or something besides a game.

        • DP

          I think you are missing the point. The scenarios you describe might be possible in today’s video games, but they really aren’t realistic, and they tend to take away from the immersion that VR enables. In real life, when you wield a gun, you have to keep a hand on it, and cannot switch rapidly between guns and throw grenades all at once. We all enjoy those fast paced games sometimes, but the point of VR is to make it seem more real, and would be improved by more realistic controls. I think an FPS in which you have a few weapons and can only use two hands in real time would be awesome. It would feel so much more realistic and bring a sense of urgency like never before. At the same time, you could get pretty fast and it would be more satisfying, because it would be so intuitive that the learning curve would be virtually 0. No buttons to learn, no combos to set into muscle memory. Just see a gun on the ground, bend down, reach out, pick it up, hold it like a real gun, and fire by pulling the trigger. Out of ammo, with another gun in your pack? Toss the gun and do a reaching motion over your shoulder to pull a shotgun from your back. Sure, it is never as fast as pressing sequences of buttons, but damn if it isn’t more immersive and bad ass. I think this is the experience most people want from VR. One of the reasons I haven’t bothered adopting VR is that while it is a pretty huge step forward, it is still pretty much playing regular video games with your eyes closer to the screen, and nice head and controller motion tracking. Again, a cool innovation, but still very much feels like regular video games with a cool headset. I’d also argue that the current experience is actually a sub-par hybrid right now, since you don’t quite feel immersed, but the controls are less tight than in regular video games. Once we have 8k res per eye, full peripheral FOV, wireless and lightweight headsets, perfect tracking of your whole body without worn sensors/controllers, and some way to feel more like we are really walking around to explore VR worlds, then we will REALLY feel immersed. I figure that is at least 5 years off, maybe a bit more since we don’t have processors good enough to handle that yet. After that, it will be about making it feel like you can touch things with some sort of feedback system that doesn’t require putting on a full body suit (maybe some sort of unit that sends electrical signals?).

  • fuyou2

    shitty shitty shitty tracking! Low FOV, Jittery, Whats the point anyway with crappy hmd.

  • This is a step in the right direction so that one day, somebody will produce a device with full finger tracking with actual resistance that developers have use cases for. We need more developers leaning the basics now to pave that way for that in the future. And it’s free if you have a Vive Pro. Leap Motion has proved it is doable so mass adoption is great.

    Many devices like mobiles and touchscreen displays exist today so this will open doors for VR too.

    e.g. a wild concept could be a sort of rigid mini mousemat on your left/right arm, fingers are tracked and bingo, you have something that works as good as a mobile touch screen only there is no mobile device. You get needed feedback from a portable surface rather than tapping thin air.

    Holding something is not always practical either. If you are a mechanic you do not constantly hold a wrench while examining parts of a vehicle with your hands. You only hold something when you need a tool. Which can be done with a controller. For business use maybe physical controllers can evolve to something more acceptable like a stylus, a more natural device as we all know how to hold a pen. Touch controllers on my Rift are a huge barrier for non technical users. Too many buttons, never hold it right etc. Accurate finger tracking and a stylus would broaden VR to much more than just a gamers gadget.

  • daveinpublic

    Good to know this is on it’s way in a main stream product. Right now it doesn’t look too accurate, which you already have in a controller, and the latency seems high, but hopefully it gets good enough soon. Every HMD should eventually have this option, along with the ability to use a controller. Would assume the Quest could do this, if it’s already tracking the controllers using cameras, but the cameras may calibrated to look for certain wavelengths coming from the controllers.

  • jj

    This is a testament to how innovative and progressive leap motion has been. Theyre hand tracking works amazing as far as hand tracking goes, and you can see the lag in the vives hand solution. Bravo to both but itll be cool to see if vive can match leap at all

  • Byaaface

    For the gif at the start of the article: I hope the tv they are using is really laggy, otherwise the hand tracking demo looks to have an intolerable amount of lag

  • abu

    Is it possible to give hepatic feedback by using some type of small visuo motors on users fingers?