Qualcomm and Ultraleap today announced a “multi-year co-operation agreement” that will bring Ultraleap’s controllerless hand-tracking tech (formerly of Leap Motion) to XR headsets based on the Snapdragon XR2 chipset. Ultraleap claims to have the “fastest, most accurate, and most robust hand tracking.”

Snapdragon XR2 is Qualcomm’s latest made-for-XR chip which the company has touted as being the ideal foundation for standalone XR headsets.

The leading standalone VR headset, Oculus Quest, has been increasingly focusing on controllerless hand-tracking as a means of input for the device. Other major headset makers, like Microsoft and its HoloLens 2, have also honed in on hand-tracking as a key input method. As industry leaders coalesce around hand-tracking, it becomes increasingly important for competing devices to offer similar functionality.

But hand-tracking isn’t a ‘solved’ problem, making it a challenge for organizations that don’t have the resources of Facebook and Microsoft to work out their own hand-tracking solution.

Over the years Qualcomm has been working to reduce the barrier to entry to making a standalone XR headset by offering ready-made technologies—like inside-out tracking—alongside its chips. Now the company is announcing that its XR2 chip will be optimized for Ultrealeap hand-tracking out of the box.

While Qualcomm and Ultraleap have previously worked together on this front, the Ultraleap hand-tracking solution offered through Qualcomm was tied to Ultraleap’s hand-tracking hardware. The new announcement means that Ultraleap’s hand-tracking software is being offered independent of its hardware. This makes it a more flexible and cost-effective solution, with the hand-tracking software ostensibly making use of a headset’s existing inside-out tracking cameras, rather than requiring an additional cameras just for hand-tracking; this also frees up two of XR2’s seven supported camera slots for other uses like eye-tracking, mouth, tracking, and more.

Qualcomm and Ultraleap say the hand-tracking tech will be “pre-integrated” and “optimized” for XR2. It isn’t clear if this simply means that Ultraleap hand-tracking will be available as a service in the XR2 software stack, or if XR2 will include special hardware to accelerate Ultraleap hand-tracking, making it more power and resource efficient.

SEE ALSO
Leap Motion 'Virtual Wearable' AR Prototype is a Potent Glimpse at the Future of Your Smartphone

Despite being a years-long leader in hand-tracking technology, Ultraleap (formerly Leap Motion) hassn’t managed to get its solution to catch on widely in the XR space. Now that hand-tracking is seeing greater emphasis from leading companies, Ultraleap’s camera-agnostic solution on XR2 could be the moment where the company’s hand-tracking tech begins to find significant traction.

Newsletter graphic

This article may contain affiliate links. If you click an affiliate link and buy a product we may receive a small commission which helps support the publication. More information.


Ben Lang
Ben is the world's most senior professional analyst solely dedicated to the XR industry, having founded Road to VR in 2011—a year before the Oculus Kickstarter sparked a resurgence that led to the modern XR landscape. He has authored more than 3,000 articles chronicling the evolution of the XR industry over more than a decade. With that unique perspective, Ben has been consistently recognized as one of the most influential voices in XR, giving keynotes and joining panel and podcast discussions at key industry events. He is a self-described "journalist and analyst, not evangelist."

  • They might want to consider focusing on basic controller tracking, as the Cosmo used their last generation of VR tech and it failed miserably at controller tracking. Hand tracking is *kinda* fun, but ultimately limited to light VR usage.

    • Adderstone VR

      I think you may be mistaken.
      If you’re referring to the Vive Cosmos (I don’t know of a “Cosmo” VR headset)
      – it is not a stand-alone headset – it does not use a qualcomm chip to do it’s tracking – it needs a PC to be plugged into. So bad tracking on Cosmos has zero to do with Qualcomm and even less to do with Qualcomm’s partnership on hand tracking.

      Besides – the Quest uses an even older Qualcomm chip (Snapdragon 835) – and it’s tracking is best in class – controller tracking seems to be more about the OEM’s implementation and tracking algorithms than the chip it’s running on.

    • Andrew Jakobs

      Yeah, they should look to the way the Pico Neo 2 does its tracking, or a combination of optical/electromagnetic.

  • nullcodes

    Hand gestures in mid air are literally a pain — for the same reason the light-pen failed in the 70s and 80s and the mouse won. You should be able to make micro-gestures with your hands on the side while sitting on a couch. Have to hold up your hand in the air isn’t casual. It’s not relax friendly.

    • TechPassion

      EXACTLY. Hand tracking could be good for engineering, but not for casual relax.

    • silvaring

      Without hand tracking you lose two critical elements of immersion though, 1) using your hands while socializing with people, and how much immersion is lost when we interact with each other without those hand signals. And 2) Even when alone and not around others just having the freedom to see your hands moving in sync with your head may reduce the feeling of disconnection and motion sickness.

      So no, it’s not like the light pen… only if people make your hands move like a light pen to move through VR spaces, but that’s going to be up to the developers and hardware engineers to sort out.

      • nullcodes

        I am in favor of hand tracking, we should have that. I am saying it should also support microgestures while seated with your hands on the side. That’s a type of hand tracking too, maybe with wider cameras.

  • xyzs

    Japan Display made a 3 inch 2.5k low latency display 2 years ago,
    There is a Qualcomm great XR2 chip available today,
    The best made hand tracking system from Ultraleap is out there for many years already.
    Tobii is making solid eye tracking for years…

    … 2020, still no clue of where those tech are, just nobody sell them to us…

    • Andrew Jakobs

      I guess putting it all together makes is a hefty price.. Tobii eye trackers seem to around $100-$150 for each eye, those 3 inch displays might be considerably more expensive to produce than the current ones used. And I guess the XR2 chip isn’t cheap either..

  • Andrew Jakobs

    All in all nice, but when do we actually get to see all those XR2 headsets.. They have shown all the reference designs already way back in febuari, but still no announcement from ANY manufacturer of actually gonna release one, within the next six months..
    And even though handtracking can be great for a lot of applications, I find it still better to have something in my hand which gives me some form of feedback.. Also shame no other company has used anything like the Index controllers or license them (had hoped for HP with their G2)..

  • That’s a great piece of news! As a fan of the good old Leap Motion, I like this idea of becoming hardware-agnostic. Ultraleap has a great expertise in hands tracking and hands-UX, so I think it can do great things for the standalone segment

  • TheMediaman1

    Congratulations to both… and especially to users. Hand-tracking, like speech, is something intrinsic, personal, always available and -if well-implemented- immersive and satisfying. It can break the confines of windows and menus (and avoid health/cross-contamination concerns). Better yet, physical interaction becomes optional and purposeful… rather than “excluded”.. and Ultraleap also provides physical-proximity & sonic touchless feedback.

    No solution fits all people & contexts, but like speech (eye/body tracking, touch, mice, keyboards, sensors, etc.), hand-tracking presents a combine-able interaction “mode” that can significantly enhance user experience in VR & XR. Hopefully, we’ll also soon trade head-mounted cigar boxes for something more svelte, and perhaps something more virtual real-world switchable. Cost vs other hardware/software options- will also likely balance well.

    Just my two cents worth…