Ultraleap, the company behind the Leap Motion hand-tracking controller, has released a Developer Preview of its hand-tracking engine Gemini. By many accounts, Ultraleap’s latest software overhaul dramatically increases the ability of the company’s camera modules to do more precise and stable two-handed interactions.

Gemini is now available in Developer Preview for Windows 10, and is designed to work with all existent Leap Motion controllers as well as Ultraleap’s more recent Stereo IR 170 camera module.

In comparison to Orion (V4), which was released in June 2018, its Gemini (V5) engine is said to offer better smoothness, pose fidelity, and robustness. It also improves hand initialization, and brings “significantly better performance with two-hand interactions,” Ultraleap says.

As seen in the gif below, the solidity of Gemini (V5) is pretty astounding. Not only are both hands more accurately tracked, but occlusion appears to be much less of an issue too, as fingers interlock and move in front of each other with comparative ease.

Ultraleap is set to integrate Gemini into a number of XR headsets, including Varjo VR-3 and XR-3 headsets, and the Qualcomm Snapdragon XR2 5G reference design, which makes use of Ultraleap hardware.

Antony Vitillo of XR publication Skarred Ghost went hands-on with Gemini using his first-generation Leap Motion tracker. To him, the software overhaul represents “the best hands-tracking system I’ve seen until now on all headsets for what concerns the interactions between two hands.”

“What really surprised me is the stability of two hands interactions. For the first time, I’ve been able to make the fingers of my two hands cross and interweave [together], and the tracking kept working reliably.”

SEE ALSO
Vision Pro Owners Hopeful Apple Event Will Bring News of Unreleased Panoramic Display Feature

Granted, Vitillo’s five year-old Leap Motion does present somewhat of a roadblock due to its comparatively small field of view, however Ultraleap says with its updated IR 170 camera module that “hands will almost certainly be tracked before they come into your sight.”

In practice, Ultraleap hopes its new software will let developers create hand-tracking-focused applications in preparation for the next wave of AR and VR headsets to make more prominent use of the technology. Facebook’s Oculus Quest standalone notably includes hand-tracking for use within its system UI and a handful of applications, however it hasn’t become a standard input method yet.

Newsletter graphic

This article may contain affiliate links. If you click an affiliate link and buy a product we may receive a small commission which helps support the publication. More information.


Scott Hayden
Well before the first modern XR products hit the market, Scott recognized the potential of the technology and set out to understand and document its growth. He has been professionally reporting on the space for nearly a decade as Editor at Road to VR, authoring more than 4,000 articles on the topic. Scott brings that seasoned insight to his reporting from major industry events across the globe.
  • Xron

    Now we need some kind of haptic feedback aswell.

    • Alexander Grobe

      With the more reliable two hand interaction you can press virtual UI elements placed on the palm or back of a hand with the fingers of the other hand enabling haptic feedback.

    • Ad

      That’s what ultra haptics does. Doesn’t seem particularly useful though. Honestly I think knuckles are what you want if you want haptics.

  • Andrew Jakobs

    Damn they get that old device some extra life. Bought it a few years ago brand new for 15 bucks.

  • Looks incredible! Cant wait to try it out on my leap!

    • You should do it! With two hands it is very cool!

      • Gotta figure out how to mount it to my Quest 2 first XD

      • Indeed it is! Incredible how much better the tracking has gotten on this tiny little device with just some software updates. Up til now putting your hands together 100% of the time broke the simulation but post-this update, I put my fingers through each other and held my own hand… and it worked! No tracking glitch, no losing one hand or the other, nothing! Just perfect tracking (at least in the position I tried it in :D )

  • Thanks for the mention, Scott! And kudos to UltraLeap for the great work they’re doing

  • Zantetsu

    Hm I still have a Leap Motion stuck to the front of my DK2 in my closet. Brings back memories just to think of it. They had some fun demos that came with the Orion software, they were simple but I enjoyed them back when there wasn’t a whole lot to do in VR and sat for hours just flicking little cubes around in their demo apps.

    However, I would have to say that the left hand side in that video is pretty generous from what I remember. It wouldn’t take much finger motion for it to get confused, and unfortunately with that kind of tracking, the experience gets exponentially worse for every percent below perfect you are. I hope they’re much closer to perfect now.

    The hand/finger tracking on my Quest 2 is pretty good, I’d say 50% better than the Leap Motion was. I wonder how this new Gemini software compares to the Quest 2.

    Speaking of the DK2, I think it’s too bad to that so many early experiences like the Leap Motion demos are going to be lost forever in the annals of time. These were important early accomplishments in consumer VR that no one will remember.

    I can recall a simple ‘game’ or ‘experience’ where you observed a miniaturized doll-house obstacle course that was built like an old gothic ruins in a spiral around your head. It was dark and atmospheric. There was incredible lighting and shadow detail. You moved a small man around the obstacle course until he got to the very top, which was just above your head. Then you jumped him off towards your face and the surprise was that dragon jaws came out from your face and chomped him as if *you* were the dragon. It was astoundingly well done and an example of an experience that as far as I know, is lost forever.

    Also there was that one where the scene changed around you as you looked away, taking you from an outdoor nature setting to a city, to an interior banquet, to outer space, to the inside of some alien existence. I forget the name of that one. Wonder if it also will be lost forever, or will people somehow be able to find them and run them in DK2 emulators in future VR?

    • Ad

      You’re missing the 2016-2017 period it sounds like, when it improved a lot. This tracking is significantly better then quest either way.

      • Zantetsu

        The Orion update was the 2016-2017 period, and I didn’t miss it. It’s what I’m talking about.

    • Johnatan Blogins

      that was “Sightline: The Chair” the developer, Frooxius, has built Neos VR, and his latest version of The Chair is still on Steam, the Gothic Dragon one sounds good!
      Big fan of Leap Motion sensor, had an entry for their game jam, and always thought tracking was as good as VR got over the years, and now I’m looking forward to seeing where it goes!

    • Sibbo

      Would be great to have some early DK1 and Dk2 demos and games remade to work on quest. Like a retro arcade for nostalgia and to see what primitive people once did in be and how far we have come

  • Hopefully this means Facebook can get Quest 2 up to a similar level in the near future as well.

  • Ad

    It’s absolutely infuriating that they don’t at least make an official SteamVR driver. There are basic community ones but I’ve never been able to get them to work. It would give a minimum use case for these things and honestly it would have a fair amount of software that works out of the box, and a bit more that works with one controller and one tracked hand.

    I don’t know of even any demos that have come out for leap in years.

  • Patrick Hogenboom

    Wow, I wish that Gemini would work on my Index’ camera’s :)

  • Mike Dolnald

    It looks incredible! But it’s weird that I can’t have two hands intersection work correctly through Gemini V5. When I played with it, it only performed as Orion did (not supported two hands intersection). I used to work with Orion (V4) and bought Leap Motion controller years ago (I remembered at that time, the name of the company was still Leap Motion, rather than Ultraleap). I have migrated my Leap Motion software from Orion (V4.1.0) to Gemini (V5.5.3), but I still used original Leap Motion controller. Did someone have similar problems? Could you please give me some tips (like, buying the newest version of controller, since it may have something to do with the hardware, I guess)? Thanks in advance!