Oculus finally revealed that Quest will get controllerless hand-tracking input. We got to test the feature at Oculus Connect 6 last week and came away impressed. Hand-tracking will open the door to a range of casual use-cases that will broaden the headset’s appeal, but core gaming experiences will still rely on the precision and reliability of controllers.
Hand-tracking is coming to Quest early next year, Facebook announced last week at Oculus Connect 6. The appeal is clear: using your hands instead of controllers makes the headset that much easier to use; instead of learning the placement and functions of buttons and sticks, users will be able to throw on the headset and simply point and touch their way around virtual worlds.
At Connect, Oculus showed off the hand-tracking with a demo called Elixir made by VR studio Magnopus. The setting was fantastical witch’s workshop where I got to poke, prod, and play with spells and magic.
Quest Hand-tracking Demo
It was clear from the outset that the entire demo was designed and built specifically for hand-tracking. The demo revolved around touching, poking, and pinching interactions, but, notably, no ‘holding’ or direct manipulation of objects except in the case of a pen (which is a very purposeful choice that I’ll discuss more later).
Elixir is a sandbox-ish experience designed to show some of the ways that hand-tracking could be used for gaming interactions. In the experience I was asked to touch my hands to various objects which would turn them from human hands into magical hands which would give me unique powers.
For instance I was prompted to touch my hand to something which looked like a hot-plate crossed with grill which caused my hands to turn into fire-infused hands. Then when I waved my fire hands over some candles they were ignited. Another object I touched turned my hands into the hands of a creature with Wolverine-like claws which extended when I made a first.
The coolest and most inventive of these hand transformations was when I dunked my hands into a cauldron and they turned into octopus tentacles! The tentacles were longer than my actual fingers and they were physics-enabled, so when I wiggled my fingers the tentacles wobbled about in a gross but oddly satisfying way.
Designing Around Haptics Limitations
Aside from the hand transformations (and the ‘make fist’ gestures which powered them), a ‘pinch’ gesture was used at several points throughout the experience which, from a design standpoint, served as a sort of virtual button press. For instance, hanging above the cauldron was a liquid dispenser with an eye-dropper top which squirted liquid into the cauldron when squeezed. Elsewhere, a miniature bellows could be squeezed with a pinch to brighten a flame.
Because the pinching gesture naturally involves touching your own fingers, it provides a sort of self-haptic sensation which feels more natural than pressing a virtual button with no feedback at all.
This is the same reason why the magic pen—which was offered to me at the end of the experience to sign my name on a big scroll—was the only object in the experience which could actually be directly held and manipulated. It uses effectively the same pinching gesture as before (mostly replicating a real pen grip), which offers a haptic sensation since you’re actually touching your own fingers.
A good general purpose ‘grab’ gesture for arbitrarily sized and shaped objects—as has been attempted with gestures like making a fist or a ‘C’ shape with your hand—has remained quite elusive, as the lack of feedback just doesn’t feel quite right (not to mention issues with consistently detecting the pose to prevent items from dropping). That said, even a gesture-based ‘grab’ that doesn’t feel immersive could be perfectly useful for non-entertainment applications where immersion isn’t a high priority.
