Oculus brought controllerless hand-tracking to Oculus Quest as an experimental feature back in December, and while it’s useful for navigating around the headset’s menus when you don’t want to pick up the controllers, developers have begun prototyping more creative uses for the tech.
Controllerless hand-tracking on Quest is a great convenience feature. While we still expect controllers to be the go-to for hardcore games, there’s definitely opportunity for casual and novel games and experiences to prosper with hand-tracking.
Daniel Beauchamp is one VR developer that’s dreaming up creative uses for hand-tracking on Quest that go far beyond laser pointer and touchscreen interactions. He’s created a series of prototypes as a means of understanding the capabilities of the feature and what kind of interactions and experiences can bring delight to users. Here’s three that he shared recently.
Thrown Hand
Turning the tracked hand into its own independent object allows for a ‘remote control’-like capability where the hand propels itself around the environment by ‘crawling’ as the player continues to move the fingers on their real hand. Intriguingly, one could consider this a form of VR locomotion which only applies to a specific body part. This also extends the range of potential interactions in a more interesting way than 2DOF interactions like laser pointers and guns.
Finger Skateboard
Harkening back to the ’90s fad of fingerboarding (AKA Tech Decks), Beauchamp envisions a game mechanic where players control a tiny skateboard with their fingers. In later experiment he even showed some basic physics, allowing the user to flip the board to do tricks.
Hand Walking
While it’s easy to get around using VR controllers thanks to a combination of 6DOF motion and physical inputs like buttons and sticks, things get more complex with just your hands. In this prototype, Beauchamp shows a locomotion scheme where the player puppeteers a leg gesture with their hands to walk forward. While not likely a serious solution for VR locomotion, it’s a thought-provoking concept and one which could form the basis of other interesting interactions involving the personification of the user’s hand.
Experimenting Outside of the Box
Beauchamp is the Head of VR and AR at Shopify, a leading e-commerce platform. While he told us that his Quest hand-tracking experiments are done on his personal time, he applies the same concepts to how he thinks about the intersection of XR and e-commerce.
“One of the best ways to unlock new & powerful ideas is to build upon silly ideas. That’s what I’m doing with hand-tracking. Rather than starting with ‘How can I build a useful product with hand tracking’, I play around with many little ideas that bring a smile to people’s faces. They may seem random, but I’m learning a lot about what’s possible with the tech and interactions that could be applied elsewhere,” Beauchamp said. “I wish more VR devs did this. Don’t put the burden on yourself to build out a whole game or build out a whole product. Build many small things, no matter how silly they may seem. You’ll be surprised at just how much you learn.”
He stresses the need for rapid-prototyping, something he felt important enough that he and the Shopify VR team built a VR rapid-prototyping tool for rapid experimentation.
“A lot of people are surprised Shopify has a VR team. We’ve had one for the past five years. Our goal has been to figure out how VR will impact the world of commerce.
This problem space requires outside-the-box thinking. Every time I talk about VR & shopping, people immediately suggest shopping malls. And to me that’s such a narrow view,” said Beauchamp. “In VR you have the ability to do anything you want, be anyone you want, and be anywhere you want. You are a wizard…and you want to shop in a regular mall? Wizards don’t shop at boring malls. So we’ve tossed out a lot of the notions you’d expect from a brick and mortar store bound by reality. We’re looking at things that are delightful and effective that are now possible in this new medium.”
But how do you think outside-the-box if the box is all you can see? Beauchamp suggests tossing out what you think you know.
“I think you just have to consciously throw out the [interaction] tropes [of real life and traditional games]. Like just prevent yourself from using them. Come up with ideas that don’t use them first, then evaluate if you still need them.”
– – — – –
Quest hand-tracking is expected to eventually leave its experimental status behind and become a full-blown feature on Quest, and other VR headsets are likely to follow, making the time ripe for developer experimentation ahead of implementation.