If you haven’t heard of the Leap Motion, you’ve got some catching up to do. It’s an $80 motion sensing peripheral, designed to sit on your desk, that boasts ultra low latency and ultra high accuracy for tracking hands and other objects. It seems like a natural fit for the Oculus Rift and virtual reality input, but so far we’ve seen much more development happening with the Razer Hydra than the Leap. Here’s a video showing early implementation of the Leap in unit combined with the Oculus Rift and positional webcam tracking.
Harley Witt is the author of this prototype and he tells me that this prototype is built in Unity with models from Asset Store. The positional head tracking is done with the webcam, which is likely part of the reason that it’s so jumpy.
“I took a sample Unity project that had simple hand and finger tracking and with Unity Mecanim feature it has an inverse kinematic system for head, hands and feet. It was fairly simple to pick a relative pointable object (Finger) and map it to a hand position for the IK,” said Witt.
It would seem that the Leap API is not yet terribly adapted for virtual reality input.
“…there are still big issues with the way Leap Motion implements it’s pointable objects. In a nutshell there is no persistance of object order and you can’t rotate your hand beyond sideways. Thumb is missing most of the time. Unity only has tracking for the palm, not the fingers anyway. When the coding starting getting more complicated I stopped (to hopefully let Unity and Leap Motion software catch up). There is much more one could do, but my priority will likely shift to the Hydra,” said Witt.
“…I seriously doubt it,” he said when I asked if the Leap was a viable VR input option for hands. ” The dev team doesn’t seem interested in pushing their capability into adequate skeletal tracking. They have point cloud data but they aren’t sharing it through an API (a little sad about that).”
Sad to hear given the capabilities of the Leap, but it should be noted that the company shipped out developer kits several months ago and the consumer model is shipping early next month. Thus there’s likely many improvements to be made down the road. Or at least we hope!