At CES 2015 last week, we met up with Leap Motion co-founders Michael Buckwald and David Holz to try their latest prototype motion input camera, codenamed ‘Dragonfly’, and chat with the duo to learn more about their plans for making gesture input part of every VR experience.
Leap Motion is a low-cost motion input camera that was announced in May, 2012, a few months before the Oculus Rift DK2 Kickstarter, with an initial aim toward desktop applications. In August, 2014, the company ‘Set Course‘ for virtual reality, releasing a ‘VR Mount’ which conveniently attaches the sensor to the front of any VR headset. At the same time, the company teased a prototype version of their gesture camera called ‘Dragonfly’, which aims to be “embedded by VR OEMs,” with “greater-than-HD image resolution, color and infrared imagery, and a significantly larger field of view.”
See Also: Leap Motion’s Next-gen ‘Dragonfly’ Sensor is Designed for VR Headsets
At CES 2015 last week, we got to try Dragonfly for ourselves and learn more about the prototype sensor and its aspirations from the company’s co-founders, Michael Buckwald and David Holz
The Dragonfly sensor prototype we saw at CES was a naked circuit board sporting two cameras, attached to an Oculus Rift DK2. Buckwald and Holz told us that differences between the made-for-VR Dragonfly and the existing Leap Motion device (which they call the “Peripheral”) is higher resolution (they said “better than native” on a 2.5k display, but wouldn’t specify further), the distance between the cameras is set to approximately the human IPD average of 64mm (compared to 40mm of the Peripheral), the cameras are capable of a wider field of view and higher framerate, and the unit also sports IR and RGB (color) sensors (whereas the Peripheral only has IR).
See Also: Phase Between the Real and Virtual World With Leap Motion and a Swipe of Your Hand
I tried the Peripheral and the new Dragonfly sensor back-to-back mounted on a DK2. The increase in resolution was noticeable in both IR and RGB modes. The IR mode on Dragonfly actually looked better in terms of resolution than the RGB mode due to quite a bit of graininess on the latter; Buckwald and Holz told me this was because they haven’t yet implemented any image-processing, as is typical with digital cameras, to clean up some of the noise and correct for other bits. They said such filtering is planned to be added soon.
Having used the low field of view monoscopic passthrough camera on Gear VR, which felt like looking through a small 2D window into the real world, Dragonfly’s passthrough view filled the entire Rift screen and provided depth perception thanks to the dual cameras, making it easy to reach out and grab objects.
Holz walked me through a series of alternative views, first removing one of the two cameras (effectively removing my sense of depth), then bringing the field of view down to that of a standard cell phone camera (which brought the outside world back to that small window I was used to with Gear VR). From there he re-enabled the second camera, giving me a small 3D window, then jumped back up to a vision-filling field of view. The whole process was very interesting and showed how much more usable a passthrough camera can be with a wide field of view and stereoscopic imagery.
Holz also demonstrated some augmented reality functionality with Dragonfly by loading floating spheres in the environment surrounding me; I could swat my hands through the air to disrupt and move the spheres, but more interestingly, I could move my real hand behind the virtual sphere, and the sphere would occlude it. This may not sound like a big deal, but it’s actually a rather challenging problem for augmented reality, and one that’s really neat to see done well in practice.
As we talked with Buckwald and Holz, it became clear that the company’s ultimate goal is to have Dragonfly embedded in every VR headset. They seemed less interested in packaging up Dragonfly into a gen 2 Peripheral, but said they’d consider it if necessary. The duo also reinforced that the software is a huge component of Leap Motion’s functionality and strategy; even if the hardware stays the same, the software continues to improve, they said.
At the end of our interview, Buckwald said that Dragonfly is “not an ending, it’s a beginning,” and teased that in the next few weeks we’d hear announcements from partners that will be incorporating new Leap sensors into their devices.
