Leap Motion Open-sources Project North Star, An AR Headset Prototype With Impressive Specs

Earlier this year Leap Motion had been teasing some very compelling AR interface prototypes, demonstrated on an unknown headset. The company revealed that the headset is a prototype dev kit, designed in-house, offering a combined 100 degree field of view, low latency, and high resolution. Leap Motion has begun open-sourcing the design of the device, which they’re calling Project North Star.

Update (6/6/18): Leap Motion has begun the process of open-sourcing the North Star headset. Today the company published a new hub page for project North Star and detailed documentation showing how to construct the headset. Mechanical and electronic schematics and designs have been release on GitHub, and everything falls under an Apache 2.0 license.

“Our goal is for the reference design to be accessible and inexpensive to build, using off-the-shelf components and 3D-printed parts. At the same time, these are still early days and we’re looking forward to your feedback on this initial release. The mechanical parts and most of the software are ready for primetime, while other areas are less developed,” the company notes on its blog. “The reflectors and display driver board are custom-made and expensive to produce in single units, but become cost-effective at scale. We’re also exploring how the custom components might be made more accessible to everyone.”

The company promises additional details and updates to the open-sourced information “in the coming weeks.”

Original Article (4/9/18): Founded in 2010, Leap Motion develops leading hand-tracking hardware and software. Though their first piece of hardware was designed for desktop input, the company pivoted into VR, and more recently the AR space, exploring how their hand-tracking tech can enable new and intuitive means of interacting with virtual and augmented information.

With AR hardware still in its infancy, the company sought to build their own in-house prototype AR headset, targeting specifications far beyond what’s available to consumers today. This was so they could design AR interfaces, based on their hand tracking tech, targeting the capabilities of future AR headsets. They’re calling this work Project North Star, and plan to open-source the design next week, saying that such a headset could cost “under $100 dollars to produce at scale.”

The prototype headset uses side-mounted displays with large ‘bird bath’ style optics (similar to the Meta 2 approach), which afford the device a 1,600 × 1,400 per-eye resolution at 120 FPS, with over 100 degrees of combined field of view, and hand-tracking from the company’s latest hardware which tracks at 150Hz over a 180 × 180 degree area.

The version of Project North Star which Leap Motion plans to open-source is actually a pared back version of an earlier prototype which boasted greater specs, but was quite a burden to wear. The team at Leap Motion constructed this earlier version as a baseline of what could be achieved.

“[…] we wanted to create something with the highest possible technical specifications, and then work our way down until we had something that struck a balance between performance and form-factor,” the company shared on its blog today. “[…] The vertical field of view struck us most of all; we could now look down with our eyes, put our hands at our chests and still see augmented information overlaid on top of our hands. This was not the minimal functionality required for a compelling experience, this was luxury.”

With a good look at what could be achieved, the team used masking tape over the lenses to crop down the field of view to get a feel for how much they could reduce the size of the lenses before losing some of the essential experience due to the lower field of view. Once they found that balance they began the process of cutting smaller optics and shrinking the headset, moving from cell phone displays to a custom display system using a pair of 3.5″ fast-switching LCD displays.

“We ended up with something roughly the size of a virtual reality headset. In whole it has fewer parts and preserves most of our natural field of view. The combination of the open air design and the transparency generally made it feel immediately more comfortable than virtual reality systems (which was actually a bit surprising to everyone who used it),” the company writes. “[…] Putting this headset on, the resolution, latency, and field of view limitations of today’s [AR] systems melt away and you’re suddenly confronted with the question that lies at the heart of this endeavor: What shall we build?”

Indeed, with hardware in hand, the company has been focusing on that question; using a wearable camera, it was the Project North Star prototype through which Leap Motion’s VP of Design, Keiichi Matsuda, shot those tantalizing ‘virtual wearable’ prototype videos which we recently called “a potent glimpse at the future of your smartphone.”

Leap Motion says they have a few tweaks left to do before open-sourcing the Project North Star design next week, including “room for enclosed sensors and electronics, better cable management, cleaner ergonomics and better curves […] and support for off the shelf head-gear mounting systems.”

There’s also a number of areas where Leap Motion says that Project North Star is ripe for further development:

• Inward-facing embedded cameras for automatic and precise alignment of the augmented image with the user’s eyes as well as eye and face tracking.
• Head mounted ambient light sensors for 360 degree lighting estimation.
• Directional speakers near the ears for discrete, localized audio feedback
• Electrochromatic coatings on the reflectors for electrically controllable variable transparency
• Micro-actuators that move the displays by fractions of a millimeter to allow for variable and dynamic depth of field based on eye convergence

With the open-sourcing of the Project North Star hardware and software, Leap Motion hopes that the design will “spawn further endeavors that will become available to the rest of the world.”