The first version of Varjo’s ‘bionic display’ enterprise VR headset is due out later this year (see our latest hands-on), and the company has announced plans to offer an add-on upgrade in 2019 which brings high-quality pass-through AR capabilities to the headset. Varjo showed off a tech demo of the headset’s add-on capabilities this week at AWE 2018.
Given Varjo’s positioning (exclusively for non-consumer sectors), the company is aiming for maximum quality visuals for their AR add-on. As shown in their current tech demo headset, it houses a pair of 4K cameras with lenses that match the headset’s own ~95 degree field of view, supporting stereoscopic pass-through video at 90Hz. Latency for the pass-through input is presently around 40ms, the company says, but they expect to be able to achieve single-digit latency before the add-on launches.
A look through their AR tech demo headset (which is identical to the current Varjo Beta headset, with the addition of the AR add-on) showed a fair bit of noise in the low lighting of the demo space; despite this, it was clear from a visual quality standpoint that Varjo has presented the best looking pass-through AR I’ve seen yet. Inside the headset I saw the view of the real room around me, while a full-sized model of a virtual motorcycle was projected into the space. Given the headset’s unique inset display, the center of my field of view revealed crisp details on the motorcycle that couldn’t be seen otherwise. Compared to optically transparent AR headsets, the field of view of both the real world and the virtual content was wide and immersive.
The brief demo I saw was purely visual for now, and lacked interaction. Varjo isn’t talking about advanced functions which one would hope to see with a pass-through AR system, like hand-tracking or SLAM; so while it’s clear that Varjo is aiming for high quality visuals in their AR add-on, it isn’t clear how much of the input/interaction pipeline they intend to own.
Varjo released a video showcasing the upcoming AR pass-through add-on, though there’s an important caveat to note before viewing it: the company says that it was “shot entirely through Varjo’s [AR] prototype in real-time,” but this is somewhat misleading, as the video was not recorded through the headset’s lenses. Instead, it appears they recorded the input from the headset’s camera, and while it’s mostly representativie of the quality you’d see on the headset’s focus display, it doesn’t represent the quality drop off of the peripheral context display. More simply put: it’s a direct video feed instead of being shot through the lens of the headset, and the part you’re seeing represents the quality of the headset’s high resolution center display, not the lower resolution peripheral area.
Caveats aside, what they’re showing here is rendered in real-time, and the company says that their software is interpreting the scene’s real world lighting to allow the virtual components to cast convincing shadows onto the real ground, and for elements of the real scene to be cast in reflections on virtual objects (in the last shot you can see that the puddle reflections on the ground are actually virtual.
There’s pros and cons to doing pass-through AR instead of transparent AR. For one, the augmented parts of the world can look quite a bit more real because the pixels are drawn directly on top of the image of the real world, eliminating that semi-transparent ‘hologram’ look that you’ll find on transparent AR displays (caused by the fact that it’s extremely difficult to create pixels that can occlude 100% of the light from the outside world when dealing with a transparent display). It’s also much easier today to achieve an immersive, wide field of view by using pass-through AR rather than transparent AR.
On the flip side, our eyes have excellent resolution and contrast ratio, which generally means we limit our dynamic range (the ability to see dark and light areas of a scene at the same time) and visual fidelity when using pass-through AR (not to mention the potential to introduce latency).
Update (6/3/2018): A prior version of this article stated that the pass-through video was only showing the cropped view that would be displayed on the headset’s central focus display. This has been corrected to indicate that the pass-through video shows the footage that would be displayed across both displays, but is shown at a quality level which matches the headset’s cameras without showing the pixel density drop off between the focus display and the peripheral context display.