Switzerland-based CREAL is developing a light-field display which it believes will fit into VR headsets and eventually AR glasses. An earlier tech demo showed impressive fundamentals, and this week at CES 2020 the company revealed its progress toward shrinking its tech toward a practical size.
Co-founded by former CERN engineers, CREAL is building a display that’s unlike anything in AR or VR headsets on the market today. The company’s display tech is the closest thing I’ve seen to a genuine light-field.
Why Light-fields Are a Big Deal
Knowing what a light-field is and why it’s important to AR and VR is key to understanding why CREAL’s tech could be a big deal, so let me drop a quick primer here:
Light-fields are significant to AR and VR because they’re a genuine representation of how light exists in the real world, and how we perceive it. Unfortunately they’re difficult to capture or generate, and arguably even harder to display.
Every AR and VR headset on the market today uses some tricks to try to make our eyes interpret what we’re seeing as if it’s actually there in front of us. Most headsets are using basic stereoscopy and that’s about it—the 3D effect gives a sense of depth to what’s otherwise a scene projected onto a flat plane at a fixed focal length.
Such headsets support vergence (the movement of both eyes to fuse two images into one image with depth), but not accommodation (the dynamic focus of each individual eye). That means that while your eyes are constantly changing their vergence, the accommodation is stuck in one place. Normally these two eye functions work unconsciously in sync, hence the so-called ‘vergence-accommodation conflict’ when they don’t.
On more advanced headsets, ‘varifocal’ approaches dynamically shift the focal length based on where you’re looking (with eye-tracking). Magic Leap, for instance, supports two focal lengths and jumps between them as needed. Oculus’ Half Dome prototype does the same, and—from what we know so far—seems to support a wide range of continuous focal lengths. Even so, these varifocal approaches still have some inherent issues that arise because they aren’t actually displaying light-fields.
More simply put, almost all headsets on the market today are displaying imagery that’s an imperfect representation of how we see the real world. CREAL’s approach aims to get us a several steps closer.
That’s why I was impressed when I saw their tech demo at CES 2019. It was a huge, hulking box, but it generated a light-field that with one eye (and without eye-tracking) I could focus on objects of arbitrary depths (which means that accomodation, the focusing of the lens of the eye, works just like when you’re looking at the real world).
Above is raw, through-the-lens footage of the CREAL light-field display in which you can see the camera focusing on different parts of the image. (CREAL credits the 3D asset to Daniel Bystedt).
Slimming Down for AR & VR
At CES 2020 this week, CREAL showed its latest progress toward shrinking the tech to fit into AR and VR headsets.
Though the latest prototype isn’t yet on a head-mount, the company has shrunk the display and projection module (the ‘optical engine’) enough that it could reasonably fit on a heard-worn device. The current bottleneck which is keeping it on a static mount is the electronics required to drive the optical engine which are housed in a large box.
Shrinking those driving electronics is the next step; on that front, the company told me it already has a significantly reduced board which in the future will give way to an ASIC (a tiny chip) which could fit into a glasses-sized AR headset.
Looking through their CES 2020 demo, the company showed that they had replicated their light-field technology in a much smaller package, though the demo had a much smaller eye-box, field of view, and lower resolution than what could be seen through their much larger demo.
CREAL told me it intends to expand the field of view on the compact optical engine by projecting additional non-light-field imagery around the periphery.
This is very similar to the concept behind Varjo’s ‘retina resolution’ headset, which puts a high resolution display in the center of the view while filling out the periphery with lower resolution imagery. Except, where Varjo needs additional displays, CREAL says it can project the lower fidelity peripheral views from the same optical engine as the light-field itself.
The company explained that the reason for doing it this way (rather than simply showing a larger light-field) is that it reduces the computational complexity of the scene by shrinking the portion of the image which is a genuine light-field. This is ‘foveated rendering’, light-field style.
CREAL hopes to cover the entire fovea—the small portion in the center of your eye’s view which can see in high detail and color—with the light-field. The ultimate goal, then, would be to use eye-tracking to keep the central light-field portion of the view exactly aligned with the eye as it moves. If done right, his could make it feel like the entire field of view is covered by a light-field.
That’s all theoretically possible, but execution will be anything but easy.
A growing question is what level of quality the display tech can ultimately achieve. While the light-field itself is impressive, the demoes so far don’t show good color representation or particularly high resolution. CREAL has been somewhat hesitant to detail exactly how their light-field display works, which makes it difficult for me to tell what might be a fundamental limitation rather than a straightforward optimization.
VR Before AR
The immediate next step, the company tells me, is to move from the current static demo to a head-mounted prototype. Further in the future the goal is to shrink things toward a truly glasses-sized AR device.
Before the tech hits AR glasses though, CREAL thinks that VR headsets will be the first stop for its light-field tech considering more generous space requirements and a number of other challenges facing AR glasses (power, compute, tracking, etc).
CREAL doesn’t expect to bring its own headset to market, but is instead positioning itself to work with partners and eventually license its technology for use in their headsets. Development kits are available today for select partners, the company says, though it will likely still be a few years yet before the tech will be ready for prime time.