Facebook Reality Labs, the company's R&D department, previously revealed its 'Half Dome' prototype headsets which demonstrated functional varifocal optics small enough for a consumer VR headset. At a conference earlier this year, the Lab's Director of Display Systems Research said the latest system is "almost ready for primetime," and also detailed the Lab's research into HDR (high-dynamic range) and pupil-steering displays for XR headsets. Technological Readiness Douglas Lanman, Director of Display Systems Research at Facebook Reality Labs, gave a keynote presentation at the SPIE AR VR MR 2020 conference earlier this year. In his presentation, which was recently posted online, Lanman introduced a scale of 'technological readiness': Basic Research – Basic principles observed Technology Formulation – Technology concept and application formulated Initial Validation – Experimental proof of concept Small Scale Prototype – Technology validated in lab Large Scale Prototype – Technology initially validated in intended environment Prototype System – Technology robustly demonstrated in intended environment Demonstration System – System prototype demonstrated in operational environment First of a Kind Commercial System – System complete and qualified Generally Available Commercial System – Actual system proven in operational environment While the scale was originally used by NASA, Lanman likened it to the journey that research (level one) takes all the way through widespread availability of a product (level nine). Lanman explained that the work of researchers tends to focus on levels 2 through 4, at which point the research is published and the researchers move onto another project, but rarely see their work reach the higher levels on the scale. The Display Systems Research team at Facebook Reality Labs is unique, Lanman said, because the group has the capacity to work between levels 1 to 6, taking research all the way from "first principles" through to polished prototypes; much closer to a finished product than researchers typically see their work carried. "So what's really unique about this Display Systems Research team is that we're not quite a startup, we're not quite a major company, and we're not quite academics. We really play from the absolute fundamental vision science through very polished prototypes—more polished than you'd see from most startups—to try to do one thing, which is [have a genuine impact on future products]." Half Dome "almost ready for primetime" [caption id="attachment_90849" align="aligncenter" width="640"] Image courtesy Oculus[/caption] The team created a series of prototypes dubbed 'Half Dome', which employ varifocal displays that allow the headset to correctly support both vergence and accommodation together—something no consumer VR headset does to date. Half Dome 3 is the latest of the prototypes which Facebook Reality Labs has spoken about publicly. Instead of relying on a mechanically-driven varifocal display like the prior prototypes, Half Dome 3 implemented a static varifocal display which uses a series of liquid crystal lenses that allow the headset's optics to change between 64 discrete focal planes. Half Dome 3 also employs 'folded optics' which significantly reduce the size of the display module. [caption id="attachment_90846" align="aligncenter" width="640"] Size comparison: Half Dome 3 static varifocal display module with folded optics (left), Half Dome mechanical varifocal display module (right) | Image courtesy Oculus[/caption] The first Half Dome prototype was revealed back in 2018. At the time, Oculus said that customers shouldn't "expect to see these technologies in a product anytime soon." A year later the Half Dome 3 prototype was revealed, but Oculus was still tight lipped about whether or not the tech would find its way into a headset. While it's still not clear how close Oculus is to productizing a varifocal or folded optic display, Lanman ranked both Half Dome and Half Dome 3 on the technology readiness scale that he introduced earlier in this talk. He placed Half Dome, the mechanically-driven varifocal headset, at level 6 (Prototype System), and Half Dome 3, the static varifocal headset with folded optics, at level 5 (Large Scale Prototype). "It's almost ready for primetime," he said of Half Dome 3. Continue on Page 2: A Step Toward "creating the world's first HDR headset" » A Step Toward "creating the world's first HDR headset" After giving an overview of Half Dome 3, Lanman went on to share new research from Facebook Reality Labs which he called "the first step towards creating the world's first HDR headset." HDR is basically shorthand for 'very high contrast ratio capable of significantly more brightness'. The goal of an HDR display is to be able to produce a more realistic range of brightness that you'd find in the real world, all the way from dark shadows to bright reflections from the Sun. [caption id="attachment_97004" align="aligncenter" width="640"] Image courtesy Facebook Reality Labs[/caption] Lanman said that Facebook Reality Labs researchers created an optical pipeline that maintains as much of the light from the source as possible using a collimator and an engineered diffuser followed by dual modulation of the image to boost contrast, and finally passed through a lens that's specially designed to retain contrast. The team build a benchtop prototype to prove out the pipeline and created a working HDR display capable of 6,000 nits of brightness (compared to around 100 nits for the typical VR headset, according to Lanman). But, he said, the benchtop prototype was impractical to be miniaturized for use in a headset, largely because the contrast-preserving lens was far too large. [caption id="attachment_97007" align="aligncenter" width="640"] Image courtesy Facebook Reality Labs[/caption] So the team went about creating a 'folded' version of the lens. "This you could get on your head," Lanman said. The work is detailed in a paper titled High Dynamic Range Near-eye Displays which was published during the conference. Pupil Steering Theory Lanman also touched on 'pupil steering' retinal displays for AR headsets. Pupil steering is moving the retinal display to stay aligned with the movements of your eyes so that the headset can have a wide field of view without sacrificing a large eyebox. Before building a pupil steering system, Researchers at Facebook Reality Labs wanted to understand the fundamental requirements of such a system—in how many degrees of freedom must the retinal display be moved to achieve optimal performance? [caption id="attachment_97008" align="aligncenter" width="640"] Image courtesy Facebook Reality Labs[/caption] To answer that question, the researchers built a simulated retinal display to answer basic questions like 'how bright would the image be?' 'What will happen to the image during fast eye movements if the pupil steering can't keep up?' The findings were published last year in a paper titled Retinal Image Quality in Near-eye Pupil-steered Systems. - - — - - "After five and a half years we're having a lot of fun working across a broader scope of research than many of us have in the past," Lanman said, wrapping up his presentation. "Starting at level 1, we're trying to push fundamental vision science to drive requirements for our systems. We still have plenty of not-quite-ready for primetime ideas at levels 2 through 4. And occasionally, we dive into startup mode, and we do things at level 5 and beyond, like Half Dome. We're so happy to get to pull back the curtain here at [the conference]. And of course, we hope to inspire others." Interested in R&D? Check out our recent coverage of Facebook Reality Labs' holographic folded optics and field of view expansion for holographic displays.