With Quest 3 now officially announced, Meta is emphasizing the device’s improved MR capabilities.
Meta CEO Mark Zuckerberg took to Instagram to share a first look at mixed reality gameplay on Quest 3 which was announced yesterday.
The video shows the headset’s full color passthrough MR mode, which allows it to present a view of the outside world while selectively adding virtual content to the scene.
We also see some shots of virtual objects attached to the wall, like a glass window into an undersea world, or a zombie jumping through a window into the room to attack the player. While Quest 2 and Quest Pro have done the same in the past, Quest 3’s new depth sensor should make attaching virtual objects to walls, floors, and ceilings more convincing thanks to a more precise map of the world around the headset.
We also see Meta CTO Andrew “Boz” Bosworth jump into the action, showcasing a co-presence experience where both Zuckerberg and Bosworth battle each other virtually but in the same physical space.
Beyond Quest Pro
It’s difficult to tell from the footage how Quest 3’s passthrough resolution compares to Quest Pro. However, it’s notable that the footage doesn’t show any of the obvious color fringing that was an artifact of Quest Pro’s passthrough architecture, which used multiple black-and-white cameras that were fused with the color from a single RGB camera. That ought to be solved now that Quest 3 will include two RGB cameras which will allow stereoscopic capture of color information, rather than monoscopic like with Quest Pro.
Another common artifact of Quest Pro (and Quest 2) passthrough is the warping of objects (especially hands) that are close to the headset. This is caused by a breakdown of the computer-vision depth estimation which struggles with near-field objects, especially when they’re moving.
It’s difficult to tell from the footage we have so far, but there’s a good chance that Quest 3 significantly reduces these passthrough warping artifacts thanks to its included depth sensor. Whereas Quest 2 and Quest Pro estimate the distance to objects and surfaces around the headset with computer vision, Quest 3’s depth sensor will provide much more reliable distance measurements which the system can use to judge how far it should render each part of the scene.
It will be interesting to see if the prior issue with color fringing on Quest Pro manifests in the same way with depth. With a single depth sensor, the headset only has a monoscopic depth view, whereas it will have a stereoscopic visual of the real world. Ostensibly the stereoscopic view of the world will be projected onto the depth map, and ‘depth fringing’ may occur around near field objects for the same reason that we saw color fringing on Quest Pro.