Meta Confirms Next-gen Consumer Quest Headset Planned for 2023

No insider info needed. Just take a look at the Snapdragon 865 and then the Snapdragon 8 Gen1. About double the GPU performance. And with the XR2 Gen2 being based on the SD8 Gen2, expect around 2.5 times the GPU performance.

Now they’ll probably use the same (highly praised) pancake lenses of Quest Pro together with 2 separate screens (which makes for a better panel utilization than Quest 2) and you’ll have a more comfortable, significantly better looking headset with smooth IPD adjustment with WiFi6E for better wireless PCVR.

These sensors shouldn’t tax the GPU though, rather the CPU. Although there’s a good chance it will have some specialized units / features to help with that, as Carmack hinted.

“Performance gain per month” is a really strange metric though. Quest 1 simply started out with an older SOC. We should compare SoC release vs. SoC release.

Honestly, it might aswell go the other way: A performance gap that’s even bigger. Just speculation of course, but the GPU clock on the XR2 in the Quest 2 is heavily restrained due to the thermals. With the XR2+, they changed the package so that it’s easier to cool and can therefore sustain higher clocks.

What if that knowledge went into designing the XR2 Gen2 aswell, giving it an even bigger advantage than the raw specs suggest?

At least we can pretty sure that Metas involvement in designing the XR2 Gen2 was even bigger than with the XR2 Gen1.

Not confirmed of course, but the Quest 3 design leak claims no face or eye tracking and I think that’s very plausible considering the price focus.

Don’t count on the Q3 to be $800, it will be around the same current price as the Q2 (don’t believe they will go down to the initial Q2 release price).

I don’t think they will (have to) subsidize the PSVR 2. With a 2K resolution, USB-C connection, Fresnel lenses, b/w passthrough, no built-in audio and other costs reducing measures it will be rather cheap to produce, way below what a Quest 2 or Pico 4 costs to build, allowing them to sell the PSVR 2 for much less than those. Whether they will do that remains to be seen, but my guess would be that they intend to at least tripple or quadruple the 5m PSVR 1 sales, and that requires offering the headset and bundles at a low price. Sony stopped selling consoles below production costs after the PS3, since then they always made a (tiny) profit from the start which rose over time due to falling production costs.

Skipping the holiday season isn’t necessarily due to building a large PSVR 2 launch inventory. Sony said they expect to solve their component shortage problems currently still constraining PS5 availability in the first half of 2023, so delaying the PSVR 2 is more about not adding to the expected shortage of PS5 during the 2022 holiday season and being able to actually have enough PS5 to sell to new PSVR 2 costumers on launch. So it is more about building a PS5 inventory. And no matter how good the PSVR 2 will be, VR is still a niche product and people will take some time to get convinced, so I wouldn’t expect them to sell millions on the first day.

Unfortunately Sony just announced the prices as USD 550 a few minutes ago, so while Sony now definitely doesn’t have to subsidize the HMD, they obviously will not pass that to customers.

The specs are somewhat improved over the Quest 2, but not a lot when considering that the PSVR 2 is released three years later. The most expensive parts of a phone are usually the SoC and the screen. The SoC should be simpler in the PSVR 2 compared to Quest 2, depending on how much of the tracking can be delegated to the PS5. OLED screens are more expensive, but not significantly more, and Sony is one of the main users of OLED in smartphones, designed by Sony, produced by Samsung or LG. Sony actually produces 4K 120Hz HDR OLED phones, most likely based on the same panels as the PSVR 2 with only 16% more pixels than the 2020 Quest 2. The battery and the electronics needed to charge it add minor costs, but not needing any on the PSVR 2 of course reduces cost. The improved haptics are based on LRAs and voice coil actuators, more expensive than the usual vibration motors, but are also not very expensive, nor is the hardware for eye tracking. There is no audio on the HMD itself, though the PSVR 2 now seems to come bundled with headphones, and the Fresnel lenses should be cheap too.

I did a breakdown of all the components, tried to find sources for them with actual prices, and they are simply not particularly expensive. I even made estimates regarding the license cost for the Tobii eye tracking (< USD 18) based on Tobii's stock price.. Sony benefits from waiting till 2023, with a lot more technology for VR now being more easily available. Which is why I came to the conclusion that the main design objective was low production costs, and I still would expect the HMD and the controllers to be (significantly) cheaper to produce than USD 250. I was hoping that their main reason to do this was to keep the sales price very low too, in order to reach a broader audience by lowering the entry bar. At least that last part turned out to be very wrong.

It is the same type of sensor, near infrared cameras. These are actually just regular b/w cameras without a filter to remove non-visible light, which is usually integrated as the nIR light the camera sensors are sensitive to would otherwise distort the image. This is why they work in the Quest for both scanning the room and providing a grayscale passthrough image, while at the same time helping with controller and hand tracking, as the nIR LEDs on the Touch controllers and the warmer hands will glow brighter, making their position and shape easier to detect.

The Leap Motion runs two cameras at 120Hz compared to 30Hz/60Hz on the Quest, most likely due to having the extra computational power of PCs. Ultraleap now also offers a tracking module for integration into HMDs with a larger FoV, but limited to 90Hz, so I’d assume that the 120Hz isn’t strictly necessary or worth the extra power it needs.

Meta only warned that 60Hz hand tracking wouldn’t work with the highest performance level, but many apps run just fine at the recommended levels 2 and 3. As decoding video streams only requires very little energy due to the dedicated hardware decoders, it shouldn’t be any problem to use the faster tracking frequency when using the Quest 2 with (Air) Link. I’m not 100% sure here, because AFAIK the h264/h265 decoder is located on the DSP also used for the tracking, but I doubt that they will interfere. If you have a headset which can display a native DP signal like the Pico Neo 3 Link, there has to be a separate decoder chip for that, as the XR2 is lacking native DP decoding capability. In this case the XR2 will experience even less load, so the DP decoder would have to be terribly inefficient to run hot enough to cause the whole device to have to throttle down or reduce the hand tracking frequency.

As video streaming is a very common use case on mobile phones, this is highly optimized to increase battery life. The decoders themselves take only a tiny part of the silicon, and John Carmack managed to get even the SD821 in the Oculus Go to decode 5K@60Hz h264 video to be used as 360° background video in VR apps, despite Qualcomm engineers telling him that this was impossible. The XR2/SD865 is much faster and has another specialized subprocessor for video capture that can record and compress 8K@24Hz or 720p@960Hz video, and obviously has to be able play these as well. So it should be quite a challenge to get it to overheat with decoding any type of video stream.

I don’t know why they don’t allow it, but it is not strictly due to technical limitations. There is a fork of the open source ALVR streaming alternative to Air Link or VirtualDesktop called ALXR that allows sending the hand tracking information to the PC for OpenXR applications.

Hand tracking is currently an extension in the OpenXR specification, so there has to be a translation layer that takes the Meta specific XR_FB_hand_tracking_xxx data and spits out XR_EXT_hand_tracking, XR_HTC_hand_integration, XR_MSFT_hand_interaction, XR_ULTRALEAP_hand_tracking_XXX or whatever the app is able to handle. So the main reason for (currently) only allowing it in standalone might simply be that it is not yet part of the regular OpenXR specifications, forcing developers to deal with vendor specific extensions. A problem that is similar for eye tracking and other features, something future versions of OpenXR will most likely resolve.

We are talking about different things here. I was referring to only sending the results of the face/eye/hand tracking from the Quest back to the PC, which shouldn’t cause bandwidth issues, as it would be just a few vectors for eye tracking and a few more for the rig of the skeletal model of the hands, but the total amount of data should be negligible. It should also be feasible as long as the type of data is defined in OpenXR in some way, and both the client on the Quest and the receiving PC host software can produce/interpret OpenXR compliant data.

You are talking about sending (raw) sensor data from the Quest to the PC and doing the image analysis and tracking there, which is much more difficult. It would seem a good idea due to the higher processing power on the PC, but besides actual bandwidth issues you first of all have to run matching tracking software on the PC, which usually isn’t available. Pretty much the only VR hand tracking software on PC comes bundled with software from Ultraleap or HTC, and these don’t accept raw image feeds, instead directly query their sensors. Most PC eye tracking will be based on proprietary software by Tobii, also bound to their sensors. So even if you could get the sensor data from Quest Pro to PC, requiring raw access on the Quest which we don’t get due to privacy issues, you’d currently have to create your own tracking software to then again create matching, OpenXR compliant data and still need apps that will actually accept this type of OpenXR tracking data and do something useful with it.

Bandwidth could be an issue when trying to send multiple camera feeds, though the resolution of the nIR cameras used for tracking is rather low. AFAIK the Leap Motion sensors resolved 640*240 pixels, the eye/face tracking sensors on the Quest Pro 400*400, all in b/w. The hand/room tracking cameras on the Quest 2 aren’t highres either, the only multi-megapixel sensor is the RGB camera. Looking only at the five face and eye tracking sensors, combining their data into a single image and sending it to the PC would result in a smaller images than the 1280*720p many webcams can send back, usually encoded frame by frame as Motion JPEG. It would be difficult to send raw, uncompressed sensor data, but if the Quest can do at least some image preprocessing the bandwidth needs will drop drastically. The XR2 is obviously capable of live recording the current Quest app, encoding the image as a 1080p h264/h265 video stream and sending it to the Oculus app or another streaming target for spectators, thanks to the previously mentioned Spectra image subprocessor that could actually handle a lot more pixels.

I’ve made a similar argument for the PSVR2, which uses DisplayPort over USB-C to send the image from the PS5 to the headset by rededicating the four USB superspeed data line pairs to DisplayPort lines thanks to the so called DP Alternate Mode that is part of the USB specification. This results in only a USB-2 speed data channel remaining available to send back data from the PSVR 2 to the PS5, not enough for raw camera sensor data. So the PSVR2 has to have some type of SoC onboard to do either the tracking locally or at least preprocess and compress the image sensor data to reduce its size to fit through the narrow USB-2 connection.

On the PSVR2 this might actually make sense and allow for a cheaper HMD, as the tracking software on the PS5 would obviously work with the PSVR2 sensor data, just like Tobii eye tracking on the PC works with Tobii eye tracking modules on the HTC Vive Pro Eye. I doubt that we will see any type of open solutions that will allow the Quest tracking camera feed to be analyzed on the PC for the above mentioned privacy issues and lacking software to do that. Someone could try to implement it with e.g. existing hand and eye tracking solutions based on OpenCV, and calibrate it for well known sensor types like those in the Quest Pro, but it will be much, much easier to just translate the OpenXR results from the Quest Pro internal tracking to something an app on the PC can use directly.

Yes, using the data depends on the manufacturers providing access to the tracking devices via OpenXR, but that shouldn’t be an issues, as this is the actual purpose of OpenXR, which every significant XR player besides Apple supports. Sony hasn’t officially announced an OpenXR compliant SDK yet, but Quest, Steam and Pico as major consumer platforms all have started switching from their similar, but incompatible SDKs to OpenXR. All the ports of Quest apps to Pico are based on this and the fact that most VR apps are created in Unity, which adds its own, now also OpenXR based abstraction layer.

So the incompatibility is not due to Meta’s reluctance to grant access to the data. They already provide it, otherwise ALXR couldn’t have released a client app for the Quest allowing to pass the results of the Quest hand tracking to the PC. Usually any function accessible to Quest developers could be forwarded to a PC, unless restricted by the license agreement similar to what HP did with the Reverb G2 Omnicept that requires an extra Inference Engine SDK subscription for advanced eye, face and heart rate data.

The problem is more that OpenXR is still new and somewhat incomplete. Quest and Rift got full support for the current version in July 2021, and that version hasn’t standardized either hand nor face nor eye tracking yet. It just acknowledges that these (will) exist and included a way for manufacturers to provide their own, again incompatible extensions. This way developers can at least use them in combination with the main OpenXR specs instead of being forced to use proprietary SDKs. If Pico or anybody else really wanted, they could implement their own versions of the Meta, Ultraleap or HTC hand and eye tracking extensions, allowing developers to port Quest Pro apps by basically recompiling them. As these extensions (currently 101 from multiple vendors) are described in the OpenXR spec, Meta couldn’t stop them even if the wanted to.

A future version of OpenXR will define a standard way to access the new tracking data, which will incorporate what has been learned from the current vendor extensions, and manufacturers will start to switch from their proprietary extensions to this standard. This is very similar to how OpenGL developed over decades. It makes more sense for Pico and others to wait for the improved spec instead of now reimplementing multiple extensions from other vendors, but this process will take some time. In the mean time it is at least possible to either port apps from for example Quest to WMR and only have to alter the parts that currently still require vendor specific extensions, or write an abstraction layer that translates one extension to another, if their features match.

VR is still such a tiny niche that the different vendors are forced to play somewhat nice with each others. It is hard enough to get larger studios to invest into VR, so making it even harder by sticking to a lot of incompatible SDKs would pretty much hurt everybody. Many problems with incompatibility come from the tech simply being still new and the standardization processes both taking time and first having to wait until standard industry practices have established themselves, which the next specifications then try to match.

The Khronos group that manages OpenXR isn’t a specific team, it is a consortium of 170 organizations defining standards for all things 3D, and most of the members are from the industry. So there isn’t necessarily a sort of planed roadmap and a conscious decision when to include things, it is an ongoing process. OpenXR 1.0 was released in July 2019, we are currently at OpenXR 1.0.25. The changes are minor and mostly happen in the extensions, so the core remains stable, but for example the XR_EXT_hand_tracking is now in rev 4 and was last modified 2021-04-15. XR_EXT_ extensions are already a step further from vendor specific ones, they try to create a common reference even though it is not yet part of the main definitions. These XR_EXT_ will probably go through a couple of iterations and become part of the spec with an OpenXR 1.1 spec.

If the XR_EXT_hand_tracking meets all the needs at Meta, they might decide to switch to it and deprecate their own extensions, but they don’t have to while things are still developing. And as far as I can see, they already did that, as rev2 of XR_FB_hand_tracking_aim from 2022-04-20 already refers to XR_EXT_hand_tracking, though I don’t follow it close enough to be sure. The OpenXR spec is intended for developers and vendors, not for end users, so its purpose is to make development easier without complicating it. While it would be great for users if we already had a final definition of how hand or eye tracking works across all platforms, so you could connect all the parts anyway you like, the current approach with vendor extensions for testing out things, the XR_EXT_ as a voluntary shared common and then finally an updated specification is a good compromise considering that there we are just seeing the first implementations, often still buggy like with hand tracking or yet unproven benefits like with eye tracking.