Qualcomm’s New Mobile Tech Unlocks Impressive Inside-Out Tracking

At CES 2017 Qualcomm unveiled its Snapdragon 835 processor and along with it a solid focus on the mobile VR market. We got a chance to test that new focus by going hands on with Qualcomm’s latest reference hardware and find impressive inside-out tracking using just the phone’s onboard sensors.

Qualcomm’s processors and chips power perhaps the majority of smartphones today, and the company often creates reference hardware designs to demonstrate and provide guidance to their OEM partners when developing these next generation devices. At CES 2017, the company showed off a mobile VR experience powered by reference hardware sporting their latest Snapdragon 835 platform.

The experience featured a tie-in with the new Power Rangers movie, due for release this year, and while we could talk about that, it’s the tracking technology that deserves the most attention. I was able to try it out, and I can say that it may, potentially, represent a breakthrough development for the VR industry.

The demo experience was prominent at Qualcomm’s space on the CES show floor, but anyone walking by may have been fooled into thinking that the experience was primarily about Power Rangers, not realizing that a hugely important development in technology was also being demonstrated. The company had several small demo areas, each complete with a VR headset and each with space enough to take a few steps around whilst inside the experience.

The headset units themselves comprised the reference Snapdragon 835 phones mounted inside third-party Cardboard-style “passive” headsets. From the front of the headset you could see the reference phone’s rear camera sensors. We have no details on these just yet, but as you can see from the images, the phone is packing more than just a single RGB sensor. We’d hazard a guess that, similar to Asus’ ZenFone AR, also announced at CES this year, Qualcomm’s reference device also sported additional depth sensors.

Putting on the HMD and headphones, there was also a focus knob that the headset, pretty standard stuff. Paying little attention the content at hand, I began looking around to test the positional tracking. The view changed as my brain expected it to. Then I began quickly moving and leaning my head around to further test the system, and it managed to keep up, although perhaps not as quickly as systems like SteamVR’s Lighthouse or Oculus’ Constellation. It felt very slightly behind in positional tracking, but for rotational tracking, it was just as good as any of the PC VR headsets.

I explored as much and as far as I could within the small demo area, and it held tracking throughout the entire volume, from standing to crouching. There was a railing that was slightly reflective, so I tried going really close to it in an attempt to freak out the tracking, and it stayed stable until I was around a foot away, where it began to hiccup. Throughout the whole demo however, there were only a few instances where the view would drift a touch or perhaps became “floaty”.

Knowing that it this was inside-out tracking powered by the phone’s front-facing camera sensor array, I tried waving my hands around the front of the headset, yet the view stayed stable for the most part, right up until I put my hand right in front of the camera to keep it from seeing anything. Unsurprisingly, my virtual world view would stay stable for a second before positional tracking was lost completely.

Despite some flaws, the tracking performance demonstrated by Qualcomm was extremely impressive, especially considering that the entire positional experience was tracked by an onboard camera array, all powered by a mobile phone. Overall, the quality of the tracking felt similar to that of Sony’s PlayStation VR (PSVR). But unlike the PSVR, and most other VR headsets with positional tracking, you don’t have to mount external cameras, or laser base stations. What’s more, have could have a potentially unlimited play area to walk around in.

The positional tracking forms part of Qualcomm’s reference design, so partner OEMs utilising the company’s SoC’s (System on Chip) will have the ability to take advantage of it. This means that we may very soon see this level of positional tracking in mobile VR headsets at last. More than that though, Qualcomm’s advances here may also have implications for other areas of virtual reality, augmented reality, and beyond.

All of that said, my demo was (as ever) best case scenario and we should bear that there may still myriad other limitations. How will the system deal with differing lighting conditions and environmental make up. We don’t have definite knowledge of how reflective surfaces and different kinds of light in the environment may affect tracking. However, talking with a representative at the booth, solid colors and reflective surfaces were in fact cited as problematic. To what degree, we’ll have to see, but the representative did say that the system detects edges and corners. Further, we don’t yet know how demanding this kind of mobile-powered computer vision is and how quickly it would drain battery life.

Still, even with some these potential limitations in mind, it’s possible that this development could be considered a real breakthrough. If it really does work this well and OEMs are quick to adopt (as well as willing and able), then we’ll get the fabled positional tracking we’ve always hoped for in mobile devices quite soon. If rumors are true, then we may even see it powering the Samsung Galaxy S8, due for release later this year.