While it’s exciting to start seeing 6DOF standalone VR headsets like the Vive Focus and Mirage Solo, both unfortunately include a 3DOF controller which in many cases functions as an immersion barrier compared to a fully 6DOF VR system. Today at its Vive Ecosystem Conference 2018 in China, HTC said that the 3DOF Vive Focus controller will see a 6DOF mode, and further said that the headset will soon include controller-less hand-tracking for gesture input.
3DOF to 6DOF Controller Upgrade
Following interesting research from Google on the topic of 3DOF controllers being used for 6DOF input with the aid of on-board cameras and some computer-vision processing, HTC today teased that the Vive Focus controller could be used as a 6DOF controller by leveraging the headset’s on-board cameras and the company’s “proprietary AI computer vision technology.”
The Windows VR headsets also use their on-board cameras to track their controllers, but have the advantage of the controllers being covered in LEDs to give the cameras a much easier target to track. When it comes to the Vive Focus’ simple 3DOF controller, it seems more complex computer vision processing would need to be employed considering that the device lacks and explicit tracking markers. Presumably HTC also has the option of launching a new Focus controller with markers for enhanced accuracy, though the company says that being able to roll out a 6DOF mode for the controller without additional cost to end-users is a key benefit of their approach—though it isn’t clear strategically why the company waited until after the headset’s launch to add such a game changing (literally) feature.
Similar to the Windows VR headsets, tracking of the controller can only be done when the controller is inside the field of view of the cameras, so the 6DOF tracking area will largely depend on the area which the Focus’ cameras can see. Outside of that area, the position of the controllers can be inferred from the IMU, but this becomes inaccurate quickly, making games where players hands aren’t always in front of them potentially problematic for such tracking. Key to making this kind of controller tracking feel decent will be how quickly the system can relocate an accurate controller position once it comes back into view.
HTC has also confirmed to Road to VR that the Vive Focus’ new PC VR streaming functionality will eventually also support the 6DOF controller tracking mode.
Hand-tracking for Gesture Input
In addition to the 6DOF mode for the Vive Focus controller, HTC has announced that developers will soon be able to use gesture recognition in their applications thanks to hand-tracking through the Vive Focus’ cameras. HTC tells Road to VR that this function is specifically for coarse gesture recognition, not finger-level tracking like you might expect from something like Leap Motion. The company says it will launch the gesture recognition SDK to Viveport developers in the coming weeks.
