Nexus Interactive Arts, an immersive media division of VFX production studio Nexus Studios, have used Apple’s ARKit working on an iPhone 7 in an experiment that creates basic inside-out positional tracking and pass-through AR for a Google Cardboard headset.
Announced last month at Apple’s annual Worldwide Developer Conference (WWDC), ARKit is an iOS 11 tool allowing developers to create AR applications thanks to the device’s computer vision capabilities. With ARKit, iOS 11 devices are able to map surfaces in real time, and allow users to superimpose digital objects onto the physical world—replete with interactive animations and dynamic lighting.
Using ARKit, the team reports their inside-out positional tracking solution clocks in “at around 60 frames per second,” or right around mobile VR’s current target framerate. This, according to the team, means Apple has created the foundations for a cheap, but still ultimately reliable positional tracking solution for mobile VR headsets.
In the video, they demonstrate inside-out positional tracking for VR and pass-through AR by touring a conceptual ‘art museum’ in a park. When in VR, walking close to a boundary like a tree results in a point cloud materializing into the otherwise closed-off experience—essentially acting as a guardian system to keep you from bumping into things as explore the infinite (or sufficiently large) tracking volume afforded by the device’s machine vision. In the AR demonstration, the digital skybox is lifted to reveal digital scenery affixed to the park’s trees and landscape.
The AR headset capabilities presented in the video, while an impressive use of ARKit, are less useful in this case because of the lack of stereoscopic vision afforded by the iPhone 7’s monoscopic rear-mounted camera. The developers aren’t couching this as a verified AR headset solution however, but rather showing the versatility of ARKit itself.
Allowing developers free reign to create applications for AR—and thanks to this experiment, now free-roaming VR experiences currently puts Apple back into competition despite its lack of discrete AR/VR headsets.