HypeVR is developing volumetric video tech which we called “a glimpse at the future of VR video,” when we got to see it back in January. Now the company has adapted their system for Apple’s ARKit, showing how phone-based augmented reality can be used to view volumetric footage, and allow users to step through a portal into the footage.
Volumetric video is that which contains spatial data and allows users to physically move around inside of it, similar to real-time rendered VR games. Volumetric video is far more immersive than the 360 video footage you can readily find today, because the perspective isn’t locked into one static location.
I was impressed with the level of immersion and quality when I got to see some of HypeVR’s volumetric video footage running through a VR headset, but with new AR tracking tools coming from Apple (ARKit), Google (ARCore), and others, there’s an opportunity to bring a huge audience part way into these experiences.
HypeVR is experimenting on that front, and recently integrated their tech with ARKit’s tracking, giving any capable iPhone the ability to view the company’s volumetric video footage on the phone’s display.
The video above shows a volumetric video scene from HypeVR being played back on an iPhone SE (which is of the ‘6S’ generation) The company is employing a ‘portal’ as a way of navigating from the real-world into the volumetric space; it’s a neat trick and definitely a cool demo, if quite novel, though a similar mechanism might be an interesting way to transition between volumetric video scenes in the future, or as a central hub for ‘browsing’ from one volumetric video to the next.
The AR tracking seen here is indeed cool to see in action, but what’s happening under the hood is equally interesting. Since rendering volumetric video can be challenging for a mobile device (not to mention, take up a ton of storage), CEO Tonaci Tran says the company has devised a streaming scheme; the phone is actually relaying its movements to a cloud server which then renders the appropriate video frame and sends it back to the phone, all fast enough for a hand-held AR experience.
That means the output is being drawn from the same source data set that would playback on a high-end VR headset and require a beefy GPU. This not only lowers the computational bar enough that even a last-gen iPhone can play back the volumetric video, but it also means users don’t need to download a massive file.
Tran tells me that the company also plans to support volumetric video playback via Google’s ARCore. Between ARKit and ARCore, there’s expected to soon be hundreds of millions of devices out there that are capable of this sort of tracking, and HypeVR intends to launch an ARKit app in early 2018.
Tran says that the ultimate vision for HypeVR revolves around distribution and monetization of volumetric video. The company is in the process of raising a Series A investment and encourages inquiries to be sent through their website.