In this technical deep-dive, D ‘eVRydayVR’ Coetzee explores in detail the technical challenges preventing us from sharing our favourite VR experiences with others and the solutions available to overcome them.
Sharing your experiences of virtual worlds is a fundamental part of Internet culture. Through screenshots and videos on sites like Imgur and YouTube, captured with tools like Fraps, OBS, and Shadowplay, we let others see our creations, our reactions, our strategies, and our shared moments with other players, in precisely the same manner that we saw them on our monitors during play. Videos of content can be viewed without installing anything, even on machines that can’t run the original application.
Virtual reality complicates this: if you record a video of the content displayed on your VR headset and simply play it back on someone else’s headset, a host of problems arise. First, differences between headsets (or even user settings) can result in distortion, artifacts, and problems with convergence. Second, because the original player is in control of head movement, not the person viewing the content, they feel as though their head is being forcibly turned to look around the scene, a sensation that often induces VR sickness. The problem can be circumvented by simply watching the recorded content on your monitor instead, but that’s no fun – what you really want is for others to be able to experience those virtual worlds in VR, the same way that you did during play.
This is where 360 capture comes in: by capturing your complete surroundings, anyone with a VR headset can be completely immersed in the recorded environment, just as you were, regardless as what hardware they use, from a top-of-the-line PC HMD right down to a Google Cardboard. Only minimal processing power is required, making it great for mobile VR. They have full control over their own head movement, and retain the same powerful sense of scale and distance.
However, there are a number of technical challenges to solve before capturing and sharing high-quality 360 content with other VR users is as easy as capturing and sharing video. This article describes what solutions are available today, how they work and what their limitations are, and the problems that are still under investigation.
What we can do today
If you are a VR developer creating an application in Unity, the most popular engine for VR applications, there is a free script that I created on the Unity Asset Store called 360 Panorama Capture. By simply dropping this script onto an object in your application, you activate a hotkey that will take a 360 degree snapshot from the player’s perspective and save it to an image file on disk, like the equirectangular image shown below. This image can then be viewed using 360 degree panorama viewing tools on a variety of platforms, and can be uploaded to 360 degree panorama sharing websites like VRCHIVE – visit the image below on VRCHIVE and use your mouse or mobile device to explore it in 360 degrees.
The 360 Panorama Capture script can capture both monoscopic panoramas (where both eyes see the same image) and stereoscopic panoramas, where you get an effect like in 3D films. (If you have a WebVR-capable browser, you can view a stereo version of the above panorama.) Certain VR applications like the social VR world VR Chat have already made this feature available to all users.
It can also be used to produce 360 videos which can be uploaded to YouTube, like the one below. Most Android phones can use the YouTube for Android app to view them on Google Cardboard, and on the PC, Virtual Desktop can download and view 360 YouTube videos on the Rift.
