LISTEN TO THE VOICES OF VR PODCAST
There has been a debate within the virtual reality community around the validity of 360-degree videos, and whether or not they should even be considered a legitimate part of the virtual reality landscape. This sentiment has been vocalized by Valve hardware engineer Alan Yates, who declared that spherical videos are not VR & that they “basically suck.”
I understand the technical complaint about the limited stereoscopic effects of 360-degree video, as well as my personal experience of them not being as immersive or interactive as a completely computer-generated VR environment. Some of my deepest experiences of presence have come from CGI experiences with stylized art and highly dynamic and interactive environments. But just because I might prefer computer-generated VR experiences over a lot of the more passive 360-video that I’ve seen, then should that mean that a whole range of immersive video experiences shouldn’t be considered as valid VR? How should we think about all of these passive narrative video experiences that are being created? What are the critical components of what is a VR experience and what isn’t? How can we make sense of this emerging landscape taking that makes sense of all of the different levels of interactivity and storytelling potential within VR?
These were some of the questions that Devon Dolan sought out to answer with his thought-piece on “Redefining the Axiom of Story” co-written with FilmNation Entertainment’s Michael Parets. They propose a four-quadrant system that categorizes stories told within VR on two different axes.
One one axis they are determining your level of existence within the story, and so in other words whether you’re a character within the plot who is integrated within the story or whether you’re just an omniscient ghost who is observing what’s happening around you. The other axis is the level of influence that you have on the story and whether you have an active impact on it or whether you’re passive and have no real impact on the outcome of the story.
Here’s an image of the original grid that they used in their essay:
I personally find their original language to be abstract and confusing, and so Devon and I came up with a new iteration of this grid that’s a little bit more concrete. Instead of “Observant” or “Participant” Existence, we changed it whether you are a ghost or a character within the story. And instead of having “Active” or “Passive” Influence, we changed it to whether or not your character has any impact within the story or within how you experience the story.
Here’s our updated grid detailing the four different types of story in VR:
We go into more detail about each quadrant within the interview, and unpack some different examples and insights in trying to figure out the boundaries and differentiations within each category.
We start with the least amount of agency, which is the “Observant Passive” / “Ghost without Impact” quadrant. This is where most of our existing media and many of the current 360-degree videos exist. Then we move towards the quadrant with the most agency, the “Participant Active” / “Character with Impact” quadrant, which is where pure interactive fiction experiences like Facade or perhaps open world games like Grand Theft Auto might fit.
The level of impact can be confusing at first. For example, what does it really mean to be a ghost but to still have impact on the story? One way that I found it helpful to differentiate between these quadrants is to determine whether you have either local or global agency within the experience. Local agency is where you could control the outcomes of your own experience in small ways, but these small actions many have no real impact on the overall outcome of the story. In order to really change the course of the story, then you’d need to also have global agency. Most interactive VR experiences will probably have a dimension of local agency, but that doesn’t mean that you’re actions will necessarily have any consequence to the overall story that’s unfolding.
In order to help explain the different between local and global agency, then I’d highly recommend watching this excellent talk by Nicky Case from the 2015 XOXO Festival where he talks about the different small decisions that end up “flavoring” his life experience versus what end up being huge decisions that completely change the course of his life sending him a completely new branch. The amazing thing is that we often have no idea whether or not we’re making a big or small decision in any given moment, and I think Nicky’s story and how he structured the Coming Out Simulator 2014 is a great example of this concept:
Let’s apply this concept of local and global agency to this grid of four different types of VR stories. In an “Observant Passive” / “Ghost without Impact” experience, then you would not have either local or global agency. You may be able to look anywhere around within a 360-degree video, but where you look has no impact as to how the story ultimately unfolds. It’s completely on rails, and none of your actions can really change anything about your experience.
Adding gaze-based triggers within an experience could turn it into an “Observant Active” / “Ghost with Impact” experience. Or perhaps you might have limited interactivity with being able to explore an environment, but your interactivity has no real consequence to the story that’s unfolding. Exploring an environment would impact how you personally experience the story within the constraints of your local agency, but if this doesn’t change the story at all then you have no real global agency as to how the overall story unfolds.
The immersive theater piece Sleep No More is a really good example of a “Observant Active” / “Ghost with Impact” story since you are able to explore 100 different rooms and choose which of the 21 parallel stories to follow, but your choices aren’t really impacting the overall arc of what the actors are doing. While some characters may have some limited one-on-one interactions with participants, for the most part the audience is ignored as invisible ghosts within the realm of telling the story of MacBeth through interpretive dance.
On the extreme end of interactivity is a “Participant Active” / “Character with Impact” experience where each of your small actions would both cause something within story to respond to you, but also your participation would be crucial to how the overall story ultimately unfolds. The holy grail of VR experiences might eventually be where you’re interacting with convincing artificially intelligent characters, and the nature of these small interactions would be contributing in some way to a range of vastly different outcomes.
I think that the interactive fiction game of Facade is probably the best example of an experience that has a true combination of both local and global agency. Tomorrow I’ll be featuring an interview with Facade co-creator Andrew Stern and one of his current collaborators Larry LeBron. Here’s a trailer for Facade that gives you a taste of a “Participant Active” / “Character with Impact” experience:
The “Participant Passive” / “Character without Impact” quadrant is probably the most confusing one. You’re not really given any choices to change the outcome of the story, but yet you’re addressed as if you were a character within the story. There were a number of examples of this quadrant that I’ve saw at Sundance, and most of them were shot within the first-person perspective. These are experiences where you have actors who are directly addressing your character and provide a voyeuristic experience that can transport you into the shoes of another person, but yet you’re essentially a mute with no way to respond or react within the story.
Three examples of “Participant Passive” / “Character without Impact” experiences at Sundance New Frontier are Defrost, Perspective Chapter 2: The Misdemeanor, and Across the Line. Defrost is a 360-degree video that was shot in the first-person perspective of a woman who’s been cryogenically unfrozen and you have a discussion with your family who is suddenly many years older than you’ve last seen them. Perspective Chapter 2: The Misdemeanor cuts between four different first-person perspectives in four segments where you witness a police shooting from the perspective of two cops and two black adolescents. Across the Line has a portion of the experience where you’re walking by a line of pro-life protesters on the way to walking into getting an abortion at a Planned Parenthood clinic.
I do notice a big difference between a 360-degree video where I’m a character in the story and one where I’m just a ghost and observing a scene. There was a moment in Defrost where I kind of felt like this what it must feel like to come out of being cryogenically frozen after a few decades, and it was definitely a surreal moment. But the pieces where I felt the most empathy were ones where I’m merely witnessing a scene unfold as if I were a ghost because there’s a different quality of experience when I’m being directly addressed as a character in the story. This seems to reinforce Eric Darnell’s thoughts that there is indeed a tradeoff between interactivity and empathy. Not having my ego involved in the story does indeed allow me to receive the story of other characters without worrying how I should respond. There’s nothing to do but just receive the story in an “Observant Passive” / “Ghost without Impact” experience.
As with any model that’s trying to describe reality, there are going to be imperfect mappings of the complexities of the full landscape of VR. I found myself having to add additional qualifiers about local and global agency to really flesh out what having “impact” really means.
I also found that the line between whether or not you have “impact” or “influence” on how the story unfolds can actually be blurred even within the constraints of a 360-degree video. One could argue that what you choose to look at within a spherical video can impact your experience, but the video is still on rails. Rose Troche’s thoughts about the vulnerability of the first-person perspective illustrate to my how my own biases and pre-existing narratives can shape how I experience Perspective, which is essentially an simulator for how flawed eye witness testimony can be.
But it’s possible to use positional audio to vastly change your experience in an otherwise completely on-the-rails, “Observant Passive” / “Ghost without Impact” 360 video. For example, the live action VR experience Hard World for Small Things has a portion of the video that has multiple conversations happening that are about 90 degrees apart. Depending on whether you’re looking at the conversation to your right or in front of you will determine which conversation you’ll be able to hear. So even though the video will be the same, you could get an entirely different experience of it based upon what audio you hear. You could feasibly choreograph a 360-video with four or more different audio tracks that could be triggered when looking in one of the four cardinal directions. Using audio alone could essentially transform an experience from a being an “Observant Passive” / “Ghost without Impact” into a “Observant Active” / “Ghost with Impact” experience.
Some experiences will cross over in being categorized into different quadrants, and I don’t think that need a model that’s able to fit existing experiences neatly into a box. It’s more about creating a tool that can help storytellers figure out the strengths and weaknesses of telling a story within the medium of VR.
Devon thinks that ultimately content will prove king in VR, just as it has in every other medium. So why not look at and understand what type of unique opportunities that VR provides for storytellers to be able to tell new and different stories? I can see storytellers would find it useful to use this framework to plan out the best way to tell their story within VR.
This “Dolan Parets Framework for VR storytelling” has also helped me understand and contextualize where 360-video fits within the overall landscape of storytelling within VR. Before Sundance, I was actually fairly skeptical about the validity of passive 360-degree videos within the VR landscape. But after watching all 37 VR experiences within the Sundance New Frontier program, I discovered that 360-videos do offer a very unique storytelling medium that does use enough of the essential components of what we define as being “virtual reality.”
It’s true that some passive 360-videos may appear to just be replicating the film medium without really adding anything new to the language of VR. These may end up feeling flat or boring to people. But ultimately, it will be a process of figuring out the strengths and weaknesses of the language of storytelling within the VR medium and as a result some boring videos and stories will indeed be created. But take a look at Sonar or Perspective to see where narrative in VR is going.
Baobab Studios’ Eric Darnell made the distinction to me that that film is a medium where someone can share the story of an experience, whereas virtual reality is a medium where you can give someone an experience that they can generate their own stories from. This is a powerful insight, and it’s something that’s gets a lot of independent film storytellers really excited about the potential of VR.
The Dolan Parets Framework for VR storytelling should caution us that these “Observant Passive” or “Ghost without Impact” type of experiences still have a valid place within the ecosystem of VR storytelling. And we may discover that it’s this currently discounted quadrant that provide some of the most powerful explorations of empathy within VR, and unlock new genres of immersive stories & 360-degree videos that help kickstart this VR revolution.
Become a Patron! Support The Voices of VR Podcast Patreon
Theme music: “Fatality” by Tigoolio