VR Time Perception Insights from Filmmaking & Cognitive Science

Voices of VR Podcast – Episode #379

Jun 11, 2016

Xárene Eskandar is an artist who has been exploring representations of time through the mediums of film and photography for many years, and she’s has been starting to design VR experiences that alter the user’s perception of time based upon insights from cognitive science research. She’s also a Ph.D. candidate at UC Santa Barbara Department of Media, Arts, & Technology, and she was presenting a paper titled “Aesthetics of temporal perception and scale in virtual reality” at the Experiential Technology & Neurogaming Conference in San Francisco in May. I had a chance to catch up with Xárene to talk about filmmakers Béla Tarr & Andrei Tarkovsky’s experiments in time perception, as well as the cognitive science research into why awe, self-regulation, and The Extended Now impacts time perception.

LISTEN TO THE VOICES OF VR PODCAST

Here’s Xárene’s poster titled “Aesthetics of temporal perception and scale in virtual reality”:

Filmmaker Andrei Tarkovsky wrote a book titled “Sculpting in Time” where he argues that people go to see films in order to experience time, and that the role of a director is to sculpt the time that the audience experiences:

What is the essence of the director’s work? We could define it as sculpting in time. Just as a sculptor takes a lump of marble, and, inwardly conscious of the features of his finished piece, removes everything that is not part of it—so the film-maker, from a ‘lump of time’ made up of an enormous, solid cluster of living facts, cuts off and discards whatever he does not need, leaving only what is to be an element of the finished film, what will prove to be integral to the cinematic image.

Here’s the opening scene of Andrei Tarkovsky’s The Sacrifice, which feels about 2-3 times longer than the actual run time.

And here are two of the potato peeling scenes in Béla Tarr’s The Turin Horse that also give an expanded sense of time:

For more Voices of VR interviews about time perception and time dilation in VR, then be sure to check out my interviews with Gerd Bruder, Owlchemy Labs, Karl Krantz, and Sarah Northway.

Below are some links and excerpts from some of the cognitive science research that Xárene cites.

Rudd, Vohs, and Aaker connect awe and time perception in their paper titled “Awe Expands People’s Perception of Time, Alters Decision Making, and Enhances Well-Being”

Two psychological theories are also relevant. The first involves the extended-now theory (Vohs & Schmeichel, 2003), which suggests focusing on the present moment elongates time perception. Awe captivates people’s attention on what is unfolding before them, which the extended-now theory predicts would expand the perception of time. The second is Socioemotional Selectivity Theory (SST), which posits that people are motivated to acquire new knowledge when time feels expansive (Carstensen, Isaacowitz, & Charles, 1999). Awe triggers in people a desire to make new knowledge structures (Keltner & Haidt, 2003). Thus, a speculative suggestion from SST is that awe’s ability to stimulate the creation of mental schemas may be a signal that the mind perceives an expanded amount of time in response to awe.

Rudd, Vohs, and Aaker also provide a specific definition of awe with references to previous research on the topic:

One, awe involves perceptual vastness, which is the sense one has come upon something immense in size, number, scope, complexity, ability, or social bearing (e.g., fame, authority). Two, awe stimulates a need for accommodation, meaning it alters one’s understanding of the world (Keltner & Haidt, 2003)… Experiences involving awe, such as optimal athletic performances (Ravizza, 1977), peak experiences (Maslow, 1964), and spiritual or mystical events (Fredrickson & Anderson, 1999), often also involve a sense of timelessness (Csikszentmihalyi & Hunter, 2003). The phenomenology of awe, therefore, suggests it might expand perceptions of time.

Vohs & Schmeichel make the argument that “regulating the self can elongate the felt duration of time” in their paper titled “Self-Regulation and the Extended Now: Controlling the Self Alters the Subjective Experience of Time.” They define self-regulation as “operations by the self to alter its own habitual or unwanted responses to achieve a conscious or nonconscious goal.”

Because people who are self-regulating tend to monitor their behavior (Baumeister, Heatherton, & Tice, 1994), they are likely to be attuned to the passage of time (“How long has it been?”). These monitoring responses and resultant attention to time are not found among people who are not regulating. Stated simply, it is likely that the act of self-regulation is associated with close attention to time.

Does our conscious perception have framerate? It turns out that according to the model of Herzog, Kammer, & Scharnowski that it does. They say that our perceptions show signs that they’re “quasi-continuously and unconsciously analyzed with high temporal resolution” and that “consciousness arises only in time intervals of up to 400 milliseconds, with gaps of unconsciousness in between.” They elaborate on their model within their paper titled “Time Slices: What Is the Duration of a Percept?.”

Even though we may only be synthesizing all of the unconscious streams of perceptual input every 50ms to 400ms, our ability to detect discrete changes across different senses differs across the different senses:

Whereas two clicks can already be separated if they are only 1–3 ms apart [33], two taps need to be about 10 ms apart [33], and two flashes about 25 ms [34]. However, for trains of stimuli, the presentation rate at which the sensation of flicker ceases is similar for the visual and auditory systems, i.e., at around 16 ms inter-stimulus-interval (ISI) [35,36].

Finally, The Atlantic did a profile on Donald D. Hoffman, a professor of cognitive science at the University of California, titled The Case Against Reality, A professor of cognitive science argues that the world is nothing like the one we experience through our senses. Our perception of time and reality has been shaped by many thousands of years of evolution of our senses by interacting with the real world.

After reading this profile about Donald D. Hoffman, then it’s really started to make me question the underlying fabric of our reality, and how much we’ve trained ourselves to completely ignore. Does VR have the capacity to provide virtual experiences that requires our perceptual systems to evolve before its ready to receive them?

Virtual reality is uniquely poised to serve as a medium to continue to explore the nature of time perception and our perception of reality. There’s still a lot to learn, and this cognitive science research into awe and self-regulation is providing a lot of different ways to start putting these theories into practice.

UPDATE: Xárene wanted to make the following clarification:
I meant to say that there is not any research that relates body scale to time perception rather than time perception research in VR doesn’t exist. Specifically, there’s 2D research where the movement of a dot is used for the perception of duration, or the movement of the sun, and we accept those objects scaling on the screen but not relative to our body scale. What if instead we view those objects the same size relative to each other with the body scaling to bring them to their correctly understood scale? Then what is the relationship of the scaling body with time and how is duration perceived?

In psychology research where the scale of the body is studied, it is studied as a mental disorder, such as Alice in Wonderland syndrome or body dysmorphic disorder. So any view of the body other than what is considered ‘normal’ is a disorder. I’m looking for research that acknowledges that our body exists in different scales just as time and space have different scales. This discussion also brings up another topic on how the trans body is seen.