Exploring AR and VR’s Potential: Education and Changing Human Behavior With Virtual Experiences

A woman peers through goggles embedded in a large black helmet. Forest sounds emanate from various corners of the room: a bird chirping here, a breeze whispering there. She moves slowly around the room. On the wall, a flat digital forest is projected so observers can get a rough idea of her surroundings, but in her mind’s eye, this undergrad is no longer pacing a small, cramped room in a university lab. Thanks to that black helmet, she’s walking through the woods.

In a minute, she’s handed a joystick that looks and vibrates like a chainsaw, and she’s asked to cut down a tree. As she completes the task, she feels the same sort of resistance she might feel if she were cutting down a real tree. When she leaves this forest, and re-enters the “real” world, her paper consumption will drop by 20 percent and she will show a measurable preference for recycled paper products. Those effects will continue into the next few weeks and researchers hypothesize it will be a fairly permanent shift. By comparison, students who watch a video about deforestation or read an article on the subject will show heightened awareness of paper waste through that day — but they will return to their baseline behavior by the end of the week.

The tree-cutting study is one of many that Stanford University has conducted in its Virtual Human Interaction Lab over the last several years in an attempt to figure out the extent to which a simulated experience can affect behavior. And it’s part of a growing body of research that suggests virtual experiences may offer a powerful catalyst for otherwise apathetic groups to begin caring about issues and taking action, including on climate change. That’s important because while time spent in nature has been proven to be quite beneficial to human health, whether or not humans repay the favor tends to rely on the type of nature experiences they have in their youth. In a 2009 study published in the journal PLoS ONE, researchers from the University of Pretoria in South Africa found that while people who spent time hiking and backpacking were more willing to support conservation efforts a decade or more later, those who had visited national parks or spent time fishing as kids were actually less inclined to do anything to support the environment. An earlier (2006) study on the relationship between nature experiences and environmentalism found that while those who had spent their youth in “wild” nature, defined as hiking or playing in the woods, were more likely to be environmentalists as adults, those who had been exposed to “domesticated” nature — defined as visits to parks, picking flowers, planting seeds, or tending to gardens — were not. Given the unlikelihood of every child having a “wild” nature experience, researchers are on the hunt for other ways to cultivate environmentally responsible behavior.

The latest work with virtual reality builds upon roughly half a century of behavioral studies that indicate humans’ willingness to shift behavior is directly correlated to our sense of control.

Climate change, like many large-scale environmental issues, is a problem over which few people feel they have a direct impact — for better or worse. As researchers Sun Joo (Grace) Ahn and Jeremy Bailenson wrote in a forthcoming paper in the journal Computers and Human Behavior, individual actions taken at a micro-scale, like failing to recycle paper or support certain policies, can contribute over time to negative environmental consequences, like deforestation, which in turn affects climate trends over many years. But the long time frames and vast scale create a dangerous disconnect. While 97 percent of peer-reviewed scientific research points to human activities as a primary contributor to climate change, only half of Americans see the link.

Proponents of virtual reality think it could help drive home the impacts of climate change and make people feel empowered to do something about it. “When individuals feel that their behaviors directly influence the well-being of the environment, they are more likely to be concerned about and actively care for the environment,” Ahn and Bailenson wrote.

Bailenson, a cognitive psychologist and founding director of Stanford’s Virtual Human Interaction Lab, sees particular value in virtual reality related to climate change because it allows for a combination of real experience with boundless possibilities: The brain treats the virtual experience as real but, at the same time, knows that anything is possible in the simulation.

“One can viscerally experience disparate futures and get firsthand experience about the consequences of human behavior,” Bailenson said.

The Link Between Virtual Reality and Environmentalism on Vimeo.

Teacher Tech

Researchers working on both virtual and augmented reality — in which mobile apps on either smartphones or tablets overlay information on reality — are increasingly experimenting with these technologies as learning tools. Multiple universities, including Stanford, Harvard, and MIT, are piloting the use of these augmented and virtual reality in middle and high schools. And museums, which enjoy more flexibility, operating outside the realm of curricula requirements and test scores, have wholeheartedly embraced the idea. Science museums and zoos on both coasts are using the technology in exhibits and deploying augmented reality apps that visitors can use on their phones or on museum-issue mobile devices to learn more about what they’re seeing.

“Understanding complicated issues like climate change requires a shift in perspective in terms of how you’re willing to see the problem,” said Amy Kamarainen, PhD, co-director of Harvard’s EcoMOBILE project. “We’re trying to do that by immersing kids in environments that have elements similar to real-world systems but are somewhat simplified to meet kids where they are. We put them in complex worlds but give them the tools to be able to unpack what’s happening.”

EcoMUVE’s pond interface

EcoMUVE, a multi-user, desktop computer-based virtual environment that features a simulated pond ecosystem, was developed by Harvard University to teach students basic biological processes like photosynthesis and decomposition as well as systems thinking about complex environmental issues. The Harvard team recently launched EcoMOBILE, a corresponding augmented reality app, which enables students to take the EcoMUVE experience with them, collect data out in the field, and “see” what’s going on below the surface and what happened in an ecosystem in the past. EcoMUVE was initially piloted in schools in Massachusetts and New York, but is now available for download by any school, and is being used across the United States and in other countries as well, including India and Mexico. EcoMOBIL is currently being piloted at schools in Massachusetts and New York.

A handful of Massachusetts high schools have also piloted an MIT-developed augmented reality app called Time Lapse 2100, which requires users to set various policies that would affect the environment and then shows them what would happen if those policies were enacted. This fall, Bay Area schools will be pilot-testing Stanford’s Coral Reef, a virtual reality game in which participants become a piece of coral in a reef affected by ocean acidification. All three universities are also working with museums and science learning centers to deploy their technology in learning experiences.

“I was initially not sold on the idea of augmented reality,” said cognitive scientist Tina Grotzer, a professor in Harvard’s graduate school of education and the co-principal investigator for both the EcoMUVE and EcoMobile projects. Grotzer spent several years as a teacher herself before heading to Harvard to research how kids learn, particularly how they learn science. Grotzer said it was the technology’s potential to drive home environmental science lessons that won her over. “With physics, you can do an experiment, and kids can see instantly what you’re talking about. With environmental science, we tried to do a decomposition experiment, but you set the experiment up and then 12 weeks later something happens. By then the kids have completely lost interest.”

That’s because it’s difficult for kids to grasp anything that they cannot immediately see, Grotzer explained. Augmented reality enables teachers to extend that vision, or what scholars call an attentional frame, and make the unseen more tangible. For example, teachers take kids to a nearby pond and use EcoMOBILE to show them how the town dumped garbage there 60 years ago and nearly filled in what is today a pristine, natural pond. The app shows them how plants around the pond are turning sunlight into energy and reveals what microscopic pond life is doing under the water’s surface. It also walks them through the real-world collection of water samples, which it helps them to analyze.

Harvard’s EcoMOBILE app includes a “submarine” tool, which lets students examine microscopic pond life,

helping them to understand that ecosystems involve non-obvious causes hard to detect with the naked eye.

“I’ve tagged along on these field trips and have seen how the technology actually immerses them more in the surroundings, rather than distracting them,” Grotzer said. Students use smartphones to take photographs and notes, documenting what they’re seeing: the clarity of the pond water, the weather, descriptions of their samples, different species of bugs and birds. And they can learn at their own pace too. “On a regular field trip, if a student had a question they’d have to leave that moment that spurred the question and go ask the teacher,” Grotzer said. “The teacher would be facilitating the needs of 30 kids. This way they can find the answer themselves and stay in the moment, stay engaged with what they’re looking at.”

In Stanford’s Coral Reef  students embody a tall piece of purple coral off the coast of Italy, near Ischia. Over the course of a 14-minute lesson,  they are taken through the experience of being coral in a body of water affected by ocean acidification. At first, the surrounding ocean is filled with an abundance of sea life. Waves around the reef are simulated by floor vibrations and ocean sounds. A lab technician periodically touches the participant with a stick in synchronized motions to coincide with what he sees as a fishing net hitting the reef. Then acidification sets in. Sea life begins to die off all around. The reef begins to lose its color, as does the piece of coral the participant has embodied.

Bailenson and his team have tested the simulation with college students and shown that it resulted in students caring more about what is happening to coral reefs. The team followed those participants over weeks, compared them with a group that had simply watched a video about how ocean acidification affects coral reefs, and found the change in attitude catalyzed by the virtual reality experience lasted longer than any shifts stirred by the video.

Smartphones for All

Whether schools opt for an augmented reality tablet app that leads students around the schoolyard pointing out, say, the the biological process at work in the compost pile, or a landscape-based smartphone app (like EcoMOBILE or Time Lapse 2100) for use on a field trip, or a desktop experience (like EcoMUVE) that can be used in the school’s computer lab they face steep tab for both hardware and software. Hardware for virtual reality simulations remains cost prohibitive for most schools, although costs are coming down: virtual headsets like the Oculus Rift now cost consumers $350. A school could potentially purchase a few headsets for a multiuser virtual reality game that four students could play at a time while the rest of the class engages with an augmented reality component on desktops nearby.

Still, despite an increasing variety of options and declining prices, schools looking to put these technologies to use in the classroom face a number of challenges.

If virtual and augmented reality are to have a measurable impact on how future generations understand and approach climate change, access across all socioeconomic classes will be key. Kamarainen said that in some higher-income school districts students could use their own devices.

In many school districts around the country, however, the majority of students do not have smartphones. Mobile phone company Kajeet has begun to address this issue by offering schools data packages that provide WiFi with school-managed filtering so they can set time limits for usage, enabling kids to take home school-provided tablets for only school-related work.

In the schools where Kamarainen works, Harvard provides smartphones to students for use on field trips and pays for Kajeet’s WiFi and data service (two to three cents per megabyte per device). The Harvard apps work on both smartphones and tablets, so it’s feasible that any of the thousands of U.S. schools that have either purchased or been awarded tablets over the past two years could sign up with Kajeet to enable the use of these apps on and off campus. Industry analysts estimate that U.S. schools will purchase an additional 3.5 million tablets by the end of 2014, and multiple companies, including Intel, AT&T, Fox, and Qualcomm have launched nonprofit initiatives to dole out tablets in schools.

The Principal’s Office

Even if companies like Kajeet succeed in making hardware more affordable for schools, virtual and augmented reality developers still face a long road to see their programs widely adopted in education. Logistical challenges include securing funding for pilot tests, budgeting funds to purchase new technology, training staff, and winning buy-in from parents, teachers, and administrators.

“There are clashes all the time between the reality of what goes on in a classroom and what researchers would like to see happen in a classroom,” said Paul Olson, an outreach specialist at the Games Learning Society, or GLS, at the University of Wisconsin at Madison, who taught seventh grade for more than three decades. He said that a lot of his time these days is spent explaining to researchers what life is like “in the trenches” and encouraging teachers to experiment with GLS games to motivate those students who “really don’t respond to a lecture or a chapter in a book but are all over programming something.”

This is where museums incorporating these technologies might fill some gaps. “A museum has the freedom to step outside the rigid guidelines and requirements that schools are held to,” said Dan Wempa, vice president of external affairs for the New York Hall of Science in Queens, which sees roughly 1,200 students per day on field trips during the school year. The museum’s latest exhibit Connected Worlds, created with input from Kamarainen, will immerse visitors in a digital, interactive world that shows how their actions affect the environment. In one part of the exhibit, visitors add water to the environment and a plant flourishes. In another, they add too much and cause flooding. Taken together, the exhibit puts nature into fast forward to help students see how their individual and communal actions hurt or sustain plant and animal life, clean water, and fresh air.

“Students have a germ of knowing that water is important, but they say ‘I didn’t realize that it’s THAT important, and I didn’t realize that what I do over here affects someone way over there,’” Wempa said.

The PTA

“I’m not keen on my kids being immersed in this type of technology,” said Megy Karydes, a marketing consultant and mother of two (ages 7 and 9) in Chicago. “We very much limit our kids’ electronics exposure because I don’t want them addicted. On the other hand, I realize they need to be aware of what’s going on in the world too. I balance it, but if I had to err on the side of caution, I’d rather we go hiking than have them staring at a screen.”

Karydes’ concerns are common among parents. “There are two ways that parents tend to look at these games,” said Eric Klopfer, who directs MIT’s Scheller Teacher Education Program, developed Time Lapse 2100, and has been researching the use of augmented reality in education since 2009. “One is, ‘Great. My kid is outside, but he still has the phone in his hand,’ and the other is that the mobile device and the game are actually getting their kid outside.”

Kamarainen and Grotzer have also heard parental concerns about technology interrupting kids’ experience of nature, and they have worked hard to design games that they feel complement a relationship with nature rather than detract from it.

The EcoMOBILE pilot has included around 1,000 students so far, and Kamarainen said they consistently talk about how the augmented reality piece helps them to see things going on in their communities that they never paid attention to before. “They say this helps open their eyes about the environment that’s around them,” Kamarainen said. “They’re more aware and conscious of it, and they’re paying closer attention to the natural world.”

Ultimately, proponents say that these games not only complement and improve students’ relationship with nature but also teach them how to think systematically and to see their own roles in harming or improving their world.

“The younger kids say, ‘I get to create a world!’” Wempa said, “and the older kids say, ‘I like this because it felt like I was in control and, as a kid, I’m never in control of anything.’ That carries over. They understand that actions have consequences and that they can affect outcomes.”