In ‘Horizon’ Facebook Can Invisibly Observe Users in Real-time to Spot Rule Violations

Facebook Horizon is shaping up to be an interesting social VR offering with powerful building tools that will allow players to create and share their own worlds. But equally powerful are tools the company has baked into Horizon for monitoring users and enforcing Facebook’s version of appropriate behavior.

While Facebook would like users to think that hanging out in Horizon is no less private than being in a public space, there’s several huge differences.

First, all the users in Horizon are involuntarily recording each other. The last few minutes of everything that users see and hear is recorded on a rolling basis. Facebook says this recording is stored on the headset itself, unless one user reports another, at which point the recording may be sent to Facebook to check for rule violations. The company says that the recording will be deleted once the report is concluded.

Second, anyone you interact with can invite an invisible observer from Facebook to come surveil you and your conversations in real-time to make sure you don’t break any rules. The company says this can happen when one user reports another or when other “signals” are detected, such as several players blocking or muting each other in quick succession. Users will not be notified when they’re being watched.

And third, everything you say, do, and build in Horizon is subject to Facebook’s Community Standards. So while in a public space you’re free to talk about anything you want, in Horizon there a many perfectly legal topics that you can’t discuss without fear of punitive action being taken against your account.

Facebook laid out these observation and moderation tools in a “Horizon Safety Video” and explained them in further detail in an interview with Road to VR.

Transparency & Authenticity

Facebook loves to throw around the word “transparent” with regard to its stand on privacy and user tracking, and they seem to have truly taken the word to heart… after all, what’s more “transparent” than an invisible stranger that may or may not be watching you at any given moment?

Facebook also loves to use the word “authentic.” And what could make a community more authentic than ensuring that all users are constantly recording each other and are just a click away from sending behavior they don’t like to a corporation for analysis?

“Unsettling” is the word that comes to my mind when I think about these features. Yes, Horizon should be a place where people can come and have fun without fear of being trolled or exposed to vile behavior, but is the introduction of another fear—potentially being monitored at any given moment without your knowledge—really the best answer? I don’t think so.

Corporate-approved Behavior

The thing about real public spaces is that what happens in them—beyond what’s outright illegal—is up to the people inhabiting the space. In Horizon it feels much more like the space is making the rules, not the people. And in this case, the space is Facebook.

Facebook’s approach to privacy in Horizon is not much different than if the company tried to police real public spaces by using the Facebook app on everyone’s phone to constantly record their conversations just on the off chance that someone breaks the corporation’s approved social guidelines.

Before I thought about it much, when it came to privacy, I was roughly in the camp of ‘what’s to worry about if you have nothing to hide?’. But at one point someone said something utterly simple that made me understand the naivety of that position: “If you knew you were being watched every time you searched for something on Google, would it change what you searched for?”

My answer is a definitive yes, and I think any honest person would agree. Even though I’m not searching for anything illegal, simply the act of being watched would change my behavior. And this is precisely the issue with Horizon’s privacy model… even if nobody is breaking the rules, being watched—or the possibility of unknowingly being watched—changes behavior.

Don’t Blink—This is the Start of the Metaverse

And you still might say “so what, Horizon is just a game.” But the most important thing to understand about all of this is that Horizon is Facebook’s proto-metaverse. The company has been quite public about its goal of bringing one billion people into VR; the privacy norms Facebook is establishing now will deeply influence the way those billion people interact with each other in the future. Should Facebook get to write the guidelines for how they communicate and decide how private their conversations should be?