Google Announces Android XR Operating System Alongside Samsung MR Headset

By

-

Dec 12, 2024

Google today announced Android XR, a new core branch of Android, designed as a spatial operating system for XR headsets and glasses. The company is pitching this as a comprehensive spatial computing platform, and hopes to establish its own territory in the XR landscape against incumbents Meta and Apple.

Google has revealed Android XR and it’s basically what the name implies: a full-blown version of Android that’s been adapted to run on XR headsets, supports the entire existing library of flat Android apps, and opens the door to “spatialized” versions of those apps, as well as completely immersive VR content.

Samsung’s newly announced headset, codenamed Project Moohan, will be the first MR headset to launch with Android XR next year. Check out our early hands-on with the headset.

Samsung Project Moohan | Image courtesy Google

Google tells us that Android apps currently on the Play Store will be available on immersive Android XR headsets by default, with developers able to opt-out if they choose. That means a huge library of existing flat apps will be available on the device on day one—great for giving the headset a baseline level of productivity.

That includes all of Google’s major first-party apps like Chrome, Gmail, Calendar, Drive, and more. Some of Google’s apps have been updated to uniquely take advantage of Android XR (or, as Google says, they have been “spatialized”).

Google TV, for instance, can be watched on a large, curved screen, with info panels popping out of the main window for better use of real-estate.

Google Photos has been redesigned with a layout that’s unique to Android XR, and the app can automatically convert photos and videos to 3D (with pretty impressive results).

Hands-on: New VR Shooter 'VENDETTA FOREVER' Mashes 'Pistol Whip' & 'Superhot' Into Something Unique

YouTube not only supports a large curved screen for viewing but also supports the platform’s existing library of 360, 180, and 3D content.

Chrome supports multiple browser windows for multi-tasking while web-browsing, which pairs nicely with Android XR’s built-in support for bluetooth mice and keyboards.

And Google Maps has a fully immersive view that’s very similar to Google Earth VR, including the ability to view Street View photography and newly added volumetric captures of business interiors and other places (based on gaussian splats).

Functionally, this is all pretty similar to what Apple is doing with VisionOS, but Android flavored.

Where Android XR significantly differentiates itself is through its AI integration. Gemini is built right into Android XR. But this goes far beyond a chat agent. Gemini on Android XR is a conversational agent which allows you to have free-form voice conversations about what you see in both the real world and the virtual world. That means you can ask it for help in an app that’s floating in front of you, or ask it something about things you see around you via passthrough.

Apple has Siri on VisionOS, but it can’t see anything in or out of the headset. Meta has an experimental AI on Horizon OS that can see things in the real world around you, but it can’t see things in the virtual world. Gemini’s ability to consider both real and virtual content makes it feel more seamlessly integrated into the system and more useful.

Android XR is designed to power not only immersive MR headsets, but smartglasses too. In the near-term, Google envisions Android XR smartglasses as HUD-like companions to your smartphone, rather than full AR.

'Skydance's Behemoth' Won’t Miss Dec 5 Launch Despite Technical Setbacks, Week-1 Patches Announced

Prototype Android XR smartglasses | Image courtesy Google

And it’s Gemini that forms the core of Google’s plans for Android XR on smartglasses. The near-term devices for this use-case are compact glasses that can actually pass for regular-looking glasses, and offer small displays floating in your field-of-view for HUD-like informational purposes, as well as audio feedback for conversations with Gemini. Uses like showing texts, directions, or translations are being shown. Similar to Android XR on an MR headset, these smartglasses are almost certain to be equipped with cameras, giving Gemini the ability to see and respond to things you see.

It’s a lot like what Google Glass was doing a decade ago, but sleeker and much smarter.

While no specific smartglasses products have been announced for Android XR yet, Google and Samsung have been collaborating on an MR headset called “Project Moohan,” which Samsung will launch to consumers next year.

When it comes to development, Google is supporting a wide gamut of dev pathways. For devs building with Android Studio, a new Jetpack XR SDK extends that workflow to help developers create spatial versions of their existing flat apps. This includes a new Android XR Emulator for testing Android XR apps without a headset. Unity is also supported through a new Android XR Extension, as well as WebXR and OpenXR.

Google also says it’s bringing new capabilities to OpenXR through vendor extensions, including the following:

AI-powered hand mesh, designed to adapt to the shape and size of hands to better represent the diversity of your users
Detailed depth textures that allow real world objects to occlude virtual content
Sophisticated light estimation, for lighting your digital content to match real-world lighting conditions
New trackables that let you bring real world objects like laptops, phones, keyboards, and mice into a virtual environment

New 'Batman: Arkham Shadow' Trailer Reveals Gameplay, Quest 3 Graphics, & Fall Release Date

On the design side, Google has updated its ‘Material Design’ to include new components and layouts that automatically adapt for spatial apps.

Developers interested in building for Android XR can reach out via this form to express interest in an Android XR Developer Bootcamp coming in 2025.

xyzs

I am not a fan of Zuck but at least he showed commitment toward XR.
Google didn’t.
And they are known for killing their product overnight even if they are popular but not popular enough for them…
Pass.
Michael Speth

Google wants to join the battle to control what you see and hear in the future. This is the wrong direction for VR – we don't want global companies filtering our real world through their tech especially AI.
g-man

Like I’m going to use something made by the world’s largest ad network lol
Christian Schildwaechter

TL;DR: AndroidXR opens a whole new battleground for the future of XR due to Google binding it to their Play services, which a lot of large companies like Meta or Huawei cannot accept. This is a dangerous gamble, as the XR relevant parts (minus AI etc.) should be rather easy to implement in a framework not controlled by Google. Which could become the standard used instead of AndroidXR, throwing a big and potential fatal wrench into Google's attempt to forcefully extend their Play store money printing machine into XR, and could end up "accidentally" giving Meta the access to 2D Android apps it desperately seeks to compete with Apple and Google.

One of the IMHO most critical aspects of AndroidXR is missing from the article: compared to regular Android, it is bound to Google Play services, which could have dire consequences for its spread.

To catch up with Apple's iPhone, Google offered Android not only for free, but as the ASOP (Android Open Source Project) with no strings attached. That's the version HorizonOS, PicoOS and billions of cheap TV set-top boxes, phones, tablets or smart fridges and toasters are based on, leading to about 80% market share on smartphones and even more for other devices.

The Android version most of us are familiar with is a different one including lots of extra functionality bound to Google Play services, incl. access to the Play store and all its apps. To be allowed to install for example Google Maps, a phone maker has to agree to also let the Play store funneling all software revenue to Google. The agreements include clauses forbidding installing alternative stores, which courts and regulators now started to declare invalid. And obviously companies like Meta or Huawei went with ASOP instead and reimplemented missing functions themselves to keep the money or work in places without Google Play services.

With AndroidXR (and GoogleTV and others before), Google tries to prevent anyone escaping their licenses as they still could with Android thanks to ASOP. Companies like Samsung already using Android with Google Play services will again go for the restricted AndroidXR, but significant parts of the world simply cannot even access Play services, and many companies won't like being forced into them.

Google is betting on their existing user base demanding access to the Play store and thereby forcing XR HMD manufacturers to fold into the Google Android(XR) ecosystem, but that is far from a safe bet. It's not that difficult for a large company to implement their own OpenXR stack on top of ASOP. Meta, Pico, Qualcomm and others already did that, which makes porting XR apps rather easy. Similarly Amazon, Huawei, Epic and many others offer stores for apps not relying on Play services. And replicating the new APIs Google now introduced for integrating Android apps into XR shouldn't be that hard either, so existing non-Google app stores could achieve a similar integration.

Whether this succeeds will depend on Google's competitors coming up with a unified approach that provides sort of an open source version of the XR related part of AndroidXR, not depending things like the Gemini AI that Google certainly also added to introduce more dependencies. Then the whole thing could horribly backfire for Google, as a freely available alternative to AndroidXR with compatibility for the main features would be a lot more attractive to many XR companies. And could even allow Meta to escape the dilemma of not having Play store access by integrating this compatible API working with numerous existing app stores into HorizonOS.

visionOS and AndroidXR are basically Meta's worst nightmare: the existing giant mobiles leveraging their billions of users tied up in their ecosystems to win a similar dominance in XR, before Meta has grown their platform enough for developers to release apps there too. But this provides Meta with a huge motivation to make sure there is an open alternative to AndroidXR, even if they cannot tightly control it as part of HorizonOS.

So Google trying to force future XR HMDs under their control by legally binding AndroidXR to Play services might push companies like Meta with billions of Facebook money and Pico/ByteDance with billions of TikTok money to cooperate and make sure that just like with ASOP, the future of Android based XR is a free no-strings-attached open source implementation. Which I'm pretty sure would be the better option for users.

It also wouldn't be the first time Google's XR attempts failed due to them restricting the platform. Daydream at least partly failed because Google, wanting to up the quality compared to Cardboard to compete with Meta, required very high system requirements, the use of Google's Daydream viewer and each phone to be certified for Daydream, leading to very few phones being compatible and following lack of developer and user interest. Other factors like Meta selling hardware at cost certainly added to Daydream's demise, but Google switching from an open to a closed platform, even for valid reason, definitely backfired with Daydream.
Mandub

Meta is doomed.