An In-depth Look at Microsoft’s HoloLens & Mixed Reality Ecosystem

The HoloLens is the most impressive augmented reality headset on the market today, and their developer kit is already being deployed into industries ranging from architecture, engineering, design, sales, medicine, and education. Microsoft is taking a holistic approach with Windows Mixed Reality being baked into Windows 10, meaning that developers can create a single application that can run on the HoloLens, on one of their partner VR headsets, on a Surface tablet, or as a desktop app. At Microsoft’s Build conference today, the company is announcing a new OEM VR partner with HP as well as inside-out, six degree-of-freedom input controllers for VR headsets made by Microsoft partners like Acer.

LISTEN TO THE VOICES OF VR PODCAST

I had a chance to sit down for an hour with two representatives from Microsoft to talk about the latest HoloLens updates, the VR headsets coming from their partners, as well their overall Mixed Reality strategy. Brandon Bray leads the Mixed Reality developer ecosystem, and Greg Sullivan is on the marketing team for the Windows & Devices Group. We cover a lot of the high-level mixed reality strategies as well as the low-level details for developers, as well as a wide range of topics from AI integrations with Microsoft Cognitive Services to the technical details of their new motion-tracked controllers.

Microsoft has leapfrogged the augmented reality competition with the combination of having the best AR headset with the HoloLens, a healthy ecosystem of enterprise developers, a suite of AI-driven cognitive service APIs, and a forward-looking Mixed Reality strategy (which the company uses to describe both VR and AR headsets). They have impeccable timing with taking a leap of faith to solve a lot of really hard problems in order to have created the HoloLens in the first place.

Bray admits that there’s still a lot of remaining problems to be solved with the limited field of view, but that there was a tradeoff for being able to even create a battery-driven, tetherless, holographic computing platform that you can wear on your head that can do inside-out positional tracking.

The HoloLens developer kits are priced at $3,000, and so they’re targeting enterprise use-cases for now. But their VR strategy seems to be aiming for the bottom to low-end portion of the market with their $400 price point for an Acer VR headset bundled with motion-tracked controllers. I had a chance to have some hands-on time with the Acer VR headset, and I was not impressed with the motion-to-photon latency of the LCD screen, poor quality optics, build quality, or user experience of putting the headset on.

These tradeoffs in comfort were made in order to bring the price down, but the overall experience feels like it’s a small step up from a Daydream, but perhaps on par with the GearVR or possibly even worse. The high resolution of the Acer VR headset makes it one of the best VR HMDs to read text in and the inside-out tracking works pretty well with occasional judder. But the LCD screen is not a blur-free low persistence screen that seasoned VR veterans have grown used to, and so the resulting Rift DK1-era blurring when turning your head makes it feel worse than a Gear VR. But as long as you’re not quickly moving your head around, then you’ll minimize the motion sickness triggers.

The 6 DOF motion controllers are tracked inside-out and Bray said that they rely upon a sensor fusion combination of having a direct line of sight with the front-facing cameras on the VR headset, IMU sensors, and inverse kinematic probabilities. There were not any prototypes available for testing, and so I don’t have any direct experience with how they actually work. But I do have some concerns with their approach based upon my experiences with other line-of-sight controllers such as the Leap Motion. With the Leap Motion, you have to hold your hands up so that they can be seen by the cameras on the HMD, which will likely require developers to specifically design applications that optimize for this constraint.

This limitation of the input controllers may mean that it could limit existing room-scale Vive and Rift VR experiences from being easily ported. If existing Vive or Rift applications aren’t a good experience on these lower-end VR HMDs, then there’s going to be a huge gap of content to drive consumer adoption. If this lower price point is going to attract more consumer-grade users, then they’re going to need content. If custom entertainment content is need, then I doubt that the Microsoft enterprise developers are going to generate a lot of compelling and entertaining content.

But it could be that Microsoft isn’t concerned about having a library of entertainment for regular consumers of these VR headsets, and maybe they’re more interested in creating data visualization and enterprise applications. But if that was the case, then why not create something on par with the Vive and charge enterprise prices? Most of the mobile VR content designed for a 3 DOF controller hasn’t been nearly as compelling as the full room-scale and 6 DOF experience provided by the Rift and Vive. These Microsoft VR headsets look to be in yet another category of quality & performance that’s slightly better than mobile, but a lot worse than the best high-end systems.

If Windows Mixed Reality VR headsets are going to go anywhere, then there’s going to need to be content that’s compelling and drives adoption. Will these VR systems meet the needs of whatever Microsoft has decided is their target market? If it does, then all of this discussion is moot. But if not, then we’ll have another platform that could creature the developer ecosystem and is left without a critical mass of compelling content.

Overall, I’m really impressed with Microsoft’s holistic approach to mixed reality. The HoloLens is the market leader for head-mounted AR that’s actually being deployed into enterprise. They are positioned to really own the enterprise and professional AR market as they create more integrations between Windows Mixed Reality, their cloud hosting, and AI-driven cognitive services.

There’s a lot of long-term promise in tetherless VR with inside-out tracking, but the early Acer VR prototypes are disappointing and risk fracturing the VR ecosystem in potentially needing specially designed experiences in order to really use the strengths of the platform.

Here’s a number of Twitter threads with more thoughts and impressions from Microsoft Build so far:

Live tweets of first day keynote of Microsoft Build Conference

Thread with highlights from the HoloLens YouTube channel

Twitter Thread of Hands-On Impressions from Acer headset

Support Voices of VR

• Subscribe on iTunes
• Donate to the Voices of VR Podcast Patreon

Music: Fatality & Summer Trip