Keeping an eye on your phone when you’re in a VR headset is an absolute chore: place it in on vibrate in your pocket, glance at the screen through the gap between your nose and facial interface when it buzzes, and prop the headset on your forehead to respond to calls and text messages. To address this excruciating first-world malady, Dell is including a new VR function in its Mobile Connect software that lets you bring your entire Android phone—and some of your iPhone—into any VR app running on SteamVR.
Mobile Connect has been available exclusively to Dell and Alienware users for about a year now, downloadable on branded systems purchased after January 2018. Before now, Mobile Connect was intended for desktop and laptop users looking for a way to keep an eye on (and interact with) every notification from their Android or iOS smartphone, but it’s soon to include the most unique and clever use-case to date: smartphone integration with VR.
I got a chance to try out the software at CES 2019, getting a full demo of the VR side of Mobile Connect using a HTC Vive Pro and a pair of Android smartphones—one for my far-flung contact, and the other for me, which normally sits tight in my pocket so I can keep my notification anxiety in check.
In the demo, I learned how to wield the mighty virtual smartphone, which mirrors your Android phone’s screen via WiFi and transmogrifies it into a virtual doppelganger, letting you use the touchpad or analog stick on your VR controller to thumb through it like you might if it were in your hand. You can alternatively use the other controller as a laser pointer for finer interactions.
Propping the physical Samsung Galaxy S8 on the keyboard facing me, I was able to unlock the screen, open any app on the phone, respond to calls and talk through the headset’s integrated microphone and headphones. That last bit alone was the most impressive, as I’ve always hated simultaneously juggling my phone and motion controllers just to answer a quick call.
All of this—without understatement—worked flawlessly in the demo. Not only could I do something as silly as open the camera app and take a selfie while in VR, I could more practically follow a YouTube link from a Facebook message and view it right then and there. While blowing the screen up to an impossibly large size and playing the mobile version of Fruit Ninja sitting in a photogrammetry scene of the Icelandic countryside, it struck me just how much I’d missed the sheer capability of my phone in VR.
Ok, maybe I won’t be playing a mobile version of Fruit Ninja when I could be playing the real VR version, Fruit Ninja VR (2016), but the fact that latency didn’t seem to break the illusion that I had my phone transported to Valve’s ever-impressive The Lab, well, that’s something worth mentioning.
Since Dell Mobile Connect VR is a disparate layer projected on top of the SteamVR app, your virtual phone can’t interact with other virtual objects, although you can anchor it to a position within the bounds of your guardian system and leave it there. Effectively, this lets you rifle through your email while flying the long and boring trade routes of Elite Dangerous, or furtively glancing at a floating NHL hockey game in the middle of an Onward firefight. Also, because it’s a local layer you won’t be able to show other VR users what you’re looking at.
Moving on to notifications: If I didn’t want to get handsy with the virtual phone, I could also banish it with the click of the Vive controllers menu button, allowing me to solely rely on the incoming notifications which are displayed at the lower register of the 3D environment. I found this much more useful than operating the virtual phone itself, giving me the ability to quickly send out the preset message “I’m in VR. Talk to you later.” or a custom preset like “I’m in VR. Your problems are more boring and less important to me than playing that K-POP level on Beat Saber.” You can of course type that out using SteamVR’s virtual keyboard on the fly, but the message may be useful more than once during your gaming session. This goes for regular SMS text messages and other messaging apps like WhatsApp.
Something that’s coming at launch, presumably sometime in the near future (still no word on exactly when), is voice commands. I didn’t get a chance to try these out, although I was told that it would work like Cortana, Siri, or Google Assistant, but with a flair for accessibility commands like “Call Mom,” or “Tell Bob congratulations on the new baby.” I can’t really imagine needing it since I rarely ever use my various digital voice assistants that way, but it could be a nice touch.
The big issue with Dell Mobile Connect is pretty obvious: only recent model Dell and Alienware computers need apply. This, I was told by a Dell spokesperson, wasn’t based on dogma or for the blatant sake of brand exclusivity, but rather to make sure every supported computer can both mirror the Android phone’s screen in the lowest possible latency via WiFi, and stream audio and receive notifications via Bluetooth. Dell understandably knows every WiFi and Bluetooth antenna in each of its computers, so it stands to reason that the company would first create for those and work on edge cases later, including Dell and Alienware computers older than January 2018. That said, not all computers have built-in Bluetooth, so it’s clear Dell has some room to speak here in its opening bid for supposed incidental exclusivity.
The same Dell spokesperson told me that similar to Alienware Control Center, a proprietary software for managing Dell hardware, that the company isn’t opposed to eventually making the software available on non-Dell computers. The parallels here aren’t exactly 1:1 though, as Alienware Contol Center is specifically for Dell accessories, and isn’t some charitable piece of freeware. For now, the company is offering the VR part of the software in beta exclusively on recent Dell and Alienware computers as it addresses intermediary edge cases in mobile phone model compatibility and the aforementioned PC hardware. The spokesperson stressed to me that ‘this isn’t a product’ and there was still more work to be done.
Notice how I’ve been talking a lot about Android phones? The companion mobile app is compatible with Android 5.0 and above, and iOS 10 and above, although that last bit has an asterisk. Thanks to the locked-down nature of iOS, Dell wasn’t able to include screen mirroring on iOS devices, however responding to first and third-party apps via the SteamVR menu and virtual keyboard is supposed to work just like it does on Android. There wasn’t an iOS device at the demo, so I can’t be sure, but iOS device owners using the non-VR app seems to like it well enough, giving it an aggregate [4.1/5] stars on the App Store.
I also never got a chance to see the software in action in a game without teleportation, so I can’t say for sure how well a game with smooth locomotion could affect the app’s performance in the event you actually want to move around while having your phone anchored to a fixed point in the game. I wouldn’t suspect it would change by much, although it’s certainly something to keep an eye on when the update finally releases at some point. Again, we still don’t know exactly when.
In the end, I got the impression that Dell Mobile Connect VR was the result of a small, but focused team. To my knowledge, there’s no secret sauce that wouldn’t make this a viable project, possibly even an open sourced one, for a similarly minded group looking for some name recognition. If Dell does drag their feet in releasing to other PC brands in the near future, we’ll be keeping our eyes out for the next best thing, because the concept is just too awesome to foist on a niche of a modest, but burgeoning userbase.