apple vision pro-2 — Image courtesy Apple

Apple Reportedly Has No Plans to Make or Support VR Controllers for Vision Pro

By

-

Jul 4, 2023

If a recent Bloomberg report from Mark Gurman holds true, not only is Apple not planning to release a motion controller for Vision Pro in the future, but it may not even support third-party VR controllers at all.

When the Cupertino tech giant unveiled Vision Pro last month, it didn’t emphasize the headset’s ability to potentially support VR games, which have typically been designed around motion controllers like Meta Touch or Valve’s Index controller for SteamVR headsets.

Among Vision Pro’s lineup of content, which features a standard suite of Apple ecosystem and standard content viewing apps, the studio only off showed a single VR app, Rec Room, the prolific social VR app that supports most major VR headsets (excluding PSVR 2 for now) in addition to consoles, desktop, and both iOS and Android mobile devices.

Apple Vision Pro | Image courtesy Apple

Mark Gurman, one of the leading journalists reporting on unreleased Apple tech, maintains that Apple is neither actively planning a dedicated controller, nor planning support for third-party VR accessories.

When the $3,500 headset launches in early 2024, this would leave Vision Pro users relying on the headset’s built-in hand and eye-tracking, which admittedly worked very well in our hands-on. It’s also using Siri-driven voice input, Bluetooth and Mac keyboard support, and PlayStation 5 and Xbox controllers for traditional flatscreen games.

For VR gaming though, hand and eye-tracking lack the haptic feedback required for many game genres, meaning what VR games do come to Vision Pro will likely require overhauls to make sure hand-tracking is fully baked in.

'Bounce Arcade' is Like VR Pinball for Your Fists—And Exactly the Kind of Creativity VR Needs to Thrive

Provided Apple sticks with its purported internal plan to not support VR controllers, that would essentially shunt development away from VR gaming and towards the headset’s AR abilities. For Apple, that’s where the ‘real’ money presumably lies.

Denny Unger, founder and lead of pioneering VR studio Cloudhead Games, explains the move as a way to provide a strong development foundation now for Apple’s AR glasses of the future, which will be both more affordable and more capable of replacing a standard smartphone than the admittedly bulky MR headsets of today.

For more from Unger, who heads one of the most successful VR studios, check out his Road to VR guest article to learn more about Vision Pro and why Apple may be launching an AR headset in VR clothing.

impurekind

Apple has been ignoring/dismissing hardcore gamers for decades. Looks like nothing much is changing in the VR space for them.
- Christian Schildwaechter
  
  Hardcore gamers have been ignoring/dismissing mobile games for decades, [despite those being played way more and making way money.]
  
  Looks like nothing much is changing in the XR space for them, [and even if Apple becomes the largest gaming company in XR too, hardcore gamers will still continue to claim that Apple ignores gaming.]
  - Arno van Wingerde
    
    I disagree here Christian: impurekind did not talk about gamer, but hardcore gamers (as opposed to mobile gamers, I assume), so that his statement is fully correct. “Alyx” versus “Candy Crush”.
    - Christian Schildwaechter
      
      My answer was somewhat polemic, but the problem here is that hardcore gamers define hardcore mostly as “what I play”, which often boils down to multiplayer FPS. The argument for hardcore being that the players are “serious/skill-driven” and not “casual”, but that is all self-referential. I’d argue that someone with 1000h in Candy Crush is more of a hardcore player than someone who pops into COD on the weekend to hang out with friends. I’d argue that the skillset you need for precision platformers exceeds that of COD, and that the tactical skills of most larger RTS gamers will wipe the floor with the FPS crowd.
      
      I spend a significant time with games, I use a lot of game engines and am technically a game developer, but I don’t like to call myself a gamer because of a very elitist self-perception of lots of gamers of themselves as “true/hardcore” gamers, looking down on billions of others. Usually not by any type of objective criteria (like time spent, money spent, skills required, required investment for progression, required communication with others), but just by making an arbitrary us/them division, with “us” being superior gamers by some kind of default. And there are games that fulfill all the fuzzy requirements of “hardcore” gamers on iOS, they just ignore them.
      - Pab
        
        Hardcore gamers play games, say, 3-5 hours per day, for years. It gets deeper, you just can´t play candy crush or plants vs zombies for that long without feel them dumb. So HC gamers look for immerssion, deep stories, eyetearing graphics, all aspects that are rarely found on mobile.
        Most people I know, play mobile games just to pass the time, while waiting for the bus. That´s just not hardcore. You can´t call a bookworm to someone that just reads the news.
      - Christian Schildwaechter
        
        The arbitrary argument here is “deeper”, without any explanation. I’m not sure what qualifies long COD sessions as “without feel them dumb”, and I’m pretty sure “hardcore” Clash of Clans would have something to say about tactics, clan building and the depth of the game.
        
        I’m not the only one who criticizes that the “hardcore” definition is completely made up. A “hardcore fan” is someone who spends way above average time on a subject, usually knows a lot more about it and is very passionate about it, but you can be a hardcore VR fan or a hardcore Hello Kitty fan. Only the “hardcore gamers” assume that it is not the intensity of the activity itself, but somehow the genre, so a 3000h+ Clash of Clan veteran is still a “casual gamer”, while someone who died for two hours in Dark Soul is a hardcore gamer. Someone who reached the top at “Getting over it” is a casual, even though only about 8% of all players ever make it, and it is a horrible gruesome process that requires a lot of training and skill, while 28%/18% make it to the Link the Fire/Dark Lord endings in Dark souls.
        
        I get what people basically mean when they say “hardcore gamer”, and I accept it as a rough description of a type of games and a level of interest. But it is a completely fuzzy definition that is most certainly not based on intensity of gameplay, skill level required, narrative depth or graphical fidelity, because there are many games that are considered “casual” and fulfill all of these. Most certainly not enough to put games and whole genres into hardcore/casual boxes, completely ignoring that all of them allow for very different levels of interaction with them.
        
        The basic problem starts with whether “hardcore” is a quality of the game or of the player. If it is a quality of the game and someone who plays 30min of Dark Souls qualifies, while someone who intensively plays Plants vs Zombies for hours every day does not, then it usually boils down to mostly a feature list like “requires keyboard/mouse or gamepad” controls, and doesn’t really say anything about the experience. If it is a quality of the gamer and the intensity of their gameplay, than there is no way how you can exclude the 3000h Clash of Clan players.
        
        Sure, there are millions of players that only game for a few minutes during their commute to combat boredom. But there are also a lot of COD players who say they only play the game because that’s where their friends hang out, so it is mostly a chat room with some action in the background, the game itself isn’t what they are coming for. If the intensity is the criterium, these are all casual gamers.
        
        I’m not the only one who thinks that “hardcore gamer” is an attribute that a subset of gamers have given themselves mostly to distance themselves from others for not particularly mature reasons, in a way that seriously lacks self-reflection and (intentionally) neglects looking at what types of games are actually available on e.g. mobile platforms and how people actually interact with them. “Hardcore” is basically just a label for “we are the real ones, everyone else are just noobs” without much evidence behind it, a way to pat yourself on the back for being part of a group. I wouldn’t be so annoyed by it of they would call themselves “deep lore”, “high adrenaline”, “long play session” seeking gamers for a description that didn’t imply that they are going for the true “hard core” of gaming, as these classifications would both be less arbitrary and less excluding.
        
        From the Wikipedia entry on Gamer#Dedication_spectrum:
        
        Hardcore gamer: Ernest Adams and Scott Kim have proposed classification metrics to distinguish “hardcore gamers” from casual gamers,[32] emphasizing action, competition, complexity, gaming communities, and staying abreast of developments in hardware and software. Others have attempted to draw the distinction based primarily on which platforms a gamer prefers,[33] or to decry the entire concept of delineating casual from hardcore as divisive and vague.[34]
        
        I’m clearly with the latter group.
      - Pab
        
        Men, you really have time for this.
        Most definitions, overall, social definitions, are subjective, and in most cases really, no one can force their meaning on YOU. However, among Hardcore gamers, we just recognoice ourselves. Certainly, that is a subjectivity of us. And if you play candy crush, you are just not part of this subjective definition, because WE made it up. Sure, you can play candy crush and call yourself a HC gamer, but we just wont. We might in the future, if this subjective deffinition changes, but so far (since the 80s), it hasn´t.
- Mattphoto
  
  Yet within 6 months this headset will have 100x more games than Quest and guarantee it. Many won’t be any good, but developers will flock to this.
  - Christian Schildwaechter
    
    After six months Apple will have sold about 200K headsets, as that’s all they are currently able to produce. A lot of these will have been sold to developers, very few of which will release commercial games within just a few months, because a) development takes longer and b) there will be basically no one that could buy them.
    
    We’ll probably see a number of ports from Quest for titles that are already a good match for hand tracking and passthrough like Puzzling Places, but I’m pretty sure developers will stick to developing for Quest with currently 6.5m+ active users who mostly bought it for gaming and may actually buy newly developed games.
Christian Schildwaechter

Provided Apple sticks with its purported internal plan to not support VR controllers, that would essentially shunt development away from VR gaming and towards the headset’s AR abilities. For Apple, that’s where the ‘real’ money presumably lies.

That is not true, was never true and will never be true. I’m sure there were a lot of Nintendo DS fans and Nokia Snake players that predicted mobile games on smartphones would never catch on, because you just need haptic buttons for any decent reaction time. They were dead wrong too.

Apple is now third in place after Tencent and Sony in revenue from games, first in place in earnings from games, with games generating about 2/3 of all revenue from mobile apps, split between sales, DLC, micro-transactions and ads. And all that without any physical buttons or controllers. Game designers simply adapted and came up with mechanics that worked with tipping, dragging, tilting and shaking the devices.

So “no VR controllers” doesn’t mean “no VR games”, it means “different VR games”. Yes, hand tracking is a worse solution than a VR controllers for all types of games that use the VR controller as a gun substitute, of which there are a lot. But there are a lot of those, because VR controllers aren’t good for much else other than a gun substitute. You basically have to point and pull a trigger, often a rather clunky “grab/don’t grab” function and joysticks adapted from gamepads for a type of movement that makes a lot of people sick. Their main benefits are precise button presses and “feeling something”.

VR controllers are absolute crap for anything that involves interacting with small objects: placing a key in a lock, writing something down, turning a switch, stacking objects. Things you do all the time in your real life, so they could make for great game mechanics, but with VR controllers you throw out all the hand-eye coordination that occupies a significant part of our brain activity and replace it with the precision of large, clunky toddler toys.

So we should fully expect VR games, but not necessarily games that are basically still adaptions of desktop games played with a mouse for aiming and a keyboard for running, interacting indirectly with rather distant objects, only now you aim with your arm. Instead games that can utilize good hand tracking at close distances will implement lock picking by having you place two pieces of virtual wire you hold with your actual fingers into a rather small lock and moving them, instead of turning two sticks until they vibrate. You’ll solve Indiana Jones’ style weight puzzles by carefully putting small amounts of virtual sand in a bag with your hand instead of tugging on a giant lever. You’ll put your inventory items in your bag and have to pull them out of there, instead of being transported into a 2D puzzle where you have to place everything in Tetris style Tetromino shaped compartments for its own spatial optimization meta game.

Of course haptics can add a lot, but Apple seems to have done a pretty good job with “faking” haptics on the virtual keyboards they demonstrated during WWDC. The keys react to fingers approaching them by highlighting, then move with the finger in a way your brain expects, then emit a pulse of light once they have been pressed down far enough and a click sound that again matches your expectation, causing several people to say they “felt” the keys click, even though that was impossible. If enough things indicate that an event happened, your brain just fills in the missing details. That’s basically how all of VR works, and it works on the Vision Pro too.

So instead of moaning that Apple not supporting traditional VR controllers will cause all the VR developers to be lost in the desert, maybe look at what we might gain once we are no longer forced to interact with the virtual world with what amounts to the agility of glued-on boxing gloves. How about looking back a couple of years in the past, when developers were suddenly forced to deal with touchscreens instead of buttons, and somehow managed to turn this into the largest gaming market ever, now generating more than half of all revenue and still growing fast?

Doom was great in 1993, its mechanics were an excellent match for mouse and WASD, but who knows what John Carmack would have come up with if eye tracking as an input method had already been established back then. Genres are very sticky, and I was pretty shocked when a redaction error in Sony court documents revealed that currently about 1mn Playstation users play nothing but Call of Duty, and another 20m play mostly Call of Duty (partly just because that’s where everybody hangs out), so there are a lot of FPS gamers that need their controls to be gun substitutes, and these won’t be happy with any substitutes. But there is really more to XR than that, so there is no need to panic that a lack of buttons will end VR gaming. And Apple will go for whatever kind of XR gaming developers will come up with, because it makes them more money than their whole Mac business.
- STL
  
  Skyrim, Fallout, Half Life.
- Jistuce
  
  Even on games that don’t use buttons or thumbsticks at all, VR controllers have a major advantage over image-recognition hand-tracking in terms of accuracy and responsiveness.
Andrew Jakobs

Big fail. But luckily developers can support it themselves through BT.
- ViRGiN
  
  Oh noes, can’t play muh steamvr games
  No display port no buy!
Mario Baldi

I think it makes sense for now. They probably want to push their vision of an interaction that is as simple as possible. Usually when you force some strong limits on your design (UX), the result is more creative.
They may come around at a certain point, it happened before. Jobs was against the use of any sort of pen for the iPad, but eventually Apple introduced the iPencil.
STL

This thing would sell like sliced bread with Elder Scrolls 6 and Motion Controllers. Maybe Apple is not that smart…
- Christian Schildwaechter
  
  So becoming the most valuable company in the world and selling more than 1.5bn smartphones didn’t require Apple to be smart, and not aligning their business priorities with the wishes of VR gamers makes them dumb? Because after a decade the VR market is so huge with almost 10mn active users, Skyrim VR alone made PCVR and PSVR a big success among gamers, and Skyrim VR players routinely buy multi-thousand dollar headsets for games that will not release for years? Strange, I always thought pretty much the opposite was true.
  - STL
    
    Exactly. They sell the same product since 2007 with only minor innovations. They are Steve Jobs zombies. Powerful, but deprived of innovation. AVP could be big and one day become a contact lense or a brain implant, but they make it an iPhone strapped in front of your face.
    - DeepWired
      
      Indeed. I don’t think Jobs would ever launched this device, there is nothing super interesting or revolutionary in it. All those videos with 2d apps floating do not make me running to the store.
      - Guest
        
        It really is sad. They kept saying spatial computing, all while showing 2D squares of everything. Just how unaware is Tim Cook?
      - DeepWired
        
        Even sadder make me all these fanbois who think this is the best thing ever released…
- Mattphoto
  
  Apple doesn’t need Elder Scrolls to sell Apple hardware. it’s pretty evident they never did.
Iain Mac

It isn’t even AR.

Digitizing your environment then viewing in VR, is not AR…. It’s something, but at the end of the day how is this not technically just VR?

I may be biased as a Hololens dev, but I don’t see this as actually being augmented reality, if you arern’t actually looking at reality.

Looking forward to finding ways to screw with the virtualization though
BryanGlasgow

It seems to me that Apple has chosen the path of fully integrating Mixed and Virtual Reality into the daily experience. With sufficient image clarity and comfort of use, such a headset can completely replace the desktop.

If that’s the case, then it’s understandable why they’re ignoring controllers for now. They don’t yet know what kind of controllers users will need in such a working environment.
Walter Sharrow

Pretty typical. Like I’ve always said, they show up to every market late, with inferior hardware and outrageous prices, and yet their cult members rain praise down on them and the technologically ignorant are amazed by a poor copy of better hardware they could have picked up years before. Their MP3 players were weak compared to Rio, their smartphone couldn’t handle a 1/10th the things a Microsoft CE device could, and their $3000+ headset doesn’t have basic hand-controllers like a Quest 2 has. And no matter yet they still fail upwards, coddled in the arms of their faithful toadies, always eager to ignore the vast shortcomings for that vile logo.

Seriously, what is wrong with the world? I really want to know.
DeenR

This will be so stupid if they don’t make or at least support vr motion controllers. It will limit and niche the device drastically, not just for gaming. It would be the same as them ignoring the stylus/apple pencil for years and it was so frustrating trying to take notes on the first iPads, I gave up. Years later they came out with the Apple pencil and I was just pissed by then, had already graduated college. I hated Apple for a while because of it Lol..why limit the device like that. I want the VP but I won’t buy it until they support vr controller/experiences.