Apple’s annual Worldwide Developer Conference (WWDC) is here, and today’s keynote saw a number of VR-specific announcements including Apple’s first VR-ready computers to go along with the launch of the company’s newest macOS High Sierra. While the company is finally going ‘VR-native’ for desktop, Apple is also zeroing in on augmented reality for iOS 11 with the entrance of their newly revealed app developer kit ‘ARKit’.

Possibly taking a swipe at Facebook’s latest AR demo at F8 in April, Senior VP of software engineering Craig Federighi said: “We’ve all seen a lot of carefully edited vision videos on this topic recently, but in this case, I’d like to show you something for real.”

image courtesy Apple

Starting up a test application that will be made available to developers, Federighi explains that with the iPhone’s computer visions capabilities it’s able to map surfaces and add digital objects—replete with interactive animations and dynamic lighting. Adding a steaming coffee cup, a lamp and a vase to a bare, marker-less table, the tracking proves to be relatively solid.

Federighi says that ARKit provides fast and stable motion-tracking, plane estimation with basic boundaries, ambient lighting estimation, scale estimation, support for Unity, Unreal, SceneKit and Xcode app templates—all available on “hundred of millions of iPhones and iPads […] making overnight ARKit the largest AR platform in the world.”

Apple says iOS 11 will be made available to iPhone 5s and later, all iPad Air and iPad Pro models, iPad 5th generation, iPad mini 2 and later, and iPod touch 6th generation. iOS 11 will be released this fall, likely in tandem with iPhone 8 and iPhone 7S smartphones. A public beta is coming in June.

image courtesy Apple

Apple is working with third-parties such as IKEA, Lego, and Niantic to use ARKit, with the company showing an improved Pokémon Go on stage that looks to actually utilize augmented reality to bring the game to life. Because ARKit uses computer vision that relies on the device’s onboard sensors and CPU/GPU, no external equipment is required to run these sorts of AR experiences.

The keynote also revealed a new AR-focused company from critically-acclaimed director and FX guru Peter Jackson called ‘Wingnut AR’. A special demo showed off the graphical and camera-based tracking capabilities of Apple’s hardware featuring a complex, real-time rendered scene digitally placed on a tabletop using an iPad. Wingnut AR is bringing an AR experience to the App Store later this year. Check out the video below to see Wingnut AR’s special Apple demo.

Newsletter graphic

This article may contain affiliate links. If you click an affiliate link and buy a product we may receive a small commission which helps support the publication. More information.


Scott Hayden
Well before the first modern XR products hit the market, Scott recognized the potential of the technology and set out to understand and document its growth. He has been professionally reporting on the space for nearly a decade as Editor at Road to VR, authoring more than 4,000 articles on the topic. Scott brings that seasoned insight to his reporting from major industry events across the globe.
  • Ian Shook

    I remember back when AR apps were popping up all over, pushing the boundaries with 2D scanning tech and later 3D scanning tech (object recognition). You could do some neat things, but the graphics were nothing short of terrible – even if the tracking was good which was only sometimes. This was in 2012. It’s a little sad that apple is entering the AR game so late – especially with crap useless demos like placing coffee cups. Even that game was sort of awkward to watch if you looked at the livesteam. All this was possible and has been done 5 years ago!

    I think we’re sort of done with AR – at least how apple is using it (watching an iphone or ipad as you walk around a low poly object). Needless to say I was really disappointed with Apple’s latest offering and I’ve lost all hope that their future VR tech is going to be groundbreaking.

    • VirtualBro

      Maybe they have more impressive stuff in the works but don’t want to reveal it until it’s consumer-grade, and this is just something to muddy the waters so investors don’t think Apple is being completely left behind

    • daveinpublic

      Apple did this demo for other reasons. They weren’t trying to show off seemingly old technology, they were positioning themselves. Believe me, later this year, or sometime next year, they’re going to release an AR headset. I haven’t heard anything, I just know Apple. They just want as much software available as possible to take advantage of AR when they do. Just like Facebook’s event, where you can use your phone to leave notes in AR, it’s not so you can hold your phone up when walking around the mall, it’s for when AR glasses hit the market, and they even said as much. Why do you think Facebook is putting so much time into this? Apple has probably shown them prototypes of the AR glasses they’re going to release, and told them to develop software for it. How do you think they have software from 3rd parties ready to go every time they release a new product or product category?

      • Vern_S

        You are totally correct. I have been developing AR and VR apps for the past three years and the focus for everyone is Wearable Hands Free AR, ie: glasses/headset. Hololens is great for Enterprise applications but too costly for consumers. Android/iOS based consumer AR will change everything but there needs to be apps, apps, and more apps readily available.

        I for one will continue developing AR on phones and tablets to push exposure and capabilities of AR forward. My clients understand where the technology is going and move forward with that ultimate reality in mind.

    • Get Schwifty!

      Sort of done with AR – are you on crack? That is where the huge application set is going to be, full on VR will be a small subset in comparison… what hey showed is one application of it. Their point is the massive install base that can begin taking advantage of it. If your focused on the cup you are seriously missing the point here – imagine the collaboration possibilities and entertainment options this opens up. I do think as DaveInPublic thinks, there is certainly a VR headset or AR/MR glasses setup in the works.

      • Ian Shook

        I hope you’re right! – Regarding my ‘done with AR’ comment, I don’t think gaming in AR (holding an ipad or holding a iphone) will be very big. I realize glasses based AR is inevitable – as there becomes a whole new world of way’s to target consumers with advertising. I think Apple’s glasses-based AR and VR are in the works, and that they put this out there since the high-end final version of the AR devices are not ready for release. It’s just not very Apple to show or release a silly demo of something they’re toying with. I think if they’d said “we’re working on an incredible AR solution you’re going to love” would’ve generated more hype than showing what they did. And that game was silly. Sorry. :) I know apple didn’t develop it but it wasn’t even that well done.

    • Get Schwifty!

      I have to add – did you even see the demo of the game? Far beyond a single coffee cup…

    • Oriato

      Totally agree.
      – No preservation of state. Means: You can’t put something on your wall, quit the app’s session, restart it later and find the object still attached to your wall.
      – No object occlusion.
      – No geometry creation, as with the M in SLAM. Means: You can’t go through your house with an AR Kit ready device and have a 3D model/map of it afterwards.

      Tango and Hololens are both capable of all of the above, but that isn’t any better as they are not really available or/and will not gain any significant spread before 2020(?) -> no apps.

  • Master E

    Huge move

    AR legos with friends and no clean up?

    Yes!!!!!!

    Hopefully they’ll have a good headset to slide my iPhone in and a pair of Bluetooth gloves to interact with the AR…and then it’s all over

    • Get Schwifty!

      As a VR enthusiast and a LEGO enthusiast, I am very excited!

  • daveinpublic

    How difficult do you think it would be to transfer this technology into an AR headset? It would be nothing. The hard work is done. Guess what, Apple has inside out tracking. They just demonstrated it. Now guess what they’re going to release later this year or sometime next year?

    • Lucidfeuer

      See-through camera feedback is not that easy.

      • With plenty of AR SDK’s out there already and now Apple releasing their own library it is very easy to do. Unless you mean something else?

        • Lucidfeuer

          Smartphone AR is bullshit, it’s been around for years indeed but that’s a bullshit gimmick. It might have worked for Pokemon once, because on the face surface that’s a free pokemon game for smartphone, but the only way to do AR is through a headset or glasses.

          I was talking about headset: adjusting lenses, perspective, video feedback is possible, it’s been done by some start-up and I believe it’s implemented on the newest Microsoft VR headset. But for it to be practical and usable it needs to be so stable that people won’t look back on using any other screen interface, so it’s the same problem as VR headset overall: they’re not practical yet, in fact there’s not even one that is.

    • Good question. The tracking will use the sensors in an iOS device to make it work. Saying that, third party AR SDK’s work fine via the Apple Desktop/Laptop cameras already, but they are not Apple SDK’s and the subscription models are pretty high. Anyway, It would not be that difficult for Apple to release this as a macOS/iOS SDK rather than lock it to iOS only but this would be dependent on Apple releasing their own HMD or supporting third party headsets.

  • Maybe I need to see if I can add a AR mode to my SceneKit VR app. But have they said anything about integrating VR with AR?