Showcased at Apple’s Worldwide Developer Conference this week, ARKit is a new core technology for iOS 11, due to launch this Fall, soon to enable augmented reality features on hundreds of millions of iPhones and iPads. As the iOS 11 developer beta is already available, we’re starting to see some interesting real-world tests of ARKit, showing off the tracking that’s achievable with nothing more than a camera.

Developer Cody Brown hacked together a quick demo using Overwatch assets as a ‘hello world’ test of ARKit running on an iPhone 6S:

Apple’s keynote included a couple of impressive live demonstrations of screen-based AR on the stage, including a sneak peek at an Unreal-powered experience from developer Wingnut AR:

Perhaps the most impressive aspect of that demo was that it was running on an iPad, using the single camera on the back of the device for tracking. This appeared to deliver fairly stable tracking, without the need for dedicated hardware, unlike Google Tango, which uses a suite of cameras and sensors.

Now that developers have their hands on ARKit, the early real-world tests are very promising, such as this clip of the Unity sample demo showing tracking points and plane estimation:

This video from Austrian augmented reality company ViewAR puts the technology through a demanding tracking test, covering the camera, moving quickly away from the virtual object, through multiple rooms with different lighting conditions to check for drift. The result is remarkable considering the limitations of using a single camera:

Apple is believed to be hard at work on AR technologies, and is likely to make screen-based AR a key selling point of the next iPhone, which is anticipated to have a near bezel-free design, which would certainly enhance the appearance of AR features.

Newsletter graphic

This article may contain affiliate links. If you click an affiliate link and buy a product we may receive a small commission which helps support the publication. More information.


Dominic Brennan
The trial version of Microsoft’s Monster Truck Madness probably had something to do with it. And certainly the original Super Mario Kart and Gran Turismo. A car nut from an early age, Dominic was always drawn to racing games above all other genres. Now a seasoned driving simulation enthusiast, and former editor of Sim Racer magazine, Dominic has followed virtual reality developments with keen interest, as cockpit-based simulation is a perfect match for the technology. Conditions could hardly be more ideal, a scientist once said. Writing about simulators lead him to Road to VR, whose broad coverage of the industry revealed the bigger picture and limitless potential of the medium. Passionate about technology and a lifelong PC gamer, Dominic suffers from the ‘tweak for days’ PC gaming condition, where he plays the same section over and over at every possible combination of visual settings to find the right balance between fidelity and performance. Based within The Fens of Lincolnshire (it’s very flat), Dominic can sometimes be found marvelling at the real world’s ‘draw distance’, wishing virtual technologies would catch up.
  • Me

    VERY impressive stuff here. This is promising, and I wonder what this tech might be able to do with the help of a second camera for VR. Looks like inside-out tracking for the masses to me, and considering how powerful mobile GPUs are these days, it’s definitely where things will be going.

    • Get Schwifty!

      No question now that mobile will be the platform for “everyday” AR/MR. Full-on VR will of course require more resources, etc.

  • Stefan Küppers

    I wonder if its a technical limit of the iphone/ipad cameras or the image processing that physical objects like the vase when walking around the goblin won’t conceal part of the virtual ones. I think I saw some hololense/ magic leap demos where that was’t the case.
    Maybe more/ different sensors are needed to do this kind of stuff more realistically?

    • benz145

      What you’re talking about is called ‘occlusion’ and it’s definitely not easy with a single camera. I believe most AR devices doing occlusion are using depth-sensors to in addition to cameras. We could see depth-sensing tech on Apple’s next iPhone and iPad to assist in AR use-cases.

    • James Allan

      The MS Hololens and Occipital Bridge use a sensor to map the environment creating a mesh that is then used for occlusion and collision. The ARkit and other SLAM based approaches are wonderful in that they don’t require additional hardware. As mentioned below, this is great entry point for “everyday” AR/VR/MR experiences and will accelerate the adoption and application of the the technology.

  • LarZen

    It’s really cool. I just hope it wont take to long before you can see AR trough some glasses. Because no matter how cool the AR functionality is your to restricted by looking at an mobile/tablet’s display.

    • NooYawker

      Lots of companies are working on AR glasses but it’s a long way away from mobile AR glasses. The battery restriction itself is a pretty big hurdle.

    • Lucidfeuer

      Through VR, not through glasses. Glasses are a decade away.

      • Lucas Rizzotto

        Nope, much less. More like 4-5 years away from market.

        • Lucidfeuer

          Explain to me exactly what technology will be ready in 5 years for them to hit consumer markets?

  • Diego Cesaretti

    Wow that last video it’s really impressive… Hope this ends the discussion about inside out tracking being the future…

    • Firestorm185

      Yeah, and the drift for that wasn’t even that bad, and was easily able to recenter itself. When technology gets just a little bit better it’ll probably be able to not drift at all!

  • Michael Oder

    The tracking quality looks good enough that I would love to try it out with a full immersion cardboard-style experience.

  • DaKangaroo

    Widowmaker in my bedroom? Where do I sign?

  • Diego Cesaretti

    What’s stopping
    Apple from using this as tracking for VR? iPhone 8 maybe?

    • Jim Cherry

      we dont have a 7s

  • I really do think that AR is going to takeover, side stepping VR entirely. Mixed reality will eventually become the new VR experience, as with normal AR will replace our current smartphone experience. Which is why I tell developers, designers and Entrepreneurs to start thinking about how your current Mobile App experience is going to have to play out in the AR market. If your looking to tap into this “Gold Mine” – then you may want to take advantage of this course – https://www.udemy.com/create-a-mockup-storyboard-for-an-augmented-reality-app/?couponCode=AR_STRYBRD_235 , as it will teach anyone without any technical or designing skills how to prototype an effective mockup storyboard for your Augmented Reality experience. Another tool to have, just saying…