Niantic, the developer behind Pokémon Go and the upcoming Harry Potter AR game, announced they’ll be opening up access to their latest work in AR, dubbed the ‘Niantic Real World Platform’, to third-party developers soon. To boot, the company also introduced a few key AR technologies that will have you salivating over the possibilities of actually chasing down pocket monsters on your commute to that next Pokéstop.
Niantic says their Real World Platform blends machine learning and computer vision to tackle the classic challenge of building a useful and realistic AR experience on mobile devices—something that can sense small details, understand surroundings, and model them in an interactive 3D space that a smartphone can digest.
One area of research Niantic has been work on is proper occlusion, or making sure digital imagery fits into the physical world correctly, and allowing it to be obscured naturally by objects in the environment. The company published a quick video on their blog, showing off their latest work in the area of AR occlusion. What better test subject than Pikachu?
Creating correct occlusion in AR requires that the computer, in this case a smartphone, contextually understands the world around it. Slowing down the video some, it becomes a little more clear however that the company still has a ways to go, as the occlusion masks oftentimes overcompensate, or misjudge the alignment of objects as Pikachu scampers about. While a proof-of-concept, it’s definitely a tantalizing look at the near future of smartphone AR, and a clear departure from what we saw at Pokémon Go’s launch back in Summer 2016.
The Niantic Real World Platform is also focusing on cross-platform AR for shared, multiplayer experiences. The biggest obstacle, the company says, is invariably latency. To this effect, the company says they’ve developed “proprietary, low-latency AR networking techniques” to overcome this problem, which allowed them to realize a unified, cross-platform solution with a single code base. To demonstrate, Niantic built a multiplayer smartphone AR shooter, dubbed ‘Neon’, which shows six users playing at once.
We can attribute some of this to the company’s recent acquisitions; Niantic recently acquired Escher Reality, a studio touted for its cross-platform, multi-user AR platform, and the computer vision and machine learning company Matrix Mill—two decisive moves forward after the company’s $200 million Series B finance round.
“It’s through the coordination of these teams that we’ve been able to establish what the Niantic Real World Platform looks like today, and what it will be in the future,” Niantic says in a statement.
As for third-party developers looking to get in on Niantic’s platform, the company says they’ll be picking a “handful” of devs to begin working with their tools later this year. To receive more information, sign up here.