As HoloLens filters its way into more developers’ hands, we’re beginning to see some of the true potential in Microsoft’s ‘Holographic’ augmented reality headset. This early new project from Jasper Brekelmans shows the potential, practical applications for SFX artists to build digital visual effects against reality in real time.
The separation of the digital and physical world is one digital artists in the entertainment industry have to tackle regularly. So what if there was a tool which allowed an SFX artists to build his effects and try them out, in real time, overlaid into reality?
Jasper Brekelmans thinks he has a solution, or at least the beginnings of one. He’s using Microsoft’s augmented reality visor HoloLens to build an AR integration with Autodesk’s popular MotionBuilder character animation studio.
“Virtual Production (the ability to see virtual 3D content through the camera lens while shooting) has become a standard tool in many big movie productions these days,” says Brekelmans, “This early prototype shows how to live stream a 3D character with motion capture data into the Microsoft HoloLens from Autodesk Motionbuilder [a popular professional 3D character animation tool], using a simple WiFi connection.”
The main thrust of Brekelmans’ project is to take SFX pre-vis to new levels. “Integrating a HoloLens in this process would allow a visual effects supervisor or director to see these 3D assets while walking around a live-action movie set, without the need for looking through a camera/tablet screen. Giving them a more natural way to get a feel for scene blocking and finding the best shot.”
Brekelmans has released a video (embedded at the top of this page), demonstrating the new project in action, filmed through the HoloLens visor, aptly demonstrating the HoloLens’ tracking abilities. “The HoloLens is a revolutionary device, it’s rotational & positional tracking are amazingly robust. It’s environmental understanding (it can detect where the floor, walls, ceiling and tables are for example) combined with being a fully standalone wearable Windows 10 computer open up possibilities that weren’t possible before,” states Brekelmans.
Of course, working with what is essentially just one step beyond a prototype device has its challenges. “Being a 1st generation developer kit has it’s limitations, interaction (gaze, speech & gesture) can be somewhat limiting, field of view is not extremely wide yet and being a mobile device there are CPU/GPU constraints,” he says. “Still it allows for experimenting with the future in the present day and just like we’ve seen after the first Oculus Rift release things will only improve in the next few years and iterations.”
It’s an impressive achievement, and seeing it in action really does show HoloLens playing to its strengths. Crafting fantasy characters and objects and at the same time understanding how they look once placed into the real world is a potentially extremely powerful tool, which is heartening to see given the scattershot direction MS appear to be taking with pitching the HoloLens up to now.