Packing a 7-megapixel front-facing depth camera, Apple’s iPhone X can do some pretty impressive things with its facial recognition capabilities. While unlocking your phone and embodying an AR poop emoji are fun features though, the developers at Kite & Lightning just published a video of an interesting experiment that aims to use the iPhone X as a “cheap and fast” facial motion capture camera for VR game development.
Created by Kite & Lightning dev Cory Strassburger, the video uses one of the studio’s Bebylon Battle Royale characters (work in progress) to demonstrate just how robust a capture the iPhone X can provide. Flexing through several facial movements, replete with hammy New York(ish) accent, Strassburger shows off some silly sneers and a few cheeky smiles that really show the potential for capturing expressive facial movement.
While still a quick first test, Strassburger says that even though the iPhone X can drive a character’s blendshapes at 60fps while it tracks 52 motion groups across the face, “there’s a bit more to be done before I hit the quality ceiling in regards to the captured data.”
On the docket before the iPhone X’s TrueDepth camera can be levied as a VR game development workhorse, Strassburger says his next steps will include getting the eyes properly tracked, figure out why blinking causes the whole head to jitter, re-sculpting some of the blend shapes from the Beby rig to be better suited for this setup, visually tune characters, and add features to record the data.
To top it off, Strassburger is thinking about creating a harness system to mount the iPhone into a mocap helmet so both face and body (with the help of a mocap suit) can be recorded simultaneously.
Bebylon Battle Royale, a comedic multiplayer arena brawler, is due out sometime in 2018 on Rift and Vive via Steam. We can’t wait to see what the devs have come up with, as the game already promises to be one of the silliest games in VR.