The Kinect was a great example of cutting edge technology coming from R&D into entertainment. First released as a peripheral for the Xbox 360, it quickly became one of the most successful console peripheral ever launched and then promptly completely failed to deliver on its promise to bring natural human gesture recognition to compelling gaming experiences.
Arguably, the Kinect’s most interesting projects have almost all come from hackers and developers on the PC, after Microsoft released official driver and SDK support for the device back in 2011. The Kinect’s (at the time unique), RGB plus Depth sensors in particular gave those interested in Augmented and Virtual reality development ways to quickly capture the boundaries of physical spaces and project them into the virtual.
One of the projects that was borne from within Microsoft’s own research labs was KinectFusion, which undertook research into grabbing high quality 3D environment models captured from cameras with depth sensors, just like Kinect. The upshot of the project was the ability to scan an environment and reconstruct the geometry digitally in 3D models.
Now, a team of researchers have taken KinectFusion and combined it with the Oculus Rift. Kintinuous captures your surrounding environment as KinectFusion did, but then renders the output directly to an Oculus Rift. What’s more, when the viewer moves their head, the render is transformed based on depth information (and an implicitly aligned point in space, in this case the desk) from the Kinect’s depth sensor and not the Rift’s onboard IMU. The results are an impressive way to capture an environment and move through it physically with positional tracking.
According to Hack a Day, the project is not free for release due to copyright issues. However, the research papers here, here and here. As Hack a Day states, nothing prevents a suitably smart individual from taking the research and producing their own Kintinuous implementation. Fingers crossed the results of the project see the light of day. We’ll let you know if we hear anything.
Of course, now the Xbox One is out, the original’s Kinect’s upgraded offspring imaginatively entitled ‘Kinect Sensor for Windows 2’ is available for Pre-order now. With MS promising better SDK support and the upgraded sensor hardware, Kinect 2 could be an interesting device to watch if you’re a VR hacker.