Hacker turns Kinect into 3D camera

A UC Davis researcher has managed to turn the Xbox Kinect into a 3D video camera.

After the device was hacked just over a week ago, Oliver Kreylos figured he could use the Kinect’s two cameras to capture objects in three dimensions,  and then re-create them in their original orientation and size. The feat took him just three days.

Data from the two cameras – one providing a photographic video stream, the other the depth video stream – is merged to create a 3D model of the room and everything in it.

Kreylos says he relied on the work of Hector Martin – marcan42. “I didn’t use any of his code, but the ‘magic incantations’ that need to be sent to the Kinect to enable the cameras and start streaming,” he says. “Those incantations were essential, because I don’t own an Xbox myself, so I couldn’t snoop its USB protocol.”

Kreylos wrote his own 3D reconstruction code from scratch in C++, using his own Vrui VR toolkit for 3D rendering management and interaction.

He was able to measure the reconstructed objects and show that they exactly match their real-life counterparts.

Kreylos’ next project, he says, will be to incorporate real and computer-generated imagery as a single feed, creating an augmented reality environment.

There’s video, here; Kreylos is promising more information soon, including source code.