NASA Control Robot With Kinect 2 And Oculus Rift

NASA are no stranger to innovation, which is exactly why they signed up for the Kinect for Windows developer program in order to get their hands on the Kinect 2. It wouldn’t be the first time the team have tinkered with gaming peripherals for their experiments either as the team previously used the Leap Motion controller, Oculus Rift and Virtuix Omni to allow them to take a virtual walk around tour of the red planet.

Using the Oculus Rift and Kinect 2 the team at Nasa’s Jet Propulsion Laboratory were able to get the two devices controlling an off-the-shelf robotic arm. Using a head mounted display and the motion sensors lead the team to state that the resulting interface was their “most immersive interface to date”.

“It allowed us to track open and closed states, and the rotation of the wrist,” says Human Interfaces Engineer Victor Luo. “With all of these new tracking points and rotational degrees of freedom, we were able to better manipulate the arm.”

Latency is an issue for the team, especially given they have to simulate the delay incurred by remote controlling missions on other planets such as Mars rovers. They work around this issue in their simulations by having a ghost image of the control arm which you control, while the solid image of the arm catches up, allowing you to plot moves in advance for the robot to then carry out.

Thank you Engadget for providing us with this information.

Image courtesy of Engadget.