RoboKinect

Post your NXJ projects, project ideas, etc here!

Moderators: 99jonathan, roger, imaqine

RoboKinect

Postby Starky » Wed Dec 18, 2013 4:28 pm

So I've recently purchased an EV3 kit and can't wait to be working with leJOS again when I get some time off over Christmas. In the meantime I thought it would be great to share my last leJOS project with the community here.

For my Masters project at University, two of my colleagues and I worked on a project called RoboKinect. The idea was to create an open robotic platform for 2.5D object recognition.

Having previous experience with MINDSTORMS and leJOS, I kicked things off by building a simple robot which could support the weight of the Kinect and even rotate it. A sensor was placed at the front of RoboKinect to stop it from bashing into things - the Kinect itself was not used for proximity detection because it has a minimum range of some tens of centimeters. At this point of the project, I was controlling RoboKinect from my laptop using the USB connectivity features within leJOS.

Image

The Kinect was also attached to my laptop and image data was pulled from it using OpenKinect, which allowed us to capture depth information as well as regular color images. You can see below how depth is represented in a 2D array of distances, in this case converted to greyscale levels.

Image

In order to perform object recognition we made use of the OpenCV project, which accomplishes this task using Haar-like features. We used training algorithms to teach RoboKinect how to recognize human hands.

Image

At this point, everything was still plugged into a laptop. An aim of the project was for RoboKinect to be more autonomous, so we moved all of the software onto a Linux-based BeagleBoard. On this board we were able to acquire images from the Kinect, detect objects and then command the NXT brick to move around based on what it could see. The basic demo which we set up had the robot following a person's hand.

The Kinect and Beagleboard were powered from a 12V battery and a wireless module was also added to allow commands to be sent to RoboKinect from a laptop.

Image

The final challenge was getting everything mounted onto RoboKinect. The weight of all of the equipment pushed the NXT kit to it's limits but in the end it was fully capable of movement, even if it did look like a bit of a monstrosity!

Image

The final product:

Image

We had a great time working on this project and I believe it really showed what's possible with leJOS and the MINDSTORMS platform. Thanks to everyone who contributes to this community, I'm very excited for the next project! If anyone has any questions I'd be more than happy to answer them.

Technologies

OpenKinect - http://openkinect.org/wiki/Main_Page
OpenCV - http://opencv.org/
BeagleBoard - http://beagleboard.org/
Starky
New User
 
Posts: 8
Joined: Fri Nov 18, 2011 11:37 am

Return to NXJ Projects

Who is online

Users browsing this forum: Google [Bot], Yahoo [Bot] and 1 guest

more stuff