RoboKinect

Post your NXJ projects, project ideas, etc here!

Moderators: 99jonathan, roger, imaqine

RoboKinect

Postby Starky » Sun Nov 27, 2011 4:24 pm

For our Masters project in University myself and 2 of my colleagues are working on an autonomous robot capable of 3d object recognition. We study a mix of Electronic and Electrical Engineering and Computer Science.

Our project is making use of the LEGO NXT kit, a MIcrosoft Kinect camera and a BeagleBoard. The idea is that the Beagleboard is the brains of the operation and is able to obtain depth images from the Kinect camera and use them to do some simple object recognition. It can also control the NXT robot over USB to explore areas.

The software we are using comprises of LEJOS, OpenKinect and OpenCV. OpenCV (Computer Vision) is a framework which will allow us to perform transforms on images taken by the Kinect to hopefully do some object recognition. All of this is run on the Beagleboard which uses Ubuntu. The code for OpenKinect and OpenCV is in C/C++ and we're using a JNI wrapper to control the robot over USB with a simple send/receive protocol.

I'll add more information as the project progresses including images, videos and source code. We're currently at the point of configuring everything on the Beagleboard after doing proof of concepts on more powerful machines. This can be quite slow because of the large amount of dependencies we are working with and the slow processor on the board.

Would love to hear any questions/suggestions, we're quite excited about this!

EDIT: Here are a few pictures of the initial attempt to attach the Kinect to a robot, if you peel off the padded part on the bottom you can get access to the screw holes - normal LEGO plugs kind of fit into these - will drill into the Kinect later on to get a better fit. However this is fairly sturdy for now and the robot can handle the weight without any issues.

Apologies for the poor quality, one of my colleagues has a DSLR so we'll get some better ones when we've made more progress.

Image
Image
Image

An example of the images we're capturing

Image

Looks like we're also going to get the chance to write a paper on this for an upcoming conference if things go well, here's hoping!
Last edited by Starky on Fri Dec 02, 2011 6:36 pm, edited 2 times in total.
Starky
New User
 
Posts: 8
Joined: Fri Nov 18, 2011 11:37 am

Re: RoboKinect

Postby PapaTom » Fri Dec 02, 2011 9:12 am

have a look at andrew davinson's work at

fivedots.coe.psu.ac.th/~ad/jg

chapters 13 to 16.6

best of luck

papatom
PapaTom
New User
 
Posts: 2
Joined: Fri Nov 26, 2010 11:31 am
Location: UK

Re: RoboKinect

Postby Starky » Fri Dec 02, 2011 6:32 pm

Thanks for the link PapaTom, looks like something we'd definitely be interested in. Right now we're trying to figure out how to get the Texas Instruments image processing libraries working - the OMAP chip on the Beagleboard has a dedicated DSP unit so we're going to try to make use of that rather than running OpenCV on the ARM core. We have some contacts at TI through the University so hopefully they can help us sort it out!
Starky
New User
 
Posts: 8
Joined: Fri Nov 18, 2011 11:37 am


Return to NXJ Projects

Who is online

Users browsing this forum: Yahoo [Bot] and 2 guests

more stuff