Autonomous Vehicle

Pursuit of Target Through Visual Recognition

Daniel Vinson
xcdan1@sbcglobal.net
1022 Waverley St
Palo Alto, CA
94301, USA

Sergei Djuvinov
amorphous@abv.bg
“Mladost 2” no. 206 entr 3
Sofia 1799
Bulgaria

Abstract
A Lego® car was built and equipped with a stationary camera and a programmable
RCX 2.0. A program was written in Visual C++® to search for a certain red target, and
maneuver the car towards the red, keeping it in the center of the camera’s view. This
technology can be used for following a certain object (moving or stationary) with an
autonomous vehicle.

Introduction
The aim of this project was to create a computer aided vehicle that navigates itself on
a playground through computer-analyzed vision. The vehicle had to be able to follow a
moving Lego®. In our work a Lego® vehicle with an on-board camera was created. The
camera had to supply a desktop computer with visual data. The computer had to make
decisions and send commands to the Lego® car according to the data supplied by the camera.
Such a system could be very useful in any modern car or even in military vehicles. The
system could be applied to vehicles that follow targets or leading vehicles without human
drivers controlling them.
There were three main problems that had to be solved. First, a moving vehicle had to
be built. Second, for us humans it is very easy to understand an image as soon as we see it.
However, even for the latest computer this seems an unbearable task. Therefore, the second
problem that had to be overcome was to enable the computer understand what it sees.
The third problem was to write a computer program that would enable the Lego®
vehicle to be controlled on its way to the target.
 Methods and Materials
Lego Mindstorms™ Robotic Invention System set was used for the creation of a
moving vehicle. The main parts of the robot were a Programmable RCX 2.0 controller and
two Lego® electrical motors. MARMITEK camera with wireless transmitter and receiver was
used for the input of visual information. For the control of the robot a computer program was
written. The software was created in Visual C++ 6.0 programming language.

The Car

IR Tower, Camera, Receiver RCX 2.0

The Car with RCX 2.0

RCX 2.0

The Car with The Camera

The movable Lego® robot enabled the camera to move and chase the target. The
images from the camera were transmitted to a computer. The communication between the
Lego® vehicle and the computer was achieved through an infrared tower, part of the Lego
Mindstorms™ Robotic Invention System set.
After solving one of the problems - building a vehicle, a solution for the problem
with the computer vision had to be found. Processing the image taken from the on-board
camera solved the problem. The image taken by the camera is in the RGB color space. In this
case every pixel of the image consists of a mixture of the three colors (red, green, blue). For
the computer every color is represented by a numerical value (see picture).
 In this case long time is needed for the
computer to recognize the color of every
pixel and find the the target that has to be
chased. Therefore image processing is
necessary before the computer can use the
data from the image. Most appropriate was a
conversion into HSV (Hue, Saturation,
Value) color space.

The picture on the left is the actual RGB

HSV

RGB HSV

image taken by the camera. The cone in the middle is visual representation of the HSV color
space. The image on the right is the HSV version of the picture. A number of test pictures
were taken. The test pictures were processed through Matlab 6.0. Matlab 6.0 turned the
pictures from RGB to HSV color space. Using Matlab 6.0 the HSV values of different colors
in different situations and environment were determined. After comparison of the values for
the different colors, the color that would be used for target was determined. More pictures of
the future target were taken for determining the min. and max. values of HSV of the given
color in various conditions (light, distance, surrounding colors and materials).
When the color of the target was determined, tests with the vehicle were made for
writing the software. The vehicle was set to run and the camera was set to take pictures. Every
picture was sent to the computer and converted to HSV by our software.
The picture above is processed through Matlab 6.0. The program is set to show in white only
pixels with saturation >0.5 & hue <0.4. The algorithm used in our software was similar to that
Only pixels with
Matlab 6.0 uses for the task. After thesaturation
conversion of &
>0.5 the picture, only the hue and saturation
hue <0.4 are left
values were extracted from the data. The software used the data. First, every third pixel of the
white
picture was checked for hue and saturation values. If the saturation was > 0.5 and the hue <

RGB Hue Saturation

0.4, the data was remembered, if not- the numbers were discarded. The difference between
HSV and RGB is seen very well in the pictures above.
The red sheet on the left of the picture is easily recognizable in both the hue and saturation
representations of the picture.
If the target appeared on the image, its center was found. A decision weather to turn,
move forward, or stop was taken. The program was set to repeat this steps over and over
again. A decision for stopping the repetitions was taken when the number of target pixels in a
row exceeded 275.
Or in few words:

Processing Pseudo-code
Capture an image

Using every 3rd pixel, find if the saturation is > 0.5 and the hue is <0.04

If it is, remember it, otherwise discard the numbers

Find center of target

Move, turn or stop according to location and distance

For the final version of the software the engines were set to work in turns. This
means than they both work for a certain period at the same time. Then the engines stop, a
picture is taken, processed, analyzed, and the computer takes a decision. After the decision is
sent to the vehicle, the process repeats itself.

Results
After long hours of tests and many changes in the code of the software and various
upgrades of the vehicle, the robot was enabled to properly find and chase a slowly moving red
target. The time for which the engines work for moving forward is 200 milliseconds, and for
turn- 125 milliseconds. The reaction time needed by the computer to analyze the picture, send
a signal to the RCX 2.0, receive a new picture, and for the robot itself to move in total, was 2-
4 s. on average.
 Discussion
The low speed of analysis and reaction of the computer and the vehicle itself forced
us to make many changes in the way the car moves. First, the speed of rotation of the motors
had to be decreased. Second, the robot had to stop every time the camera had to take a picture.
If these measures were not taken, the computer would be unable to analyze the pictures input.
As a consequence the car would move in its last direction and eventually collide into a part of
the environment or the target itself.
Another problem in the beginning of our work was the turning of the car. It used to
turn at too big angles and lose the target. Therefore we had to decrease the time for which the
engines work when turning. This decreased the angle at which the car turns. If needed the car
makes another turn to correct its position so that the target remains in the middle of the view
area. Also, the vehicle turned out to be heavy. Therefore a proper selection of gears had to be
made. Our last gear selection gave the car more torque. This enabled the engines to move the
whole weight of the vehicle and made the turns slower and more precise.
Although the image processing part was suited for working in conditions with many
background colors, the system was never tested outside the laboratory. Therefore there might
be some problems with the recognition of the target in outdoor conditions. These problems
are connected with the fact that other objects may show similar values of hue and saturation.
Choosing a unique color for the target can solve this problem. This color can be one that
doesn’t exist in nature or is very rarely met there. Another solution to the problem can be
coloring the target in a variety of colors. The computer then will have to look for a special
combination of values for hue and saturation.
Still, because the vehicle is only chasing the target, the distance between the two is
not necessary. However, in real life situation the actual distance might be needed for various
reasons. In this case the software algorithm doesn’t change much. The only thing that has to
be known is the real size of the target. Through counting the number of pixels seen by the
camera, the computer can easily calculate the actual distance between the chasing vehicle and
its target.

Conclusions
The aim of the project was to build a system that helps navigate a vehicle by
computer aided vision. Our results were satisfactory with the assumption that the vehicle and
the target move slowly. Our robot finds, chases, and reaches the target.
 However, problems like real-time processing, precision of the motor control, and high
speed of the target have to taken into account. Not all colors can be used as targets in nature
because many colors are met at many places. As a result, the computer will not be able to
recognize exactly where the target is situated. Our project can be used as basis for further
development of ‘smart’ robots for the army as well as for the civil world.

Acknowledgements
We would like to thank our mentor Johanan Erez and the whole Electrical Engineering
faculty
. Many thanks to Ronel Veksler. Many thanks also to our councilors: Koby, Noga,
Gila, and Daniel.
We are also very thankful to Technion for hosting SciTech 2003.
Many thanks to Harry Stern for funding our participation in SciTech 2003.

References:

1. MATLAB documentation.
2. http://members.cox.net/pbrick-alpha/Phantom.htm (Phantom ActiveX control for
LEGO Mindstorms).
3. http://www.lego.com/
4. http://www.legomindstorms.com/

For More Information:

http://visl.technion.ac.il/projects/scitech03