Copyright @ IFAC Telematics Applications in Automation

and Robotics. Weingarten. Germany. 2001

ROBOGUARD, A TELEOPERATED MOBILE SECURITY

ROBOT

Andreas Birk· Holger Keno •

• Vrije Universiteit Brussel, AI-Lab, birk@ieee.org

Abstract: The so-called RoboGuard is a mobile security device which is tightly integrated
into the existing surveillance framework developed and marketed by Quadrox, a Belgian
SME. RoboGuards are semi-autonomous mobile robots providing video streams via wireless
Intranet-connections to existing watchguard systems, supplemented by various basic and
optional behaviors. RoboGuards fill several market-niches. Especially, they are a serious
alternative to the standard approach of using Closed Circuit Television (CCTV) for surveil-
lance. The paper describes how the main challenges from the telematics viewpoint, namely
ensuring Quality of Service (QoS) and Fail-Safe Guarantees (FSG), are solved in this system.
: Copyright @200J IFAC

Keywords: tele-operation, realtime control, human in the loop, behavior-oriented AI

1. INfRODUCTION (QoS) and Fail-Safe Guarantees (FSG) despite the un-

predictable performance of standard InternetlIntranet-
technologies, especially when wireless components
The so-called RoboGuard is a joint development
are involved.
between Quadrox (see QUAOO), a Belgian security
SME, and two academic partners, the AI-lab of the In accordance with recent interest in service robotics
Flemish Free University of Brussels (VUB) and the (see e.g. Engelberger (1989) for an overview), there
Interuniversity Micro-Electronics Center (IMEC). A has also been previous work on security robots. This
RoboGuard allows remote monitoring through a mo- work is widely scattered, ranging from unmanned
bile platform using onboard cameras and sensors. gunned vehicles for military reconnaissance opera-
RoboGuards are supplements and often even alter- tions (see Aviles et al. (1990» to theoretical research
natives to standard surveillance technology, namely on reasoning within decision-theoretic models of se-.
Closed Circuit Television (CCTV) and sensor-triggered curity (see Massios andF. (1999)). Our approach deals
systems. RoboGuards are tightly integrated into the with a system operating in semi-structured environ-
existing range of products of Quadrox. This is an ments under human control and which is a product,
important aspect for the acceptance of any novel tech- i.e., it must be competitive to existing alternative solu-
nology in well-established markets as customers are tions for the task.
usually not willing to completely replace any existing
The rest of this article is structured as follows. In sec-
infrastructure.
tion two, the software architecture of the RoboGuard
For efficiency and security reasons, the RF-transmitted is described. In doing so, there is a special emphasis
video-stream of the on-board cameras can be com- on the main challenges from the telematics viewpoint,
pressed using a special wavelet-encoding (see Dewitte namely, how Quality of Service (QoS) and Fail-Safe
and Cornelis (1997». The IMEC is the responsible Guarantees (FSG) can be ensured when an unreliable
partner for this feature of RoboGuard. The mobile wireless network connection is a major part of the
base and its control, which are the main focus of control loop. The third section presents the hardware
this paper, are at the hands of the VUB AI-lab. From and the low-level software environment with which
the telematics viewpoint, the major challenges in this
part of the project are to ensure Quality of Service

247
 RF-connection

down-link
Internet
(Intranet)
·0
RoboGuard

Fig. L RoboGuards are teleoperated from a so-called cockpit by a human watchguard. Despite the human in
the loop, RoboGuards need quite some autonomous functionality ensuring FSG and QoS as the network
performance is unknown and can even break completely down, especially as wireless components are
involved.
the system is implemented. Section four concludes the
article. thread TO: autonomous, hard realtime control
- runs to completion « 1 msec)
- consists of sub-threads TO.x
2. THE SOFTWARE ARCHITECTURE
* scheduled offline
RoboGuards are teleoperated via standard network * TO.x ensure FSG
technologies by human watchguards (figure 1). The
human in the loop ideally feels like being in full
control of the system. But the unpredictable perfor- thread Tl: semi-autonomous, soft realtime control
mance of networks, in terms of bandwidth, latency, - runs with preemption
and even reliability, makes it necessary to implement - invokes a dedicated scheduler (B-scheduling)
quite some autonomy on teleoperated devices. In do-
* online scheduling of sub-threads Tl.x
ing so, there are two major issues, namely ensuring
FSG and QoS. FSG must never be violated at any cost * implementing a rich set of behaviors
For a mobile robot, this means for example that obsta- * one sub-thread services the operator
cles must be avoided or that the base must be stopped * Tl.x ensure QoS
to avoid serious damages. QoS in contrast defines
constraints which maximize utility as long as they thread T2: non-uniform processing
are not violated. A timely response to requests from
- runs with preemption
the operator for example ensures that the RoboGuard
moves along its path as desired. If these constraints are - services spare-time activities
occasionally violated, they should at most cause some * operator changes of mission parameters
slight inconveniences to the operator, but they never * building up environment maps
must put the whole device or mission at risk. * etc.
The main challenge for telematics applications is to
find a software architecture which supports these dif- Fig. 2. All threads running on a RoboGuards are clas-.
ferent types of processes. For RoboGuard, we use sified into three types. For each type, a respective
following approach (figure 2). The run-time system master-tread handles the invocation of its related
consists of three cyclic master threads TO, Tl, and sub-thread. This scheme allows the combined
T2 running in times lots in a 125 Hz major cycle. TO usage of hard realtime, soft realtime, and non-
includes everything dealing with FSQ. It establishes uniform processing.
a hard real time control-system. It is run to comple-
tion and its components are scheduled offiine. On the the RoboGuard can be used to keep the robot on a
RoboGuard, TO covers the motor- and basic motion- trajectory, to avoid obstacles, to approach a target,
control as well as odometry and positioning. to autonomously scan for intruders, and so on. The
steering commands from the operator are serviced in
The sub-threads of the master-thread Tl are so-called
a dedicated behavior. They are transformed to motion
behaviors. Following the field of behavior-oriented
commands and fused with the motion commands from
robotics (see e.g. Brooks (1986); Steels (1991, 1994)
all other behaviors.
for an overview), reactive control schemes are used
to establish close, dynamic couplings between sen- For the behaviors, soft real time constraints are the
sors and motors (see Brooks (1986); Steels (1991)) only possible way to go. The "ideal" deadline until
which are computed in pseudo-parallel. Behaviors on when a behaviors has to be handled is in most cases

248
 ,...-_ _- - , -rveDCY CQl1tro~ .,tion CODtrol JaOtor control
125 as lao 1 - - -.....- - - - - - - - _ - - - - - - - _ _ .

valu••

aanity
clwck

acti". brealti.Dg ca.pute

if nec••• ary target .pee4a

1IC68332 ODI>oard TPtI

CD. P

Fig. 3. The tasks for realtime control are cyclic processes running at a fixed frequency. Their target-values are
asynchronously set by higher level behaviors.

Cockpit Wirel •••

PC Bridge
~~icati

:Internet

IIetwork

Data LiDk

l'hy.ica~

:tntarnet "irue•• Date

tr~ •• iOD ftheruet

Fig. 4. The flow of the control data from the cockpit to the RoboGuard. For certain parts, the time can not be
predicted.

not known as it depends on unpredictable or simply 2.1 The hard realtime control
too many conditions. The behaviors are scheduled on-
line by a special so-called B-scheduler (see Birk and
The TO layer interacts with the higher layers via a
Keno (2000» which is invoked by Tl. Note that this
shared memory buffer that is written by a thread
scheduler as well as the behaviors are pre-empted by
from a higher layer and is read by the lower-layer
the master-scheduler. The B-scheduler guarantees a
TO thread. The write operation is made atomic by
idle-free, optimally balanced execution of the behav-
delaying the execution of the TO thread during write
iors, thus optimizing QoS.
operations. So, target-values in the motion-controller
Spare time activities, i.e., processes which neither can be asynchronously set by higher level behaviors.
contribute to the FSG nor the QoS, are handled in
The motion-controller so-ta-say transforms the target-
the master-thread TI. They can include the occasional
values on basis of odometric data to appropriate target-
change of mission parameters by the operator, the
values for the motor-control. The motion- and motor-
construction of environment maps, and so on.
control layers are based on generic software modules
for differential drive robots, featuring
• PID-control of wheel speed

249
 • odometric position- and orientation-tracking a lost packet since the following packet will contain
• rotational and translational trajectory control updated state information. By exploiting this property
• emergency breaking of the protocol, low-latency operation can be assumed.
As mentioned before, all of the involved subthreads The communication between the RoboCube and the
TO.x are scheduled off-line to achieve a hard real- onboard PC uses inband handshaking to prevent buffer
time control. This allows especially to include an overruns in the RoboCube software. The communica-
emergency-module which ensures that the robot is tion layer software in the RoboCube confirms every
stopped if it is extremely close to an obstacle. This packet with a Ox40 control code. Only if this control
sub thread uses active breaking to get the base to a code has been received, the onboard PC communica-
fast, but uncontrolled stop. Hence, the base will be tion layer software transmits the next packet. If the
protected in such circumstances from damage, but RoboCube communication layer software did not yet
valuable positioning and trajectory information will confirm a packet when a new packet arrives from the
be necessarily lost as this harsh breaking will include Internet transport layer, this packet is discarded so that
slipping motions. the control layer software only receives recent packets,
again ensuring low-latency operation.
The option of this subthread is hence explicitly for
guaranteeing failure-safety, which only kicks in on Moreover, the communication layer measures the time
extremely rare occasions. Normal obstacle avoidance, between two packets. Whenever it becomes too large,
including controlled stops which are autonomously the command information in the last packet is dis-
activated by the base, are handled on the layer of T 1. carded and the base is transfered into a safe state
depending on sensor information, i.e. stopped with the
motor controller actively holding the last position.
2.2 The soft realtime control via behaviors
Plausibility checks on the same layer can be used to
The hard realtime is needed to ensure FSG. But discard packets or to modify the implications of the
for tele-operated devices in general, network perfor- information they contain. This is done in a rule-based
mance, especially for wireless solutions, can usually module. This functionality is optional and allows a
not be predicted. Hence, hard real time conditions are convenient incorporation of background knowledge
not an option for complete control of the device. Fur- about particular application domains.
thermore, hard real time software is difficult to main-
tain and to extent.
The major trick in our software architecture is that
a soft real time scheduler for behaviors is run as part
of the hard real time schedule. In behavior-oriented
robotics, the control of a system is distributed over
various processes or behaviors running in virtual par-
allel. The different behaviors, like controlled obstacle
avoidance, ensure a smooth performance of the base.
A core behavior, especially from the viewpoint of a
teleoperated device, is operator communication, i.e.,
the transmission of control states from the operator's
console or so-called cockpit to the control hardware
(figure 4). To ensure a low-latency operation over the
Internet link, a protocol based on UDP packets has
been implemented. The protocol is completely state-
less. The packets are formed at the cockpit by syn-
chronous evaluation of the control state and transmis-
sion to the onboard PC of the RoboGuard platform
via Internet. Here, they are received and transmitted
to the RoboCube via the serial port. The communica-
tion behavior parses the packets and makes its content
available to other behaviors via shared memory. Op-
erator command-data for motion is simply fused with
the data of other autonomous behaviors.
To ensure low-latency-operation, there is no retrans-
mission on lost packets although UDP does not guar-
antee successful delivery of packets. However, since
packets are transmitted synchronously and are only
Fig. 5. The inside core of the RoboGuard.
containing state information, there is no need to resend

250
 camera mobile PC
4x USB-cameras
tower video compression
WaveLAN RF-ethemet

sensor CubeSystem
5x Ultrasound Sonar
moduls 6x Active Infrared
optional (Pyro, Temp., Smoke)
Odometry and Positioning
mobile Motioncontrol
base

Fig. 6. A schematic overview of the different components of the RoboGuard.

Fig. 7. Left: The RoboCube, an extremely compact embedded computer for robot control. Right: The mobile
base of RoboGuard. It is completely constructed from CubeSystem components including the RoboCube as
controller, the motor- and sensor-modules, as well as the battery-management hardware.

2.3 The OS support using SMD-components. Second, three boards are

stacked on each other leading to cubic design, hence
The RoboGuard control software relies on the RoboCube its name RoboCube.
controller platform, which is shortly described below,
RoboCube has a open bus architecture which allows
and on it's CubeOS operating system to implement
to add "infinitely" many sensorimotor-interfaces (at
the control application. The CubeOS nanokernel con-
the price of bandwidth). But for most applications the
tains real-time multi-threading, abstract communica-
standard set of interfaces should be more than enough.
tion interfaces and thread control primitives. On top
RoboCube's basic set of ports consists of
of the nanocore, a set of software drivers provides an
application programming interface to the RoboCube's • 24 analog/digital (AID) converter,
hardware. • 6 digitaVanalog (DI A) converter,
• 16 binary Input/Output (binl/O),
• 5 binary Inputs,
3. THE HARDWARE IMPLEMENTATION OF • 7 timer channels (TPC), and
THE SYSTEM • 3 DC-motor controller with quadrature-encoding
(QDEC).
The implementation of the RoboGuard is based on the
so-called CubeSystem, a kind of construction kit for The RoboCube is described in more detail in Birk
robotic systems, engineered at the VUB AI-lab. The et al. (2000, 1998a).
VUB AI-lab has quite some tradition in developing In addition to its central component, the RoboCube as
flexible robot control hardware. Various experimental controller hardware, the CubeSystem provides addi-
platforms have been build in the lab starting from the tional hardware, including electronics and mechanics,
mid-eighties up until now. The center of the CubeSys- and software components. The CubeSystem features
tern is the so-called RoboCube controller hardware for example a special operating system, the CubeOS
(figure 7) based on the MC68332 processor. The com- (see Kenn (2000)), which ranges from a micro-kernel
pact physical shape of RoboCube is achieved through over drivers to special high-level languages like the
several techniques. First, board-area is minimized by

251
process description language PDL (see Steels (1992» . performance. In Proceedings of the Sixth Euro-
The CubeSystem is used in basic and applied research, pean Workshop on Learning Robots, LNAI 1545.
industrial projects and academic education (see Birk Springer, 1998.
et al. (1999, 1998b); Birk and Belpaeme (1998); Birk Andreas Birk and Tony Belpaeme. A multi-agent-
(1998». Therefore, a wide range of sensor- and motor- system based on heterogeneous robots. In Alexis
components exists. The CubeSystem also includes Drogoul, Milind Tambe, and Toshio Fukuda, ed-
dedicated RF-network components. For compatibil- itors, Collective Robotics, CRW'98, LNAI 1456.
ity reasons, radio-ethemet serviced via a mobile PC Springer, 1998.
is used in the RoboGuard. This PC is also used to Andreas Birk and Holger Kenn. Programming with
compute the video-compression. All control and ser- behavior-processes. In 8th International Sympo-
vice related data going to and coming from the cock- sium on Intelligent Robotic Systems, SIRS '00,2000.
pit is directly relayed from the RF-connection to the Andreas Birk, Holger Kenn, and Thomas Walle.
RoboCube which handles all service and control re- Robocube: an "universal" "special-purpose" hard-
lated tasks on the RoboGuard. ware for the robocup small robots league. In 4th In-
ternational Symposium on Distributed Autonomous
Robotic Systems. Springer, 1998a.
Andreas Birk, Holger Kenn, and Thomas Walle. On-
4. CONCLUSION board control in the robocup small robots league.
Advanced Robotics Journal, 14(1):27 - 36,2000.
The paper described the RoboGuard, a commercial Andreas Birk, Thomas Walle, Tony Belpaeme, and
surveillance robot teleoperated via Intranets. On its Holger Kenn. The vub ai-lab robocup ' 99 small
hardware side, the implementation of the RoboGuard league team. In Proc. of the Third RoboCup.
is based on the CubeSystem, a kind of construction Springer, 1999.
kit for robotic systems. Its use in the design of the Andreas Birk, Thomas Walle, Tony Belpaeme, Johan
RoboGuard is shortly presented in this article. Parent, Tom De Vlaminck, and Holger Kenn. The
The main focus of this article is on the general prob- small league robocup team of the vub ai-lab. In
lem of ensuring secure but convenient control of a Proc. of The Second International Workshop on
tele-operated device. We presented a special software RoboCup. Springer, 1998b.
architecture which incorporates Quality of Service Rodney Brooks. Achieving artificial intelligence
(QoS) and Fail-Safe Guarantees (FSG). The main idea through building robots. Technical Report AI memo
of the architecture is to find a suited way to combine 899, MIT AI-lab, 1986.
hard and soft real time scheduling. S. Dewitte and J. Comelis. Lossless integer wavelet
transform. IEEE Signal Processing Letters 4, pages
Concretely, we use an hierarchical scheduling struc- 158-160,1997.
ture as follows. On the highest layer, there are only Joseph F. Engelberger. Robotics in Service.
three threads TO, n , T2 running in time-slots in a MIT Press, Cambridge, Massachusetts, 1989.
fixed frequency master cycle. TO is run to comple- Holger Kenn. Cubeos, the manual. Technical Re-
tion and its subthreads TO.x establish a hard realtime port MEMO 00-04, Vrije Universiteit Brussel, AI-
control, ensuring FSG. The thread Tl , which can be Laboratory, 2000.
preempted, invokes a further soft-real time scheduler N. Massios and Voorbraak F. Hierarchical decision-
for behaviors, which provide QoS. The behaviors es- theoretic robotic surveillance. In IJCAl'99 Work-
tablish close, dynamic couplings between sensors and shop on Reasoning with Uncertainty in Robot Nav-
motors computed in pseudo-parallel. This includes the igation, pages 23-33,1999.
steering-commands from the human in the loop, which QUAOO. The quadrox website.
are simply fused with the autonomous functionalities. http://www.quadrox.be.
The third thread T2 allows optional non-uniform pro- Luc Steels. Towards a theory of emergent functional-
cessing, e.g., for operator changes of mission parame- ity. In Jean-Arcady Meyer and Steward W. Wilson,
ters. editors, From Animals to Animats. Proc. ofthe First
International Conference on Simulation ofAdaptive
Behavior. The MIT PresslBradford Books, Cam-
bridge, 1991.
References Luc Steels. The pdl reference manual. Technical
w.A. Aviles, T.W. Hughes, H.R Everett, A.Y. Umeda, Report MEMO 92-05, Vrije Universiteit Brussel,
S.w. Martin, A.H. Koyamatsu, M.R Solorzano, AI-Laboratory, 1992.
RT. Laird, and S.P. McArthur. Issues in mobile Luc Steels. The artificial life roots of artificial intelli-
robotics: The unmanned ground vehicle program gence. Artificial life Journal, 1(1), 1994.
teleoperated vehicle. In SPIE Mobile Robots V,
pages 587-597,1990.
Andreas Birk. Robot learning and self-sufficiency:
What the energy-level can tell us about a robot's

252