Collision Avoidance Road Test

for COLREGS-Constrained Autonomous Vehicles*

Kyle L. Woerner, Michael R. Benjamin, Michael Novitzky, and John J. Leonard

Abstract— Recently developed algorithms quantify and sub-

sequently evaluate COLREGS performance in collision avoid-
ance scenarios based on vessel track data. Combining these
evaluation algorithms with proposed categories of COLREGS
rules allows for testing of collision avoidance performance in
accordance with protocol requirements. This paper proposes a
“road test” framework for autonomous marine vehicles prior
to operating outside of a testing environment. Testing and
certifying agencies may adopt the proposed categories of scope
and testing attributes while determining the appropriate param-
eters for evaluation. Adapting the evaluation criteria to several
thresholds would allow for various levels of certification and
locally-tailored customs. Generalization to human operators
and other domains such as Rules of the Air is proposed.

I. I NTRODUCTION

Collision avoidance protocols such as the Collision Reg-

ulations (also referred to as COLREGS or Rules [2]) are
written primarily for human operators resulting in a rule set
that is open to some interpretation, difficult to quantify, and
challenging to evaluate. The continuing migration toward
autonomous collision avoidance has resulted in numerous Fig. 1. Autonomous surface vessels were evaluated using the COL-
implementations of the Rules with varying degrees of fidelity REGS collision avoidance protocol rules with both real-time and post-
as well as large variations in authors’ real-world experience mission analysis tools using a common library of configurable metrics.
Using significant at-sea experience, US and international case law, and
using the Rules. Prior to operations outside of a testing case studies of past collisions, an evaluation library was created to
environment, algorithms and vessels should be evaluated for quantify claims of COLREGS “compliance” within appropriate cate-
both compliance with the Rules and safety. gories of scope. Both rule violations (top left) and safety violations (top
right) were reported as part of the evaluation. On-water testing of M100
Recently developed COLREGS evaluation algorithms al- and M200 Clearpath R autonomous marine vehicles with a human-
low quantifiable and objective assessment of autonomous operated MOKAI R motorized kayak (bottom) allowed for human-
robot interactions and multi-vehicle, multi-rule simultaneous encounters
and human-operated vessels maneuvers for the Rules of the for testing the evaluation programs. Using quantifiable metrics for
Road [1]. A ship’s COLREGS compliance may now be evaluating these categories of COLREGS compliance, a first-iteration
tested, evaluated, and validated using only the vehicle track “road test” for protocol-constrained collision avoidance is proposed for
certifying autonomous marine vehicles to operate at sea in the vicinity
data of each encounter. To allow autonomy designers to more of human-present vessels.
objectively claim compliance with the written Rules and
maritime customs, this paper establishes and preliminarily
develops a COLREGS-based collision avoidance “road test”
II. COLREGS C OMPLIANCE IN THE L ITERATURE
for autonomous marine vehicles. Figure 1 demonstrates
example protocol compliance evaluation during on-water Collision avoidance protocols are prevalent in many phys-
experimentation with both autonomous and human-operated ical domains where explicit negotiation or communication is
vessels. either impractical or infeasible. In common practice, these
protocols are often communicated simply as having “right
*This work was supported by the U.S. Office of Naval Research (Code of way.” In ground transit, drivers are taught to yield to
33: Robert Brizzolara; Code 311: Behzad Kamgar-Parsi and Don Wagner) the driver on the right when arriving simultaneously at an
and Battelle (Mike Mellott). Portions of this paper first appeared in [1].
1 Kyle Woerner, Michael Benjamin, Michael Novitzky, and John intersection with stop signs [3]. Airplanes use the Rules of
Leonard are members of the Department of Mechanical Engineering the Air to determine right of way and appropriate maneuvers
and the Computer Science and Artificial Intelligence Laboratory, Mas- when not under active control of an air traffic controller [4].
sachusetts Institute of Technology, 77 Massachusetts Avenue, Cam-
bridge, MA 02139, USA k w@mit.edu, mikerb@mit.edu, Surface vessels similarly abide by the COLREGS to deter-
novitzky@mit.edu, jleonard@mit.edu mine right of way and appropriate maneuvers without ex-

978-1-5090-1537-5/16/$31.00 ©2016 IEEE

plicit communication [2]. Special rules within each protocol Case law and common practice greatly influence the re-
have evolved from real-world feedback; one such example is quirements of COLREGS despite not being found anywhere
the traffic separation schemes of COLREGS when entering within the written rules. Examples of on-water collisions
or exiting a harbor [5], [6]. While the Rules of the Air and case law provide relevant insight into nuances of the
and COLREGS are largely similar, differences in the phys- COLREGS and their evolution over the years. Areas for
ical domains manifest as differences between the collision increased scrutiny in autonomous collision avoidance solu-
avoidance protocol requirements such as maintaining altitude tions can be derived from problematic past encounters of
separation. human ship drivers. The intentional vagueness of the COL-
Collision avoidance using COLREGS has been incorpo- REGS including their underlying meaning as derived from
rated on autonomous vessels using various approaches since the evolution of protocol-constrained collision avoidance
first introduced in [7]. Throughout maritime literature dis- in maritime environments, analysis of real-world examples,
cussing COLREGS, the term “compliance” arises in varying critiques of experienced mariners, and relevant rulings from
context and meaning. Compliance, however, lacks objectivity Courts of Admiralty are presented in detail in [5], [6], [12],
partly due to the inherent vagueness of the COLREGS [15], [16].
and partly due to the varying scope of many solutions. A further complicating factor results from the disconnect
This intentional vagueness allows the human operator liberty between experienced mariners and autonomous designers.
to interpret the vast array of complex collision avoidance Few designers of marine autonomous collision avoidance
scenarios without being overly restricted from a common algorithms have demonstrated significant experience using
sense yet safe approach. COLREGS in open ocean navigation for non-academic
Power-driven collision avoidance implementations of purposes or incorporated an experienced operator into the
COLREGS (Rules 13-18) dominate the COLREGS-related algorithm design and testing process.
literature. Other non-collision avoidance rules of COLREGS
arise as claiming compliance within the literature when III. A PPROACH
discussing light configurations [8]. The varying scope of
what authors claim as compliant largely depends on the In order to certify autonomous vessels and their collision
scope of interest of a particular researcher. The notion of avoidance algorithms for use outside of a dedicated testing
compliance, however, should be amplified with the applicable environment, this paper proposes an examination framework
scope of the COLREGS. comprised of a comprehensive scope of quantifiable metrics
Testing to date in the literature fails to demonstrate what of performance. Evaluators may therefore assess the baseline
the term compliance means in any quantifiable fashion for level of knowledge, skills, and practical demonstrations nec-
the collision avoidance section of the rules. Several authors essary to certify an autonomously operated vessel to safely
claim compliance with these protocols without specifying operate in the vicinity of other vessels. This is similar to a
the degree or scope of compliance [9], [10], [11]. In [9], the human obtaining their drivers license for operating a car.
head-on rule was shown to appropriately eliminate all turns Proposed evaluation categories include general rules, con-
to port. It did not, however, appear to prefer courses that were duct of vessels, responsibilities in sight, restricted visibility,
“readily apparent” (COLREGS Rule 8) when finding a turn lights and shapes, sound and light signals, inter-vehicle
to starboard. Case law defines apparent course maneuvers to communications, and cumulative performance [1]. To be
consist of a minimum of 35◦ turn while common practice compliant with COLREGS, a satisfactory level of perfor-
often requires no less than 30◦ of heading change [5], [12], mance must be met across each category of evaluation. Local
[13], [14]. Courts have found that head-on maneuvers with customs may be achieved by using a collision avoidance
insufficient turns (i.e., not readily apparent) are in fact non- protocol evaluation library consisting of tunable penalty and
compliant and, when a collision occurs, partly to blame. With reward functions for each rule. A tunable collision avoidance
velocity vector cost functions that favor maintaining course protocol evaluation library was first presented in [1].
and speed [9], improper selection of costing weights may This paper proposes several categories of scope as a first
result in less than apparent course changes. Other authors pass means of grouping similar research and subsequent
consider breaches of COLREGS that “may be in the USV’s evaluation. With the development of metrics and evaluation
best interest” such as turning to port to avoid a collision when techniques within each category of scope, performance can
explicitly prohibited by the COLREGS [10]. Many authors be reliably demonstrated to a certifying body to a required
such as [11] simply claim COLREGS compliance without degree of satisfaction within each given category. A means
any quantification or definition of scope. would then exist to properly combine work of differing
The inconsistency of authors’ claims of COLREGS com- categories to produce more fully compliant solutions.
pliance likely results from a combination of three factors With the methods of this paper, conversations in future
including: literature can be more exact in their meaning of compliance
• the vagueness of the rules, in protocol-constrained collision avoidance research. The
• the unspecified scope of each author’s work, and scope of this road test framework includes:
• the tacit assumption that the COLREGS rules as written • the identification of various categories for collision
fully encompass all collision avoidance requirements. avoidance evaluation,
 • assignment of appropriate COLREGS rules into their Organization guidance, US Coast Guard’s local issuance of
respective categories, and inland-specific requirements, and other local guidelines can
• introduction of evaluation techniques for both human be adopted as appropriate to supplement or alter these testing
operators and autonomous collision avoidance algo- areas.
rithms based on admiralty case law and on-water ex- The categories proposed to define scope of work within the
perience. international COLREGS and compliance thereof are listed in
Table I and include:
IV. D ISCUSSION
• general requirements of vessels including Rules 1-3
A. Lexicon
• conduct of vessels in any condition of visibility includ-
Discussion of vehicles not completely controlled by a ing Rules 4-8
human present onboard in real-time often take various names • special cases for channels and separation schemes in-
with ambiguous, inconsistent titles such as “unmanned” or cluding Rules 9-10
“drone”. Vehicles generally have three major parameters that • conduct of two sailing vessels in sight of one another
may define their state for the purposes of discussion: human and operating under Rule 12
presence, element of control, and physical domain. • general vessel encounters including conduct of vessels
Human presence defines whether humans are onboard in sight of one another and operating under Rules 13-17
regardless of their contribution, if any, to the vehicle’s • responsibilities of vessels in sight of one another as
control. Human presence represents a binary state that may exhibited in Rules 11 and 18
change over time – even during a single underway in the • conduct of vessels in restricted visibility under Rule 19
case of boarding a pilot for a return to port. The element of • lights and shapes required of vessels under Rules 20-31
control is defined on a spectrum of human controlled to fully • sound and light signals required of vessels under
autonomous where it is understood that this is often neither Rules 32-37
a discrete nor permanent state. • inter-vehicle communications to include sending, re-
Finally, the physical domain specifies the type of environ- ceiving, interpreting, and appropriately acting on mes-
ment in which the vehicles operate (e.g., sea surface, sub- sages to/from other vessels or third parties (e.g., USCG
marine, aerial, ground, etc.). For the purposes of COLREGS district)
road tests, this paper assumes vessels operating on the sea • cumulative performance of the above categories to en-
surface with or without human presence. While developed sure a satisfactory holistic approach to safe navigation
for the purpose of evaluating autonomous vessels of varying and collision avoidance
degrees of the element of control, the road test concept may
be applied to human operators as well – whether present on TABLE I. Categories of Scope for COLREGS Compliance Evaluation
the vessel or operating remotely.
I General Rules (Rules 1-3)
B. Categories for COLREGS Scope II General Conduct of Vessels (Rules 4-8)
III Special Traffic Schemes (Rules 9-10)
Collision avoidance compliance in the most general sense IV Sailing in Sight of Another Sailing Vessel (Rule 12)
involves maneuvering one’s vessel to properly interact with V Vessel Encounters in Sight of One Another (Rules 13-17)
VI Responsibilities in Sight of One Another (Rules 11, 18)
a contact for a given initial geometry. In domains that re- VII Restricted Visibility (Rule 19)
quire a protocol-based solution, specific vehicles assume the VIII Lights and Shapes (Rules 20-31)
roles of stand-on or give-way depending on their geometric, IX Sound and Light Signals (Rules 32-37)
X Inter-vehicle Communications
propulsion, and operating conditions relative to the other. XI Cumulative Performance Including Local Customs
A stand-on vessel is generally required to maintain her
course and speed consistent with reasonable navigational
requirements as though the contact were not present (e.g., Rule categorization allows one designer to claim compli-
slowing to board a pilot) [5], [12], [16]. The give-way ance within one or more categories (for example, maneuver-
vessel similarly yields right of way to the stand-on vessel. ing requirements of power driven vessels) while deferring
The geometric, propulsion, and operating conditions used evaluation of rules related to other areas (for example, sound
to determine stand-on and give-way assignments include identification and response) to other authors. By defining the
relative geometry (e.g., being overtaken, holding a vessel scope of applicable rules and demonstrating quantifiable lev-
off her starboard side, etc.), propulsion state (e.g., under els of compliance within each category of rules, autonomous
sail, power-driven, etc.), and operating state (e.g., engaged collision avoidance algorithm designers can more exactly
in fishing, restricted in ability to maneuver, etc.). articulate their contributions to the literature. It should be
To counter the disparity between compliance claims and noted that evaluation within the scope of one category may
actual performance, scope may be compartmentalized to rely on compliance of another category to some degree. For
allow quantification and certification within similar areas of example, because Category II includes maintaining a lookout,
requirements. COLREGS rules may therefore be separated determining safe speed, determining risk of collision, and
into categories to allow a vehicle to demonstrate compliance taking action to avoid a collision, it heavily influences
of appropriate COLREGS subsets. International Maritime evaluation of Categories III-VII.
 V. ROAD T EST F RAMEWORK C. Multi-Vehicle, Multi-Rule Test Scenarios
In order to certify autonomous collision avoidance al- Multi-vehicle, multi-rule scenarios create situations where
gorithms for on-water use outside of a testing environ- precedence must be determined by the collision avoidance
ment, vehicles should complete a road test comprised of algorithms. In some situations, an action for one vehicle
a comprehensive scope of examination using quantifiable of higher precedence may be an appropriate maneuver for
metrics of performance. To be compliant with the appropriate the other vehicle as well. In other situations, action for one
protocol rule set, a satisfactory level of performance must vehicle may directly conflict with the preferred behavior for
be met across each category of evaluation as defined in the second if it were encountered alone. An appropriate
Table I. Differing degrees of road tests may be possible for decision, however, is required to appropriately determine
various operational levels of certification. A nominal road which vehicle, if any, has the higher precedence or greater
test consists of satisfactorily obtaining passing scores for all risk of collision. Maneuvering for a head-on vehicle while
applicable categories of scope identified in Table I as set by ownship is simultaneously crossing stand-on for the second
the appropriate certification authority. contact is an example of precedence, as shown in Figure 2.
Other scenarios require cautious action such as when
A. Collision Avoidance Algorithm Examination overtaking an already-overtaking vessel. Care must be taken
Sections of the road test that require involvement of to ensure no hindrance of either of the other two vessels,
the collision avoidance algorithms should include detailed especially in the choice of overtaking side. Figure 3 demon-
examination of the following: strates an example nested overtaking situation.
• determination of obligations and governing rule(s) for In situations where ownship may be overtaking another
both ownship and the contact for various geometric vessel and find itself simultaneously in a crossing situation,
configurations at time of visual or radar detection a decision must be made as to whether to safely continue
• determination of appropriate precedence (e.g., sailing the overtaking maneuver or to break off and maneuver for
vessels, etc.) the crossing contact. This largely depends on collision risk,
• determination of role hierarchy when nested rules ap- initial geometry, and subsequent assignment of stand-on
ply (e.g., ownship overtaking a vessel that is already or give-way responsibilities. Examples for these scenarios
overtaking a third (slower) vessel) include Figures 4 and 5.
• identification of redundant contacts and performing con- Crossing while simultaneously head-on presents a signifi-
tact fusion cant challenge to elementary collision avoidance algorithms.
• releasing priority for contacts past CPA and opening This geometry checks the collision avoidance algorithm
or otherwise no longer posing a risk of collision (e.g., to ensure that a stand-on vessel will indeed relax itself
agreement or signalling of right-of-way passage) of maintaining course and speed when prudent navigation
• verification of unsaturated CPU loading for up to N - concerns warrant deviation. Example canonical situations of
simultaneous contacts, where N is set by the certifica- crossing while simultaneously head-on are shown in Figure 6
tion authority (give-way) and Figure 7 (stand-on).
A situation that accounts for both contact lumping and
B. Attributes of the Road Test
To ensure a sufficiently robust algorithm for on-water in-
teractions with other vessels, the road test should incorporate
the following attributes:
• non-canonical and reasonably exhaustive geometries
(thorough geometric testing approaches such as the
iterative geometric testing of [17] are encouraged)
• multi-vehicle, multi-rule scenarios
• conflicting simultaneous collision avoidance rules
• conflicting mission, rule, and navigational priorities
• various initial ownship and contact speeds
• non-compliant contacts (delayed / insufficient action or
entirely disregarding Rules)
• over-constrained encounters
• robot-robot and robot-human interactions
1
• exercise of a default safe mode Crossing - Head-on
• statistical significance of testing encounters
• broadcasting appropriate signals (including distress) to
other vessels or shoreside entities as necessary Fig. 2. When a vehicle finds itself crossing as the stand-
on vessel while also head-on with a second contact, it must
1 A default safe mode might represent a turn to starboard or taking all maneuver for the head-on vessel in a safe and prudent
way off the vessel and sounding an appropriate signal. fashion. Ownship is the dark blue vessel.
 Nested Overtaking Crossing-Overtaking

Fig. 3. Nested overtaking vessels must decide how best to Fig. 5. Nested crossing-overtaking vessels must account for
safely pass without interfering with the already-maneuvering an approaching give-way vessel while likely continuing the
vehicles. Ownship is the dark blue vessel. overtaking of the slower vessel ahead. Ownship is the dark
blue vessel.

Crossing-Overtaking
Crossing - Head-on

Fig. 4. Nested crossing-overtaking vessels must account for

an approaching stand-on vessel while aborting or safely con- Fig. 6. When a vehicle finds itself crossing as the give-
tinuing the overtaking of the slower vessel ahead. Ownship way vessel while also head-on with a second contact, it must
is the dark blue vessel. maneuver such as to take appropriate head-on action and
properly give-way in a safe and prudent fashion. Ownship is
the dark blue vessel.

ensuring a robust response to Rule 14 is the multi-contact

head-on scenario such as the one of Figure 8. An example of This constitutes the first known road test framework for
a robust response to Rule 14 includes a vessel taking head- autonomous collision avoidance and would allow consistent
on action even if the contact is initially slightly starboard of testing and certification of algorithms with configurable
track [5]. metrics prior to fielding. Differing degrees of road tests
may be possible for various levels of certification for op-
VI. C ONCLUSION
eration including initial testing, initial fielding, supervised
The International Maritime Organization and other govern- operations, and unrestricted operations. Performance areas,
ing bodies may choose to include these metrics as a means attributes, and a general framework important for a collision
to inform both regulation and policy in maritime collision avoidance road test are presented. Evaluation of humans to
avoidance protocols. the same standards as robots naturally extends from this
Further discussion and research is needed to fully incor- paper including standardization of practical examinations
porate local customs and laws within COLREGS (including throughout the maritime community.
U.S. Inland Rules), alternative protocols, and special arrange- While the motivation of these techniques applies to im-
ments such as those made by bridge-to-bridge radio. provement of autonomous marine collision avoidance under
 CPA - closest point of approach; point of global min. range
USV - uninhabited/unmanned surface vessel
R EFERENCES
[1] K. Woerner, “Multi-Contact Protocol-Constrained Collision Avoidance
for Autonomous Marine Vehicles,” Ph.D. dissertation, Massachusetts
Institute of Technology, 2016.
[2] United States Coast Guard, Navigation Rules, International-Inland,
ser. 1977-19. United States Department of Transportation, 1999.
[3] Commonwealth of Massachusetts, Right-of-way at Intersecting Ways;
Turning on Red Signals, Part 1 Title XIV Chapter 89 Section 8.
[4] International Civil Aviation Organization, Rules of the Air, Interna-
tional Civil Aviation Organization, Ed. Convention on International
Civil Aviation, 2005.
[5] A. N. Cockcroft and J. N. F. Lameijer, Guide to the Collision
Avoidance Rules. Butterworth-Heinemann, 2012.
[6] D. Thomas, The Fatal Flaw: Collision at Sea and the Failure of the
Crossing - Head-on Rules. Carmarthenshire, United Kingdom: Phaiacia, 2001.
[7] M. R. Benjamin, J. J. Leonard, J. A. Curcio, and P. M. Newman, “A
Method for Protocol-based Collision Avoidance Between Autonomous
Marine Surface Craft,” Journal of Field Robotics, vol. 23, no. 5, pp.
Fig. 7. When a vehicle finds itself crossing as the stand- 333 – 346, 2006.
on vessel while also head-on with a second contact, it must [8] M. Fisher, “Evaluation of The Vertical Sector Light Requirements for
maneuver such as to take appropriate action for the head-on Unmanned Barges; Final Rept,” US Coast Guard, Tech. Rep., 1991.
vessel. Ownship is the dark blue vessel. [9] Y. Kuwata, M. Wolf, D. Zarzhitsky, and T. Huntsberger, “Safe
Maritime Autonomous Navigation With COLREGS, Using Velocity
Obstacles,” Oceanic Engineering, IEEE Journal of, vol. 39, no. 1, pp.
110–119, Jan 2014.
[10] B. C. Shah, “Trajectory Planning with Adaptive Control Primitives
for Autonomous Surface Vehicles Operating in Congested Civilian
Traffic,” IEEE International Conference on Intelligent Robots and
Systems, 2014.
[11] K. L. Woerner, “COLREGS-Compliant Autonomous Collision Avoid-
ance Using Multi-Objective Optimization with Interval Program-
ming,” Master’s thesis, Massachusetts Institute of Technology, 77
Massachusetts Avenue, Cambridge, MA 02139, 6 2014.
[12] C. Allen, Farwell’s Rules of the Nautical Road, ser. Blue and gold
professional library series. Naval Institute Press, 2005.
[13] United States Coast Guard, “Standing Orders for The Officers of The
Deck (Freedom of Information Act),” USCGC HEALY Instruction
M1603.1C, 6 2006.
[14] United States Navy, “Standing Orders for USS Greeneville (Freedom
of Information Act),” USS Greeneville Instruction C3120.25, 12 1999.
[15] A. Henderson, “Murky Waters: The Legal Status of Unmanned Un-
dersea Vehicles,” Naval Law Review, pp. 55–72, 2006.
Head-on - Head-on [16] J. Zhao, “The Legal Interpretation of Keeping Course and Speed,”
Journal of Maritime Law & Commerce, vol. 41, p. 85, 2010.
[17] K. L. Woerner and M. R. Benjamin, “Autonomous Collision Avoidance
Tradespace Analysis for High-Speed Vessels,” in 13th International
Fig. 8. Head-on encounters with a risk of collision must Conference on Fast Sea Transportation. Society of Naval Architects
maneuver such as to pass port-to-port even when a contact and Marine Engineers, 2015.
may initially be starboard of track at time of detection.
Ownship is the dark blue vessel.

the protocol constraints of COLREGS, the concepts for

protocol evaluation extend naturally to alternative protocols
and other physical domains with protocol constraints such as
the Rules of the Air.

A. Abbreviations and Acronyms

COLREGS - international rules as formalized at the Con-
vention on the International Rules for Preventing Collisions
at Sea, developed by the International Maritime Organiza-
tion, and ratified as an international treaty by Congress.
These rules were further formalized by the U.S. International
Navigational Rules Act of 1977 [2], and are sometimes
referred to as the Collision Regulations outside the United
States.