HANDOUT

Defensive flying for pilots: An

introduction to Threat and Error
Management (TEM)

The easiest way to understand threat and error management (TEM) is to liken it to defensive
driving for a motorist. The purpose of defensive driving is not to teach people how to drive a
vehicle (e.g., how to shift a manual transmission) but to emphasise driving techniques that
people can use to minimise safety risks (e.g., techniques to control rear-wheel skids). Similarly,
TEM does not teach pilots how to technically fly an airplane; instead, it promotes a proactive
philosophy and provides techniques for maximising safety margins despite the complexity of
one’s flying environment. In this sense, TEM training can be framed as defensive flying for
pilots.

TEM proposes that threats (such as adverse weather), errors (such as a pilot selecting a wrong
automation mode), and undesired aircraft states (such as an altitude deviation) are everyday
events that flight crews must manage to maintain safety. Therefore, flight crews that
successfully manage these events regardless of occurrence are assumed to increase their
potential for maintaining adequate safety margins. It is this notion that provides the overarching
objective of TEM—to provide the best possible support for flight crews in managing threats,
errors, and undesired aircraft states.

Origin and development of TEM

The origin of TEM is inextricably tied to the origin of Line Operations Safety Audits (LOSA). It
began with a simple question: “Do the concepts taught in training transfer to normal, everyday
flight operations?”

The question prompted a partnership between The University of Texas Human Factors
Research Project (UT) and Delta Airlines in 1994 to develop a line audit methodology utilising
jump-seat observations on regularly scheduled flights. All parties realised that in order for the
audit to work, i.e., to really see what happened on the line, there had to be a guarantee of
confidentiality with no regulatory or organisational jeopardy for the crews that were observed.
Crews had to believe there would be no individual repercussions; otherwise, they would revert
to their best “angel performance” when being observed and the audit would uncover nothing
more than what was learned from line check or training data.

The first observation form was designed by the University of Texas (UT) researchers to
evaluate Crew Resource Management (CRM) behaviours. The form was then expanded to

© Global Air Training Limited 2018 1.1

HANDOUT

address error and its management. As well as type of error committed, the form prompted
observers to note who caused the error, the response to the error (i.e., whether the error was
detected and by whom), and the outcome of the error. Knowing an error occurred without really
knowing the conditions under which it occurred seemed to tell only part of the story. Hence, the
researchers developed and included the concepts of threat and threat management in the
observation form to capture the full operational complexity of a flight.

The first full TEM-based LOSA was conducted at Continental Airlines in 1996. Data from the
observation forms was aggregated to develop an airline profile. As well as the original CRM
indicators, such as leadership, communication, and monitoring/cross-checking, the TEM
organisational profile highlighted the most frequent threats, threats that were well-managed
versus more problematic threats (i.e., those that were mismanaged at higher rates than other
threats), the most common errors, the least versus more problematic errors, and the rate of
Undesired Aircraft States, including unstable approaches. Among other things, the airline
learned that it had issues with its checklists. It also realised there were no clear guidelines on
when to execute a missed approach, which could explain the rate of unstable approaches. With
a data-driven report that highlighted operational strengths and weaknesses, the airline set up
cross-departmental committees from Flight Operations, Ground Operations, Training, and the
Safety Department to work on solutions. The company also instigated a one-day TEM training
course for all its pilots.

Trainers introduced the concepts of Threat and Error and then debriefed the LOSA findings. As
a result, pilots were able to see a different perspective of safety performance at their airline as
reflected in organisational threat and error prevalence and management rates. The pilots
responded positively, analysing the data for reasons, and using what they learned to
proactively enhance their own performance. Using the 1996 LOSA results as a baseline,
Continental conducted a follow-up LOSA in 2000.

To quote Captain Don Gunther, Senior Director of Safety & Regulatory Compliance at
Continental Airlines:

“The 2000 LOSA, when compared to the results of 1996, showed the pilots had not
only accepted the principles of error management but incorporated them into
everyday operations. LOSA 2000 showed a sizeable improvement in the areas of
checklist usage, a 70 percent reduction in non-conforming approaches (i.e., those
not meeting stabilized approach criteria), and an increase in overall crew
performance. It could be said that Continental had taken a turn in the right
direction.”

© Global Air Training Limited 2018 1.2

HANDOUT

TEM: Definitions, examples, and quizzes

The Threat and Error Management (TEM) framework focuses simultaneously on the operating
environment and the humans working in that environment. Because the framework captures
performance in its “natural” or normal operating context, the resulting description is realistic,
dynamic, and holistic. Because the TEM taxonomy can also quantify the specifics of the
environment and the effectiveness of performance in that environment, the results are also
highly diagnostic.

Threats and their management

Pilots have to manage various complexities in the operating environment on a typical day of
flying. In TEM, such complexities are known as threats.

Threat definition
Threats are defined as events or errors that:

▪ Occur outside the influence of the flight crew (i.e., not caused by the crew).

▪ Increase the operational complexity of a flight.

▪ Require crew attention and management if safety margins are to be

maintained.

Using this definition, a threat can be; high terrain, icing conditions, an aircraft malfunction (e.g.,
inoperative thrust reverser), or other people’s errors, such as an inaccurate recording of a fuel
load by a dispatcher. All these events occur independently of the flight crew, yet they add to the
crew’s workload and need to be managed. Sometimes they can be managed discreetly and
sometimes they interact with one another, further complicating the necessary management. In
commercial airlines, threats can be divided into two categories:

▪ Environmental threats, which are outside the airline’s direct control, such as
weather and ATC.

▪ Airline threats, which originate within flight operations, such as aircraft

malfunctions and ground problems.

The table below shows the various threat types with examples.

© Global Air Training Limited 2018 1.3

HANDOUT

Threat types with examples

Environmental Threats Examples

Adverse Weather Thunderstorms, turbulence, poor visibility, wind shear, icing, IMC

Poor signage, faint markings, runway/taxiway closures, INOP

Airport
navigational aids, poor braking action, contaminated runways

Tough to meet clearances/restrictions, re-routes, language

ATC
difficulties, controller errors

Environmental Operational Pressure Terrain, traffic, TCAS, TA/RA, radio congestion

Airline Threats Examples

Systems, engines, flight controls, automation anomalies or

Aircraft malfunctions, MEL items with operational implications, other aircraft
threats requiring crew attention

On time performance pressures, delays, late-arriving aircraft or flight

Airline Operational Pressure
crew

Cabin Cabin events, flight attendant errors, distractions, interruptions

Load sheet errors, crew scheduling events, late paperwork, changes

Dispatch/Paperwork
or errors

Aircraft loading events, fuelling errors, agent interruptions, improper

Ground/Ramp
ground support, de-icing

Aircraft repairs on the ground, maintenance log problems,

Ground Maintenance
maintenance errors

Manuals/Charts Missing information or documentation errors

Threat management can be broadly defined as how crews anticipate and/or respond to threats.
A mismanaged threat is defined as a threat that is linked to or induces flight crew error. Some
of the common tools and techniques used in commercial aviation to manage threats and
prevent crew errors include reading weather advisories, turning weather radar on early,
thorough walk-rounds during pre-departure, correct use of procedures to diagnose unexpected
aircraft malfunctions, briefing an alternate runway in case of a late runway change, briefing
cabin crew as to acceptable times and reasons for interruptions, and loading extra fuel when
the destination airport is in question due to poor weather or restricted access.

Just how common are threats and when do they occur? Take the quiz below to find out.

© Global Air Training Limited 2018 1.4

HANDOUT

Threat Management Quiz

Test your knowledge of threats and their management by circling your best guess to the
following questions about findings from the LOSA Archive of more than 4500 observations
across 25 airlines.

Correct answers with discussion will be provided at the end of the quiz.

1) On average, how many threats per flight (regularly scheduled, normal operations) are
encountered by flight crews in the LOSA Archive?

a) One threat every 2-3 flights

b) One threat per flight
c) 1-3 threats per flight
d) 4-6 threats per flight

2) In what phase of flight do most threats occur in the LOSA Archive?

a) Pre-departure/Taxi-out
b) Takeoff/Climb
c) Cruise
d) Descent/Approach/Land

3) What are the most frequently encountered threats by flight crews in the LOSA Archive?

a) Adverse weather (e.g., thunderstorms)

b) ATC (e.g., challenging clearances)
c) Aircraft (e.g., malfunctions/anomalies)
d) Airport (e.g., poor signage/construction)

4) What percent of threats are successfully managed by flight crews in the LOSA Archive?
(i.e., percentage of threats not contributing to a flight crew error)

a) 95-100%
b) 85-95%
c) 75-85%
d) Less than 75%

5) Of all threats encountered by flight crews in the LOSA Archive, which are the most
problematic?

a) Adverse weather (e.g., thunderstorms)

b) ATC (e.g., challenging clearances)
c) Aircraft (e.g., malfunctions/anomalies)
d) Airport (e.g., poor signage/construction)

© Global Air Training Limited 2018 1.5

HANDOUT

Threat management quiz - Answers and discussion

1) The correct answer is (D). Based on the last 25 LOSAs (over 4500 flights in total) in the
LOSA Archive, the typical flight (regularly scheduled, normal operations) encounters an
average of 4.2 threats per flight. Of those, three are likely to be Environmental threats
and one is likely to be an Airline threat. Only 3% of flights encounter no threats
whatsoever, while 17% of flights encounter seven or more threats per flight. In other
words, multiple threats are the standard and should be considered as such in every
flight.

2) The correct answer is (A). Overall, about 40% of all threats occur during Pre-
departure/Taxi-out and 30% occur during Descent/Approach/Land. Different types of
threats are more prevalent during different phases of flight. For Environmental threats
(weather, ATC, terrain, traffic, airport conditions), the busiest phase of flight is
Descent/Approach/Land, while for Airline threats, the busiest phase is Pre-
departure/Taxi-out. In percentage terms, 43% of all Environmental threats occur during
Descent/Approach/Land, while 73% of all Airline threats occur during Pre-
departure/Taxi-out.

3) The correct answer is (A or B). With 4500 flights having an average of 4.2 threats per
flight, there are 19,000 logged threats in the LOSA Archive. So, which are the most
common? Actually, Adverse Weather and ATC both account for about one quarter of all
observed threats, followed by Aircraft Threats (about 13% of all observed threats) and
Airport Conditions (about 7% of all observed threats).

4) The correct answer is (B). 85-95% of all threats are successfully managed. The
average across the Archive is 90%. Put another way, about one-tenth of all threats are
mismanaged by the crews, leading to some form of crew error.

5) The correct answer is (B). Mismanagement rates are actually very close for the top
three “offenders”. Thirteen percent of Aircraft threats, 12% of ATC threats, and 11% of
Adverse Weather threats are typically mismanaged.
However, when you combine these mismanagement rates with the frequency with which
different threats occur, ATC threats emerge as the most problematic threat. In particular,
challenging clearances and late changes from ATC are the most problematic of all
threats for flight crews.

© Global Air Training Limited 2018 1.6

HANDOUT

Errors and their management

From the TEM perspective, error is a crew action or inaction that leads to a deviation from crew
or organisational intentions or expectations. Put simply, threats come “at” the crew, while errors
come “from” the crew. Flight crew errors can be the result of a momentary slip or lapse, or
induced by an expected or unexpected threat. For example, a late runway change might induce
a procedural shortcut that results in further error, just as a gate agent interruption could distract
the flight crew from completing a checklist, causing them to miss an incorrect flaps setting for
takeoff. Other errors are more deliberate. Known as intentional noncompliance errors in the
TEM taxonomy, these errors are often proven shortcuts used by flight crews to increase
operational efficiency even though they are in violation of Standard Operating Procedures. High
rates of noncompliance at an airline can often indicate systemic over-proceduralisation.

Error definition
Errors are defined as flight crew actions or inactions that:

▪ Lead to a deviation from crew or organisational intentions or expectations.

▪ Reduce safety margins.

▪ Increase the probability of adverse operational events on the ground or

during flight.

Flight crew errors can be divided into three types: aircraft handling, procedural and
communication errors. Aircraft handling errors are those deviations associated with the
direction, speed and configuration of the aircraft. They can involve automation errors, such as
dialling an incorrect altitude, or hand-flying errors, such as getting too fast and high during an
approach. Procedural errors are flight crew deviations from regulations, flight manual
requirements or airline standard operating procedures. Lastly, communication errors involve a
miscommunication between the pilots, or between the crew and external agents such as ATC
controllers, flight attendants, and ground personnel.

The following table shows the various error types with examples:

© Global Air Training Limited 2018 1.7

HANDOUT

Aircraft Handling Errors Examples

Incorrect altitude, speed, heading, autothrottle settings, mode executed

Automation
or entries

Incorrect flaps, speedbrake, autobrake, thrust reverser or power

Flight Control
settings

Attempting to turn down the wrong runway/taxiway. Missed

Ground Navigation
runway/taxiway/gate

Hand flying vertical, lateral or speed deviations. Missed runway/taxiway

Manual Flying
failure to hold short, or taxi above speed limit

Systems/Radio/Instruments Incorrect pack, altimeter, fuel switch, or radio frequency settings

Procedural Errors Examples

Missed items in the brief, omitted departure, takeoff, approach, or

Briefings
handover briefing

Callouts Omitted takeoff, descent, or approach callouts

Performed checklist from memory or omitted checklist.

Checklist Missed items, wrong challenge and response, performed late or at
wrong time

Wrong weight and balance, fuel information, ATIS, or clearance

Documentation
recorded. Misinterpreted items on paperwork

Pilot Flying (PF)/Pilot Not Flying (PNF) PF makes own automation changes, PNF doing PF duties, PF doing
Duty PNF duties

SOP Cross-verification Intentional and unintentional failure to cross-verify automation inputs

Other deviations from government regulations, flight manual

Other Procedural
requirements or standard operating procedures

Communication Errors Examples

Missed calls, misinterpretation of instructions, or incorrect read-backs to

Crew to External
ATC. Wrong clearance, taxiway, gate or runway communicated

Pilot to Pilot Within-crew miscommunication or misinterpretation

© Global Air Training Limited 2018 1.8

HANDOUT

Error management is now recognised as an inevitable part of learning, adaptation, and skill
maintenance; hence, a primary driving force behind TEM is to understand what types of errors
are made under what circumstances (i.e., the presence or absence of which threats) and how
crews respond in those situations. For example, do crews detect and recover the error quickly,
do they acknowledge the error but do nothing, perhaps because they believe it is
inconsequential or will be trapped later, or do they only “see” the error when it escalates to a
more serious undesired aircraft state? This is the heart of error management: detecting and
correcting errors. However, approximately 45% of the observed errors in the LOSA Archive
were errors that went undetected or were not responded to by the flight crew, which gives
credence to an important point for effective error management: an error that is not detected
cannot be managed.

An error that is detected and effectively managed has no adverse impact on the flight. On the
other hand, a mismanaged error reduces safety margins by linking to or inducing additional
error or an undesired aircraft state.

Just how common are mismanaged errors and when do they occur? The LOSA Archive
provides some insight, as shown in the quiz that follows:

Error management quiz

Test your knowledge of flight crew errors and their management by circling your best guess to
the following questions. As with the Threat Management Quiz, correct answers with discussion
will be provided at the end of the quiz.

1) Of flights in the LOSA Archive, how common is flight crew error?

a) Approximately 5% of flights have some form of observable crew error

b) Approximately 50% of flights have some form of observable crew error
c) Approximately 80% of flights have some form of observable crew error
d) All LOSA flights (100%) have at least one observable crew error

2) In what phase of flight do most flight crew errors occur in the LOSA Archive? When do the
mismanaged errors occur? (Hint: The answer is the same phase of flight for both questions)

a) Predeparture/Taxi-out
b) Takeoff/Climb
c) Descent/Approach/Land
d) Taxi-in/Park

© Global Air Training Limited 2018 1.9

HANDOUT

3) What are the most frequently committed flight crew errors in the LOSA Archive?

a) Aircraft Handling (e.g., wrong automation setting)

b) Procedural (e.g., omitted callout)
c) Communication (e.g., incorrect ATC readback)

4) What are the most common procedural errors observed in the LOSA Archive?

a) Briefing
b) SOP Cross-verification
c) Callout
d) Checklist

5) What percentage of errors are mismanaged by flight crews in the LOSA Archive (i.e.,
percentage of errors linking to an additional error or undesired aircraft state)

a) 20-30%
b) 30-40%
c) 40-50%
d) More than 50%

6) What are the most frequently mismanaged flight crew errors in the LOSA Archive?

a) Manual Handling/Flight Control

b) Automation
c) System/Instrument/Radio
d) Checklist

© Global Air Training Limited 2018 1.10

HANDOUT

Error management quiz answers and discussion

1) The correct answer is (C). Based on the last 25 LOSAs (over 4500 flights in total) in the
LOSA Archive, about 80% of flights have one or more errors – the average is about three
errors per flight. Twenty percent of flights have no observable errors.

2) The correct answer is (C). The busiest phase of flight for errors is Descent/Approach/Land.
About 40% of all observed errors occur during this phase. Another 30% of errors occur
during Predeparture/Taxi-out when crews are preparing the flight. If you look at the sub-set
of errors that are mismanaged, then the rate for Descent/Approach/Land jumps to 55%.
Therefore, the most problematic phase of flight where more errors, and more mismanaged
errors, are likely to occur is Descent/Approach/Land. This likely makes intuitive sense—
errors on the ground aren’t as difficult to manage as errors coming down.

3) The correct answer is (B). About one-half of all observed errors are Procedural errors, one-
third are Aircraft Handling, and one-sixth are Communication errors. However, this ratio
changes dramatically for mismanaged errors. Procedural errors make up half of all errors,
but a little less than one-quarter of the mismanaged errors.
Three-quarters of all mismanaged errors are Aircraft Handling errors, with Communication
errors comprising the remaining few percent.

4) The correct answer is (D). Checklist errors are the most common procedural error, followed
closely by Callout and SOP cross-verification errors. Briefing errors are less common.

5) The correct answer is (A). About 25% of all errors are mismanaged—6% of all errors lead to
additional error and 19% result directly in an undesired aircraft state.

6) The correct answer is (A). Manual handling/flight control errors make up 36% of all
mismanaged errors.
Automation and System/Instrument/Radio errors each make up 16% of the mismanaged
errors. Checklist errors make up 5% of the mismanaged errors; Crew-ATC communication
errors make up 3% of the mismanaged errors.

© Global Air Training Limited 2018 1.11

HANDOUT

Undesired aircraft states and their management

Unfortunately, not all errors are well managed. Sometimes they lead to another error or a
safety-compromising event called an undesired aircraft state (UAS).

Undesired aircraft state definition

An undesired aircraft state (UAS) is defined as a position, speed, attitude, or configuration of an
aircraft that:

▪ results from flight crew error, actions, or inaction;

and
▪ clearly reduces safety margins.

In other words, a UAS is a safety-compromising state that results from ineffective error
management. Examples include unstable approaches, lateral deviations from track, firm
landings, and proceeding towards the wrong taxiway/runway. Events such as malfunctions or
ATC controller errors can also place the aircraft in a compromised position; however, in the
TEM taxonomy, these events are considered threats as they are not the result of actions by the
flight crew.

UAS types with examples

UAS Types Examples

Vertical, lateral or speed deviations

Unnecessary weather penetration
Aircraft Handling
Unstable approach
Long, floated, firm or off-centreline landings

Runway/taxiway incursions
Ground Navigation Wrong taxiway, ramp, gate, or hold spot
Taxi above speed limit

Incorrect Aircraft Configuration Automation, engine, flight control, systems, or weight/balance events

As with errors, UASs can be managed effectively, returning the aircraft to optimally safe flight,
or mismanaged, leading to an additional error, undesired aircraft state, or worse, an incident or
accident. The last quiz sheds light on the prevalence and mismanagement of undesired aircraft
states in the LOSA Archive.

© Global Air Training Limited 2018 1.12

HANDOUT

Undesired aircraft state management quiz

Test your knowledge of undesired aircraft states and their management by circling your best
guess to the following questions. As with the previous quizzes, correct answers with discussion
will be provided at the end of the quiz.

1) Of flights in the LOSA Archive, how common are undesired aircraft states (UAS)?

a) Less than 1% of flights have a UAS

b) 15% of flights have a UAS
c) 35% of flights have a UAS
d) 50% of flights have a UAS

2) What are the most frequent UASs observed in the LOSA Archive?

a) Incorrect systems configurations (e.g., wrong anti-ice setting in icing conditions)

b) Speed deviations
c) Lateral and vertical deviations
d) Incorrect automation configurations (e.g., wrong altitude dialled after cross-check)

3) How common are unstable approaches in the LOSA Archive and how often do they result in
a missed approach?

a) < 1% of flights have an unstable approach; of those, 95% result in a missed

approach
b) 5% of flights have an unstable approach; of those, 5% result in a missed approach
c) > 15% of flights have an unstable approach; of those, 50% result in a missed
approach
4) How many UASs in the LOSA Archive can be linked back, via mismanaged crew error, to a
mismanaged threat?

a) Virtually all UASs come about because of a threat that was mismanaged (95-100%)
b) About 70% of all UASs are linked to a mismanaged threat; the rest emerge from
“spontaneous” crew errors that were mismanaged (“spontaneous” = not linked to a
threat)
c) About 30% of all UASs are linked to a mismanaged threat.

© Global Air Training Limited 2018 1.13

HANDOUT

Undesired aircraft states quiz answers and discussion

1) The correct answer is (C). Despite being the safest form of transport, fully one-third of all
flights in the LOSA Archive have an undesired aircraft state. Numbers such as these remind
us there is still room for improvement!

2) The correct answer is (A). Almost 20% of all UASs involve an incorrect aircraft system
configuration (they occur on approximately 9% of flights). Speed deviations are next at
16%, followed by lateral/vertical deviations and incorrect automation configuration (each
comprises about 13% of all UASs). These UAS types each occur on approximately 7% of
flights.

3) The correct answer is (B). In regularly scheduled, normal operations, 5% of flights involve
an unstable approach. What is disconcerting is that only 5% of those unstable approaches
result in a go-around, meaning the vast majority of crews decide to continue with the
landing, even though they know they are not within specified parameters. Are they choosing
to continue the approach because of operational pressure (wanting to save time and fuel),
poor airmanship, or foolish bravado? Perhaps some of all three - what do you think?

4) The correct answer is (C). About 30% of all UASs occur as part of a chain of events that
starts with a threat that is not managed well and leads to a crew error, which in turn is
mismanaged to a UAS. An example would be an Airport Conditions threat such as poor or
faded signage (threat) that confuses the crew, leading them to turn down the wrong runway
(error), which results in a runway incursion (UAS).

TEM tools & techniques

The principles of TEM are not new to aviation. In fact, Orville and Wilbur Wright no doubt
practiced threat and error management when they took their first controlled flight with the
Wright Flyer in 1903. Since then, various tools and techniques have been developed over the
past century to help flight crews manage threats, errors, and undesired aircraft states.

Some tools—the “hard” safeguards—are associated with aircraft design, and include
automated systems, instrument displays, and aircraft warnings. The Traffic Collision Avoidance
System (TCAS), which provides flight crews with visual and audio warnings of nearby airplanes
to prevent midair collisions, is a good example of a “hard” TEM safeguard.

Even with the best designed equipment however, these “hard” safeguards are not enough to
ensure effective TEM performance.

Other tools—the “soft” safeguards—are very common in aviation (and other high-risk
industries). They include regulations, standard operating procedures, and checklists to direct
pilots and maintain equipment; and licensing standards, checks, and training to maintain

© Global Air Training Limited 2018 1.14

HANDOUT

proficiency. With the hard and soft safeguards in place, the last line of defence against threat,
error, and undesired aircraft states, is still, ultimately, the flight crew. Checklists only work if
flight crews use them; the autopilot only works when engaged in the correct mode. Therefore,
TEM tools work best when pilots adopt TEM techniques.

The TEM philosophy stresses three basic concepts: anticipation, recognition, and recovery. The
key to anticipation is accepting that while something is likely to go wrong, you can’t know
exactly what it will be or when it will happen. Hence, a chronic unease reinforces the vigilance
that is necessary in all safety-critical professions. Anticipation builds vigilance, and vigilance is
the key to recognising adverse events and error. Logically, recognition leads to recovery. In
some cases, particularly when an error escalates to an undesired aircraft state, recovering
adequate safety margins is the first line of action: recover first, analyse the causes later. For
example, a crew enters a Flight Management System (FMS) approach to runway 26L; however,
they mistakenly enter data for 26R. Furthermore, the error is not detected by the flight crew on
a SOP required cross-verification. Once the flight crew executes the incorrect entry and the
airplane starts flying on a profile to the wrong runway, the flight is considered to be in an
undesired aircraft state. At this point, the crew can either analyse what’s wrong with the
automation and fix the problem or save valuable time by simply disconnecting the autopilot and
hand-flying the approach to the correct runway.

The latter option is more effective from the TEM perspective because it focuses effort on
recovering from the undesired aircraft state rather than analysing its causes.

While “hard” and “soft” safeguards help support pilots to best anticipate, recognise and recover
from threats, errors, and undesired aircraft states, there is arguably no better way to manage
these events in multi-pilot cockpits than through effective crew coordination.

Many of the best practices advocated by Crew Resource Management (CRM) can be
considered TEM countermeasures.

▪ Planning countermeasures—planning, preparation, briefings, contingency

management—are essential for managing anticipated and unexpected
threats.

▪ Execution countermeasures—monitor/cross-check, taxiway/runway

management, workload and automation management—are essential for error
detection and error response.

▪ Review/Modify countermeasures—evaluation of plans, inquiry—are essential

for managing the changing conditions of a flight, such as undesired aircraft
states.

© Global Air Training Limited 2018 1.15

HANDOUT

Initial research in the LOSA Archive has supported links between TEM and CRM.

For example, crews that develop contingency management plans, such as proactively
discussing strategies for anticipated threats, tend to have fewer mismanaged threats; crews
that exhibit good monitoring and cross-checking usually commit fewer errors and have fewer
mismanaged errors; and finally, crews that exhibit strong leadership, inquiry and workload
management are typically observed to have fewer mismanaged errors and undesired aircraft
states than other crews.

Conclusion: Applications of TEM

TEM is both a philosophy of safety and a practical set of techniques. Originally designed to
simultaneously capture performance and the context in which it occurs, TEM has demonstrated
its usefulness in many settings.

Training:
The International Civil Aviation Organization (ICAO) has introduced a standard making TEM
training mandatory for airline flight crews engaged in international operations. TEM training
must now be delivered during initial as well as during recurrent training. ICAO has also
introduced standards making TEM training mandatory for licensing and training requirements of
private and commercial pilots and air traffic controllers.

In order to support these standards, ICAO is continually developing guidance material on TEM
which reflects and is aligned with the concepts discussed in this paper (Human Factors Training
Manual, Procedures for Air Navigation Services, Training, PANS/TRG, and An Introduction to
TEM in ATC). In addition, the Australian Transport Safety Bureau and Australian Civil Aviation
Safety Authority are facilitating TEM training courses for pilots.

Line Operations Safety Audits (LOSA):

Considered a best practice for normal operations monitoring and aviation safety by both ICAO
and the FAA, TEM-based LOSAs continue to provide valuable diagnostic information about an
airline’s safety strengths and vulnerabilities.

Incident reporting:

Several US airlines now use TEM as the conceptual structure for their incident reporting
systems. Reporting forms prompt pilots to report the threats that were present, the errors they
may have made, how the event was managed, and how the event may have been avoided or
handled better. Even pilots who have not had training in TEM are able to complete the reporting
form, a fact that speaks to the intuitive nature of the TEM framework.

© Global Air Training Limited 2018 1.16

HANDOUT

Incident and accident analysis:

The International Air Transport Association (IATA) Safety Committee adopted the TEM model
as an analysis framework for its Incident Review Meetings, based on its ease of use and utility
of the extracted data. IATA has also created the Integrated Threat Analysis Task Force
(ITATF). This group analyses data from accidents, incidents, and normal operations using TEM
as the common framework. By selecting specific scenarios, for example, runway excursions
from the incident and accident databases, and precursors to runway excursions from the LOSA
Archive, it is possible to provide a more complete picture of safety issues within the aviation
system.

Other aviation settings:

Studies are underway to adapt TEM to Air Traffic Control, Flight Dispatch, and Ramp. Of
interest, the first ATC trials, called the Normal Operations Safety Survey (NOSS), were
conducted under ICAO sponsorship in Australia, Canada, and New Zealand, and were well-
received. The ICAO sponsored NOSS manual explaining how to conduct normal operations
monitoring in Air Traffic Control, is now available.

TEM has proved its utility in many safety management applications. As organisations and
individuals continue to adopt TEM as a way to understand and enhance their performance, we
hope that you too will see the utility of the TEM framework and find ways to incorporate TEM
techniques into your own personal philosophy of safety.

Reference:
1) Ashleigh Merritt, Ph.D. & James Klinect, Ph.D.

2) The University of Texas Human Factors Research Project

3) The LOSA Collaborative - December 12, 2006

© Global Air Training Limited 2018 1.17