Safety Performance
Indicators
Stefano PROLA
Manager Safety and Flight Ops EUR IATA
Overview
SMS > why SPIs
Measuring Safety
Examples
SMS Basics
Safety is dependent on this System
working
SMS requires organizations to make
data-based decisions and actions
Dependent on the collection and analysis of correct
information
Must understand the system as a whole and the risks
associated with it
Shift of responsibility for safety risk mitigation
Proactive safety: from theory to practice
29 October 2015
Fundamental Shift
Reactive to proactive management
Prescriptive to performance based management and
oversight
Proactive safety: from theory to practice
29 October 2015
Shift to Performance Based
Management
New requirements
New roles
New needs
Proactive safety: from theory to practice
29 October 2015
Performance Based Management
In context of operation
Domestic / International / Regional
Wide-body / Narrow-body
Bulk load / Container load
Cargo / Pax / Combi
Risk tolerance of the organization
Complexity of the operation
An alternative means to prescriptive compliance
Proactive safety: from theory to practice
29 October 2015
Why?
Proactive management
Technology improvements
Vast majority of accidents / incidents related to
technology / human interface
Improvements in the understanding of
human factors
Drill into causal factors
Proactive safety: from theory to practice
29 October 2015
SMS purpose
Safety Management processes identify hazards with the
potential to adversely affect safety ~ ICAO SMM
Purpose is to improve safety performance
Also requirement for continuous improvement, system
must continue to evolve
Proactive safety: from theory to practice
29 October 2015
ICAO SMS Framework - Annex 19
Safety Policy and Objectives
1.1 Management commitment and responsibility
1.2 Safety accountabilities
1.3 Appointment of key safety personnel
1.4 Coordination of emergency response planning
1.5 SMS documentation
Safety Risk Management
2.1 Hazard identification
2.2 Risk assessment and mitigation
Proactive safety: from theory to practice
29 October 2015
ICAO SMS Framework Annex 19
Safety Assurance
3.1 Safety performance monitoring and measurement
3.2 The management of change
3.3 Continuous improvement of the SMS
Safety Promotion
4.1 Training and education
4.2 Safety communication
Proactive safety: from theory to practice
10
29 October 2015
Measuring Safety
Performance
You cannot manage what you dont
measure
How do you measure safety performance?
Definitions
Safety ~ the state in which risks associated with aviation
activities, related to, or in direct support of the operation
of aircraft, are reduced and controlled to an acceptable
level - ICAO
Safety Performance ~ a States or Service Providers
safety achievement as defined by its safety performance
targets and safety performance indicators - ICAO SMM
Ed3 Effective 2013
Proactive safety: from theory to practice
13
29 October 2015
Definitions
Safety Performance Indicator ~ a data based safety
parameter used for monitoring and assessing safety
performance - ICAO SMM
Performance Measures ~ Metrics (or values) that are
set as a target (usually a number or rate) in order to
measure the level of operational performance being
achieved - IATA Reference Manual for Audit Programs
(IRM)
Proactive safety: from theory to practice
14
29 October 2015
IOSA Reference
ORG 3.2.1 The Operator should have
processes for setting performance
measures as a means to monitor the
operational safety performance of the
organization and to validate the
effectiveness of safety risk controls.
[SMS] (GM)
1 September 2016 should > shall
Proactive safety: from theory to practice
15
29 October 2015
Performance Measurement and
Monitoring
Dependent on all SMS framework
requirements met
Managing operational risks
Maintaining regulatory compliance
Proactive safety: from theory to practice
16
29 October 2015
Measuring Safety Performance
Set measureable (SMART) safety objectives
Track performance
Compare against targets
Achievement of a target > improvement in performance
Proactive safety: from theory to practice
17
29 October 2015
IOSA examples
Flight operations (e.g., landing tail strikes, unsatisfactory
line or training evaluations)
Operational control (e.g., fuel diversions due to fuel)
Engineering and maintenance (in-flight engine
shutdowns, aircraft component/equipment failures)
Cabin operations (inadvertent slide deployments)
Ground handling (aircraft damages due to vehicles or
equipment)
Proactive safety: from theory to practice
18
29 October 2015
SMS 3rd phase Doc 9859 SMM
Develop high-consequence SPIs and associated targets
and alert settings.
Typically focused on number of serious accidents /
incidents
High profile
Easy to measure
Reactive
Does not expose systemic issues or hazards
Proactive safety: from theory to practice
19
29 October 2015
Proactive Shift
SPIs measure performance of safety controls
Shift focus to precursors of serious events
Proactive safety: from theory to practice
20
29 October 2015
SMS 4th phase Doc 9859 SMM
Develop lower-consequence SPIs and associated
targets/alert settings
Enhance the safety data collection and processing
system to include lower-consequence events.
Establish lower-consequence safety/quality indicators
with target/alert level monitoring as appropriate.
Reach an agreement with the State oversight authority
on lower-consequence safety performance.
Proactive safety: from theory to practice
21
29 October 2015
Data
Must measure leading and lagging indicators
Leading indicator ~ Metrics that provide information on
the current situation that may affect future performance
Lagging Indicator ~ Metrics that measure safety events
that have already occurred including those unwanted
safety events you are trying to prevent
Established for any operational or maintenance area
Proactive safety: from theory to practice
22
29 October 2015
Example ~ Long Landing (leading)
Identify touchdown points of ALL flights
Proactive safety: from theory to practice
23
29 October 2015
Example Unstable approach paradox
Extract from Data Report for Evidence-Based Training
While airline Standard Operating Procedures (SOPs)
mandate a go-around if an approach is unstable, data
indicates that landing from an unstable approach may be
less risky.
Landings that follow an unstable approach are usually
uneventful (98% land, 90% uneventful)
Missed approaches are rarely handled well
Proactive safety: from theory to practice
24
29 October 2015
Summary
SPIs not driven to simply get a good score
Must improve overall safety performance
Process approach focusing on inputs and their direct
impact on output(s)
Results to be regularly reviewed and communicated
Can be established for virtually any operational or
maintenance area
Must be challenging and result in enhancing the SMS as
a whole
Proactive safety: from theory to practice
25
29 October 2015
IATA Activity
Creation of SPI Guidance Material
Sample SPIs
Supporting methodology
Assist organizations in developing the means to verify
their safety performance against meaningful safety
objectives
Facilitate understanding of the need to look beyond
events
Proactive safety: from theory to practice
26
29 October 2015
Thank you!
Proactive safety: from theory to practice
27
29 October 2015
