6.

6 SMS SAFETY PERFORMANCE (SMM extract, 2nd edition 2009)

6.6.1 Annexes 1, 6, 8, 11, 13 and 14 establish that a service provider’s SMS shall ensure remedial
action to maintain safety performance and shall continuously monitor and shall regularly assess such
safety performance.

6.6.2 The notion of safety performance is an essential ingredient in the effective operation of an SMS as
well as progressing towards a performance-based regulatory environment. It assists in monitoring actual
performance of the SMS, and in avoiding just simply “ticking the appropriate boxes”. It is necessary for
an SMS to define a set of measurable performance outcomes in order to determine whether the system
is truly operating in accordance with design expectations — not simply meeting regulatory requirements
— and to identify where action may be required to bring the performance of the SMS to the level of
design expectations. These measurable performance outcomes permit the actual performance of
activities critical to safety to be assessed against existing organizational controls so that necessary
corrective action is taken and safety risks can be maintained ALARP.

6.6.3 A performance-based regulatory approach will assess the actual performance of activities critical
to safety against existing organizational controls. Furthermore, only through assurance of the effective
safety performance of the SMS — through the establishment and measurement of specific safety
performance outcomes — can the objective of continuous improvement of safety underlying safety
management be achieved.

6.6.4 The safety performance of an SMS is not related to the quantification of high-consequence
outcomes (safety measurement) but rather to the quantification of low-consequence processes (safety
performance measurement). The safety performance of an SMS represents safety performance
measurement exclusively. Safety performance expresses the safety objectives of a service provider, in
the form of measurable safety outcomes of specific low-level processes of the SMS. From the
perspective of the relationship between the State and service providers, safety performance provides
objective evidence for the State to measure the effectiveness and efficiency that the SMS of service
providers should achieve while the service providers conduct their core business functions. Such safety
performance must be agreed between the State and service providers, as the minimum acceptable the
service provider must achieve during the delivery of services. The safety performance of an SMS is thus
a reference against which the State can measure the safety performance of the SMS, that is, that the
SMS works above and beyond regulatory compliance. In agreeing to the safety performance of an SMS,
it is necessary to consider such factors as the level of safety risk that applies the cost/benefits of
improvements to the system, and public expectations about the safety of the aviation industry.

6.6.5 Within each State, the safety performance of each SMS will be agreed separately between the
State and individual aviation organizations. Agreed safety performance should be commensurate with
the complexity of an individual aviation organization’s specific operational contexts, and the availability
of an aviation organization’s resources to address them. In practice, the safety performance of an SMS
is expressed by safety performance indicator values and safety performance target values and is
implemented through action plans.

6.6.6 The safety performance indicator values are short-term, measurable objectives reflecting the
safety performance of an SMS. They are expressed in numerical terms; they should be obvious,
measurable and linked to the safety concerns of an SMS. Safety performance indicator values reflect
safety performance measurement exclusively. The safety performance indicator values of an SMS
should not reflect safety measurement. Since the safety performance of each SMS will be agreed
separately between the State and individual aviation organizations, the safety performance indicator
values will therefore differ between segments of the aviation industry, such as aircraft operators,
certified aerodrome operators and ATS providers. An example is provided.

6.6.7 Through its SMS, a certified aerodrome operator has identified safety concerns regarding foreign
object debris (FOD) in ramp operations. It has also identified safety concerns regarding traffic of
unauthorized vehicles on taxiways. It therefore defines the following safety performance indicator values,
following agreement with the State’s civil aviation oversight authority: 15 FOD events in the apron per 10
000 operations and 20 events of unauthorized vehicles on the taxiways per 10 000 operations. These
safety performance indicator values fulfil the conditions discussed in 6.6.6: they are expressed in
numerical terms; they are obvious, measurable and linked to the safety concerns of the aerodrome SMS.
Furthermore, both safety performance indicators reflect safety performance measurement.

6.6.8 Safety performance target values are long-term, measurable objectives reflecting the safety
performance of an SMS. Safety performance target values are expressed in numerical terms; they
should be obvious, measurable, and acceptable to stakeholders and linked to the safety performance
indicator (short-term objective) of an SMS.
6.6.9 Continuing with the example discussed in 6.6.7, the aerodrome defines the following safety
performance target values, following agreement with the State’s civil aviation oversight authority: by
January 2009, reduce FOD events in the apron to 8 per 10 000 operations, and maintain 20 events of
unauthorized vehicles on the taxiways per 10 000 operations. These safety performance target values
fulfil the conditions discussed in 6.6.6: they are expressed in numerical terms; they are obvious,
measurable and linked to the safety performance indicators of the aerodrome SMS. Furthermore, both
safety performance target values reflect safety performance measurement.

6.6.10 Action plans are the tools and means needed to achieve the safety performance indicator values
and safety performance target values of an SMS. They include the operational procedures, technology,
systems and programmes to which measures of reliability, availability, performance and/or accuracy can
be specified. An example of an action plan to achieve the safety performance indicator values and
safety performance target values of an SMS discussed above would be as follows: implement a thrice-
daily walk-in ramp inspection programme, develop and implement a training course for drivers and
install (aerodrome-specific) taxiway signage.

6.6.11 The safety performance indicator values and safety performance target values of the safety
performance of an SMS may be different, or they may be the same. Three aspects must be considered
when assessing whether specific safety performance indicator values and safety performance target
values of the safety performance of an SMS are different or the same. First, consideration must be given
to the availability of resources within the service provider to turn the safety performance indicator value
into a more demanding safety performance target value. Second, consideration must be given to how
expensive the action plans deemed necessary to change the value of the safety performance indicator
into a more demanding value of the safety performance target are. Third, and most importantly,
consideration must be given to whether the assessment of the safety risks of the consequences of the
hazard addressed by the safety performance indicator and safety performance target falls in the
tolerable region of the safety risk management process discussed in Chapter 5, should the safety
performance indicator value and the safety performance target value remain the same. The safety
performance indicator value may reflect a safety risk assessment that falls in the tolerable region under
prevailing circumstances. However, changes in the system, growth and so forth may render such safety
risk assessment invalid. The safety performance indicator value must in this case be turned into a more
demanding target value to be valid in the changed environment.

6.6.12 A range of different safety performance indicators and safety performance targets will provide a
better insight into the safety performance of the SMS of an aviation organization than the use of a single
indicator or target. In other words, the safety performance of an SMS will always be expressed by a
number of safety performance indicators and safety performance targets, never by a single one.
Additional examples follow.

6.6.13 An aircraft operator has identified the approach and landing phases of flight operations as one
major safety concern to be addressed by its SMS. It has also identified, though the safety risk
management component of its SMS, a safety concern regarding unstable (or non-conforming)
approaches at those aerodromes of the network served by non precision approaches. It therefore
defines the following safety performance indicator value, following agreement with the State’s civil
aviation oversight authority: 10 unstable (or non-conforming) approaches per 1 000 landing operations
at aerodromes of the network served by non-precision approaches. Subsequently, the aircraft operator
defines the following safety performance target value, following agreement with the State’s civil aviation
oversight authority: within the next three years, reduce by fifty per cent the number of unstable (or non-
conforming) approaches per 1 000 landing operations at aerodromes of the network served by non-
precision approaches. The action plan to achieve the safety performance indicator value and the safety
performance target value discussed above would be as follows: development of constant descent angle
(CDA) GPS approaches at aerodromes of the network served by non-precision approaches.

6.6.14 An ATS provider has identified airport operations safety as one major safety concern to be
addressed by its SMS. It has identified, though the safety risk management component of its SMS, a
concern regarding runway incursions and has defined the following safety performance indicator value:
0.8 Cat A and B (most serious) runway incursions per million operations through 2009. Subsequently,
the ATS provider defines the following safety performance target value: by 2010 reduce Cat A and B
(most serious) runway incursions to a rate of not more than 0.5 per million operations.

6.6.15 The safety performance of an SMS should be defined, to the extent possible, through quantitative
safety performance indicators and safety performance targets. It is recognized, however, that in many
States the safety data collection and analysis capabilities of services providers may not be fully
developed. Therefore, while such capabilities are developed, the safety performance of an SMS can be
defined through a combination of quantitative and qualitative safety performance indicators and safety
performance targets. The objective should nevertheless remain the definition of safety performance of
an SMS through quantitative measures only.
6.6.16 The definition of the safety performance of an SMS is a requirement that goes above and beyond
regulatory compliance with national and international requirements. Establishing safety performance for
an SMS does not replace legal, regulatory or other established requirements, nor does it relieve service
providers from their obligations under relevant national regulations, and those arising from the
Convention on International Civil Aviation (ICAO Doc 7300) and its related provisions contained in the
Annexes to the Convention.