Introduction to

Safety Science

Pieter van Gelder

Professor of Safety Science
Delft University of Technology
TU Delft Safety and Security Science Section
WM0801TU
Lecture of
Join Zoom Meeting 23 Dec. 2021
https://tudelft.zoom.us/j/92118224070
Delft
University of 1/50
Technology

Challenge the future

• Week 1, 11/11: Introduction to safety and security (Pieter)
• Week 2, 18/11: System safety assessment using Fault/Event Trees (Pieter)
• Week 3, 25/11: Human error and the psychology of safety and security (Dr. Peter Roelofsma)
• Week 4, 2/12: Decision trees and risk optimization (Pieter)
• Week 5, 9/12:Safety management and the Machine directive (Prof. Coen van Gulijk)
• Week 6, 16/12: Numerical safety and security science (Pieter)
• Week 7, 23/12: Bayesian Networks. security assessment and examples of examination questions
(Pieter)

Delft
University of 2/50
Technology

Challenge the future

 Examination preparation
• Written exam via Mobius on Campus on Jan. 17th 18.30 – 21.30h
• Exam is based on lecture slides
• Exam is open book.

• Extra individual exam opportunity?

• Discuss the matter with an academic counsellor before sending a request in
writing to the Board of Examiners.
• https://www.tudelft.nl/en/student/faculties/3me-student-
portal/organisation/boards-of-examiners/procedures/extra-individual-exam-
opportunity

Delft
University of 3/50
Technology

Challenge the future

Upload tool in Mobius

Delft
University of 4/50
Technology

Challenge the future

Outline for today

• Bayesian networks
• Security risk analysis
• Old examination questions

Delft
University of 5/50
Technology

Challenge the future

6

Cause – consequence relationships

in Safety and Security domains
• Slow internet connection may be caused by a Malware Attack
(amongst others)
• Popups may be caused by a Denial of Service Attack or a
Malware Attack (amongst others)

• A fault tree could represent this as:

• Slow internet connection = OR{Malware, Other attacks}
• Popups = OR{DoS, Malware, Other attacks}

Delft
University of 6/50
Technology

Challenge the future

7

A directed acyclic graph showing

causes and effects consisting
of nodes and arcs

Malware attack Denial of service

attack

Internet connection Popups

Delft
University of 7/50
Technology

Challenge the future

8

Bayesian network example

Delft
University of 8/50
Technology

Challenge the future

9

Let’s assume prior probabilities for the

causes and conditional probabilities
for the effects
• P(DoS) = 0.01
• P(Malware attack) = 0.05

• P(Slow internet connection | Malware attack) = 0.9

• P(Slow internet connection | No malware attack) = 0.1
• P(Popups | Malware and DoS) = 0.9
• P(Popups | no Malware and no DoS) = 0.1
• P(Popups | Malware and no DoS) = 0.8
• P(Popups | no Malware and DoS) = 0.2

Delft
University of 9/50
Technology

Challenge the future

10

Probabilistic inference
Malware attack Denial of service ...

True 5% True 1%
False 95% False 99%

Internet connection Popups

Slow 14% False 86%

Normal86% True 14%

Delft
University of 10/50
Technology

Challenge the future

11

Theorem of total probability

• P(IC = slow) = P(IC=slow|MA=true) * P(MA=true) +
P(IC=slow|MA=false) * P(MA=false)

• P(IC = slow) = 0.9*0.05 + 0.1*0.95 = 0.14

Delft
University of 11/50
Technology

Challenge the future

12

Predictive reasoning

Malware attack Denial of service ...

True 100% True 1%

False 0% False 99%

Internet connection Popups

Slow 90% False 20%

Normal10% True 80%
Delft
University of 12/50
Technology

Challenge the future

13

Predictive reasoning

Malware attack Denial of service ...

True 100% True 100%

False 0% False 0%

Internet connection Popups

Slow 90% False 10%

Normal10% True 90%
Delft
University of 13/50
Technology

Challenge the future

14

Diagnostic reasoning

Malware attack Denial of service ...

True 32% True 1%

False 68% False 99%

Internet connection Popups

Slow 100% False 67%

Normal 0% True 33%
Delft
University of 14/50
Technology

Challenge the future

15

Bayes theorem
• P(MA=true | IC=slow) = P(IC = slow | MA =true) *
P(MA=true)/P(IC=slow)

• P(MA=true | IC=slow) = 0.9 *0.05/0.14 = 0.32

Delft
University of 15/50
Technology

Challenge the future

16

Inter-causal reasoning

Malware attack Denial of service ...

True 32% True 100%

False 68% False 0%

Internet connection Popups

Slow 100% False 58%

Normal 0% True 43%

Delft
University of 16/50
Technology

Challenge the future

17

Back to the medical example

• https://demo.bayesfusion.com/bayesbox.html
• Select: diagnosis of liver disorders

• The rectangular orange colored nodes represent diseases, the

blue nodes represent patient history, including risk factors, and
green nodes represent observations and test results.

Delft
University of 17/50
Technology

Challenge the future

18

Observations
• A Bayesian network is a probabilistic graphical model that
represents a set of random variables and their conditional
dependencies via a directed acyclic graph. It is very suitable to
represent the probabilistic relationships between causes
(attacks) and consequences (symptoms), indicated by the arcs in
the graph.
• The Bayesian network can be used for probabilistic inference,
predictive reasoning, diagnostic reasoning and intercausal
reasoning.

Delft
University of 18/50
Technology

Challenge the future

19

Observations
• Bayesian networks are updated, and new (failure) probabilities
can be recalculated, when new data is observed, which can be
called a learning process.
• Software is available at:
• https://download.bayesfusion.com/files.html?category=Academ
ia
• Background information available at:
• Bayesian network models in cyber security: a systematic review, S
Chockalingam, W Pieters, A Teixeira, P van Gelder, 2017 Nordic
Conference on Secure IT Systems, 105-122.

Delft
University of 19/50
Technology

Challenge the future

 Safety analysis vs Security
analysis
Safety analysis Security analysis
Accidental events Intentional events
Random failure/natural Attacks by adversaries
hazards/human error
Probability of failure Likelihood of attack
Historical data Threats (types of adversary and
Experimental data attack)
Expert judgment Attractiveness
Vulnerability

Delft
University of 20/50
Technology

Challenge the future

20
Security Risk Analysis
Consequence 1.1. Critical units
assessment 1.2. Severity of consequences
1

Delft
University of 21/50
Technology

Challenge the future

21
Consequence Severity Table
Description Rank
• On-site minor injury; no fatalities Minor
• Up to $100,000 property damage
• A few days of business interruption

• On-site serious injuries; no fatalities Moderate

• $100, 000 – $1,000,000 property damage
• A few weeks of business interruption

• On-site several fatalities; off-site injuries Major

• $1,000, 000 – $10,000,000 property damage
• A few months of business interruption

• On-site multiple fatalities; off-site several fatalities Catastrophic

• More than $10,000,000 property damage
• A few years of business interruption

Delft
University of 22/50
Technology 22
Challenge the future
Security Risk Analysis
Consequence 1.1. Critical units
assessment 1.2. Severity of consequences
1

Threat 2.1. Type of adversary

assessment 2.2. Type of attack
2

Delft
University of 23/50
Technology 23
Challenge the future
Threat assessment

Type of Adversary Type of Attack

External • Release of hazardous materials
Terrorists • Theft of hazardous materials
Criminals • Major damage to target’s infrastructures
Thieves
• Theft of confidential information
Internal
• Damage to equipment
Employees
Visitors
Contractors
Hybrid

Delft
University of 24/50
Technology 24
Challenge the future
Security Risk Analysis
Consequence 1.1. Critical units
assessment 1.2. Severity of consequences
1

Threat 2.1. Type of adversary

assessment 2.2. Type of attack
2

Attractiveness
analysis 3. Evaluate attractiveness of the target
3

Delft
University of 25/50
Technology 25
Challenge the future
Attractiveness analysis
• Potential for causing maximum casualties
• Potential for causing maximum economic damage
• Proximity of the target to densely populated area
• Proximity of critical units to the object’s boundary
• High reputation of the target
• Recognizability of critical units

Delft
University of 26/50
Technology 26
Challenge the future
Security Risk Analysis
Consequence 1.1. Critical units
assessment 1.2. Severity of consequences
1

Threat 2.1. Type of adversary

assessment 2.2. Type of attack
2

Attractiveness
assessment 3. Evaluate attractiveness of the target
3

Vulnerability
analysis 4. Evaluate vulnerability of the target
4

Delft
University of 27/50
Technology 27
Challenge the future
Vulnerability analysis

• Ease of access to/escape from the target

• Security barriers (fence, surveillance, guards)

• Control systems can easily be tampered

• Etc.

Delft
University of 28/50
Technology 28
Challenge the future
 Security Risk Analysis
Consequence 1.1. Critical units
assessment 1.2. Severity of consequences
1

Threat 2.1. Type of adversary

assessment 2.2. Type of attack
2
Attack Likelihood

Attractiveness
assessment 3. Evaluate attractiveness of the target
3

Vulnerability
assessment 4. Evaluate vulnerability of the target
4

Security risk
analysis Security Risk = F (Attack likelihood , consequence)
5
Delft
University of 29/50
Technology 29
Challenge the future
Risk Matrix

Severity
Catastrophic Major Moderate Minor
Likelihood

High High High High Medium

Medium High High Medium Low
Low High Medium Low Low
Very low Medium Low Low Low

Delft
University of 30/50
Technology 30
Challenge the future
Nash Equilibrium as an input for
security risk analysis

Delft
University of 31/50
Technology

Challenge the future

Attacker defender game with
two targets A and B
• It costs the defender $10 to defend A and $20 to defend B.
• The cost of failing to defend each target, if it is then attacked, is
$100 for B and $60 for A.
• The defender can only afford to defend one of the two targets, and
defense is completely effective, so an attack on a defended target
does no damage.

Delft
University of 32/50
Technology

Challenge the future

 33

Payoff matrix

Defender moves first, then attacker.

What is the best strategy for the defender, assuming that the
attacker will know the defender’s choice?
If I defend A, then B will be attacked. If I defend B, then A will be attacked. So it is better to defend B.

Delft
University of 33/50
Technology

Challenge the future

34

Pay off matrix

If the two players must make their choices simultaneously,

each in ignorance of what the other has done, then the unique
mixed-strategy Nash equilibrium can be found.
A mixed strategy is one in which each player chooses
probabilistically among his or her strategies.

Delft
University of 34/50
Technology

Challenge the future

35

Deriving the mixed-strategy Nash

equilibrium

Solution:
• Attacker attacks A with probability 9/16 = 0.5625
• Defender defends A with probability 6/16 = 0.375

Delft
University of 35/50
Technology

Challenge the future

 36

• It is easy to verify that, for these choices:

• the expected return from defending A is 0.5625 × (−10) + (1 − 0.5625) ×
(−110) = −53.75
• the expected return from defending B is 0.5625 × (−80) + (1 − 0.5625) ×
(−20)=−53.75

• The mixed-strategy Nash equilibrium condition of equal

expected payoffs is satisfied for the defender; a similar
calculation holds for the attacker.

Delft
University of 36/50
Technology

Challenge the future

37

Observation
Game-theoretic models and methods can help
analysts think more clearly and effectively about
the risks of adversarial situations by clarifying what
should be modeled as decision variables for different
players (i.e., the strategy sets of the players, which
may include which targets to attack, under what conditions,
when, and how) and what should be modeled
as chance or consequence variables.

Delft
University of 37/50
Technology

Challenge the future

 38

Threat, vulnerability,
impact
“The probability of attack on each target (sometimes referred to
as “threat” probabilities in the terrorism risk analysis literature) is
an output of the analysis, rather than an unknown input to be
guessed at (e.g., via expert elicitation).”

See the mixed-strategy Nash equilibrium in which each player

optimizes probabilistically among his or her strategies.

Delft
University of 38/50
Technology

Challenge the future

39

Similarly, instead of regarding “vulnerability” as

a single conditional probability number, or as the
outcome of a probabilistic (event tree) process, game
theory models encourage the defender to recognize
that the probability that an attack succeeds depends
on how hard the attacker attacks and the defender
defends—that is, on the magnitude of resources invested
by each in preparing for (and, perhaps, executing)
the attack and defense strategies at various
targets.

Delft
University of 39/50
Technology

Challenge the future

 40

Game-theoretic Patrolling in
Areas with Complex
Terrain to Combat
Poaching

Prof. Tambe,
University of
• University of Southern California, USA Southern
California,
USA

http://teamcore.usc.edu/papers/2016/AAAI16Demo_PAWS.pdf

Delft
University of 40/50
Technology

Challenge the future

Security risk analysis with Bayes

•An IDS is available to detect possible

intrusions.
•If the system has an intrusion, it is indicated in
99% of the cases
•If a system does not have an intrusion, the IDS
indicates it correctly in 98% of the cases
•Intrusion is present in 1 in 2000 systems
•The IDS indicates to Bibi’s system has an
intrusion
•Give the probability that Bibi indeed has the
intrusion, given that the IDS indicates that his
system has an intrusion

Delft
University of 41/50
Technology

Challenge the future

 Define the conditional
probabilities
• An IDS is available to detect possible intrusions.
• If the system has an intrusion, it is indicated in 99% of the cases ->
P( IDS | In) = 0.99
• If a system does not have an intrusion, the IDS indicates it in 98% of
the cases -> P( not IDS | not In) = 0.98 -> P(IDS | not In) = 0.02
• Intrusion is present in 1 in 2000 systems -> P( In )=1/2000
• The IDS indicates to Bibi that his system has an intrusion
• Give the probability that Bibi indeed has the intrusion, given that
the IDS indicates that his system has an intrusion -> P( In | IDS)

Delft
University of 42/50
Technology

Challenge the future

Use Bayes’ theorem and
theorem of total probability

• P(A | B) = P(B | A) * P(A)/P(B)

• P(A) = P(A|B)*P(B) + P(A| not B)*P(not B)

It follows that Bibi’s probability of having an infected system goes up

from 1/2000 to 1/41 (still quite small)

P(IDS) 0,020485

P(IDS|In)*P(In) 0,000495

P(In|IDS) 0,024164

Delft
University of 43/50
Technology

Challenge the future

 A tabular presentation

Infected IDS results

Investigated systems 1 1 positive 0 negative
2000 Not Infected
1999 40 positive 1959 negative

Delft
University of 44/50
Technology

Challenge the future

 Quiz dilemma

• Car in A, B of C

A B C

Delft
University of 45/50
Technology

Challenge the future

Quiz-dilemma

• Theorem of Bayes:

Pinf o | A  PA  1/ 2 * 1/ 3
PA | inf o     1/ 3
Pinf o  Pinf o 

Pinf o | C PC 1 * 1/ 3
PC | inf o     2/3
Pinf o  Pinf o 

Delft
University of 46/50
Technology

Challenge the future

 No info
1
Simulation result
0.9 P = 1/3

0.8

0.7

0.6

0.5

0.4

0.3

0.2

0.1

0
0 200 400 600 800 1000 1200 1400 1600 1800 2000

Delft
University of 47/50
Technology

Challenge the future

 With info from QM
1
"Stay" stragegy (P success = 0.3295)
0.9 "Switch" strategy (P success = 0.6705)

0.8

0.7

0.6

0.5

0.4

0.3

0.2

0.1

0
0 200 400 600 800 1000 1200 1400 1600 1800 2000

Delft
University of 48/50
Technology

Challenge the future

 Examples of examination
questions

Delft
University of 49/50
Technology

Challenge the future

An examination question about
fault tree analysis

Bulb 1
Fuse

Switch
Bulb 2

Power
Source

Delft
University of 50/50
Technology

Challenge the future

 Room Dark

Power off Both bulbs

burned out

Delft
University of 51/50
Technology

Challenge the future

 Room Dark

Power off Both bulbs

burned out

B1 B2
Power source
Fuse Switch

Delft
University of 52/50
Technology

Challenge the future

 The fault tree can be represented as:

TE = OR{Fuse, Switch, Power source, AND{B1,B2}}

Given component probabilities of failure of 10% for

each component, the probability of the occurrence of
the top event becomes, under the assumption of
independent component failures:

1-(1-0.1)*(1-0.1)*(1-0.1)*(1-0.1*0.1) = 0.278

Delft
University of 53/50
Technology

Challenge the future

 An examination question about
Event tree analysis

Start of Fire Extinction by Extinction by Extinction by

Sprinkler Personnel Fire brigade
Minimal
Yes
Moderate loss
Yes
No Large loss
Yes
No
Catastrophic
No

Delft
University of 54/50
Technology

Challenge the future

 Event tree

Start of Fire Extinction by Extinction by Extinction by Probability

Sprinkler Personnel Fire brigade
p=10-4 0.6 10-4
Yes p=0.6
0.32 10-4
Yes p= 0.8
No p=0.4 0.72 10-5
Yes p= 0.9
No p= 0.2
0.8 10-6
No p= 0.1

Delft
University of 55/50
Technology

Challenge the future

 An examination question
about security analysis:
Infrared cameras for
detection of terrorists at
airports

Delft
University of 56/50
Technology

Challenge the future

An experiment with 20 persons

A correct prediction of 75% and 92% (average 85%)

Delft
University of 57/50
Technology

Challenge the future

This seems like a reasonable
test, but when applied at
Schiphol, it delivers:

- Even if the test is 99.9% reliable, 40,000 people per year

remain detected as suspicious while innocent
- The false alarm rate will weaken the attention of the
Marechaussee.
Delft
University of 58/50
Technology

Challenge the future

 An examination question about
decision tree analysis
In a jewelry store, there are N independent
burglary detection systems operating
independently from each other. Every installation
works properly in the event of a burglary with a
90% chance (ringing).
A burglary is taking place.
- What is the probability that an alarm will be
raised in this accident?
- Should we now install as many installations as
possible? This is a design question.

Delft
University of 59/50
Technology

Challenge the future

Storm surge barrier in the
Nieuwe Waterweg

• The storm surge barrier in the Nieuwe Waterweg was built to

protect the area behind it against high water levels. The barrier is
closed in the event of a severe storm at sea. The water level on the
inside of the barrier is determined by the river water and will
continue to increase when the barrier is closed. If the river water
level is more than 1 meter higher than the sea water level, the
barrier will fail with enormous consequences. Correct water level
measurements are therefore of crucial importance and
independent water level measurements are necessary to build in
extra safety.

Delft
University of 60/50
Technology

Challenge the future

Storm surge barrier in the
Nieuwe Waterweg

Delft
University of 61/50
Technology

Challenge the future

• A water level measuring system has been installed to measure the
water levels on both sides of the barrier after closing, so that the
barrier can be opened as soon as the water level difference
becomes too large. The barrier is equipped with 4 measuring
points. There are measuring points on both banks of the river
(north and south), and both upstream (landside) and downstream
(seaside) of the barrier. The measuring points are vulnerable for
cyber attacks and it is assumed that they have a failure probability
due to a successful attack of 10% each. However, the failure events
of the 4 measuring points are independent from eachother.

Delft
University of 62/50
Technology

Challenge the future

• The data of the 2 measuring points on the north bank of the river are transfered to a server
with a physical cable (which cannot be hacked). The data of the 2 measuring points on the
south bank of the river are transfered to another server with another physical cable (which can
also not be hacked). The 2 servers operate independently from eachother. In principle, only
one of the servers is sufficient to determine the water level difference. For the exchange of
information between the 2 servers, a wireless connection is available to digitally transmit the
data. However, this connection is vulnerable for a cyber attack and it is assumed that it has a
failure probability due to a successful attack of 20%.
• In this assignment we will construct a risk model for the above setting with 5 basic events that
can lead to the unwanted top event “No water level difference measurement”:

Delft
University of 63/50
Technology

Challenge the future

• NS: water level measurement on north bank / sea side fails
• SS: water level measurement on south bank / sea side fails
• NL: water level measurement on north bank / land side fails
• SL: water level measurement on south bank / land side fails
• C: wireless connection between servers 1 and 2 fails

Delft
University of 64/50
Technology

Challenge the future

Block diagram can be constructed
for the above system

Delft
University of 65/50
Technology

Challenge the future

Fill in the 5 basic events in the attack
tree below to reach the attacker’s
goal “No water level difference
measurement

Delft
University of 66/50
Technology

Challenge the future

Probability of the Top Event?

• Estimate the probability of reaching the attacker’s goal, given the

above probabilities of succesfully attacking the 5 components
within this system, by using the probability calculation rules for
AND and OR gates.

Delft
University of 67/50
Technology

Challenge the future

Fault tree of SSB NW

Delft
University of 68/50
Technology

Challenge the future

Probability propagation to SF

Delft
University of 69/50
Technology

Challenge the future

Fault tree of a Defibrilator

Delft
University of 70/50
Technology

Challenge the future

Schematic diagram of
defibrilator

Delft
University of 71/50
Technology

Challenge the future

Defibrilator :
failure of electricity supply
Geen stroom
naar electr. blok

STROOMVOORZIENING

Batterijlader Laadknop faalt Batterijen falen

faalt

LADER LAADKNOP BATTERIJEN

Batterij faalt Reserve batterij

faalt

BATTERIJ RES. BATTERIJ

Delft
University of 72/50
Technology

Challenge the future

Calculate the probability of the
failure of electricity supply
Given component probabilities of
failure:
•Batterijlader(s) : Pl =0,1
•Batterij : Pb = 0,05
•Res. Batterij : Pr = 0,1
•Laadknop : Pk = 5. 10-3

Delft
University of 73/50
Technology

Challenge the future

Example – PB Design
A company is producing a new chemical product that might be exploded by
three potential factors, including temperature, humidity, and the density of
catalyst. Using a Plackett-Burman design, identify the 4 experiments that
need to be conducted in order to investigate which of the three potential
factors contribute most to the explosion risk.
Take the upper-left 3x4 matrix
Temperature Humidity Catalyst
From the PB design with 11 factors
density
Low Dry <30%
High Humid >30%

1. High, humid, >30%

2. High, dry, <30%
3. Low, humid, <30%
4. Low, Dry, >30%

Delft
University of 74/50
Technology

Challenge the future

Example – Bayesian Probability
According to the reports by the Dutch police, not wearing seatbelt contributes
to 85% of road fatalities, while the overall seatbelt use rate in the
Netherlands is 80%. What is the probability of a road fatality while driving in
highway A12 and not wearing seatbelt and given that we know from the past
data that the probability of road fatality on this highway is 1%.?

P(A|B) = P(B|A)P(A)/P(B)
P(fatality | no seatbelt) =
P(no seatbelt | fatality) P(fatality) / P(no seatbelt)
= 0.85*0.01/0.20 = 0.042

Delft
University of 75/50
Technology

Challenge the future

Example – Block Diagram

Given that the reliability for each system A C

component is 90%, what is the most
accurate probability of failure in this system? D

System failure = (A ⋂ B) U (B ⋂ C) U D B
= [(A ⋂ B) U (B ⋂ C)] U D
= [(0.1*0.1)+(0.1*0.1)-0.1*0.1*0.1]
+ 0.1 = 0.19+0.1 = 0.29

Delft
University of 76/50
Technology

Challenge the future

Example – Pay-off Matrix

S1: hot S2: cold

Given the following pay-off matrix with weather, weather, no
scenarios S1 and S2 and safety measures flammable flammable
conditions conditions
D1 and D2:
D1: separate
storage barrels 670 840
Assuming 0.35 as the coefficient of
optimism and using Hurwicz Criterion, D2: place
which decision should be made for the barriers
990 700
between
safety measures? barrels

D1: ω (maximum pay-off) + (1- ω) (minimum pay-off) = 0.35*(840)+0.65*(670) = 729.5

D2: same way: value = (0.35*990)+(0.65*700) = 801.5

Choose D2: place barriers between barrels

Delft
University of 77/50
Technology

Challenge the future

Example – System failure
Top
Event
What is the most accurate
description of this fault tree and
what is the equivalent block
diagram?
TE = B or (A and C) E1 E2

B
C
A B B C
Delft
University of 78/50
Technology

Challenge the future

Example – Fault tree

What are the minimum cut sets of this

fault tree? And what is the probability of
the top event, given that the probability of
basic events are 20% -use MC to calculate
the probability of the top event.

Minimum cut sets:

{A} {B} {C} {ADE} {ADEC}

Delft
University of 79/50
Technology

Challenge the future

Example – MCMC Simulation

Delft
University of 80/50
Technology

Challenge the future

Thank you and success with your exam!

Delft
University of 81/50
Technology

Challenge the future