Batch3

Subject CS2
Revision Notes
For the 2019 exams

Markov models
Booklet 2

Covering

Chapter 2 Markov chains

Chapter 3 The two-state Markov model and the Poisson
model

The Actuarial Education Company

Batch3
 Batch3

CONTENTS

Contents Page

Links to the Course Notes and Syllabus 2

Overview 4
Core Reading 5
Past Exam Questions 38
Solutions to Past Exam Questions 74
Factsheet 172

Copyright agreement

All of this material is copyright. The copyright belongs to Institute and

Faculty Education Ltd, a subsidiary of the Institute and Faculty of Actuaries.
The material is sold to you for your own exclusive use. You may not hire
out, lend, give, sell, transmit electronically, store electronically or photocopy
any part of it. You must take care of your material to ensure it is not used or
copied by anyone at any time.

Legal action will be taken if these terms are infringed. In addition, we may
seek to take disciplinary action through the profession or through your
employer.

These conditions remain in force after you have finished using the course.

© IFE: 2019 Examinations Page 1

 Batch3

LINKS TO THE COURSE NOTES AND SYLLABUS

Material covered in this booklet

Chapter 2 Markov chains

Chapter 3 The two-state Markov model and the Poisson model

These chapter numbers refer to the 2019 edition of the ActEd course notes.

Syllabus objectives covered in this booklet

The numbering of the syllabus items is the same as that used by the Institute
and Faculty of Actuaries.

3.2 Define and apply a Markov chain.

3.2.1 State the essential features of a Markov chain model.

3.2.2 State the Chapman-Kolmogorov equations that represent
a Markov chain.
3.2.3 Calculate the stationary distribution for a Markov chain in
simple cases.
3.2.4 Describe a system of frequency-based experience rating in
terms of a Markov chain and describe other simple
applications.
3.2.5 Describe a time-inhomogeneous Markov chain model and
describe simple applications.
3.2.6 Demonstrate how Markov chains can be used as a tool for
modelling and how they can be simulated.

4.1 Explain the concept of survival models.

4.1.8 Describe the two-state model of a single decrement and

compare its assumptions with those of the random lifetime
model. (The random lifetime model will be discussed in
detail in Booklet 4.)

Page 2 © IFE: 2019 Examinations

 Batch3

4.3 Derive maximum likelihood estimators for transition intensities.

4.3.1 Describe an observational plan in respect of a finite

number of individuals observed during a finite period of
time, and define the resulting statistics, including the
waiting times.
4.3.2 Derive the likelihood function for constant transition
intensities in a Markov model of transfers between states
given the statistics in 4.3.1.
4.3.3 Derive maximum likelihood estimators for the transition
intensities in 4.3.2 and state their asymptotic joint
distribution.
4.3.4 State the Poisson approximation to the estimator in 4.3.3
in the case of a single decrement.

© IFE: 2019 Examinations Page 3

 Batch3

OVERVIEW

This booklet covers Syllabus objectives 3.2, 4.1.8 and 4.3, which relate to
Markov chains, the two-state Markov model and the Poisson model.

Markov chain models are stochastic processes with a discrete time set and a
discrete state space (usually a series of integers or labels). It is difficult to
use time-inhomogeneous models effectively, so we concentrate mainly on
time-homogeneous models.

An important theme is the conditions necessary for a Markov chain to have a

long-term stationary distribution. The concepts of the irreducibility of a chain
and the period of a state are needed here.

Simple methods for the simulation of a Markov chain model complete the
discussion of this topic.

In exam questions on Markov chains, you might be given a situation in which

a Markov chain model can be used, and be asked various questions about it.
Often the situation is described verbally.

You may also be asked to determine if the model is irreducible and/or

aperiodic and to calculate the long-term stationary distribution of the
process.

Sometimes the model you are given initially may not actually be a Markov
model and you have to modify it so that it is Markov.

The two-state Markov model and the Poisson model are parametric mortality
models. We can use these models to estimate m x based on the results of a
mortality investigation.

In the exam, you could be asked to write down the likelihood function
associated with a model and then derive the maximum likelihood estimate of
the parameter to be estimated.

Page 4 © IFE: 2019 Examinations

 Batch3

CORE READING

All of the Core Reading for the topics covered in this booklet is contained in
this section.

We have inserted paragraph numbers in some places, such as 1, 2, 3 …, to

help break up the text. These numbers do not form part of the Core
Reading.

The text given in Arial Bold font is Core Reading.

The text given in Arial Bold Italic font is additional Core Reading that is not
directly related to the topic being discussed.
____________

Chapter 2 – Markov chains

The Chapman-Kolmogorov equations

1 The term Markov chain is reserved for discrete-time Markov processes

with a finite or countable state space S .
____________

2 So a Markov chain is a sequence of random variables X 0 , X 1, ... , X n , ...

with the following property:

P  X n  jX 0  i0 , X 1  i1,..., X m 1  im 1, X m  i   P  X n  j  X m  i 

   
(2.1)

for all integer times n > m and states i0 , i1,..., im 1, i , j in S .

____________

3 The Markov property (2.1) has the following interpretation: given the
present state of the process X m  i , the additional knowledge of the
past is irrelevant for the calculation of the probability distribution of
future values of the process.
____________

© IFE: 2019 Examinations Page 5

 Batch3

4 The conditional probabilities on the right-hand side of (2.1) are the key
objects for the description of a Markov chain; we call them transition
probabilities, and we denote them by:

P  X n  jX m  i   pi(m ,n )
  j

____________

5 The transition probabilities of a discrete-time Markov chain obey the

Chapman-Kolmogorov equations:

 m ,l  p l ,n 
pi(m
j
,n )
  pi k kj
k S

for all states i , j in S and all integer times m  l  n .

____________

Proof
[Students should understand this proof, but they will not be expected
to reproduce it in the examination.]

This is based on the Markov property (2.1) and on the law of total
probability in its conditional form.

If A1, A2 ,..., Ak ,... form a complete set of disjoint events, ie:


 Ak   , Ak  A j  , k  j
k 1

then for any two events B, C :


P BC    P BC , Ak  P  AkC 
 
k 1

Page 6 © IFE: 2019 Examinations

 Batch3

Thus:

P  X n  jX m  i    P  X n  jX m  i , X l  k  P  X l  kX m  i 

     
k S

  P  X n  jX l  k  P  X l  kX m  i 
   
k S

using the Markov property (note l  m ) .

This is the stated result.

____________

6 The Chapman-Kolmogorov equations allow us to calculate general

transition probabilities in terms of the one-step transition
probabilities pij(n ,n 1) .
____________

Hence the distribution of a Markov chain is fully determined once the

following are specified:

 the one-step transition probabilities pij(n ,n 1)

 the initial probability distribution qk  P  X 0  k  .

____________

7 Indeed we can deduce from these the probability of any path:

P  X 0  i0 , X 1  i1,..., X n  in   qi0 pi
 0,1 p(1,2) ... p(n 1,n )
i
0 1 i ,i
1 2 i i
n  1, n

It is therefore convenient, where possible, to determine states in a

manner that forms a Markov chain. The model on page 12 illustrates
this.
____________

© IFE: 2019 Examinations Page 7

 Batch3

Time-homogeneous Markov chains

8 A simplification occurs if the one-step transition probabilities are time-

independent:

pij(n ,n 1)  pij (2.2)

In this case, we say that the Markov chain is time-homogeneous.

____________

9 It follows easily from (2.2) that general transition probabilities depend

only on time differences:

l 
P  X l  m  jX m  i   pij (2.3)
 

We refer to (2.3) as the l -step transition probability.

____________

10 For time-homogeneous Markov chains, the Chapman-Kolmogorov

equations read:

nm    l  m  p n  l 
pij  pik kj for m < l < n
k S
____________

This has a very simple interpretation. The transition matrix P of a

time-homogeneous Markov chain is a square N ¥ N matrix where N is
the number of states in S (possibly infinite), with the elements Pij being
the one-step transition probabilities pij :

Pij  pij
____________

Page 8 © IFE: 2019 Examinations

 Batch3

11 The l -step transition probability pij

 l  can be obtained by calculating

the entry  i , j  of the l -th power of the matrix P :

l 
pij  P l   ij
____________

12 The normalisation condition  pij  1 holds for all i , ie each row of

j S

P must add up to one. More generally:

l 
 pij  1 for all i
j S
____________

13 It is often revealing to draw the transition graph of a Markov chain: this

is a diagram in which each state in S is represented as a node of the
graph and an arrow is drawn from node i to node j whenever pij  0 ,
indicating that a direct transition from state i to state j is possible.

The value of pij can be recorded above the arrow.

____________

Time-inhomogeneous Markov chains

14 For a time-inhomogeneous Markov chain, the transition probabilities

cannot simply be denoted by pij because they will depend on the
absolute values of time, rather than just the time difference. The value
of ‘time’ can be represented by many factors, for example the time of
year, age or duration.
____________

© IFE: 2019 Examinations Page 9

 Batch3

Models

15 A simple model of a No Claims Discount (NCD) policy

The No Claims Discount system (NCD) in motor insurance whereby the

premium charged depends on the driver’s claim record is a prime
application of Markov chains. We present two simple models and we
suggest various possible improvements.

A motor insurance company grants its customers either no discount

(state 0) or 25% discount (state 1) or 50% discount (state 2). A
claim-free year results in a transition to the next higher state the
following year (or in the retention of the maximum discount); similarly,
a year with one claim or more causes a transition to the next lower
state (or the retention of the zero discount status).

Under these rules, the discount status of a policyholder forms a

Markov chain with state space S  0,1, 2 ; if the probability of a
claim-free year is 3 the transition graph and transition matrix are:
4

3/4 3/4

3/4
State 0 State 1 State 2
0% 25% 50%
discount discount discount

1/4

1/4
1/4

The transition matrix is given by:

 14 3
4 0
 
P   14 0 3
4
0 1 3 
 4 4

Page 10 © IFE: 2019 Examinations

 Batch3

The probability of holding the maximum discount in year n  3 given

that you do not qualify for any discount in year n is:

3
 
p0,2  P 3
1,3
 9 16

____________

16 Time-inhomogeneous NCD model

For a time inhomogeneous case of this model, the probability of an

accident would be time-dependent to reflect changes in traffic
conditions. This could be due to general annual trends in the density
of traffic and/or propensity to claim.

The transition graph and matrix would then become:

P01(t) P12(t)

State 0 State 1 State 2

50% P22(t)
0% 25%
discount discount discount

P00(t)

P10(t) P21(t)

and:

P00 t  P01 t  P02 t 

 
P t    P10 t  P11 t  P12 t 
P t  P t  P t 
 20   21   22  
____________

© IFE: 2019 Examinations Page 11

 Batch3

Another model of an NCD policy

17 Modify the previous model as follows: there are now four levels of
discount:

0 : no discount

1 : 25% discount

2 : 40% discount

3 : 60% discount

The rules for moving up the discount scale are as before, but in the
case of a claim during the current year, the discount status moves
down one or two steps (if this is possible) according to whether or not
the previous year was claim-free.

Under these rules, the discount status X n of a policyholder does not

form a Markov chain on S  0,1,2,3 because:

P  X n  1  0X n  2, X n 1  1  0
 

whereas:

P  X n 1  0X n  2, X n 1  3   0 .
 
____________

18 To construct a Markov chain {Yn , n = 0, 1, 2, } , one needs to

incorporate some information on the previous year into the state; in
fact this is necessary only for state 2, which we split as:

2  : 40% discount and no claim in the previous year

2  : 40% discount and claim in the previous year

____________

Page 12 © IFE: 2019 Examinations

 Batch3

19 Assuming as before a probability of 3 of no claim in any given year,

4

we have a Markov chain on the state space S  0 ,1, 2 , 2 , 3   with

transition graph:

3/4 3/4 3/4

3/4
State 0 State 1 State 2+ State 3
0% 25% 40% 60%
discount discount discount 1/4 discount

1/4
1/4 1/4
3/4
State 2-
40%
discount
1/4

____________

20 The transition matrix is:

 14 3
4 0 0 0
1 3

 4 0 4 0 0
P  0 1
4 0 0 3 
4

 14 0 0 0 3 
4
0 0 0 1 3 
 4 4
____________

21 The probability of being at the 60% discount level in year n  3 given

that you hold 25% in year n is:

3
 
p1,3  P 3
2,5
 27 64

____________

© IFE: 2019 Examinations Page 13

 Batch3

22 This basic model is amenable to numerous improvements. For

instance the accident probability can be made to depend on the
discount status to reflect the influence of the latter on driver care.
Also the accident probability can be time-dependent (leading to a
time-inhomogeneous chain) to reflect changes in traffic conditions.
____________

Random walks

23 Simple random walk on S = { - 2, - 1, 0, 1, 2, }

This is defined as X n  Y1  Y2  ...  Yn where the random variables Y j

(the steps of the walk) are mutually independent with the common
probability distribution:

P Y j  1  p, P Y j   1  1  p

The Markov property holds because the process has independent

increments.
____________

24 The transition graph and the transition matrix are infinite.

The transition graph is:

p p p p p p

-2 -1 0 1 2

1–p 1–p 1–p 1–p 1–p 1–p

Page 14 © IFE: 2019 Examinations

 Batch3

The transition matrix is:

È        ˘
Í ˙
Í        ˙
Í 1 - p 0 p     ˙
Í ˙
Í  1 - p 0 p    ˙
P = Í        ˙
Í ˙
Í    1- p 0 p  ˙
Í     1- p 0 p ˙˙
Í
Í        ˙
Í ˙
Î        ˚
____________

25 In order to get from i to j in n steps the random walk must make

u 1
2 n  j  i  steps in an upward direction, n  u in a downward
direction. Since the distribution of the number of upward jumps in n
steps is Binomial with parameters n and p , the n -step transition
probabilities can be calculated as:

 n  u n u
  p 1  p  if 0  n  j  i  2n and n  j  i is even
pij(n )   u 

0 otherwise
____________

26 Note that in addition to being time-homogeneous a simple random walk

is also space-homogeneous:

n  n
pi j  pi  r j r
____________

© IFE: 2019 Examinations Page 15

 Batch3

Simple random walk on {0, 1, 2,  , b}

27 This is similar to the previous model, except that boundary conditions

have to be specified at 0 and b ; these will depend on the interpretation
given to the chain. Commonly used boundary conditions include:

Reflecting boundary:

P  X n  1  1X n  0   1
 

Absorbing boundary:

P  X n  1  0X n  0   1
 

Mixed boundary:

P  X  0X n  0    ,
  n 1 

P  X n 1  1X n  0   1  
  
____________

A random walk with absorbing boundary conditions at 0 and b can be

used to describe the wealth of a gambler who will continue to gamble
until either his fortune reaches a target b or his fortune hits 0 and he
is ruined; in either case, reaching the boundary means staying there
forever.
____________

28 In the general case, with mixed boundary conditions, the transition

graph is:

1– p p p p p p p


0 1 2 j –1 j j +1 b–2 b–1 b

1–p 1–p 1–p 1–p 1–p 1–p 1–p 1–

Page 16 © IFE: 2019 Examinations

 Batch3

The transition matrix is:

1 2 3    b-2 b -1 b
È a 1- a       ˘
Í ˙
Í1 - p 0 p      ˙
Í  1- p 0 p     ˙
Í ˙
Í   1 - p 0 p    ˙
Í
P=         ˙
Í ˙
Í     1- p 0 p  ˙
Í      1- p 0 p ˙˙
Í
Í       1- p 0 p˙
Í ˙
Î        1- b b˚
____________

Reflecting and absorbing boundary conditions are obtained as special

cases, taking  ,  equal to 0 or 1.
____________

29 The simple NCD model on page 10 is another practical example of a

bounded random walk.
____________

A model of accident proneness

30 For a given driver, any period j is either accident free ( Y j  0 ) or gives

rise to exactly one accident ( Y j  1 ).

The probability of an accident in the next period is estimated using the

driver’s past record as follows (all variables y j are either 0 or 1):

f ( y 1 + y 2 + ... + y n )
P ÈYn +1 = 1 ΩY1 = y 1, Y2 = y 2 , ... , Yn = y n ˘ =
Î ˚ g (n )

where f, g are two given increasing functions satisfying

0 £ f (m ) £ g (m ) .

© IFE: 2019 Examinations Page 17

 Batch3

Of course:

f ( y 1 + y 2 + ... + y n )
P ÈYn +1 = 0ΩY1 = y 1, Y2 = y 2 , ... , Yn = y n ˘ = 1 -
Î ˚ g (n )
____________

31 The dependence on the past record means that Y1, Y2 , ... , Yn , ... does
not have the Markov property (it depends on all previous values of Y j ).
____________

32 Consider, however, the cumulative number of accidents suffered by the

driver:

n
Xn   Yj
j 1

This is a Markov chain with state space S  0, 1, 2, ... .

It possesses the Markov property because:

P  X n  1  1  x nX 1  x1, X 2  x 2 , ... , X n  x n 

 

 P  X n  1  1  x n Y1  x1, Y2  x 2  x1, ... , Yn  x n  x n 1 

 

n
Since  Yj  X n , the condition Y1  x1, Y2  x 2  x1, ... , Yn  x n  x n 1
j 1

is a function only of X n and hence:

P  X n  1  1  x nX 1  x1, X 2  x 2 , ... , X n  x n 

 
f  xn 
 P  X n  1  1  x nX n  x n  
  g n
____________

Note that the chain is only time-homogeneous if g  n  is constant.

____________

Page 18 © IFE: 2019 Examinations

 Batch3

The long-term distribution of a Markov chain

The stationary probability distribution

33 We say that  j , j  S is a stationary probability distribution for a

Markov chain with transition matrix P if the following conditions hold
for all j in S :

j    i pij (2.4)
iS

j 0

j 1
jS

Note how (2.4) can be stated in the compact form    P where  is

viewed as a row vector.
____________

34 The interpretation of (2.4) is that, if we take  as our initial probability

distribution, that is to say P  X 0  i    i , then the distribution at
time 1 is again given by  :

P  X 1  j    P  X 1  jX 0  i  P  X 0  i  
  pij i   j
i S i S

The same is true at all times n  1 , so that  is an invariant probability

distribution; in fact the chain is then a stationary process in the sense
of Booklet 1.
____________

35 In general a Markov chain need not have a stationary probability

distribution, and if it exists it need not be unique.
____________

© IFE: 2019 Examinations Page 19

 Batch3

For instance no stationary probability distribution exists for the simple

random walk model on page 14, whereas in the simple random walk
with upper and lower boundaries on page 16, uniqueness depends on
the values of a , b . When the state space S is finite, the situation is
simple.
____________

36 A Markov chain with a finite state space has at least one stationary
probability distribution.
____________

The proof of this is beyond the syllabus.

As an example, we will compute a stationary probability for the NCD

model on page 12. The equations (2.4) read:

0  1
4 0  1
4 1 
1
4  2

1  3 4 0  1
4  2

 2  3 4  1 (2.5)

 2  1
4 3

 3  3 4  2  3 4  2  3 4  3
____________

37 This linear system is not linearly independent since adding up all the
equations results in an identity (this is a general feature of equations
   P due to the property  pij  1 ). Because of this we can
jS
discard any one of the equations, say the last one.

Note also that by linearity, any multiple of a solution of (2.5) is again a

solution; uniqueness comes only as a result of the normalisation
  j  1.
jS
____________

Page 20 © IFE: 2019 Examinations

 Batch3

For this reason, it is good practice to solve for the components of  in

terms of one of them (say  1 here), which we will refer to as the
working variable. The value of the working variable is determined at
the last step by normalisation.
____________

38 We now summarise the method and apply it to the above example.

Step 1: Discard one of the equations. Here the first or the last one are
obvious choices; delete the final one say.

Step 2: Select one of the  j ’s as a working variable. Here  1,  2  ,  2 

or  3 are reasonable choices; choose  1 .

Step 3: Rewrite remaining equations in terms of the working variable.

3 0   2  1 (a)

3 0   2   4 1 (b)

 2  3 4 1 (c)

4 2    3  0 (d)

Step 4: Solve the equations in terms of the working variable.

In general we might do this by Gaussian elimination, but here the

equations are so simple that the solution can be read off if we take
them in the right order:

p 2+ = 3 4 p 1

p0 =
p1
3 (4 - 3 4) = 13 12 p 1
p 2 - = p 1 ( -1 + 13 4 ) = 9 4 p 1

p 3 = 9p 1

© IFE: 2019 Examinations Page 21

 Batch3

Step 5: Solve for the working variable.

p = p 1 (13 12 ,1, 3 4 , 9 4 ,9)

p1
Âp j = 12 (13 + 12 + 9 + 27 + 108) = 169 12 p 1 = 1
j

p 1 = 12 169

Step 6: Combine the results of the last two steps to obtain the
solution.

   169 169 169 169 169 

13 , 12 , 9 , 27 , 108

____________

39 It is good practice to use the equation discarded earlier to verify that

the calculated solution is correct.
____________

40 The question of uniqueness of the stationary distribution is more

delicate than existence; we shall consider only irreducible chains. A
Markov chain is said to be irreducible if any state j can be reached
from any other state i . In other words, a chain is irreducible if, given
any pair of states i , j there exists an integer n with pij
n  0 .

____________

41 This is a property that can normally be judged from the transition

graph alone.
____________

The Markov chains of the models of no claims discount (page 10), no

claims discount (page 12) and the simple random walk (page 14) are
irreducible; so is the simple random walk with boundaries (page 16)
except when either boundary is absorbing (   1 or   1 ). Such
absorbing states occur in many practical situations (eg ruin).
____________

Page 22 © IFE: 2019 Examinations

 Batch3

42 An irreducible Markov chain with a finite state space has a unique

stationary probability distribution.
____________

The proof of this result is beyond the syllabus.

It is common for Markov chains with infinite state spaces to have no

stationary probability distribution, even if the chain is irreducible; this
is the case for the simple random walk on page 14.

The long-term behaviour of Markov chains

It is natural to expect the distribution of a Markov chain to tend to the

invariant distribution  for large times. This is why the stationary
distribution is so important: if the above convergence holds, pij
 n  will

be close to  j for an overwhelming fraction of the time in the long run.

Certain phenomena complicate the above picture somewhat.

____________

43 A state i is said to be periodic with period d  1 if a return to i is

possible only in a number of steps that is a multiple of d (ie pii
n  0

unless n  md for some integer m ).

____________

 n  can exist.
It is only for aperiodic states that lim pii
n 

One can check using the transition graphs that, in the no claims
discount models on pages 10 and page 12, all states are aperiodic
whereas in the model of the simple random walk without boundaries on
page 14, all states have period 2. Finally in the simple random walk
model with boundaries on page 16 all states are aperiodic unless both
 and  are either 0 or 1.
____________

44 If a Markov chain is irreducible all its states have the same period (or
all are aperiodic).
____________

© IFE: 2019 Examinations Page 23

 Batch3

We can now state a result on convergence to the stationary probability

distribution.
____________

45
 n  be the n -step transition probability of an irreducible aperiodic
Let pij
Markov chain on a finite state space. Then for every i and j :

lim pij
n  
j
n 

where  is the stationary probability distribution.

____________

Note how the above limit is independent of the starting state i . This
proof is beyond the syllabus.

Modelling using Markov chains

Using the principle of economy of effort, it is common to start the

modelling process by attempting to fit a simple stochastic model, such
as a Markov chain, to a set of observations. If tests show that this is
inadequate, a more sophisticated model can be attempted at the next
stage of the modelling process.

This section assumes that the model being fitted is

time-homogeneous. The situation is generally more complicated when
fitting a time-inhomogeneous model.
____________

Estimating transition probabilities

46 The first thing to fix when setting up a Markov model is the state space.
As shown by the example of a no claims discount model on page 12,
the state space which first springs to mind may not be the most
suitable and may need some modification before a Markov model can
be fitted.

Page 24 © IFE: 2019 Examinations

 Batch3

Once the state space is determined, however, the Markov model must
be fitted to the data by estimating the transition probabilities pij .
____________

47 Denote by x1, x 2 ,  x N the available observations and define:

 ni as the number of times t (1  t  N  1) such that x t = i ;

 nij as the number of times t (1  t  N  1) such that x t = i

and x t +1 = j .
____________

48 Thus nij is the observed number of transitions from state i to j , and

ni is the observed number of transitions from state i .

nij
Then the best estimate of pij is pˆ ij = .
ni
____________

49 If a confidence interval is required for a transition probability, the fact

that the conditional distribution of Nij given Ni is Binomial Ni , pij  
means that a confidence interval may be obtained by standard
techniques.
____________

Assessing the fit

The next step is to ensure that the fit of the model to the data is
adequate, or in other words to check that the Markov property seems
to hold.
____________

50 For a general Markov chain model a full verification of the Markov

property would involve a great deal of work and a voluminous supply
of data. In practice it is generally considered sufficient to look at
triplets of successive observations.
____________

© IFE: 2019 Examinations Page 25

 Batch3

51 Denote by nijk the number of times t (1  t  N  2) such that x t  i ,

x t 1  j and xt  2  k . If the Markov property holds we expect nijk to
be an observation from a binomial distribution with parameters nij and
p jk . A simple but effective test, therefore, is the chi-square goodness-
of-fit test based on the test statistic:

 nijk  nij pˆ jk 
2

  
2
i j k nij pˆ jk
____________

52 An additional method in frequent use for assessing goodness of fit is

to run some simulations of the fitted chain and to compare graphs of
the resulting trajectories with a graph of the process actually observed.
This method often highlights deficiencies that are missed by the
chi-square test.
____________

For example, given a sequence y 1, y 2 ,  y N of closing values of an

exchange rate, one model which suggests itself is to let x t be the
nearest integer to K log y t where K is a scaling constant of suitable
magnitude, and to model x1, x 2 ,  , x N as a random walk, with
transition probabilities:

pi ,i +1 = q , pi ,i -1 = f , pi ,i = 1 - q - f

The parameters q and f can be estimated quite satisfactorily in

practice, but a visual comparison of a simulated random walk with the
observed trajectory of x tends to show that the real exchange rate
remains relatively constant for long periods with occasional bursts of
increased volatility, whereas the Markov chain model is incapable of
simulating such behaviour.
____________

Page 26 © IFE: 2019 Examinations

 Batch3

Simulation

53 Simulating a time-homogeneous Markov chain is fairly straightforward,

as the Markov property means that the conditional distribution of X t 1
given the history of X up until time t is only dependent on X t .
____________

54 If the state space of X is finite, there are only a limited number of

distributions, all discrete, from which the program needs to be able to
sample; these can be listed individually, along with instructions telling
the program which distribution to use for each step.
____________

Models that assume an infinite state space usually have a simple

transition structure, often based on the distribution of the increments.
The random walk, which has independent increments, is one such
example; another might be a process which can only make transitions
of the form x  x + 1 or x  x - 1 , with respective probabilities q x
and 1 - q x .

In addition to commercial simulation packages, which are able to

simulate Markov chains without difficulty, even standard spreadsheet
software can easily cope with the practical aspects of estimating
transition probabilities and performing a simulation.

In R, the package markovchain can be used to create/simulate a

Markov chain.

(This package must be preloaded to produce the results shown below.)

As an example consider a Markov chain with three states: Employed

(Emp), Unemployed (claiming benefit) (Unemp) and Inactive in the
labour force (Inactive), measured at the end of each month. Suppose
the transition matrix, with the states in the order given above, is:

 0.8 0.1 0.1 

 
 0.5 0.4 0.1 
 0.4 0.0 0.6 
 

© IFE: 2019 Examinations Page 27

 Batch3

To create a Markov chain in R use:

Employment = new("markovchain", states = c("Emp",

"Unemp", "Inactive"),
transitionMatrix = matrix(data = c(0.8, 0.1,0.1,
0.5,0.4, 0.1,
0.4,0.0,0.6),byrow = byRow, nrow = 3),
name = "Employmt")

Suppose the process begins in state Emp. To see the probability

distribution after 3 and 6 months, use:

InitialState = c(1,0,0)
After3Months = InitialState * (Employment *
Employment * Employment)
After6Months = InitialState * (Employment^6)

After 6 months the probabilities are:

Employed 0.687, Unemployed 0.116, Inactive 0.197

____________

Page 28 © IFE: 2019 Examinations

 Batch3

Chapter 3 – The two-state Markov model and the Poisson model

55 The two-state model is illustrated in the figure below. There is an alive

state and a dead state, with transitions in one direction only.

x
a = alive d = dead

____________

The probability that a life alive at a given age will be dead at any
subsequent age is governed by the age-dependent transition
intensity m x +t (t ≥ 0) , in a way made precise by Assumption 2 below.
____________

56 Assumption 1

The probabilities that a life at any given age will be found in either state
at any subsequent age depend only on the ages involved and on the
state currently occupied. This is the Markov assumption.
____________

57 In particular, the past history of an individual – for example, spells of

sickness, occupation – is excluded from the model. If we knew these
factors, we could:

(a) treat each combination of factors as a separate model; in other

words, stratify the problem; or

(b) specify a model which took them into account; in other words,
treat the problem as one of regression.
____________

58 Assumption 2

dt q x +t = m x +t dt + o(dt ) (t ≥ 0 )
____________

59 For the purpose of inference, we restrict our attention to ages between

x and x + 1 , and introduce a further assumption.
____________

© IFE: 2019 Examinations Page 29

 Batch3

60 Assumption 3

m x +t is a constant m for 0 £ t < 1 .

____________

61 It is important to emphasise that this two-state model is not the same

as the model based on the future lifetime random variable Tx , which is
discussed in Booklet 4; we start with different assumptions. The
model in Booklet 4 is formulated in terms of a random variable T
representing the lifetime. The model is this booklet is in terms of a
transition intensity between states. It is easy to impose some mild
conditions under which the models are equivalent, but when we
consider more than one decrement these two formulations lead in
different directions.
____________

62 Since we have specified the model in terms of a transition intensity, we

must see how to compute probabilities. Consider the survival
probability t +dt px , and condition on the state occupied at age x + t .
By the Markov assumption (Assumption 1), nothing else affects the
probabilities of death or survival after age x + t .

t +dt p x = t px ¥ P [Alive at x + t + dt | Alive at x + t ]

+ t q x ¥ P [Alive at x + t + dt | Dead at x + t ]

= ( t px ¥ dt px +t ) + ( t q x ¥ 0)

= t px ¥ (1 - m x +t dt + o(dt ))

Therefore:

∂ t + dt p x- t px
p = lim
∂ t t x dt Æ0 + dt
o(dt )
= - t px m x +t + lim
dt Æ0 + dt
= - t p x m x +t (3.1)

Page 30 © IFE: 2019 Examinations

 Batch3

Ê t ˆ
So t px = exp Á - Ú m x + s ds ˜ as before.
Ë 0 ¯
____________

The important point is that it has been derived here strictly from the
assumptions of the two-state model, and that the method is easily
extended to models with more states. In the Markov framework, (3.1) is
an example of the Kolmogorov forward equations. These are
discussed in detail in Booklet 3.
____________

63 Next we define our observations. We suppose that we observe a total

of N lives during some finite period of observation, between the ages
of x and x + 1 .

We could suppose that lives were observed, or not, as a result of some

random mechanism (not depending on any parameter of interest), but
here, we suppose that data are analysed retrospectively, so we
regard N as a non-random quantity. We need not assume that we
observe the N lives simultaneously, nor need we assume that we
observe each life for the complete year of age. We do assume that all
N lives are identical and statistically independent.
____________

64 For i = 1, ..., N define:

 x + ai to be the age at which observation of the i th life starts

 x + bi to be the age at which observation of the i th life must

cease if the life survives to that age.

x + bi will be either x + 1 , or the age of the i th life when the

investigation ends, whichever is smaller.
____________

65 For simplicity we consider Type I censoring, but the approach can be

extended to more realistic forms of censoring.
____________

© IFE: 2019 Examinations Page 31

 Batch3

66 Define a random variable Di as follows:

Ï1 if the i th life is observed to die

Di = Ì
Ó0 if the i th life is not observed to die

Di is an example of an indicator random variable; it indicates the

occurrence of death.
____________

67 Define a random variable Ti as follows:

x + Ti = the age at which observation of the i th life ends.

Notice that Di and Ti are not independent, since:

Di = 0 ¤ Ti = bi , Di = 1 ¤ ai < Ti < bi
____________

68 It will often be useful to work with the time spent under observation, so
define Vi = Ti - ai . Vi is called the waiting time. It has a mixed
distribution, with a probability mass at the point bi - ai .
____________

69 The pair (Di ,Vi ) comprise a statistic, meaning that the outcome of our
observation is a sample (d i , v i ) drawn from the distribution of (Di ,Vi ) .

Page 32 © IFE: 2019 Examinations

 Batch3

Let fi (d i , v i ) be the joint distribution (Di ,Vi ) . It is easily written down

by considering the two cases Di = 0 and Di = 1 .

ÏÔ bi - ai px + ai (d i = 0)
fi (d i , v i ) = Ì
ÔÓ v i px + ai m x + ai +v i (d i = 1)

Ï Ê bi - ai ˆ
Ôexp Á - Ú m x + a +t dt ˜ (d i = 0)
Ô ÁË i ˜¯
Ô 0
=Ì
Ô Ê vi ˆ
Ô exp Á - Ú m x + ai +t dt ˜ m x + ai +v i (d i = 1)
ÔÓ ÁË 0 ˜¯

Ê vi ˆ
d
= exp Á - Ú m x + ai +t dt ˜ m x i+ a +v
ÁË 0 ˜¯ i i

Now assume that m x +t is a constant m for 0 £ t < 1 (this is the first

time we have needed Assumption 3) and fi (d i , v i ) takes on the simple
form:

fi (d i , v i ) = e - mv i m d i
____________

70 The joint probability function of all the (Di ,Vi ) , by independence, is:

i =N
’ e - mv i m d i = e - m (v1 + ... + v N ) m d i + ... + d N = e - m v m d
i =1

N N
where d = Â d i and v = Â v i . In other words define random variables
i =1 i =1
D and V to be the total number of deaths and the total waiting time,
respectively, and the joint probability function of all the (Di ,Vi ) can be
simply expressed in terms of D and V .
____________

© IFE: 2019 Examinations Page 33

 Batch3

71 The probability function immediately furnishes the likelihood for m :

L ( m ; d , v ) = e - mv m d

which yields the maximum likelihood estimate (MLE) for m :

mˆ = d / v

The estimate m̂ , being a function of the sample values d and v , can

itself be regarded as a sample drawn from the distribution of the
corresponding estimator:

m = D / V

As usual we are using capitals letters to denote random variables, and

lower case letters to denote sample values.
____________

72 It is important in applications to be able to estimate the moments of the

estimator m , for example to compare the experience with that of a
standard table. At least, we need to estimate E ÎÈ m ˚˘ and var ÎÈ m ˚˘ . The
following exact results are obtained:

E ÎÈDi - m Vi ˚˘ = 0 (3.2)

var ÈÎDi - m Vi ˘˚ = E ÈÎDi ˘˚ (3.3)

Note that the first of these can also be written as E ÈÎDi ˘˚ = m E ÈÎVi ˘˚ .
____________

Page 34 © IFE: 2019 Examinations

 Batch3

73 In the case where the {ai } and {bi } are known constants, this follows
from integrating/summing the probability function of (Di , Vi ) over all
possible events to obtain:

bi - ai
Ú e - m v i m dv i + e - m (bi - ai ) = 1
0

and then differentiating with respect to m , once to obtain the mean

and twice to obtain the variance.
____________

74 To find the asymptotic distribution of m , consider:

1 1 N

N
(D - m V ) = N Â (Di - mVi )
i =1

By the Central Limit Theorem:

(D - mV ) ~ Normal ÊËÁ 0, 2 ˆ¯˜

1 E [D ]
N N

Then note that (not rigorously):

N Ê D mV ˆ
lim ( m - m ) = lim ÁË N - N ˜¯
N Æ• N Æ• V

By the law of large numbers, V N Æ E ÎÈVi ˚˘ . So, asymptotically:

Ê m ˆ
m ~ Normal Á m ,
Ë E [V ] ˜¯
____________

75 In actuarial terminology, the observed waiting time at age x , which we

have denoted v , is often called the central exposed to risk and is
denoted E cx .
____________

© IFE: 2019 Examinations Page 35

 Batch3

The Poisson model

76 If we assume that we observe N individuals as before, for a total

of E xc person-years, and that the force of mortality is a constant m ,
then a Poisson model is given by the assumption that D has a
Poisson distribution with parameter m E xc .
____________

77 That is:

c
e - m E x ( m E xc )d
P[D = d ] =
d!
____________

78 Under the observational plan described above, the Poisson model is

not an exact model, since it allows a non-zero probability of more
than N deaths, but it is often a very good approximation.
____________

79 The Poisson likelihood leads to the following estimator of

(constant) m :

D
m =
E xc

The estimator m has the following properties:

m
E [ m ] = m and var[ m ] =
E xc

In practice, we will substitute m̂ for m to estimate these from the data.

____________

Page 36 © IFE: 2019 Examinations

 Batch3

m
80 Under the two-state model, E [ m ] = m and var[ m ] = , but the true
E[V ]
values of m and E [V ] are unknown and must be estimated from the
data as m̂ and E xc respectively.

So although the estimators are different, we obtain the same numerical

estimates of the parameter and of the first two moments of the
estimator, in either case.
____________

81 Once we have calculated m̂ , the estimated (constant) force of mortality

that applies over the age range x to x + 1 , we can use this to
estimate q x (the probability of dying over that year of age) as follows:

qˆ x = 1 - e - m
ˆ

____________

Comment on application

Having estimated piecewise constant intensities over single years of

age, we can use these (if required) to estimate the function m x as a
smooth function of age (the process of smoothing is called
graduation). For this purpose we usually assume that m̂
estimates m x +½ .

We can calculate any required probabilities from:

Ê t ˆ
t px = exp Á - Ú m x + s ds ˜
Ë 0 ¯

using numerical methods if necessary.

____________

© IFE: 2019 Examinations Page 37

 Batch3

PAST EXAM QUESTIONS

This section contains all the relevant exam questions from 2008 to 2017 that
are related to the topics covered in this booklet.

Solutions are given after the questions. These give enough information for
you to check your answer, including working, and also show you what an
outline examination answer should look like. Further information may be
available in the Examiners’ Report, ASET or Course Notes. (ASET can be
ordered from ActEd.)

We first provide you with a cross-reference grid that indicates the main
subject areas of each exam question. You can use this, if you wish, to
select the questions that relate just to those aspects of the topic that you
may be particularly interested in reviewing.

Alternatively, you can choose to ignore the grid, and attempt each question
without having any clues as to its content.

Page 38 © IFE: 2019 Examinations

 Batch3

Cross-reference grid

Question attempted
Time-homog / time-

Two-state Markov
Transition matrix
Markov property

Transition graph

Poisson model
stationary dist
Probabilities /
Irreducibility
State space

Periodicity
Question

inhomog

model
1   
2  
3     
4   
5  
6  
7  
8   
9  
10  
11   
12  
13 
14     
15  
16    
17     
18   
19 
20  
21     
22 
23    
24 
25    
26       
27     
28   
29   
30  
31   
32    
33  

© IFE: 2019 Examinations Page 39

 37
36
35
34
Question

Page 40

Markov property


State space




Transition graph




Transition matrix

Irreducibility

Periodicity
Probabilities /




stationary dist
Batch3

Time-homog /
Time-inhomog
Two-state Markov
model
Poisson model

Question attempted

© IFE: 2019 Examinations

 Batch3

1 Subject CT4 April 2008 Question 7

In a certain small country all listed companies are required to have their
accounts audited on an annual basis by one of the three authorised audit
firms (A, B and C). The terms of engagement of each of the audit firms
require that a minimum of two annual audits must be conducted by the newly
appointed firm. Whenever a company is able to choose to change auditors,
the likelihood that it will retain its auditors for a further year is (80%, 70%,
90%) where the current auditor is (A,B,C) respectively. If changing auditors
a company is equally likely to choose either of the alternative firms.

(i) A company has just changed auditors to firm A. Calculate the expected
number of audits which will be undertaken before the company changes
auditors again. [2]

(ii) Formulate a Markov chain which can be used to model the audit firm
used by a company, specifying:

(a) the state space

(b) the transition matrix [4]

(iii) Calculate the expected proportion of companies using each audit firm in
the long term. [5]
[Total 11]

2 Subject CT4 September 2008 Question 8

A No-Claims Discount system operated by a motor insurer has the following

four levels:

Level 1: 0% discount
Level 2: 25% discount
Level 3: 40% discount
Level 4: 60% discount

The rules for moving between these levels are as follows:

 Following a year with no claims, move to the next higher level, or remain
at Level 4.
 Following a year with one claim, move to the next lower level, or remain
at Level 1.
 Following a year with two or more claims, move down two levels, or
move to Level 1 (from level 2) or remain at Level 1.

© IFE: 2019 Examinations Page 41

 Batch3

For a given policyholder in a given year the probability of no claims is 0.85

and the probability of making one claim is 0.12.

(i) Write down the transition matrix of this No-Claims Discount process. [1]

(ii) Calculate the probability that a policyholder who is currently at Level 2

will be at level 2 after:

(a) one year

(b) two years. [3]

(iii) Calculate the long-run probability that a policyholder is in discount

Level 2. [5]
[Total 9]

3 Subject CT4 September 2008 Question 11

n
Consider the random variable defined by X n = Â Yi with each Yi mutually
i =1
independent with probability:

P ÈÎYi = 1˘˚ = p, P ÈÎYi = -1˘˚ = 1 - p 0 < p <1

(i) Write down the state space and transition graph of the sequence X n .
[2]

(ii) State, with reasons, whether the process:

(a) is aperiodic

(b) is reducible

(c) admits a stationary distribution. [3]

Consider j > i > 0 .

(iii) Derive an expression for the number of upward movements in the

sequence X n between t and (t + m ) if X t = i and X t + m = j . [2]

(iv) Derive expressions for the m-step transition probabilities pij( m ) . [3]

Page 42 © IFE: 2019 Examinations

 Batch3

(v) Show how the one-step transition probabilities would alter if X n was
restricted to non-negative numbers by introducing:

(a) a reflecting boundary at zero

(b) an absorbing boundary at zero. [2]

(vi) For each of the examples in part (v), explain whether the transition
probabilities pij( m ) would increase, decrease or stay the same.
(Calculation of the transition probabilities is not required.) [3]
[Total 15]

4 Subject CT4 April 2009 Question 2

(i) Explain what is meant by a time-homogeneous Markov chain. [2]

Consider the time-homogeneous two-state Markov chain with transition

matrix:

Ê1 - a a ˆ
ÁË b 1 - b ˜¯

(ii) Explain the range of values that a and b can take which result in this
being a valid Markov chain which is:

(a) irreducible

(b) periodic. [3]

[Total 5]

© IFE: 2019 Examinations Page 43

 Batch3

5 Subject CT4 April 2009 Question 12

A motor insurer operates a no claims discount system with the following

levels of discount {0%, 25%, 50%, 60%}.

The rules governing a policyholder’s discount level, based upon the number
of claims made in the previous year, are as follows:
 Following a year with no claims, the policyholder moves up one discount
level, or remains at the 60% level.
 Following a year with one claim, the policyholder moves down one
discount level, or remains at the 0% level.
 Following a year with two or more claims, the policyholder moves down
two discount levels (subject to a limit of the 0% discount level).

The number of claims made by a policyholder in a year is assumed to follow

a Poisson distribution with mean 0.30.

(i) Determine the transition matrix for the no claims discount system. [3]

(ii) Calculate the stationary distribution of the system, p . [5]

(iii) Calculate the expected average long-term level of discount. [1]

The following data shows the number of the insurer’s 130,200 policyholders
in the portfolio classified by the number of claims each policyholder made in
the last year. This information was used to estimate the mean of 0.30.

No claims 96,632
One claim 28,648
Two claims 4,400
Three claims 476
Four claims 36
Five claims 8

(iv) Test the goodness of fit of these data to a Poisson distribution with
mean 0.30. [5]

(v) Comment on the implications of your conclusion in (iv) for the average
level of discount applied. [2]
[Total 16]

Page 44 © IFE: 2019 Examinations

 Batch3

6 Subject CT4 September 2009 Question 5

(i) State the Markov property. [1]

A stochastic process X (t ) operates with state space S .

(ii) Prove that if the process has independent increments it satisfies the
Markov property. [3]

(iii) (a) Describe the difference between a Markov chain and a Markov
jump process.

(b) Explain what is meant by a Markov chain being irreducible. [2]

An actuarial student can see the office lift (elevator) from his desk. The lift
has an indicator which displays on which of the office’s five floors it is at any
point in time. For light relief the student decides to construct a model to
predict the movements of the lift.

(iv) Explain whether it would be appropriate to select a model which is:

(a) irreducible

(b) has the Markov property. [3]

[Total 9]

7 Subject CT4 September 2009 Question 7

A firm rents cars and operates from three locations – the Airport, the Beach
and the City. Customers may return vehicles to any of the three locations.

The company estimates that the probability of a car being returned to each
location is as follows:

Car returned to
Car hired from Airport Beach City

Airport 0.5 0.25 0.25

Beach 0.25 0.75 0
City 0.25 0.25 0.5

(i) Calculate the 2-step transition matrix. [2]

(ii) Calculate the stationary distribution p . [3]

© IFE: 2019 Examinations Page 45

 Batch3

It is suggested that the cars should be based at each location in proportion

to the stationary distribution.

(iii) Comment on this suggestion. [2]

(iv) Sketch, using your answers to parts (i) and (ii), a graph showing the
probability that a car currently located at the Airport is subsequently at
the Airport, Beach or City against the number of times the car has been
rented. [3]
[Total 10]

8 Subject CT4 April 2010 Question 4

A Markov Chain with state space {A, B, C} has the following properties:
 it is irreducible
 it is periodic
 the probability of moving from A to B equals the probability of moving
from A to C.

(i) Show that these properties uniquely define the process. [4]

(ii) Sketch a transition diagram for the process. [1]

[Total 5]

9 Subject CT4 April 2010 Question 10

An airline runs a frequent flyer scheme with four classes of member: in

ascending order Ordinary, Bronze, Silver and Gold. Members receive
benefits according to their class. Members who book two or more flights in a
given calendar year move up one class for the following year (or remain
Gold members), members who book exactly one flight in a given calendar
year stay at the same class, and members who book no flights in a given
calendar year move down one class (or remain Ordinary members).

Let the proportions of members booking 0, 1 and 2+ flights in a given year

be p0 , p1 and p2 + respectively.

(i) (a) Explain how this scheme can be modelled as a Markov chain.

(b) Explain why there must be a unique stationary distribution for the
proportion of members in each class. [3]

Page 46 © IFE: 2019 Examinations

 Batch3

(ii) Write down the transition matrix of the process. [1]

The airline’s research has shown that in any given year, 40% of members
book no flights, 40% book exactly one flight, and 20% book two or more
flights.

(iii) Calculate the stationary probability distribution. [5]

The cost of running the scheme per member per year is as follows:
 Ordinary members £0
 Bronze members £10
 Silver members £20
 Gold members £30

The airline makes a profit of £10 per passenger for every flight before taking
into account costs associated with the frequent flyer scheme.

(iv) Assess whether the airline makes a profit on the members of the
scheme. [4]
[Total 13]

10 Subject CT4 September 2010 Question 12

A pet shop has four glass tanks in which snakes for sale are held. The shop
can stock at most four snakes at any one time because:
 if more than one snake were held in the same tank, the snakes would
attempt to eat each other and
 having snakes loose in the shop would not be popular with the
neighbours.

The number of snakes sold by the shop each day is a random variable with
the following distribution:

Number of Snakes Potentially Sold in Day Probability

(if stock is sufficient)
None 0.4
One 0.4
Two 0.2

© IFE: 2019 Examinations Page 47

 Batch3

If the shop has no snakes in stock at the end of a day, the owner contacts
his snake supplier to order four more snakes. The snakes are delivered the
following morning before the shop opens. The snake supplier makes a
charge of C for the delivery.

(i) Write down the transition matrix for the number of snakes in stock when
the shop opens in a morning, given the number in stock when the shop
opened the previous day. [2]

(ii) Calculate the stationary distribution for the number of snakes in stock
when the shop opens, using your transition matrix in part (i). [4]

(iii) Calculate the expected long term average number of restocking orders
placed by the shop owner per trading day. [2]

If a customer arrives intending to purchase a snake, and there is none in

stock, the sale is lost to a rival pet shop.

(iv) Calculate the expected long term number of sales lost per trading day.
[2]

The owner is unhappy about losing these sales as there is a profit on each
sale of P . He therefore considers changing his restocking approach to
place an order before he has run out of snakes. The charge for the delivery
remains at C irrespective of how many snakes are delivered.

(v) Evaluate the expected number of restocking orders, and number of lost
sales per trading day, if the owner decides to restock if there are fewer
than two snakes remaining in stock at the end of the day. [5]

(vi) Explain why restocking when two or more snakes remain in stock
cannot optimise the shop’s profits. [2]

The pet shop owner wishes to maximise the profit he makes on snakes.

(vii) Derive a condition in terms of C and P under which the owner should
change from only restocking where there are no snakes in stock, to
restocking when there are fewer than two snakes in stock. [2]
[Total 19]

Page 48 © IFE: 2019 Examinations

 Batch3

11 Subject CT4 April 2011 Question 2

Distinguish between the conditions under which a Markov chain:

(a) has at least one stationary distribution

(b) has a unique stationary distribution

(c) converges to a unique stationary distribution. [3]

12 Subject CT4 April 2011 Question 4

Children at a school are given weekly grade sheets, in which their effort is
graded in four levels: 1 ‘Poor’, 2 ‘Satisfactory’, 3 ‘Good’ and 4 ‘Excellent’.
Subject to a maximum level of Excellent and a minimum level of Poor,
between each week and the next, a child has:
 a 20 per cent chance of moving up one level
 a 20 per cent chance of moving down one level
 a 10 per cent chance of moving up two levels
 a 10 per cent chance of moving down two levels.

Moving up or down three levels in a single week is not possible.

(i) Write down the transition matrix of this process. [2]

Children are graded on Friday afternoon in each week. On Friday of the first
week of the school year, as there is little evidence on which to base an
assessment, all children are graded ‘Satisfactory’.

(ii) Calculate the probability distribution of the process after the grading on
Friday of the third week of the school year. [3]
[Total 5]

© IFE: 2019 Examinations Page 49

 Batch3

13 Subject CT4 April 2011 Question 8 (part)

An investigation studied the mortality of infants aged under 1 year. The

following table gives details of 10 lives involved in the investigation. Infants
with no date of death given were still alive on their first birthday.

Life Date of birth Date of death

1 1 August 2008 –
2 1 September 2008 –
3 1 December 2008 1 February 2009
4 1 January 2009 –
5 1 February 2009 –
6 1 March 2009 1 December 2009
7 1 June 2009 –
8 1 July 2009 –
9 1 September 2009 –
10 1 November 2009 1 December 2009

(ii) Calculate the maximum likelihood estimate of the force of mortality,

using a two-state model and assuming that the force is constant. [3]

(iii) Hence estimate the infant mortality rate q 0 . [1]

[Total 6]

14 Subject CT4 April 2011 Question 12

Farmer Giles makes hay each year and he makes far more than he could
possibly store and use himself, but he does not always sell it all. He has
decided to offer incentives for people to buy large quantities so it does not sit
in his field deteriorating. He has devised the following “discount” scheme.

He has a Base price B of £8 per bale. Then he has three levels of discount:
Good price G is a 10% discount, Loyalty price L is a 20% discount and
Super price S is a 25% discount on the Base price.

 Customers who increase their order compared with last year move to
one higher discount level, or remain at level S .
 Customers who maintain their order from last year stay at the same
discount level.

Page 50 © IFE: 2019 Examinations

 Batch3

 Customers who reduce their order from last year drop one level of
discount or remain at level B provided that they maintained or
increased their order the previous year.
 Customers who reduce their order from last year drop two levels of
discount if they also reduced their order last year, subject to remaining
at the lowest level B .

(i) Explain why a process with the state space {B, G, L, S} does not display
the Markov property. [2]

(ii) (a) Define any additional state(s) required to model the system with the
Markov property.

(b) Construct a transition graph of this Markov process, clearly labelling

all the states. [3]

Farmer Giles thinks that each year customers have a 60% likelihood of
increasing their order and a 30% likelihood of reducing it, irrespective of the
discount level they are currently on.

(iii) (a) Write down the transition matrix for the Markov process.

(b) Calculate the stationary distribution.

(c) Hence calculate the long-run average price he will get for each bale
of hay. [8]

(iv) Calculate the probability that a customer who is currently paying the
Loyalty price L will be paying L in two years’ time. [3]

(v) Suggest reasons why the assumptions Farmer Giles has made about
his customers’ behaviour may not be valid. [3]
[Total 19]

© IFE: 2019 Examinations Page 51

 Batch3

15 Subject CT4 September 2011 Question 1

The diagrams below show three Markov chains, where arrows indicate a
non-zero transition probability:

State whether each of the chains is:

(a) irreducible

(b) periodic, giving the period where relevant. [3]

A State 1 State 2

B State 1 State 2 State 3

State 1 State 2

State 3 State 4

16 Subject CT4 September 2011 Question 11

An actuary walks from his house to the office each morning, and walks back
again each evening. He owns two umbrellas. If it is raining at the time he
sets off, and one or both of his umbrellas is available, he takes an umbrella
with him. However if it is not raining at the time he sets off he always forgets
to take an umbrella.

Assume that the probability of it raining when he sets off on any particular
journey is a constant p , independent of other journeys.

Page 52 © IFE: 2019 Examinations

 Batch3

This situation is examined as a Markov Chain with state space {0,1, 2}

representing the number of his umbrellas at the actuary’s current location
(office or home) and each time step representing one journey.

(i) Explain why the transition graph for this process is given by:

1 p
Zero Two One 1–p

1–p p

[3]

(ii) Derive the transition matrix for the number of umbrellas at the actuary’s
house before he leaves each morning, based on the number before he
leaves the previous morning. [3]

(iii) Calculate the stationary distribution for the Markov Chain. [3]

(iv) Calculate the long run proportion of journeys (to or from the office) on
which the actuary sets out in the rain without an umbrella. [2]

The actuary considers that the weather at the start of a journey, rather than
being independent of past history, depends upon the weather at the start of
the previous journey. He believes that if it was raining at the start of a
journey the probability of it raining at the start of the next journey is r
(0 < r < 1) , and if it was not raining at the start of a journey the probability of
it raining at the start of the next journey is s (0 < s < 1, r π s ) .

(v) Write down the transition matrix for the Markov Chain for the weather. [1]

(vi) Explain why the process with three states {0,1, 2} , being the number of
his umbrellas at the actuary’s current location, would no longer satisfy
the Markov property. [2]

(vii) Describe the additional state(s) needed for the Markov property to be
satisfied, and draw a transition diagram for the expanded system. [4]
[Total 18]

© IFE: 2019 Examinations Page 53

 Batch3

17 Subject CT4 April 2012 Question 11

The series Yi records, for each time period i , whether a car driver is
accident-free during that period ( Yi = 0 ) or has at least one accident
( Yi = 1 ).
i
Define X i = Â Yj with state space {0,1,2,} .
j =1

An insurer makes an assumption about the driver’s accident proneness by

considering that the probability of a driver having at least one accident is
related to the proportion of previous time periods in which the driver had at
least one accident as follows:

1Ê X ˆ 1
P (Yn +1 = 1) = 1 + n ˜ for n ≥ 1 , with P (Y1 = 1) = .
4 ÁË n ¯ 2

(i) Demonstrate that the series X i satisfies the Markov property, whilst Yi
does not. [2]

(ii) Explain whether the series X i is:

(a) irreducible

(b) time-homogeneous. [3]

(iii) Draw the transition graph for X i covering all transitions which could
occur in the first three time periods, including the transition probabilities.
[4]

(iv) Calculate the probability that the driver has accidents during exactly two
of the first three time periods. [2]

(v) Comment on the appropriateness of the insurer’s assumption about

accident proneness. [2]
[Total 13]

Page 54 © IFE: 2019 Examinations

 Batch3

18 Subject CT4 April 2012 Question 12

A company operates a sick pay scheme as follows:

 Healthy employees pay a percentage of salary to fund the scheme.
 For the first two consecutive months an employee is sick, the sick
pay scheme pays their full salary.
 For the third and subsequent consecutive months of sickness the
sick pay is reduced to 50% of full salary.

To simplify administration the scheme operates on whole months only, that

is for a particular month’s payroll an employee is either healthy or sick for the
purpose of the scheme.

The company’s experience is that 10% of healthy employees become sick

the following month, and that sick employees have a 75% chance of being
healthy the next month.

The scheme is to be modelled using a Markov chain.

(i) Explain what is meant by a Markov chain. [1]

(ii) Identify the minimum number of states under which the payments under
the scheme can be modelled using a time-homogeneous Markov chain,
specifying these states. [2]

(iii) Draw a transition graph for this Markov chain. [2]

(iv) Derive the stationary distribution for this process. [4]

(v) Calculate the minimum percentage of salary which healthy employees

should pay for the scheme to cover the sick pay costs. [2]

(vi) Calculate the contributions required if, instead, sick pay continued at
100% of salary indefinitely. [2]

(vii) Comment on the benefit to the scheme of the reduction in sick pay to
50% from the third month. [2]
[Total 15]

© IFE: 2019 Examinations Page 55

 Batch3

19 Subject CT4 September 2012 Question 5

A no claims discount system operates with three levels of discount, 0%, 15%
and 40%. If a policyholder makes no claim during the year he moves up a
level of discount (or remains at the maximum level). If he makes one claim
during the year he moves down one level of discount (or remains at the
minimum level) and if he makes two or more claims he moves down to, or
remains at, the minimum level.

The probability for each policyholder of making two or more claims in a year
is 25% of the probability of making only one claim.

The long-term probability of being at the 15% level is the same as the
long-term probability of being at the 40% level.

(i) Derive the probability of a policyholder making only one claim in a given
year. [4]

(ii) Determine the probability that a policyholder at the 0% level this year
will be at the 40% level after three years. [2]

(iii) Estimate the probability that a policyholder at the 0% level this year will
be at the 40% level after 20 years, without calculating the associated
transition matrix. [3]
[Total 9]

20 Subject CT4 September 2012 Question 6

(i) Define the stationary distribution of a Markov chain. [2]

A baseball stadium hosts a match each evening. As matches take place in

the evening, floodlights are needed. The floodlights have a tendency to
break down. If the floodlights break down, the game has to be abandoned
and this costs the stadium $10,000. If the floodlights work throughout one
match there is a 5% chance that they will fail and lead to the abandonment
of the next match.

The stadium has an arrangement with the Floodwatch repair company who
are brought in the morning after a floodlight breakdown and charge $1,000
per day. There is a 60% chance they are able to repair the floodlights such
that the evening game can take place and be completed without needing to
be abandoned. If they are still broken the repair company is used (and paid)
again each day until the lights are fixed, with the same 60% chance of fixing
the lights each day.

Page 56 © IFE: 2019 Examinations

 Batch3

(ii) Write down the transition matrix for the process which describes
whether the floodlights are working or not. [1]

(iii) Derive the long run proportion of games which have to be abandoned.
[3]

The stadium manager is unhappy with the number of games being

abandoned, and contacts the Light Fantastic repair company who are
estimated to have an 80% chance of repairing floodlights each day.
However Light Fantastic will charge more than Floodwatch.

(iv) Calculate the maximum amount the stadium should be prepared to pay
Light Fantastic to improve profitability. [4]
[Total 10]

21 Subject CT4 April 2013 Question 11

(i) Explain what is meant by a time-inhomogeneous Markov chain and give

an example of one. [2]

A No Claims Discount system is operated by a car insurer. There are four

levels of discount: 0%, 10%, 25% and 40%. After a claim-free year a
policyholder moves up one level (or remains at the 40% level). If a
policyholder makes one claim in a year he or she moves down one level (or
remains at the 0% level). A policyholder who makes more than one claim in
a year moves down two levels (or moves to or remains at the 0% level).
Changes in level can only happen at the end of each year.

(ii) Describe, giving an example, the nature of the boundaries of this

process. [2]

(iii) (a) State how many states are required to model this as a Markov
chain.

(b) Draw the transition graph. [2]

The probability of a claim in any given month is assumed to be constant at

0.04. At most one claim can be made per month and claims are
independent.

(iv) Calculate the proportion of policyholders in the long run who are at the
25% level. [6]

(v) Discuss the appropriateness of the model. [3]

[Total 15]

© IFE: 2019 Examinations Page 57

 Batch3

22 Subject CT4 September 2013 Question 2

The two football teams in a particular city are called United and City and
there is intense rivalry between them. A researcher has collected the
following history on the results of the last 20 matches between the teams
from the earliest to the most recent, where:

U indicates a win for United

C indicates a win for City
D indicates a draw.

UCCDDUCDCUUDUDCCUDCC

The researcher has assumed that the probability of each result for the next
match depends only on the most recent result. He therefore decides to fit a
Markov chain to this data.

(i) Estimate the transition probabilities for the Markov chain. [3]

(ii) Estimate the probability that United will win at least two of the next three
matches against City. [3]
[Total 6]

Page 58 © IFE: 2019 Examinations

 Batch3

23 Subject CT4 September 2013 Question 5

A motor insurer offers a No Claims Discount scheme which operates as

follows. The discount levels are {0%, 25%, 50%, 60%}. Following a claim-
free year a policyholder moves up one discount level (or stays at the
maximum discount). After a year with one or more claims the policyholder
moves down two discount levels (or moves to, or stays in, the 0% discount
level).

The probability of making at least one claim in any year is 0.2.

(i) Write down the transition matrix of the Markov chain with state space
{0%, 25%, 50%, 60%}. [2]

(ii) State, giving reasons, whether the process is:

(a) irreducible

(b) aperiodic. [2]

(iii) Calculate the proportion of drivers in each discount level in the

stationary distribution. [4]

The insurer introduces a ‘protected’ No Claims Discount scheme, such that if

the 60% discount is reached the driver remains at that level regardless of
how many claims they subsequently make.

(iv) Explain, without doing any further calculations, how the answers to parts
(ii) and (iii) would change as a result of introducing the ‘protected’
No Claims Discount scheme. [3]
[Total 11]

© IFE: 2019 Examinations Page 59

 Batch3

24 Subject CT4 April 2014 Question 6

In the Poisson model, if the average number of events occurring to each

member of a population in a given period of time is l , then the probability of
observing exactly d events occurring to any one individual in the same
period of time is.
exp( - l )l d
P ÎÈD = d ˚˘ =
d!

(i) Derive the maximum likelihood estimator under the Poisson model of
the average rate at which events occur, m , in a population where the
exposed to risk for each person i is E . [4]

A university runs a bus service between its teaching campus and its student
halls of residence. Traffic conditions mean that the arrival of buses at the
bus stop on the teaching campus can be considered to follow a Poisson
process.

The university decided to commission a study of how long students typically

have to wait at the bus stop for a bus to arrive. Students were asked to
record the time they arrive at the stop, and the time the next bus arrived.
Students who became tired of waiting at the stop and left before the next bus
arrived were asked to record the time they left. Below are given data from
10 students.

Student Time arrived Time left (if left before Time next bus
next bus arrived) arrived
1 4:00 pm 4:05 pm
2 4:10 pm 4:35 pm
3 4:20 pm 4:30 pm
4 4:30 pm 4:35 pm
5 4:40 pm 4:50 pm
6 4:45 pm 4:50 pm
7 4:55 pm 5:05 pm
8 5:00 pm 5:20 pm
9 5:10 pm 5:40 pm
10 5:10 pm 6:10 pm

(ii) Calculate the maximum likelihood estimate of the hourly rate at which
buses arrive at the bus stop, using the Poisson estimator, and assuming
that only one bus arrived at any given time. [3]

(iii) Comment on the use of the Poisson model for this investigation. [3]
[Total 10]

Page 60 © IFE: 2019 Examinations

 Batch3

25 Subject CT4 April 2014 Question 10

An industrial kiln is used to produce batches of tiles and is run with a

standard firing cycle. After each firing cycle is finished, a maintenance
inspection is undertaken on the heating element which rates it as being in
Excellent, Good or Poor condition, or notes that the element has Failed.

The probabilities of the heating element being in each condition at the end of
a cycle, based on the condition at the start of the cycle are as follows:

START END
Excellent Good Poor Failed
Excellent 0.5 0.2 0.2 0.1
Good 0.5 0.3 0.2
Poor 0.5 0.5
Failed 1

(i) Write down the name of the stochastic process which describes the
condition of a single heating element over time. [1]

(ii) Explain whether the process describing the condition of a single heating
element is:

(a) irreducible

(b) periodic. [2]

(iii) Derive the probability that the condition of a single heating element is
assessed as being in Poor condition at the inspection after two cycles, if
the heating element is currently in Excellent condition. [2]

If the heating element fails during the firing cycle, the entire batch of tiles in
the kiln is wasted at a cost of £1,000. Additionally, a new heating element
needs to be installed at a cost of £50, which will, of course, be in Excellent
condition.

(iv) Write down the transition matrix for the condition of the heating element
in the kiln at the start of each cycle, allowing for replacement of failed
heating elements. [2]

(v) Calculate the long-term probabilities for the condition of the heating
element in the kiln at the start of a cycle. [4]

The kiln is fired 100 times per year.

© IFE: 2019 Examinations Page 61

 Batch3

(vi) Calculate the expected annual cost incurred due to failures of heating
elements. [2]

The company is concerned about the cost of ruined tiles and decides to
change its policy to replace the heating element if it is rated as in Poor
condition.

(vii) Evaluate the impact of the change in replacement policy on the

profitability of the company. [6]
[Total 19]

26 Subject CT4 September 2014 Question 5

A sports league has two divisions {1,2} with Division 1 being the higher.
Each season the bottom team in Division 1 is relegated to Division 2, and
the top team in Division 2 is promoted to Division 1.

Analysis of the movements of teams between divisions indicates that the

probabilities of finishing top or bottom of a division differs if a team has just
been promoted or relegated, compared with the probabilities in subsequent
seasons.

The probabilities are as follows:

Finishing If promoted If relegated If neither promoted

position previous previous nor relegated
season season previous season
Top 0.1 0.25 0.15
Bottom 0.3 0.25 0.15
Other 0.6 0.5 0.7

(i) Write down the minimum number of states required to model this as a
Markov chain. [1]

(ii) Draw a transition graph for the Markov chain. [3]

(iii) Write down the transition matrix for the Markov chain. [2]

(iv) Explain whether the Markov chain is:

(a) irreducible

(b) aperiodic. [2]

Page 62 © IFE: 2019 Examinations

 Batch3

Team A has just been promoted to Division 1.

(v) Calculate the minimum number of seasons before there is at least a

60% probability of Team A having been relegated to Division 2. [3]
[Total 11]

27 Subject CT4 September 2014 Question 6

A motor insurance company offers annually renewable policies. To

encourage policyholders to renew each year it offers a No Claims Discount
system which reduces the premiums for those people who claim less often.
There are four levels of premium:

0: no discount
1: 15% discount
2: 25% discount
3: 40% discount

A policyholder who does not make a claim in the year moves up one level of
discount the following year (or stays at the maximum level).

A policyholder who makes one or more claims in a year moves down one
level of discount if they did not claim in the previous year (or remains at the
lowest level) but if they made at least one claim in the previous year they
move down two levels of discount (subject to not going below the lowest
level).

(i) (a) Explain how many states are required to model this as a Markov
chain.

(b) Draw the transition graph of the process. [3]

The probability p of making at least one claim in any year is constant and
independent of whether a claim was made in the previous year.

(ii) Calculate the proportion of policyholders who are at the 25% discount
level in the long run given that the proportion at the 40% level is nine
times that at the 15% level. [6]

(iii) (a) Explain how the state space of the process would change if the
probability of making a claim in any one year depended upon
whether a claim was made in the previous year.

(b) Write down the transition matrix for this new process. [4]
[Total 13]

© IFE: 2019 Examinations Page 63

 Batch3

28 Subject CT4 April 2015 Question 7

(i) Describe what is meant by a Markov chain. [2]

A simplified model of the internet consists of the following websites with links
between the websites as shown in the diagram below.

N(ile) B(anana)

C(heep) H(andbook)

An internet user is assumed to browse by randomly clicking any of the links

on the website he is on with equal probability.

(ii) Calculate the transition matrix for the Markov chain representing which
website the internet user is on. [2]

(iii) Calculate, of the total number of visits, what proportion are made to
each website in the long term. [4]
[Total 8]

Page 64 © IFE: 2019 Examinations

 Batch3

29 Subject CT4 September 2015 Question 10

A profession has examination papers in two subjects, A and B, each of

which is marked by a team of examiners. After each examination session,
examiners are given the choice of remaining on the same team, switching to
the other team, or taking a session’s holiday.

In recent sessions, 10% of subject A’s examiners have elected to switch to

subject B and 10% to take a holiday. Subject B is more onerous to mark
than subject A, and in recent sessions, 20% of subject B’s examiners have
elected to take a holiday in the next session, with 20% moving to subject A.

After a session’s holiday, the profession allocates examiners equally

between subjects A and B. No examiner is permitted to take holiday for two
consecutive sessions.

(i) Sketch the transition graph for the process. [2]

(ii) Determine the transition matrix for this process. [2]

(iii) Calculate the proportion of the profession’s examiners marking for

subjects A and B in the long run. [4]

The profession considers that in future, an equal number of examiners is

likely to be required for each subject. It proposes to try to ensure this by
adjusting the proportion of those examiners on holiday who, when they
return to marking, are allocated to subjects A and B.

(iv) Calculate the proportion of examiners who, on returning from holiday,

should be allocated to subject B in order to have an equal number of
examiners on each subject in the long run. [4]
[Total 12]

© IFE: 2019 Examinations Page 65

 Batch3

30 Subject CT4 April 2016 Question 10

In a small country there are only four authorised insurance companies, A, B,

C and D. The law in this country requires homeowners to take out buildings
insurance from an authorised insurance company. All policies provide cover
for a period of one year.

Based on analysis of the compliance database used to check that every

home is insured, the probabilities of a homeowner transferring between the
four companies at the end of each year are considered to be described by
the following transition matrix:

A Ê 0.5 0.2 0.2 0.1 ˆ

B Á 0.2 0.6 0.1 0.1 ˜
Á ˜
C Á 0.3 0.2 0.4 0.1 ˜
Á ˜
D Ë 0 0.2 0.2 0.6¯

Yolanda has just bought her policy from Company C for the first time.

(i) Calculate the probability that Yolanda will be covered by Company C for
at least five years before she changes provider. [2]

Zachary took out a policy with Company A in January 2013. Unfortunately,

Zachary’s house burnt down on 12 March 2015.

(ii) Calculate the probability that Company A does NOT cover Zachary’s
home at the time of the fire. [2]

(iii) Calculate the expected proportions of homeowners who are covered by

each insurance company in the long run. [4]

Company A makes an offer to buy Company D. It bases its purchase price

on the assumption that homeowners who would previously have purchased
policies from Company A or Company D would now buy from the combined
company, to be called Addda.

(iv) Determine the transition matrix which will apply after the takeover if
Company A’s assumption about homeowners’ behaviour is correct. [2]

(v) Comment on the appropriateness of Company A’s assumption. [2]

[Total 12]

Page 66 © IFE: 2019 Examinations

 Batch3

31 Subject CT4 September 2016 Question 2

The diagrams below show three Markov chains, where arrows indicate a
non-zero transition probability.

A Markov Chain 1

State 1

State 2 State 3

B Markov Chain 2

State 1 State 2

State 3 State 4

C Markov Chain 3

State 1 State 2

State whether each of the chains is:

 irreducible
 periodic, giving the period. [3]

© IFE: 2019 Examinations Page 67

 Batch3

32 Subject CT4 September 2016 Question 11

An individual’s marginal tax rate depends upon his or her total income during
a calendar year and may be 0% (that is, he or she is a non-taxpayer), 20%
or 40%.

The movement in the individual’s marginal tax rate from year to year is
believed to follow a Markov chain with a transition matrix as follows:

0% Ê1 - b - b 2 b b2 ˆ
Á ˜
20% Á b 1 - 3b 2b ˜
40% Á 2 2˜
Ë b b 1- b - b ¯

(i) Draw the transition diagram of the process, including the transition
rates. [2]

(ii) Determine the range of values of b for which this is a valid transition
matrix. [3]

(iii) Explain whether the chain is:

(a) irreducible

(b) periodic

including whether this depends on the value of b . [2]

The value of b has been estimated as 0.1.

(iv) Calculate the long term proportion of taxpayers at each marginal rate. [4]

Lucy pays tax at a marginal rate of 20% in 2011.

(v) Calculate the probabilities that Lucy’s marginal tax rate in 2013 is:

(a) 0%

(b) 20%

(c) 40%. [2]

[Total 13]

Page 68 © IFE: 2019 Examinations

 Batch3

33 Subject CT4 April 2017 Question 3

(i) Define a Markov Chain. [2]

(ii) Describe the difference between a time-homogeneous and a time-

inhomogeneous Markov Chain, giving an example of each. [2]
[Total 4]

34 Subject CT4 April 2017 Question 5

A city operates a bicycle rental scheme. Bicycles are stored in racks at

locations around the city and may be rented for a fee and ridden from one
location and deposited at another, provided there is space in the rack. The
rack outside the actuarial profession’s headquarters in that city has spaces
for four bicycles.

The profession would like the city to increase the size of the rack. The city
has said it will do so if the profession can demonstrate that, in the long run,
the rack is full or empty for more than 35 per cent of the working day. The
profession commissions a study to monitor the rack every 10 minutes during
the working day.

The study shows that, on average:

 there is a probability of 0.3 that the number, m , of bicycles in the rack
will increase by 1 over a 10-minute interval (where 0 £ m < 4 ).
 there is a probability of 0.2 that the number of bicycles in the rack will
decrease by 1 over a 10-minute interval (where 0 £ m < 4 ).
 the probability of more than one increase or decrease per 10-minute
interval can be regarded as 0.

(i) Give the transition matrix for the number of bicycles in the rack. [2]

(ii) Determine whether the city will increase the size of the rack. [6]

(iii) Comment on whether an increase in the size of the rack will reduce the
proportion of time for which the rack is empty or full. [2]
[Total 10]

© IFE: 2019 Examinations Page 69

 Batch3

35 Subject CT4 April 2017 Question 9

A journalist has just been appointed as a foreign correspondent covering

news stories in three island countries Atlantis, Beachy and Coral. He will
spend each day in the country likely to have the most exciting news, taking
the flights available between each country which go once per day at the end
of the day.

The previous foreign correspondent tells him ‘If you want to know how many
flights you are likely to take, I estimate my movements have been like this’
and she drew this diagram showing the transition probabilities:

(i) Give the transition matrix for this process. [1]

On his first day in the job the new foreign correspondent will be in Atlantis.

(ii) Calculate the probability that the foreign correspondent will be in each
country in his third day in the job. [2]

The previous correspondent also reported that Beachy must be the most
interesting of the islands in terms of news because she spent 41.9% of her
time there compared with 32.6% on Atlantis and 25.6% on Coral.

Page 70 © IFE: 2019 Examinations

 Batch3

(iii) Sketch a graph showing the probability that the journalist is in each
country over time, using the information above. [3]

(iv) Calculate the proportion of days on which the foreign correspondent will
take a flight. [1]

The first time the foreign correspondent visits each of the countries he takes
a photograph to mark the occasion.

(v) Identify a suitable state space for modelling as a Markov chain which
countries he has visited so far. [2]

(vi) Draw a transition diagram for the possible transitions between these
states. [3]
[Total 12]

36 Subject CT4 September 2017 Question 1

A Markov chain has the following transition graph:

The following is a partially completed transition matrix for this Markov Chain:

A Ê 0.2 - - ˆ
B Á - - 1.0 ˜
Á ˜
C ÁË - - 0.4˜¯

© IFE: 2019 Examinations Page 71

 Batch3

(i) Determine the remaining entries in the transition matrix. [2]

(ii) Explain whether each of the following is a valid sample path for this
process.

Path 1:

Path 2:

[2]
[Total 4]

Page 72 © IFE: 2019 Examinations

 Batch3

37 Subject CT4 September 2017 Question 8

A company has for many years offered a car insurance policy with four levels
of No Claims Discount (NCD): 0%, 15%, 30% and 40%. A policyholder who
does not claim in a year moves up one level of discount, or remains at the
highest level. A policyholder who claims one or more times in a year moves
down a level of discount or remains at the lowest level. The company pays
a maximum of three claims in any year on any one policy.

The company has established that:

 the arrival of claims follows a Poisson process with a rate of 0.35 per
year
 the average cost per claim is £2,500
 the proportion of policyholders at each level of discount is as follows:

Discount level Proportion of policyholders

0% 4.4%
15% 10.5%
30% 25.1%
40% 60.0%

(i) Calculate the premium paid by a policyholder at the 40% discount level
ignoring expenses and profit. [4]

The company has decided to introduce a protected NCD feature whereby

policyholders can make one claim on their policy in a year and, rather than
move down a level of discount, remain at the level they are at. All other
features of the policy remain the same.

(ii) Draw the transition graph for this process. [2]

(iii) Calculate the premium paid, in the long term, by a policyholder at the
40% discount level of the policy with protected NCD, ignoring expenses
and profit. [6]

(iv) Discuss THREE issues with the policy with protected NCD which may
each be either a disadvantage or an advantage to the company. [3]
[Total 15]

© IFE: 2019 Examinations Page 73

 Batch3

SOLUTIONS TO PAST EXAM QUESTIONS

The solutions presented here are outline solutions for you to use to check
your answers. See ASET for full solutions.

1 Subject CT4 April 2008 Question 7

(i) Expected number

Let N denote the number of audits that are undertaken before the company
changes auditors again. The possible values of N are 2, 3, 4, …. The
associated probabilities are:

P (N = 2) = 1 ¥ 0.2
P (N = 3) = 1 ¥ 0.8 ¥ 0.2
P (N = 4) = 1 ¥ 0.82 ¥ 0.2

and so on.

Hence:

( ) (
E (N ) = 2 (0.2) + 3 (0.8 ¥ 0.2) + 4 0.82 ¥ 0.2 + 5 0.83 ¥ 0.2 +  )
(
= 0.2 2 + 3 ¥ 0.8 + 4 ¥ 0.82 + 5 ¥ 0.83 + )

We can write this as:

(
E (N ) = 0.2 È 2 + 2 ¥ 0.8 + 2 ¥ 0.82 + 2 ¥ 0.83 + 
ÍÎ )
(
+ 0.8 + 0.82 + 0.83 +  )
(
+ 0.82 + 0.83 +  )
(
+ 0.83 +  )
+ ˘˚

Page 74 © IFE: 2019 Examinations

 Batch3

Each line in the above expression is a geometric series. So:

È 2 0.8 0.82 0.83 ˘

E (N ) = 0.2 Í + + + + ˙
ÎÍ 1 - 0.8 1 - 0.8 1 - 0.8 1 - 0.8 ˚˙
= 2 + 0.8 + 0.82 + 0.83 + 
0.8
= 2+
1 - 0.8
=6

(ii) Markov chain

(a) State space

Let’s define the states as follows:

1: Current auditor is A, and A did not carry out the previous audit.
2: Current auditor is A, and A carried out the previous audit.
3: Current auditor is B, and B did not carry out the previous audit.
4: Current auditor is B, and B carried out the previous audit.
5: Current auditor is C, and C did not carry out the previous audit.
6: Current auditor is C, and C carried out the previous audit.

This process has a discrete time set (since time is measured by number of
audits), a discrete state space (as described above), and satisfies the
Markov property (since the probability of being in any given state in one time
unit from now depends only on the current state). Hence the process is a
Markov chain.

(b) Transition matrix

Keeping the states in the order given above, the transition matrix is:

Ê 0 1 0 0 0 0 ˆ
Á 0 0.8 0.1 0 0.1 0 ˜
Á ˜
Á 0 0 0 1 0 0 ˜
P=Á ˜
Á 0.15 0 0 0.7 0.15 0 ˜
Á 0 0 0 0 0 1˜
Á ˜
Ë 0.05 0 0.05 0 0 0.9¯

© IFE: 2019 Examinations Page 75

 Batch3

(iii) Long-term proportions

Let (p 1 p 2 p 3 p 4 p 5 p 6 ) denote the stationary distribution. Then:

(p 1 p 2 p 3 p 4 p 5 p 6 ) P = (p 1 p 2 p 3 p 4 p 5 p 6 )

which gives us the following system of equations:

0.15p 4 + 0.05p 6 = p 1 (1)

p 1 + 0.8p 2 = p 2 fi p 1 = 0.2p 2 fi p 2 = 5p 1 (2)

0.1p 2 + 0.05p 6 = p 3 (3)

10
p 3 + 0.7p 4 = p 4 fi p 3 = 0.3p 4 fi p 4 = p3 (4)
3

0.1p 2 + 0.15p 4 = p 5 (5)

p 5 + 0.9p 6 = p 6 fi p 5 = 0.1p 6 fi p 6 = 10p 5 (6)

Also, since the proportions have to add to 1:

p1 + p 2 + p 3 + p 4 + p 5 + p 6 = 1 (7)

We can proceed by expressing everything in terms of p 1 .

Substituting (4) and (6) into (1) gives:

Ê 10 ˆ
0.15 Á p 3 ˜ + 0.05 (10p 5 ) = p 1
Ë 3 ¯

ie:

0.5p 3 + 0.5p 5 = p 1 (8)

Page 76 © IFE: 2019 Examinations

 Batch3

Now substituting (2) and (6) into (3) gives:

0.1(5p 1) + 0.05 (10p 5 ) = p 3

ie:

0.5p 1 + 0.5p 5 = p 3 (9)

Subtracting (9) from (8):

0.5p 3 - 0.5p 1 = p 1 - p 3
fi 1.5p 3 = 1.5p 1
fi p 3 = p1 (10)

and from (4):

10
p4 = p1
3

Also, substituting (10) into (8) gives:

0.5p 1 + 0.5p 5 = p 1 fi p 5 = p 1

and from (6):

p 6 = 10p 1

Now, using the fact that the proportions add to 1, we have:

Ê 10 ˆ 3
p 1 Á1 + 5 + 1 + + 1 + 10˜ = 1 fi p 1 =
Ë 3 ¯ 64
So the stationary distribution is:

Ê 3 15 3 10 3 30 ˆ
ÁË 64 64 64 64 64 64 ˜¯

© IFE: 2019 Examinations Page 77

 Batch3

Hence the expected proportion of companies using each firm in the long
term is:

3 + 15 18
= = 0.28125 using A
64 64

3 + 10 13
= = 0.203125 using B
64 64

3 + 30 33
= = 0.515625 using C
64 64

2 Subject CT4 September 2008 Question 8

(i) Probability transition matrix

This is:

Ê 0.15 0.85 0 0 ˆ
Á 0.15 0 0.85 0 ˜
P= Á ˜
Á 0.03 0.12 0 0.85˜
Á ˜
Ë 0 0.03 0.12 0.85¯

(ii)(a) Probability that a Level 2 policy will be at Level 2 in one year’s time

This is p22 (1) = p22 , ie the same as the one-step transition probability for
staying in state 2, and from the matrix we can see that this is zero.

(ii)(b) Probability that a Level 2 policy will be at Level 2 in two years’ time

This will be the (2, 2) element of the P 2 matrix, which we can obtain by
multiplying the second row of P by the second column of P . This comes
to:

p22 (2) = 0.15 ¥ 0.85 + 0.85 ¥ 0.12 = 0.2295

Page 78 © IFE: 2019 Examinations

 Batch3

(iii) The long-run probability of being at Level 2

We can construct four equations, one for each of the four long-run
probabilities denoted by p 1, p 2 , p 3 , and p 4 :

p 1 = p 1 ¥ p11 + p 2 ¥ p21 + p 3 ¥ p31 + p 4 ¥ p41

fi p 1 = 0.15p 1 + 0.15p 2 + 0.03p 3 (1)

p 2 = p 1 ¥ p12 + p 2 ¥ p22 + p 3 ¥ p32 + p 4 ¥ p42

fi p 2 = 0.85p 1 + 0.12p 3 + 0.03p 4 (2)

p 3 = p 1 ¥ p13 + p 2 ¥ p23 + p 3 ¥ p33 + p 4 ¥ p43

fi p 3 = 0.85p 2 + 0.12p 4 (3)

p 4 = p 1 ¥ p14 + p 2 ¥ p24 + p 3 ¥ p34 + p 4 ¥ p44

fi p 4 = 0.85p 3 + 0.85p 4 (4)

We also have:

p1 + p 2 + p 3 + p 4 = 1 (5)

From equation (4) we have:

0.15p 4 = 0.85p 3

0.85
fi p4 = p
0.15 3

We can then substitute this into equation (3) to obtain:

0.12 ¥ 0.85
p 3 = 0.85p 2 + p3
0.15
0.85
fi p3 = p 2 = 2.65625p 2
0.32

© IFE: 2019 Examinations Page 79

 Batch3

Putting this back into equation (4) we get:

0.85 ¥ 0.85
p4 = p 2 + 0.85p 4
0.32

0.852
fi p4 = p 2 = 15.05208p 2
0.15 ¥ 0.32

Now substitute the above expression for p 3 into equation (1) and we get:

p 1 = 0.15p 1 + 0.15p 2 + 0.03 ¥ 2.65625p 2

fi p1 =
(0.15 + 0.0796875) p = 0.27022p 2
2
0.85

Finally, substituting into equation (5) we obtain:

p1 + p 2 + p 3 + p 4 = 1

fi p 2 (0.27022 + 1 + 2.65625 + 15.05208) = 1

fi p 2 = 0.0527

3 Subject CT4 September 2008 Question 11

(i) State space and transition graph

The process can take any integer value, so the state space is the set of
integers: {... - 2, - 1, 0, 1, 2, ...} .

The transition graph is:

p p p p p p
... -2 -1 0 1 2 ...

1–p 1–p 1–p 1–p 1–p 1–p

Page 80 © IFE: 2019 Examinations

 Batch3

(ii) Characteristics of the process

(a) Is it aperiodic?

For this process, a return to any state is possible only in an even number of
steps. So each state is periodic with period 2. Hence the process is not
aperiodic.

(b) Is it reducible?

It is possible to reach every state in the process from every other state. So
this process is irreducible.

(c) Does it admit a stationary distribution?

The state space is infinite. Therefore the process does not have a stationary
distribution.

(iii) Number of upward movements

Let u be the number of upward movements in this time, and d be the

number of downward movements.

As there are m movements altogether, we have:

u +d = m (1)

The net number of upward movements is u  d , so:

u -d = j -i (2)

Summing equations (1) and (2) gives:

2u = m + j - i

So the number of upward movements between time t and time t + m is:

u= 1
2 (m + j - i )

© IFE: 2019 Examinations Page 81

 Batch3

(iv) m-step transition probabilities

For the process to increase by a net amount of j - i , there must be at least

that number of time steps between times t and t + m . In other words:

pij( m ) = 0 for m < j - i

When m is even, the net increase must be even, and when m is odd, the
net increase must be odd. So:

pij( m ) = 0 if m - ( j - i ) is odd

For all other combinations of ( j - i ) and m , the m -step transition

probabilities are non-zero.

From part (iii), we know that if the process moves from state i to state j
over a period of length m , then the number of upward movements is given
by 21 (m + j - i ) and hence the number of downward movements is:

m- 1
2 (m + j - i ) = 21 (m - j + i )
The probability that any given movement is upward is p . So the probability
of experiencing 1
2 (m + j - i ) upward movements and 1
2 (m - j + i )
downward movements out of a total of m movements is given by the
binomial probability:

Ê m ˆ 1 (m + j - i )
(1 - p ) 2 (
1
m- j +i )
pij( m ) = Á 1 ˜ p2
Ë2( m + j - i )¯

Page 82 © IFE: 2019 Examinations

 Batch3

(v) How the one-step probabilities would alter

(a) Reflecting boundary at zero

The transition probabilities are shown on the diagram below:

1 p p
0 1 2 ...

1–p 1–p 1–p

(b) Absorbing boundary at zero

The adapted transition probabilities are shown on the diagram below:

p p
1
0 1 2 ...

1–p 1–p 1–p

(vi) Changes to the m-step transition probabilities

The probabilities that were zero under the original model will still be zero in
situations (a) and (b) as the states being considered still have period 2, and
the changes haven’t made outcomes that were impossible previously
possible now.

For both situations (a) and (b), if there are not enough time steps available to
include at least one transition out of state 0 within m steps, then the
probabilities are unchanged from before, ie the probabilities pij( m ) are
unchanged for i ≥ m .

© IFE: 2019 Examinations Page 83

 Batch3

The effects on the other non-zero pij( m ) are as follows:

(a) Reflecting boundary at zero, m > i

pij( m ) will increase for all possible end-states j that include at least one
transition out of state 0 within the sample path. This is because the
probability of transition out of state 0 in the upwards direction is higher than
before. The highest end-state that it is possible to reach that includes a
single visit to state 0 is state m - i .

Given that we are only looking at j > i , we can therefore say that the
non-zero pij( m ) are increased for all values of i < j £ m - i .

pij( m ) remains unchanged for j > m - i .

(b) Absorbing boundary at zero, m > i

pij( m ) will decrease for all possible end-states j , that could have included at
least one transition out of state 0 within the sample path, ie for i < j £ m - i .
This is because the probability of leaving state 0 in the upwards direction
has reduced from before (to zero).

Again, pij( m ) remains unchanged for j > m - i .

4 Subject CT4 April 2009 Question 2

(i) Time-homogeneous Markov chains

A Markov chain is a stochastic process with a discrete time set and a

discrete state space that satisfies the Markov property:

P ( X n = j | X 0 = i 0 , X1 = i1,..., X m -1 = i m -1, X m = i ) = P ( X n = j | X m = i )

for all integer times n > m and all states i0 , i1,..., i m -1, i , j in the state space.

Alternatively, you could describe the Markov property in words. The Markov
property says that the past history of the process is irrelevant. It is only the
current state that affects the transition probabilities.

Page 84 © IFE: 2019 Examinations

 Batch3

A Markov chain is time-homogeneous if the transition probabilities

P ( X n = j | X m = i ) depend only on the length of the time interval, n - m .

Alternatively, you could say that the one-step transition probabilities are
independent of time.

(ii)(a) Range of values for which the chain is irreducible

A Markov chain is irreducible if every state in the chain can eventually be

reached from every other state. The transition diagram for this chain is:

1–a 1 2 1–b

Provided a and b are non-zero, it is possible to move between the states.

Also, since a and b are probabilities, the requirement is 0 < a, b £ 1 .

(ii)(b) Range of values for which the chain is periodic

If it is possible to stay in a state in two consecutive time periods, then that

state is aperiodic. If the chain is irreducible, then every state in the chain
has the same period or every state is aperiodic. So the only way for the
chain to be periodic is if we have alternation between the two states, ie if
a = b = 1.

5 Subject CT4 April 2009 Question 12

(i) Transition matrix

Let N denote the number of claims made by a policyholder in a year. Then

N ~ Poi (0.3) and:

P (N = 0) = e -0.3 = 0.74082
P (N = 1) = 0.3e -0.3 = 0.22225
P (N ≥ 2) = 1 - 1.3 e -0.3 = 0.03694

© IFE: 2019 Examinations Page 85

 Batch3

So the transition matrix is:

Ê 1 - e -0.3 e -0.3 0 0 ˆ
Á ˜
Á 1 - e -0.3 0 e -0.3 0 ˜
P=Á ˜
-0.3
Á1 - 1.3e 0.3e -0.3 0 e -0.3 ˜
Á ˜
Ë 0 1 - 1.3e -0.3 0.3e -0.3 e -0.3 ¯

Ê 0.25918 0.74082 0 0 ˆ
Á 0.25918 0 0.74082 0 ˜
=Á ˜
Á 0.03694 0.22225 0 0.74082˜
Á ˜
Ë 0 0.03694 0.22225 0.74082¯

(ii) Stationary distribution

The stationary distribution p = (p 0 , p 25 , p 50 , p 60 ) satisfies the equations:

pP = p

and:

p 0 + p 25 + p 50 + p 60 = 1

From the matrix equation we have:

0.25918p 0 + 0.25918p 25 + 0.03694p 50 = p 0 (1)

0.74082p 0 + 0.22225p 50 + 0.03694p 60 = p 25 (2)
0.74082p 25 + 0.22225p 60 = p 50 (3)
0.74082p 50 + 0.74082p 60 = p 60 (4)

We only need 3 out of these 4 equations, so let’s discard equation (2). From
equation (4), we have:

0.74082p 50 = 0.25918p 60

So:

0.25918
p 50 = p 60 = 0.34986p 60
0.74082

Page 86 © IFE: 2019 Examinations

 Batch3

Now, from equation (3):

0.74082p 25 = p 50 - 0.22225p 60 = 0.12761p 60

So:

0.12761
p 25 = p 60 = 0.17226p 60
0.74082

Also, from equation (1):

(1 - 0.25918) p 0 = 0.25918p 25 + 0.03694p 50

= 0.25918 (0.17226p 60 ) + 0.03694 (0.34986p 60 )
= 0.05757p 60

So:

0.05757
p0 = p 60 = 0.07771p 60
0.74082

and since p 0 + p 25 + p 50 + p 60 = 1 , we have:

p 60 (0.07771 + 0.17226 + 0.34986 + 1) = 1

which gives:

p 60 = 0.6251
p 50 = 0.2187
p 25 = 0.1077
p 0 = 0.0486

(iii) Expected average long-term level of discount

The expected average long-term level of discount is:

0 ¥ 0.0486 + 25 ¥ 0.1077 + 50 ¥ 0.2187 + 60 ¥ 0.6251 = 51.13%

© IFE: 2019 Examinations Page 87

 Batch3

(iv) Goodness-of-fit test

The hypotheses for this test are:

H0 : N ~ Poi (0.3)
H1 : N ~ Poi (0.3)

The test statistic is:

(O - E )2
 E

where O is the observed number in the category and E is the expected

number in the category, assuming that the null hypothesis is true.

The expected number of policyholders with exactly k claims is:

e -0.3 0.3k
130, 200P (N = k | N ~ Poi (0.3)) = 130, 200 ¥
k!

So we have:

Number of claims Observed value Expected value

0 96,632 96,454.53
1 28,648 28,936.36
2 4,400 4,340.45
3 476 434.05
4 36 32.55
5 or more 8 2.06
Total 130,200 130,200

Note that the expected number of policyholders with 5 or more claims is

calculated by subtracting the sum of the other expected values from
130,200.

Page 88 © IFE: 2019 Examinations

 Batch3

Since the expected number in the 5 or more category is less than 5, we

combine the last two categories to give:

Number of claims Observed value Expected value

0 96,632 96,454.53
1 28,648 28,936.36
2 4,400 4,340.45
3 476 434.05
4 or more 44 34.61
Total 130,200 130,200

So the value of the test statistic is:

(O - E )2
 E
= 0.3265 + 2.8736 + 0.8170 + 4.0544 + 2.5476 = 10.619

If the null hypothesis is true, then the test statistic should come from a c 2
distribution with 5 - 1 - 1 = 3 degrees of freedom. One degree of freedom
has been lost by estimating the Poisson parameter and another has been
lost by constraining the totals to be 130,200.

The c 2 goodness-of-fit test is a one-tailed test. The upper 5% point of c 32

is 7.815 (and the upper 1% point is 11.34). So we reject the null hypothesis
at the 5% significance level and conclude that the model is not a good fit to
the data. (Note that we would not reject at the 1% level.)

The examiners also gave full credit to students who didn’t combine the last
two categories. If you did it this way, you would get a test statistic of 25.565,
which you would compare with the c 42 distribution. You would then reject
the null hypothesis at both the 5% and the 1% levels.

(v) Comment

Since the Poisson model is not a good fit, the expected average level of
discount calculated in part (iii) is not likely to be correct. (This is because the
transition probabilities were calculated using the Poisson model.)

The reason that the test statistic is large is mainly due to the fact that we
have more policyholders making multiple claims than expected under the
Poisson model.

© IFE: 2019 Examinations Page 89

 Batch3

We also have a higher than expected number of policyholders making no

claims at all.

These last two observations will tend to push the average discount level in
opposite directions. (More claim-free policies lead to higher average
discount but more multiple claim policies lead to lower average discount.) So
it is difficult to tell what the overall effect would be.

6 Subject CT4 September 2009 Question 5

(i) Markov property

The Markov property says that:

P ÈÎ X (t ) Œ A | X (s1) = x1, X (s2 ) = x2 , ..., X (sn ) = xn , X (s ) = x ˘˚

= P ÎÈ X (t ) Œ A | X (s ) = x ˚˘

for all times s1 < s2 <  < sn < s < t , all states x1, x2, ..., xn and x in S and
all subsets A of S .

(ii) Proof

Suppose that the process X (t ) has independent increments. Then:

P ÎÈ X (t ) Œ A | X (s1) = x1, X (s2 ) = x2, ..., X (sn ) = xn , X (s ) = x ˚˘

= P ÎÈ X (t ) - X (s ) + x Œ A | X (s1) = x1, X (s2 ) = x2, ..., X (sn ) = xn , X (s ) = x ˚˘
= P ÎÈ X (t ) - X (s ) + x Œ A | X (s ) = x ˚˘
= P ÎÈ X (t ) Œ A | X (s ) = x ˚˘

So X (t ) has the Markov property.

(iii)(a) Difference between a Markov chain and a Markov jump process

Markov chains and Markov jump processes are both types of stochastic
process with discrete state spaces and both satisfy the Markov property.
The difference between them is that a Markov chain has a discrete time set
and a Markov jump process has a continuous time set.

Page 90 © IFE: 2019 Examinations

 Batch3

(iii)(b) Irreducibility

A Markov chain is said to be irreducible if any state can eventually be

reached from any other state.

(iv)(a) Is an irreducible model appropriate?

Let X (t ) denote the floor that the lift is on at time t . The state space
consists of the 5 floors in the office, which we can label {0,1, 2, 3, 4} .

Since it is possible to get from any floor to any other floor using the lift, an
irreducible model is appropriate.

(iv)(b) Is a Markov model appropriate?

A Markov model is unlikely to be appropriate because, for example, the

probability of going from Floor 1 to Floor 2 is likely to depend on whether the
lift arrived at Floor 1 from Floor 0 or Floor 2, as it is arguably more likely that
the lift will continue in its direction of travel rather than change direction. In
other words, the past history of the process is likely to influence the future
probabilities.

We could get round this problem by using a more complicated state space,
eg defining the state space to represent the current floor and a number of
past floors.

7 Subject CT4 September 2009 Question 7

(i) Two-step transition matrix

The two-step transition matrix is:

Ê 0.5 0.25 0.25ˆ Ê 0.5 0.25 0.25ˆ

P = Á 0.25 0.75
2
0 ˜ Á 0.25 0.75 0 ˜
Á ˜Á ˜
ÁË 0.25 0.25 0.5 ˜¯ ÁË 0.25 0.25 0.5 ˜¯

Ê 0.375 0.375 0.25 ˆ

= Á 0.3125 0.625 0.0625˜
Á ˜
ÁË 0.3125 0.375 0.3125˜¯

© IFE: 2019 Examinations Page 91

 Batch3

(ii) Stationary distribution

The stationary distribution is the vector (p A p B p C ) such that:

Ê 0.5 0.25 0.25ˆ

(p A pB p C ) Á 0.25 0.75
Á
0 ˜ = (p A p B
˜
pC )
ÁË 0.25 0.25 0.5 ˜¯

and:

p A + p B + pC = 1 (1)

From the matrix equation, we have:

0.5p A + 0.25p B + 0.25p C = p A (2)

0.25p A + 0.75p B + 0.25p C = p B (3)

and:

0.25p A + 0.5p C = p C (4)

From (4):

0.25p A = 0.5p C
fi p A = 2p C

Substituting this into (3):

0.5p C + 0.75p B + 0.25p C = p B

fi 0.25p B = 0.75p C
fi p B = 3p C

It now follows from (1) that:

2p C + 3p C + p C = 1

Page 92 © IFE: 2019 Examinations

 Batch3

1
So p C = and the stationary distribution is:
6

Ê1 1 1ˆ
ÁË 3 2 6 ˜¯

(iii) Comment

The stationary distribution gives the long-term probabilities that a car will be
returned to each of the three locations: airport, beach and city. However, it
does not tell us how likely a car is to be picked up from each of the three
locations. This is more important than the stationary distribution for drop-off
points.

Other factors that should be considered when deciding how many cars
should be based at each location include the available space and facilities at
each location, the number of cars owned by the company and the cost of
transporting a car from one location to another.

(iv) Sketch

The probabilities are shown in the diagram below.

Probability of being at each location

1
0.9
0.8
0.7
Probability

Airport
0.6
0.5 Beach
0.4 City
0.3
0.2
0.1
0
0 1 2 3 4 5 6 7 8 9 10
Number of times car has been rented

© IFE: 2019 Examinations Page 93

 Batch3

The graph shows that the long-run probabilities for the airport, the beach and
1 1 1
the city are , and , respectively. Note that the beach and city
3 2 6
probabilities are the same at times 0 and 1.

8 Subject CT4 April 2010 Question 4

(i) Define the process

The process has the following three states:

B C

The third bullet point tells us that the transitions A→B and A→C are both
possible, each with the same probability, p say.

p p

B C

The question doesn’t actually say that the probability p cannot equal zero,
but we can easily see that it can’t equal zero. If it did, it would not be
possible to move from A to either of the other two states, which would make
A an absorbing state and this would contradict the irreducible property.

Page 94 © IFE: 2019 Examinations

 Batch3

Since the process is periodic and it only has 3 states, the period must be
either 2 or 3. If the period were 3, the process would have to consist of a
triangular ‘one-way system’ going in one direction or the other:

A A

B C B C

However, these models are not consistent with the two ‘p’ arrows we
established earlier, which both go out of A. So the process must have a
period of 2.

Since it is periodic, we also know that none of the states have loops going
back to themselves. So the process must have the following form:

p p

B C

We cannot allow a bridge between B and C because it would then be

possible to go ‘round the triangle’ and this process would no longer have a
period of 2.

Since there is only one exit route from each of states B and C, the
associated probabilities must equal 1. Since there are two exit routes from
state A with equal probability (and staying in A is not possible), we also know
that p = ½ .

© IFE: 2019 Examinations Page 95

 Batch3

(ii) Sketch the transition diagram

The transition diagram therefore looks like this:

½ ½

1 1
B C

9 Subject CT4 April 2010 Question 10

(i)(a) Modelling the scheme as a Markov chain

The process can be modelled as a Markov chain with four states

corresponding to the four classes (O = Ordinary, B = Bronze, S = Silver and
G = Gold):

p1 + p2 +
G

p2 + p0

p1
S

p2 + p0

p1
B

p2 + p0

p0 + p1
O

Page 96 © IFE: 2019 Examinations

 Batch3

The process has discrete states and operates in discrete time, with
movements occurring only at the end of each calendar year.

This set-up has the Markov property because all the members in a particular
state in a particular year have the same probabilities for moving between the
states in the future.

(i)(b) Unique stationary distribution

A stationary distribution is a set of probabilities p i that satisfy the matrix

equations:

p = pP , Âpi = 1, pi ≥ 0
i

This chain has a finite state space, with 4 states. So there is at least one
stationary distribution.

The chain is also irreducible, ie it is possible to move from any state i to any
state j (eventually). We can see this because the chain allows a complete
circuit through the states O→B→S→G→S→B→O). So, in fact, there is a
unique stationary distribution.

(ii) Transition matrix

The transition matrix for the model is:

O B S G
O È p0 + p1 p2 + 0 0 ˘
Í ˙
B Í p0 p1 p2 + 0 ˙
S Í 0 p 0 p1 p2 + ˙
Í ˙
G ÍÎ 0 0 p0 p1 + p2 + ˙˚

© IFE: 2019 Examinations Page 97

 Batch3

(iii) Calculate the stationary probability distribution

We are told that p0 = 0.4 , p1 = 0.4 and p2 + = 0.2 . So the transition matrix
for the model is:

O B S G
O È0.8 0.2 0 0 ˘
Í ˙
B Í 0.4 0.4 0.2 0 ˙
S Í 0 0.4 0.4 0.2 ˙
Í ˙
G ÍÎ 0 0 0.4 0.6 ˙˚

The stationary distribution is the set of probabilities p i that satisfy the matrix
equation p = p P with the additional condition Âpi = 1.
i

Written out in full, the matrix equation for the stationary distribution
(p O , p B , p S , p G ) satisfies the system of equations:

pO = 0.8p O + 0.4p B (1)

pB = 0.2p O + 0.4p B + 0.4p S (2)
pS = 0.2p B + 0.4p S + 0.4p G (3)
pG = 0.2p S + 0.6p G (4)

From Equation (1), we have:

0.2p O = 0.4p B fi p B = 0.5p O

Equation (2) then becomes:

0.5p O = 0.2p O + 0.4(0.5p O ) + 0.4p S

So:

0.1p O = 0.4p S fi p S = 0.25p O

Equation (4) then tells us that:

0.4p G = 0.2p S fi p G = 0.5p S = 0.125p O

Page 98 © IFE: 2019 Examinations

 Batch3

So the vector of stationary probabilities has the form:

(p O , p B , p S , p G ) = (p O , 0.5p O , 0.25p O , 0.125p O )

Since these probabilities must add up to 1, this tells us that:

p O + 0.5p O + 0.25p O + 0.125p O = 1

So we have:

1 8
1.875p O = 1 fi p O = =
1.875 15

The stationary distribution is therefore:

( 8 4 2 1
, , ,
15 15 15 15 )
(iv) Profitability of scheme

The expected annual cost of the scheme per member is:

4 + 20 ¥ 2 + 30 ¥ 1 = 7 1
0p O + 10p B + 20p S + 30p G = 0 + 10 ¥ 15 15 15 3

The expected profit made from bookings is:

0 ¥ pO + 10 ¥ p1 + 10n ¥ p2 +

where n is the average number of flights booked by members booking 2 or

more.

So, to make an overall profit, we need:

10 ¥ 0.4 + 10n ¥ 0.2 > 7 31

¤ 4 + 2n > 7 31

¤ n > 1 32

But, since n must be at least 2, this will always be true. So the expected
net profit of the scheme in the long-term will be positive.

© IFE: 2019 Examinations Page 99

 Batch3

10 Subject CT4 September 2010 Question 12

(i) Transition matrix

Ê 0.4 0 0 0.6ˆ
Á 0.4 0.4 0 0.2˜
Á ˜
Á 0.2 0.4 0.4 0 ˜
Á ˜
Ë 0 0.2 0.4 0.4¯

the i , j th entry = P (start in state j on day n + 1| start in state i on day n )

(ii) Stationary distribution

Ê 0.4 0 0 0.6ˆ
Á 0.4 0.4 0 0.2˜
(p 1 p2 p3 p4)Á ˜ = (p 1 p 2 p 3
Á 0.2 0.4 0.4 0 ˜
p4)
Á ˜
Ë 0 0.2 0.4 0.4¯

0.4p 1 + 0.4p 2 + 0.2p 3 = p 1 (1)

0.4p 2 + 0.4p 3 + 0.2p 4 = p 2 (2)
0.4p 3 + 0.4p 4 = p 3 (3)
0.6p 1 + 0.2p 2 + 0.4p 4 = p 4 (4)

3
(3) gives 0.4p 4 = 0.6p 3 fi p 4 = p3
2

Substituting this into (2) gives:

7
0.4p 2 + 0.4p 3 + 0.3p 3 = p 2 fi 0.6p 2 = 0.7p 3 fi p 2 = p3
6

Then, from (1):

Ê7 ˆ 2 10
0.4p 1 + 0.4 Á p 3 ˜ + 0.2p 3 = p 1 fi 0.6p 1 = p 3 fi p 1 = p
Ë6 ¯ 3 9 3

Page 100 © IFE: 2019 Examinations

 Batch3

Then, using p 1 + p 2 + p 3 + p 4 = 1 , we have:

Ê 10 7 3ˆ
p3 Á + + 1+ ˜ = 1
Ë 9 6 2¯

which gives:

9 10 21 27
p3 = , p1 = , p2 = , p4 =
43 43 86 86

Ê 10 21 9 27 ˆ
So the stationary distribution is Á .
Ë 43 86 43 86 ˜¯

(iii) Expected long-term average number of restocking orders

 If the shop has 1 snake in stock at the start of a day, the probability it
will have to restock before it opens again the next day is 0.6.
 If the shop has 2 snakes in stock at the start of a day, the probability it
will have to restock before it opens again the next day is 0.2.
 If the shop has 3 or 4 snakes in stock at the start of a day, it will not
have to restock before it opens again the next day.

So the expected long-term average number of restocking orders placed per

trading day is:

10 21 81
0.6p 1 + 0.2p 2 = 0.6 ¥ + 0.2 ¥ = = 0.1884
43 86 430

(iv) Expected long-term number of sales lost

 If the shop has 1 snake in stock at the start of a day, it loses 1 potential
sale with probability 0.2.
 If the shop has 2 or more snakes in stock at the start of a day, there are
no lost sales that day.

So the expected long-term number of sales lost per trading day is:

10 2
0.2p 1 = 0.2 ¥ = = 0.0465
43 43

© IFE: 2019 Examinations Page 101

 Batch3

(v) Expected restocking orders and lost sales under new set up

The shop will restock if the number of snakes at the end of the day is 0 or 1.
So the number of snakes in stock at the start of each trading day is now 2, 3
or 4. The new stationary distribution is found from:

Ê 0.4 0 0.6ˆ
( p2 p3 p4 ) Á 0.4 0.4 0.2˜ = ( p2
Á ˜
p3 p4 )
ÁË 0.2 0.4 0.4˜¯

and p2 + p3 + p4 = 1

The first two equations from the matrix are:

0.4 p2 + 0.4 p3 + 0.2 p4 = p2 (5)

0.4 p3 + 0.4 p4 = p3 (6)

3
From (6), 0.4 p4 = 0.6 p3 fi p4 = p3
2

Substituting this into (5) gives:

7
0.4 p2 + 0.4 p3 + 0.3 p3 = p2 fi 0.6 p2 = 0.7 p3 fi p2 = p3
6

Ê7 3ˆ
Now, using the fact that p2 + p3 + p4 = 1 , we have p3 Á + 1 + ˜ = 1
Ë6 2¯

So, the stationary distribution under the new restocking arrangement is:

Ê 7 3 9 ˆ
ÁË 22 11 22 ˜¯

The expected long-term number of restocking orders per trading day is now:

7 3 27
0.6 p2 + 0.2 p3 = 0.6 ¥ + 0.2 ¥ = = 0.2455
22 11 110

No sales will be lost because the probability of selling more than 2 snakes in
a day is 0.

Page 102 © IFE: 2019 Examinations

 Batch3

(vi) Why restocking is not optimal

Restocking when there are 2 or more snakes in stock means that there will
be no lost sales. This can also be achieved by restocking when the number
of snakes falls to 1 as we have seen in (v) (because the probability of selling
more than 2 snakes in a day is 0).

However, restocking costs money. Restocking when there are 2 or more

snakes in stock is more expensive than restocking when the stock falls to 1.
So this would reduce profits and is therefore not optimal.

(vii) Condition in terms of C and P

If the shop is restocked only when there are no snakes in stock, the
expected long-term cost per day is:

81 2
C+ P
430 43

If the shop is restocked when the number of snakes falls below 2, the
expected long-term cost per day is:

27
C
110

So the owner should change to restocking when the number of snakes falls
below 2 if:

27 81 2
C< C+ P
110 430 43

22
ie C< P
27

© IFE: 2019 Examinations Page 103

 Batch3

11 Subject CT4 April 2011 Question 2

(a) At least one stationary distribution

If a Markov chain has a finite state space, ie if the number of states is finite,
then it has at least one stationary distribution.

(b) A unique stationary distribution

If, in addition to having a finite state space, a Markov chain is also

irreducible, ie if every state j can be reached eventually from every state
i , then it has a unique stationary distribution.

(c) Convergence to a unique stationary distribution

If, in addition to having a finite state space and being irreducible, a Markov
chain is also aperiodic, ie if there does not exist an integer d > 1 such that
return to a given state is only possible in multiples of d steps, then it will
converge to the unique stationary distribution.

12 Subject CT4 April 2011 Question 4

(i) Transition matrix

If the current state is 1 ‘Poor’, then, according to the rules, there is:
 a 20% chance the process will move up one level to 2 ‘Satisfactory’
 a 20% chance the process will move down one level (which will leave it
on 1 ‘Poor’, since it is already on the lowest level)
 a 10% chance the process will move up two levels to 3 ‘Good’
 a 10% chance the process will move down two levels (which will leave it
on 1 ‘Poor’, since it is already on the lowest level).

So the probabilities for the next state are:

 70% on level 1 ‘Poor’
 20% on level 2 ‘Satisfactory’
 10% on level 3 ‘Good’
 0% on level 4 ‘Excellent’.

Page 104 © IFE: 2019 Examinations

 Batch3

Analysing each of the other states in a similar way leads to the following
one-step probability transition matrix:

1 2 3 4
1 È0.7 0.2 0.1 0 ˘
Í ˙
P= 2 Í0.3 0.4 0.2 0.1˙
3 Í 0.1 0.2 0.4 0.3 ˙
Í ˙
4 ÎÍ 0 0.1 0.2 0.7 ˚˙

(ii) Probability distribution for the third week

We know that each child starts on level 2 ‘Satisfactory’ on the first Friday.
The probabilities for the third Friday are the probabilities after two steps.
These can be found from the second row of the matrix P 2 .

È0.7 0.2 0.1 0 ˘ È0.7 0.2 0.1 0 ˘

Í ˙ Í ˙
0.3 0.4 0.2 0.1˙ Í 0.3 0.4 0.2 0.1˙
P2 = Í
Í 0.1 0.2 0.4 0.3 ˙ Í 0.1 0.2 0.4 0.3 ˙
Í ˙ Í ˙
ÎÍ 0 0.1 0.2 0.7 ˚˙ ÎÍ 0 0.1 0.2 0.7 ˚˙

È ∑ ∑ ∑ ∑ ˘
Í ˙
0.35 0.27 0.21 0.17
=Í ˙
Í ∑ ∑ ∑ ∑ ˙
Í ˙
ÎÍ ∑ ∑ ∑ ∑ ˚˙

So the probabilities for the distribution on the third Friday are:

 35% on level 1 ‘Poor’
 27% on level 2 ‘Satisfactory’
 21% on level 3 ‘Good’
 17% on level 4 ‘Excellent’.

© IFE: 2019 Examinations Page 105

 Batch3

13 Subject CT4 April 2011 Question 8 (part)

(ii) MLE of the force of mortality

According to the two-state model of mortality, the MLE of the force of

mortality is:

qx
mˆ x + f =
E xc

where the estimate derived from lives labelled x applies to age x + f .

From the table we can see that lives 3, 6 and 10 died before their 1st
birthdays. So, for the year of age (0,1) , the number of deaths is:

q0 = 3

The central exposed to risk for the year of age (0,1) for the lives who are
recorded as deaths is:
 2 months for life 3
 9 months for life 6
 1 month for life 10.

We are told that the remaining 7 lives (with no date of death shown) survived
until their 1st birthday. So their exposed to risk is a full year.

So the total central exposed to risk for the year of age (0,1) is:

E0c = 2 + 9 + 1 + 7 ¥ 12 = 96 months = 8 years

Since we are assuming a constant force of mortality over the year, the
estimate of the force of mortality for the year of age (0,1) applies to age 0.5
and is equal to:

q0 3
mˆ 0.5 = = = 0.375
E0c 8

Page 106 © IFE: 2019 Examinations

 Batch3

(iii) Hence estimate q 0

Since we are assuming that the force of mortality is constant over the year,
the values of q and m are related by the equation:

q 0 = 1 - p 0 = 1 - exp Ê - Ú ms ds ˆ = 1 - e - m0.5
1
Ë 0 ¯

So we can estimate the infant mortality rate as:

qˆ 0 = 1 - e - m0.5 = 1 - e -0.375 = 0.313

ˆ

14 Subject CT4 April 2011 Question 12

(i) Markov property

With this model, individuals in State L who reduce their order have different
probabilities for their future movements, depending on whether they:

(a) increased/maintained or

(b) reduced their order the previous year. They will drop by either one or
two levels, depending on their previous actions.

So this violates the Markov property, which requires that the current state
fully determines the future probabilities.

(ii)(a) Additional states

To preserve the Markov property, we need to split State L into two separate
states, L+ and L– (say), where:
 L+ = Members paying the Loyalty price who increased or maintained
their order the previous year
 L– = Members paying the Loyalty price who reduced their order the
previous year.

© IFE: 2019 Examinations Page 107

 Batch3

(ii)(b) Transition diagram

The transition diagram looks like this:

L+ L-

(iii)(a) Transition matrix

The transition matrix for the model is:

B G L+ L- S
B È0.4 0.6 0 0 0 ˘
Í ˙
G Í 0.3 0.1 0.6 0 0 ˙
L+ Í 0 0.3 0.1 0 0.6 ˙
Í ˙
L - Í0.3 0 0.1 0 0.6 ˙
S ÍÎ 0 0 0 0.3 0.7 ˙˚

(iii)(b) Stationary distribution

The stationary distribution is the set of probabilities p i that satisfy the matrix
equation p = p P with the additional condition Âpi = 1.
i

Page 108 © IFE: 2019 Examinations

 Batch3

Written out in full, this matrix equation is:

pB = 0.4p B + 0.3p G + 0.3p L - (1)

pG = 0.6p B + 0.1p G + 0.3p L + (2)
p L+ = 0.6p G + 0.1p L + + 0.1p L - (3)
p L- = 0.3p S (4)
pS = 0.6p L + + 0.6p L - + 0.7p S (5)

Using p S as the working variable, Equation (4) tells us that:

p L - = 0.3p S

Equation (5) can then be written as:

p S = 0.6p L + + 0.6 (0.3p S ) + 0.7p S

fi 0.12p S = 0.6p L +

0.12
fi p L+ = p S = 0.2p S
0.6

Equation (3) can then be written as:

0.2p S = 0.6p G + 0.1(0.2p S ) + 0.1(0.3p S )

fi 0.15p S = 0.6p G

0.15
fi pG = p S = 0.25p S
0.6

Equation (1) can then be written as:

p B = 0.4p B + 0.3 (0.25p S ) + 0.3 (0.3p S )

fi 0.6p B = 0.165p S

0.165
fi pB = p S = 0.275p S
0.6

© IFE: 2019 Examinations Page 109

 Batch3

So the vector of stationary probabilities has the form:

(p B , p G , p L + , p L - , p S ) = (0.275p S , 0.25p S , 0.2p S , 0.3p S , p S )

Since these probabilities must add up to 1, this tells us that:

0.275p S + 0.25p S + 0.2p S + 0.3p S + p S = 1

fi 2.025p S = 1

1 40
fi pS = =
2.025 81

The stationary distribution is therefore:

(p B , p G , p L + , p L - , p S ) = ( 11 10 8 12 40
, , , ,
81 81 81 81 81 )
(iii)(c) Long-run average price

We can calculate the long-run average price paid by applying the long-run
probabilities to the price paid in each state:

Long-run % price = p B ¥ 100% + p G ¥ 90% + (p L + + p L - ) ¥ 80% + p S ¥ 75%

= 11 ¥ 100% + 10
81 81
¥ 90% + ( 8
+ 12
81 81 ) ¥ 80% + 40
81
¥ 75%

= 81 13
27
% or 81.48%

So the long-run average price will be:

Long-run average price = 81.48% ¥ £8 = £6.52

(iv) Probability that a Loyalty customer will be paying L in 2 years’ time

Let X t denote the state the process is in at time t . The required probability
is:

pL,L (2) = P ( X t + 2 = L | X t = L )

Page 110 © IFE: 2019 Examinations

 Batch3

However, since the original State L consists of customers with different

future probabilities, we need to split out the conditional part into L+ and L–:

pL,L (2) = P ( X t + 2 = L | X t = L + or X t = L - )

Using the law of total probabilities, we can write this as:

pL,L (2) = P ( X t + 2 = L | X t = L + ) ¥ P ( X t = L + )
+P ( X t + 2 = L | X t = L - ) ¥ P ( X t = L - )

To evaluate this probability, we will assume that the customer base has
settled down with the long-run probabilities we worked out in part (iii)(b).
8
This means that the proportions in States L+ and L– will be p L + = 81
and
p L- = 12 , so that a customer in one of these two ‘L’ states has a probability
81
of 0.4 of being in State L+ and 0.6 of being in State L–.

This then gives:

p L+
pL,L (2) = P ( X t + 2 = L | X t = L + ) ¥
p L+ + p L-
p L-
+P ( X t + 2 = L | X t = L - ) ¥
p L+ + p L-
= P ( X t + 2 = L | X t = L + ) ¥ 0.4 + P ( X t + 2 = L | X t = L - ) ¥ 0.6

To work out the two-step probabilities in this expression, we can follow the
possible paths in the transition diagram:

P( Xt +2 = L | Xt = L+) = P( Xt +2 = L+ | Xt = L+) + P( Xt +2 = L- | Xt = L+)

= {P (L +, L +, L + ) + P (L +, G, L + )} + {P (L +, S, L - )}
= 0.12 + 0.3 ¥ 0.6 + 0.6 ¥ 0.3
= 0.37

and P ( X t + 2 = L | X t = L - ) = P ( X t + 2 = L + | X t = L - ) + P ( X t + 2 = L - | X t = L - )
= {P (L -, L +, L + )} + {P (L -, S, L - )}
= 0.12 + 0.6 ¥ 0.3
= 0.19

© IFE: 2019 Examinations Page 111

 Batch3

The required probability is then pL,L (2) = 0.37 ¥ 0.4 + 0.19 ¥ 0.6 = 0.262 .

There were several other acceptable approaches you could have used here:

(1) You can assume that Loyalty customers are equally likely to start in
either State L+ or L–, which gives a probability of
0.37 ¥ 0.5 + 0.19 ¥ 0.5 = 0.28 .

(2) You can express the answer as 0.37 p + 0.19(1 - p ) = 0.19 + 0.18 p ,
where p is the unknown proportion originally in state L+.

(3) You can say that the probability will lie somewhere between 19% and
37%, depending on the proportions originally in the L+ / L– states.

Note that adding together the four probabilities 0.19, 0.18, 0.01 and 0.18
does not give the right answer here. This is calculating the probability that
an L+ customer is paying the Loyalty price in 2 years’ time PLUS the
probability that an L– customer is paying the Loyalty price in 2 years’ time.

(v) Reasons why the assumptions might not be valid

Changes in the total supply of hay (eg because of adverse weather

conditions) might affect what customers are prepared to pay and how much
Farmer Giles has to sell.

Changes in the pattern of hay consumption might change the total amount of
hay required by customers.

The introduction of the discount scheme might influence customers’

decisions.

A competitor might introduce a rival scheme that takes away some types of
customers.

The relative price of hay from different sources (eg cheap imports) might
affect his total sales.

Page 112 © IFE: 2019 Examinations

 Batch3

15 Subject CT4 September 2011 Question 1

(a) Are the models irreducible?

Model A is irreducible because it is possible to move between the two states.

Model B is not irreducible because State 3 is an absorbing state and it is not

possible to get from this state to either of the other two.

Model C is irreducible because it is possible to move between any of the four

states, eg by following the circuit 1 Æ 2 Æ 4 Æ 3 Æ 1 .

(b) Are the models periodic?

Model A is periodic with period 2 because it is only possible to return to a

particular state in an even number of steps.

Model B is not periodic (ie it is aperiodic) because it is not possible to return

to State 1 or State 2 at all, and State 3 is an absorbing state.

Model C is not periodic because it is irreducible and it is possible to return to

State 1, for example, in any number of steps apart from 1.

16 Subject CT4 September 2011 Question 11

(i) Explain the diagram

State 0

State 0 means that the actuary has no umbrellas at his current location. So
he is certain to leave without one and there will be 2 at his destination. This
is represented by the arrow from state 0 to state 2 labelled with probability 1.

State 2

State 2 means that the actuary has 2 umbrellas at his current location. If it is
raining (which has probability p ) he will take 1 with him, so that there will
then be 1 at his destination. If it is not raining (which has probability 1 - p )
he will not take an umbrella and there will be none at his destination. This is
represented by the two arrows leading out of state 2.

© IFE: 2019 Examinations Page 113

 Batch3

State 1

State 1 means that the actuary has 1 umbrella at his current location (which
means that there is already 1 at his destination). If it is raining (which has
probability p ) he will take 1 umbrella with him, so that there will be 2 at his
destination. If it is not raining (which has probability 1 - p ) he will not take
an umbrella and there will be 1 at his destination. This is represented by the
two arrows leading out of state 1.

(ii) Transition matrix

The one-step transition matrix P is:

0 1 2
0 È 0 0 1˘
Í ˙
1 Í 0 1 - p p ˙
2 ÍÎ1 - p p 0 ˙˚

The two-step transition matrix is the square of this:

È 0 0 1˘ È 0 0 1˘
Í
2 ˙ Í ˙
P =Í 0 1- p p˙ Í 0 1- p p˙
ÍÎ1 - p p 0 ˙˚ ÍÎ1 - p p 0 ˙˚

È 1- p p 0 ˘
Í 2 2 ˙
= Í p(1 - p ) (1 - p ) + p p(1 - p ) ˙
Í ˙
ÍÎ 0 p(1 - p ) 1 - p + p 2 ˙˚

(iii) Stationary distribution

The stationary distribution is the set of probabilities p i that satisfy the matrix
equation p = p P with the additional condition  pi = 1.
i
Written out in full, this matrix equation is:

p 0 = (1 - p )p 2 (1)
p 1 = (1 - p )p 1 + pp 2 (2)
p 2 = p 0 + pp 1 (3)

Page 114 © IFE: 2019 Examinations

 Batch3

Equation (1) expresses p 0 in terms of p 2 .

Also, Equation (2) simplifies to p 1 = p 2 .

So the vector of stationary probabilities has the form:

(p 0 , p 1, p 2 ) = ((1 - p )p 2, p 2, p 2 )

Since these probabilities must add up to 1, this tells us that:

(1 - p )p 2 + p 2 + p 2 = 1

fi (3 - p )p 2 = 1

1
fi p2 =
3-p

Ê 1- p 1 1 ˆ
The stationary distribution is therefore (p 0 , p 1, p 2 ) = Á , , .
Ë 3 - p 3 - p 3 - p ˜¯

(iv) Long-run proportion of journeys in the rain without an umbrella

The actuary will make a journey in the rain without an umbrella if he is

currently in state 0 and it is raining. Since the probability of rain is p , the
long-run proportion of such journeys is:

1- p p(1 - p )
p0 ¥ p = ¥p=
3-p 3-p

(v) Transition matrix for the rain

In this model there are two states for the weather, say R = ‘raining’ and N =
‘not raining’.

The one-step probability transition matrix for this process is:

R N
R Èr 1- r ˘
Í ˙
N Îs 1 - s ˚

© IFE: 2019 Examinations Page 115

 Batch3

(vi) Markov property

For a Markov process, the future probabilities must be fully determined by

the current state.

However, in this new situation, state 2, for example, will include journeys
when it rained on the previous journey and ones when it didn’t. The
probabilities for moving to state 1 next in these two cases will now be r
and s , which are different. So this violates the Markov property.

(vii) Additional states

To preserve the Markov property, we need to incorporate the weather for the
previous journey in the states. This would give us 6 states, eg 2R = ‘2
umbrellas and it was raining on the previous journey’. However, there will
not be a 0R state because the actuary can only enter state 0 from state 2,
and if it was raining on the previous journey he would have brought an
umbrella with him.

So the states for the new model are:

 0N = 0 umbrellas at current location and not raining on previous journey

 2R = 2 umbrellas at current location and raining on the previous journey
 2N = 2 umbrellas at current location and not raining on the previous
journey
 1R = 1 umbrella at current location and raining on the previous journey
 1N = 1 umbrella at current location and not raining on the previous
journey.

Page 116 © IFE: 2019 Examinations

 Batch3

The revised diagram now looks likes this:

s
r
2R 1R 1N 1–s
1–r
r
s 1–r s
1–s
0N 2N
1–s

17 Subject CT4 April 2012 Question 11

(i) Markov property

From the definition given for X i , we see that:

n +1 n
X n +1 = Â Yj = Â Yj + Yn +1 = X n + Yn +1
j =1 j =1

We also know that the probabilities for Yn +1 are determined based only on
1Ê X ˆ
X n using the formula P(Yn +1 = 1) = 1 + n ˜ . So the value of X n fully
4 ÁË n ¯
determines the probability distribution for X n +1 , and hence also for
X n + 2 , X n +3 , , ie the series X i has the Markov property.

The probabilities for the value of Yn +1 , on the other hand, depend on the
value of X n , which cannot be determined based only on the value of Yn
(apart from in year 1 when the probability is always 1 ). So the series Yi
2
does not have the Markov property.

© IFE: 2019 Examinations Page 117

 Batch3

(ii)(a) Is X i irreducible?

A Markov chain is irreducible if it is possible to move from any state i to any

state j eventually. Since the values of X i form a non-decreasing series, it
is not possible to return to a lower-numbered state, so the series is not
irreducible.

(ii)(b) Is X i time-homogeneous?

A stochastic process is time-homogeneous if the probability distribution for

the transitions does not change over time. The transition
X n +1 - X n ( = Yn +1 ) is determined by the probabilities for Yn +1 , which
1Ê X ˆ
depend on the time n via the formula P(Yn +1 = 1) = 1 + n ˜ . So the
4 ÁË n ¯
series is not time-homogeneous.

(iii) Transition graph

The transition diagram looks like this:

X0 X1 X2 X3
1 3 3
2 4 4
0 0 0 0
1 1
4 4
1
2 1 1
1 1
5
2 8 3
8
1
2 2 2
1
2
1
2 3

Page 118 © IFE: 2019 Examinations

 Batch3

The probabilities are found from the relationships:

1
P( X1 = 1) = P(Y1 = 1) =
2
1Ê X ˆ
and P( X n +1 - X n = 1) = P(Yn +1 = 1) = 1 + n ˜ , n = 1,2,3,
4 ÁË n ¯

(iv) Probability

The probability that the driver has accidents during exactly two of the first
three time periods is P ( X 3 = 2) . This can be calculated by adding the
probabilities for the various paths going from X 0 = 0 to X 3 = 2 :

P ( X 3 = 2) = P (0 Æ 0 Æ 1 Æ 2) + P (0 Æ 1 Æ 1 Æ 2) + P (0 Æ 1 Æ 2 Æ 2)
= ( 21 ¥ 41 ¥ 83 ) + ( 21 ¥ 21 ¥ 83 ) + ( 21 ¥ 21 ¥ 21 )
= 3 + 3 + 1
64 32 8
= 17 ( = 0.2656)
64

(v) Comment

Different drivers do have different probabilities of having an accident and it

seems reasonable to reflect their past accident record in the calculation of
their premium.

With this model, the effect of a past accident reduces over time (because it is
averaged over all past years), which also seems reasonable.

However, the current model includes a factor of 1 in the calculation, which

n
gives equal weighting to all past years, whereas it may be a better reflection
of the driver’s accident proneness if we put more weighting on their most
recent driving experience.

This model doesn’t take into account – not directly, at least – many other
factors that are likely to have a big effect on the rate of future accidents,
such as the driver’s age and experience, their annual mileage and where
they live and work.

This model only considers ‘no accidents’ versus ‘at least one accident’. The
company might want to take into account the actual number of accidents
each year, since accident prone drivers may have several accidents in the
same year.

© IFE: 2019 Examinations Page 119

 Batch3

18 Subject CT4 April 2012 Question 12

(i) Meaning of a Markov chain

A Markov chain is a stochastic process with a discrete time set and a

discrete state space that has the Markov property, ie the future probabilities
are fully determined by the current state.

(ii) Minimum number of states

To model the payments under the scheme, we need to know whether

members are:
 healthy (ie they are contributing and not receiving any sickness benefit)
or
 sick and receiving their full salary or
 sick and receiving 50% of their salary.

We also need to distinguish between members who are in their first and
second months of sickness because the number of months for which they
can continue receiving the full rate of sickness pay is different for the two
groups (which means the model wouldn’t be Markov if we combined them).

So we need at least 4 states, which we can label as:

H = Healthy
S1 = In first month of sickness
S2 = In second month of sickness
S3+ = In third or subsequent month of sickness (receiving only half pay).

(iii) Transition diagram

The transition diagram looks like this:

0.9
0.25
0.1 0.25 0.25
H S1 S2 S3+
0.75
0.75
0.75

Page 120 © IFE: 2019 Examinations

 Batch3

(iv) Stationary distribution

The one-step transition matrix P is:

H S1 S2 S3 +
H È 0.9 0.1 0 0 ˘
Í ˙
S1 Í 0.75 0 0.25 0 ˙
S2 Í0.75 0 0 0.25˙
Í ˙
S3 + ÍÎ0.75 0 0 0.25˙˚

The stationary distribution is the set of probabilities p i that satisfy the matrix
equation p = p P with the additional condition Âpi = 1.
i

Written out in full, this matrix equation is:

pH = 0.9p H + 0.75p S1 + 0.75p S 2 + 0.75p S 3+ (1)

p S1 = 0.1p H (2)
p S2 = 0.25p S1 (3)
p S3+ = 0.25p S 2 + 0.25p S 3+ (4)

If we try to express everything in terms of p H , Equation (2) tells us that:

(
p S1 = 0.1p H = 1 p
10 H )
Equation (3) then gives:

p S 2 = 0.25p S1 = 0.25(0.1p H ) = 0.025p H = ( 1 p

40 H )
Rearranging Equation (4) tells us that:

0.75p S 3+ = 0.25p S 2

fi p S3+ =
0.25
p =
0.25
0.75 S 2 0.75
(0.025p H ) = 0.00833p H = ( 1 p
120 H )
So the vector of stationary probabilities has the form:

( 1 p , 1 p , 1 p
(p H , p S1, p S 2 , p S 3+ ) = p H , 10 H 40 H 120 H )

© IFE: 2019 Examinations Page 121

 Batch3

Since these probabilities must add up to 1, this tells us that:

p H + 10
1 p +
H
1 p
40 H
1 p =1
+ 120 H

fi 17 p =1
15 H

fi pH = 15
17

The stationary distribution is therefore:

(p H , p S1, p S 2 , p S 3+ ) = ( 1517 , 343 , 1363 , 1361 ) or (0.8824,0.0882,0.0221,0.0074)

(v) Percentage of salary to pay

The minimum contribution rate x (expressed as a proportion of salary) must

satisfy the inequality:

Expected monthly contributions ≥ Expected monthly benefits

fi xp H ≥ 0p H + 1p S1 + 1p S 2 + 0.5p S 3+

Using the probabilities for the stationary distribution, this gives:

15 x≥ 3 3 +
+ 136 1 1 =
¥ 136 31
17 34 2 272

fi x≥ 17 ¥ 31 = 31 = 0.1292
15 272 240

So the healthy employees must pay at least 12.92% of their salary.

(vi) If sick pay is increased to 100%

The minimum contribution rate y must now satisfy the inequality:

y p H ≥ 0p H + 1p S1 + 1p S 2 + 1p S 3+

Page 122 © IFE: 2019 Examinations

 Batch3

Using the probabilities for the stationary distribution, this gives:

15 y≥ 3 3 + 1 =
+ 136 2 = 0.1176
17 34 136 17

fi y≥ 17 2 =
¥ 17 2 = 0.1333
15 15

So the healthy employees must pay at least 13.33% of their salary.

(vii) Comment

The reduction to 50% from the third month results in a slightly lower
contribution rate, ie 12.92% compared to 13.33%. In terms of the actual
amounts paid, this is a reduction of only 3%. This is a small change
because the probability of being sick for 3 months in a row is relatively small
in this model.

Employees may have more peace of mind knowing that they will continue to
receive their full pay. However, this may also reduce their incentive to return
to work if they are well enough, since they will receive the same amount of
pay anyway.

19 Subject CT4 September 2012 Question 5

(i) Probability of a policyholder making only one claim in a given year

Let p be the probability of making exactly one claim in the year. Then:

The probability of making more than one claim = 0.25 p

The probability of making no claims = 1 - 1.25 p

So, the transition matrix is:

Ê 1.25 p 1 - 1.25 p 0 ˆ
Á
P = 1.25 p 0 1 - 1.25 p˜
Á ˜
ÁË 0.25 p p 1 - 1.25 p˜¯

© IFE: 2019 Examinations Page 123

 Batch3

This Markov chain has a finite state space, is irreducible and is aperiodic.
This means that the long-term behaviour of the chain can be obtained by
finding the unique solution p to the equation:

pP = p

Ê 1.25 p 1 - 1.25 p 0 ˆ
¤ p Á 1.25 p 0 1 - 1.25 p˜ = p (1)
Á ˜
ÁË 0.25 p p 1 - 1.25 p˜¯

where p = (1 - 2a, a, a ) .

So, we have:

(1 - 2a )(1 - 1.25 p ) + ap = a (2)

and a (1 - 1.25 p ) + a (1 - 1.25 p ) = a (3)

Equation (3) gives:

Ê 5p ˆ
2 Á1 - =1
Ë 4 ˜¯

1 4 2
¤ p= ¥ =
2 5 5

So, the probability of making exactly one claim is 40%.

Page 124 © IFE: 2019 Examinations

 Batch3

(ii) Probability that a policyholder at 0% is at 40% after three years

So, the transition matrix is:

Ê 0.5 0.5 0 ˆ
P = Á 0.5 0 0.5˜
Á ˜
ËÁ 0.1 0.4 0.5˜¯

Ê 0.5 0.25 0.25ˆ

P2 = Á * * * ˜
Á ˜
ÁË * * * ˜¯

Ê 0.5 0.25 0.25ˆ Ê 0.5 0.5 0 ˆ Ê * * 0.25ˆ

P =P P =Á *
3 2
* * ˜ Á 0.5 0 0.5˜ = Á * * * ˜
Á ˜Á ˜ Á ˜
ÁË * * ˜ Á ˜ Á
* ¯ Ë 0.1 0.4 0.5¯ Ë * * * ˜¯

So, the required probability is 25%.

(iii) Probability that a policyholder at 0% is at 40% after 20 years

From Equation (2) we have:

(1 - 2a )(1 - 1.25 p ) + ap = a

¤ (1 - 2a) ÊÁË1 - 21 ˆ˜¯ + 25a = a

1 2a
¤ -a+ =a
2 5

1 5 5
¤ a= ¥ =
2 8 16

So, the required probability is approximately 31.25%.

© IFE: 2019 Examinations Page 125

 Batch3

20 Subject CT4 September 2012 Question 6

(i) Stationary distribution of a Markov chain

We say that p j , j ΠS is a stationary probability distribution for a Markov

chain with transition matrix P if the following conditions hold for all j in S :

● pj = Â p i pij
i ŒS
● pj ≥0

● Â pj =1
j ŒS

(ii) Transition matrix

The one-step transition matrix is:

Ê 0.4 0.6 ˆ
P=Á
Ë 0.05 0.95¯˜

(iii) Long run proportion of games which have to be abandoned

This Markov chain has a finite state space, is irreducible and is aperiodic.
This means that the long-term behaviour of the chain can be obtained by
finding the unique solution p to the equation:

pP = p

Ê 0.4 0.6 ˆ
¤ (p 1, p 2 ) ÁË 0.05 0.95˜¯
= (p 1, p 2 )

0.4p 1 + 0.05 (1 - p 1 ) = p 1

¤ 0.05 = 0.65p 1

1 12
¤ p1 = and, hence p 2 = .
13 13

So, the required probability is 7.69%.

Page 126 © IFE: 2019 Examinations

 Batch3

(iv) Maximum amount the stadium should be prepared to pay

The expected long-term cost under the current company, Floodwatch, is

1
¥ ($10,000 + $1,000) = $846.15 per day.
13

If Light Fantastic are used, the new transition matrix is:

Ê 0.2 0.8 ˆ
P=Á
Ë 0.05 0.95¯˜

Solving p P = p , we get:

Ê 0.2 0.8 ˆ
(p 1, p 2 ) ÁË 0.05 0.95˜¯
= (p 1, p 2 )

0.2p 1 + 0.05 (1 - p 1) = p 1

¤ 0.05 = 0.85p 1

1 16
¤ p1 = and, hence p 2 = .
17 17

So, the new required probability is 5.88%.

Solving for the maximum that the stadium should be prepared to pay $M ,
we get:

1 1
¥ $11,000 > ¥ ($10,000 + $M )
13 17

¤ $M < $4,384.62

So, the maximum that the stadium should be prepared to pay is $4,384.62.

© IFE: 2019 Examinations Page 127

 Batch3

21 Subject CT4 April 2013 Question 11

(i) Meaning of a time-inhomogeneous Markov chain

A Markov chain is a stochastic process with a discrete time set and a

discrete state space that has the Markov property, ie the future probabilities
are fully determined by the current state.

A Markov chain is time-inhomogeneous if the transition probabilities for each

time step do not all remain constant. The transition probabilities are
therefore dependent on the actual starting time and finishing time, not just on
the length of the time interval being considered.

An example would be the NCD level of a young driver whose probabilities

will change over time as he/she gains more experience.

Other examples include:

 the NCD level of insured drivers as safety campaigns reduce accident

levels over time
 the health status (H, S or D) of a person recorded once a year over their
lifetime
 the weather each day (eg sunny, rainy, mixed) as we move through the
seasons.

(ii) Nature of the boundaries

The boundaries of this process are the 0% state and the 40% state. Both
are mixed because, for example, a policyholder in the 40% state may either
remain in the 40% state for the next year or move to a different state (25% or
10%) following a claim.

(iii)(a) How many states are required?

To model this as a Markov chain, we require 4 states (0%, 10%, 25% and
40%).

Page 128 © IFE: 2019 Examinations

 Batch3

(iii)(b) Transition graph

The transition graph looks like this:

p2 +

p0 p0 p0 p0
0% 10% 25% 40%

p1 + p2 + p1 + p2 + p1 p1

p2 +

where:

p0 = the probability of no claims during a particular year

p1 = the probability of exactly one claim
p2 + = 1 - p0 - p1 = the probability of two or more claims

(iv) Long-term proportion

The one-step probability transition matrix P for the process is then:

0% 10% 25% 40%

0% È1 - p0 p0 0 0˘
Í ˙
10% Í1 - p0 0 p0 0˙
25% Í p2 + p1 0 p0 ˙
Í ˙
40% ÎÍ 0 p2 + p1 p0 ˚˙

The number of claims made during the year will have a Bin(12, 0.04)
distribution. So:

p0 = 0.9612 = 0.61271
Ê12ˆ
p1 = Á ˜ ¥ 0.9611 ¥ 0.04 = 0.30635
Ë 1¯

and:

p2 + = 1 - p0 - p1 = 0.08094

© IFE: 2019 Examinations Page 129

 Batch3

The long-term probabilities for the 4 states can be found by finding the
stationary distribution, ie the set of probabilities p i that satisfy the matrix
equation p = p P with the additional condition  pi = 1.
i

Written out in full, this matrix equation is:

p0 = 0.38729p 0 + 0.38729p 10 + 0.08094p 25 (1)

p 10 = 0.61271p 0 + 0.30635p 25 + 0.08094p 40 (2)
p 25 = 0.61271p 10 + 0.30635p 40 (3)
p 40 = 0.61271p 25 + 0.61271p 40 (4)

If we try to express everything in terms of p 25 , Equation (4) tells us that:

0.61271p 25
p 40 = = 1.5820p 25
1 - 0.61271

Equation (3) then gives:

p 25 - 0.30635p 40 p 25 - 0.30635 ¥ 1.5820p 25

p 10 = = = 0.84111p 25
0.61271 0.61271

Rearranging Equation (2) tells us that:

p 10 - 0.30635p 25 - 0.08094p 40
p0 =
0.61271
0.84111p 25 - 0.30635p 25 - 0.08094 ¥ 1.5820p 25
=
0.61271
= 0.66379p 25

So the vector of stationary probabilities has the form:

(p 0 , p 10 , p 25 , p 40 ) = (0.66379p 25 , 0.84111p 25 , p 25 ,1.5820p 25 )

Page 130 © IFE: 2019 Examinations

 Batch3

Since these probabilities must add up to 1, this tells us that:

0.66379p 25 + 0.84111p 25 + p 25 + 1.5820p 25 = 1

fi 4.0869p 25 = 1

fi p = 1
25 = 0.24468
4.0869

So the proportion of policyholders at the 25% level in the long run is 24.5%.

(v) Appropriateness of the model

The model has some good points:

 A Markov chain model is a natural model to use here. It provides the
right structure, since the states are discrete and there is a discrete one-
year time step for the transitions.
 It is probably a reasonable approximation to assume that the claims in
each month are independent. However, if a vehicle is badly damaged in
an accident and is off the road for a couple of months, this would reduce
the probability of a claim in those months.
 It seems appropriate that those policyholders who make more than one
claim in a year are penalised more heavily than those who make only
one claim, as this encourages a safer driving/no claims culture.
However, it also has some bad points:
 This model assumes that all policyholders have the same probability of
making a claim in each month. This is unlikely to be true, since the
policyholders will have a range of different driving standards and will be
driving different vehicles in different driving conditions.
 The process may not be Markov because policyholders’ decisions
whether to put in a claim following an accident may be influenced by
whether they have already made a claim recently.
 The probabilities of making a claim are unlikely to be the same for
policyholders at all levels. For example, the policyholders in the 40%
level will tend to be safer drivers.
 In real life the process may not be time-homogeneous because the
probabilities may change over time, eg there are likely to be more
claims in a severe winter.

© IFE: 2019 Examinations Page 131

 Batch3

22 Subject CT4 September 2013 Question 2

(i) Estimate the transition probabilities

The table below shows the numbers of transitions between consecutive

match results.

U C D Total
U I II III 6
C II III II 7
D II III I 6

So the transition probability matrix is:

U C D U C D
 1 2 3  1 1 1
U  6 6 6 U  6 3 2
or (if we cancel the fractions)
C  2 3 2 C  2 3 2
 7 7 7  7 7 7
D  2 3 1 D  1 1 1
 6 6 6  3 2 6

(ii) Probability calculation

We can find this probability by listing the outcomes for the next 3 matches
where there are at least 2 wins for United, noting that the last match that was
played was a ‘C’. In the table below ‘*’ indicates a ‘wild card’, ie any result.

Next 3 results Probability

(C)UU* 21  2
7 6 76
(C)UCU 2  62  72  7867
7
(C)UDU 2  63  62  712
7 66
(C)CUU 3  72  61  7766
7
(C)DUU 2  62  61  7646
7

So the required probability is:

2 14 16 1 1 4 10
       0.15873
7  6 7  7  6 7  6  6 21 21 63 63

Page 132 © IFE: 2019 Examinations

 Batch3

23 Subject CT4 September 2013 Question 5

(i) Transition matrix

The one-step probability transition matrix P for the process is:

0% 25% 50% 60%

0%  0.2 0.8 0 0 
 
25%  0.2 0 0.8 0 
50%  0.2 0 0 0.8 
 
60%  0 0.2 0 0.8 

(ii)(a) Is the process irreducible?

A Markov chain model is irreducible if it is possible to reach any state j from

any state i in a finite number of moves.

We can see that this process is irreducible because it is possible to cycle

through all the states following the path
0%  25%  50%  60%  25%  0% .

(ii)(b) Is the process aperiodic?

A state in a Markov chain model is periodic with period d if it is only

possible to return to that state in multiples of d steps, where d  2 .

As this process is irreducible, all the states have the same period. We can
see that the process is aperiodic because it is possible to ‘loop’ from the 0%
state back to itself in one time step, so this state, and hence all the states,
has a period of 1.

(iii) Long-term proportion

The stationary distribution is the set of probabilities  i that satisfy the matrix
equation    P with the additional condition i  1.
i

© IFE: 2019 Examinations Page 133

 Batch3

Written out in full, this matrix equation is:

0  0.2 0  0.2 25  0.2 50 (1)

 25  0.8 0  0.2 60 (2)
 50  0.8 25 (3)
 60  0.8 50  0.8 60 (4)

If we try to express everything in terms of  25 , Equation (3) tells us that:

 50  0.8 25

Equation (4) then gives:

 60  0.8(0.8 25 )  0.8 60

 0.2 60  0.64 25

  60  3.2 25

Equation (2) then gives:

 25  0.8 0  0.2(3.2 25 )

0.36
 0   25  0.45 25
0.8

So the vector of stationary probabilities has the form:

( 0 ,  25 ,  50 ,  60 )   0.45 25 ,  25 ,0.8 25 ,3.2 25 

Since these probabilities must add up to 1, this tells us that:

0.45 25   25  0.8 25  3.2 25  1

 5.45 25  1

1
  25   0.1835
5.45

Page 134 © IFE: 2019 Examinations

 Batch3

So the stationary distribution is:

( 0 ,  25 ,  50 ,  60 )   0.0826, 0.1835, 0.1468, 0.5872 

(iv) Protected NCD scheme

This process is no longer irreducible since the 60% state is an absorbing

state and it is not possible to get from this state to any of the other states.

The 0% and 60% states are aperiodic because they have ‘loops’. The 25%
and the 50% states are also aperiodic because they interconnect with the
0% state and therefore have the same periodicity. So the whole chain is
aperiodic.

In the long run, all policyholders will now end up in the 60% state. So the
stationary distribution for the protected scheme is
( 0 ,  25 ,  50 ,  60 )   0,0,0,1 .

24 Subject CT4 April 2014 Question 6

(i) MLE under the Poisson model

Here l , the average number of events occurring to each member, is m E .

So the probability of di events occurring to the i th individual in the

population is:

e - mE ( m E )di
P (Di = di ) =
di !

Assuming that there are i independent lives in the population, the likelihood
function for the whole population is therefore:

n n
n e - mE ( m E )di  di  di n 1
L=’ = e - n m E ¥ m i =1 ¥ E i =1 ¥ ’
i =1 di ! i =1 di !

So the log-likelihood is:

n
ln L = - n m E + Â d i ln m + ... (terms not involving m )
i =1

© IFE: 2019 Examinations Page 135

 Batch3

Differentiating with respect to m to find the MLE gives:

∂ 1 n
ln L = - nE + Â di
∂m m i =1

Equating to zero and rearranging gives the maximum likelihood estimator:

1 n
m =
nE
 di
i =1

The second derivative is:

∂2 1 n
ln L = - Â di <0
∂m 2 m2 i =1

So the value we have estimated for m is indeed a maximum.

(ii) Calculate the MLE

We can calculate the waiting times for each student, ie the time between
arriving at the bus stop and the next bus arriving or the student leaving.

Student Time Time Time Waiting time

arrived left of next bus (minutes)
1 4:00 4:05 5
2 4:10 4:35 25
3 4:20 4:30 10
4 4:30 4:35 5
5 4:40 4:50 10
6 4:45 4:50 5
7 4:55 5:05 10
8 5:00 5:20 20
9 5:10 5:40 30
10 5:10 6:10 60
Total 180

Page 136 © IFE: 2019 Examinations

 Batch3

We can now calculate the MLE of the arrival rate for the buses by dividing
the total number of buses arriving by the total waiting time for the students.
(Note that only 4, not 6, buses arrived.) This gives:

4 4
mˆ = per minute = = 1.33 per hour
180 3

An alternative approach is to argue as follows:

Four buses arrived during the period 4pm to 6.10pm. However, there are
gaps in this period during which no-one was at the bus stop, so the length of
the observation period is not the full 2 hours and 10 minutes. For example, if
a bus had arrived at 4.07pm, its arrival would not have been recorded.

There is at least one person waiting at the bus stop during the following
intervals:

Interval Length of interval (minutes)

4.00pm-4.05pm 5
4.10pm-4.35pm 25
4.40pm-4.50pm 10
4.55pm-6.10pm 75

115
So the bus stop was ‘observed’ for a total of 115 minutes or hours
60
between 4pm and 6.10pm. So the estimated arrival rate of buses is:

4 48
= = 2.087 per hour
115 / 60 23

Note that dividing the number of students who caught a bus by the total
observed waiting time for all the students gives the answer:

6 6
= = 2 per hour
180 / 60 3

This is an estimate of the rate at which students catch a bus, rather than an
estimate of the arrival rate of buses.

© IFE: 2019 Examinations Page 137

 Batch3

(iii) Comment

The assumption that the arrival of buses follows a Poisson process may not
be valid. In particular:

 The model assumes that the arrival rate is constant. In reality there
may be more buses scheduled to arrive during the busiest times of day.

 The model also assumes that the arrival times of the buses are
random. In reality there is a timetable, so they will be most likely to
arrive around those times.

 The model also assumes that the times between buses are
independent. In reality, several consecutive buses may all be affected
by an obstruction such as a traffic accident and the times would not be
independent.

Also, there were periods when no students were at the bus stop, eg between
4:50 and 4:55. We do not know whether any buses arrived during these
periods. These would affect our estimate of the arrival rate.

25 Subject CT4 April 2014 Question 10

(i) Stochastic process

The condition of each heating element can be modelled as a Markov chain.

(ii) Is it irreducible / periodic?

A Markov chain model is irreducible if it is possible to reach any state j from

any state i in a finite number of moves. This chain is not irreducible since
it’s not possible for a heating element to move from Good to Excellent, for
example.

A state in a Markov chain model is periodic with period d if it is only possible

to return to that state in multiples of d steps, where d ≥ 2 . In this case it is
possible to remain in each of the four states after just one step (eg Excellent
at the beginning and end of a firing cycle). So this process is aperiodic.

Page 138 © IFE: 2019 Examinations

 Batch3

(iii) Probability

For brevity we will refer to the states by their initial letters (eg E = Excellent).

We can derive the probability of moving from state E to state P in two steps
by considering the possible paths:

0.5
¥ 0.2
 + 0.2
¥ 0.3
 + 0.2
¥ 0.5
 = 0.26
E ÆE ÆP E ÆGÆP E ÆP Æ P

(iv) Transition matrix with replacement of failed heating elements

The revised transition matrix is:

È0.6 0.2 0.2˘

Í ˙
Í0.2 0.5 0.3˙
ÍÎ0.5 0 0.5˙˚

(v) Long-term probabilities

The long-term probabilities for the 3 states can be found by finding the
stationary distribution, ie the set of probabilities p i that satisfy the matrix
equation p = p P with the additional condition Âpi = 1.
i
Written out in full, this matrix equation is:

pE = 0.6p E + 0.2p G + 0.5p P (1)

pG = 0.2p E + 0.5p G (2)
pP = 0.2p E + 0.3p G + 0.5p P (3)

If we try to express everything in terms of p E , Equation (2) can be

rearranged to give:

0.5p G = 0.2p E
fi p G = 0.4p E

© IFE: 2019 Examinations Page 139

 Batch3

Equation (3) then gives:

p P = 0.2p E + 0.3(0.4p E ) + 0.5p P

fi 0.5p P = 0.32p E
fi p P = 0.64p E

So the vector of stationary probabilities has the form:

(p E , p G , p P ) = (p E ,0.4p E ,0.64p E )

Since these probabilities must add up to 1, this tells us that:

p E + 0.4p E + 0.64p E = 1

fi 2.04p E = 1

fi pE =
1
2.04
= 0.4902 = ( 25
51 )
So the long-term proportions are:

 25 = 49.02% in the Excellent state

51
 10 = 19.61% in the Good state
51
 16 = 31.37% in the Poor state
51

(vi) Expected annual cost

The kiln is fired 100 times a year and the cost will be £1,000 + £50 = £1,050
whenever it finishes the cycle in the Failed state. So the expected annual
cost will be:

100 ¥ ( 2551 ¥ 0.1 + 1051 ¥ 0.2 + 1651 ¥ 0.5) ¥ 1,050 = £25,735

Page 140 © IFE: 2019 Examinations

 Batch3

(vii) Impact of the change

To deal with this new replacement policy, we need to combine both the
Failed and the Poor columns in the original matrix with the Excellent column,
which gives the following 2 ¥ 2 matrix:

È0.8 0.2˘
P* = Í ˙
Î0.5 0.5 ˚

The long-term probabilities can be found by solving the simultaneous

equations:

p E* = 0.8p E* + 0.5p G*
p G* = 0.2p E* + 0.5p G*

This gives:

p E* = 5
7
= 0.7143 and p G* = 2
7
= 0.2857

So the new expected annual cost is:

Cost = 100 ¥ ( 57 ¥ 0.1 + 72 ¥ 0.2) ¥ 1,050 + 100 ¥ ( 57 ¥ 0.2 + 72 ¥ 0.3) ¥ 50

= 13,500 + 1,143 = £14,643

So there is an annual saving of 25,735 - 14,643 = £11,092 .

© IFE: 2019 Examinations Page 141

 Batch3

26 Subject CT4 September 2014 Question 5

(i) Minimum number of states

To model this as a Markov chain we will need 4 states.

(ii) Transition graph

The transition graph looks like this:

0.15
0.85 1N 2R

0.3
0.7 0.75
0.25
1P 2N
0.85
0.15

The states used are:

 1N = In Division 1, Not promoted last time
 1P = In Division 1, Promoted last time
 2R = In Division 2, Relegated last time
 2N = In Division 2, Not relegated last time

(iii) Transition matrix

The transition matrix is:

1N 1P 2R 2N
1N È0.85 0 0.15 0 ˘
Í ˙
1P Í 0.7 0 0.3 0 ˙
2R Í 0 0.25 0 0.75 ˙
Í ˙
2N ÎÍ 0 0.15 0 0.85 ˚˙

Page 142 © IFE: 2019 Examinations

 Batch3

(iv)(a) Is the process irreducible?

A Markov chain model is irreducible if it is possible to reach any state j from

any state i .

We can see that this process is irreducible because it is possible to cycle

through all the states following the path 1N Æ 2R Æ 2N Æ 1P Æ 1N .

(iv)(b) Is the process aperiodic?

A state in a Markov chain model is periodic with period d if it is only

possible to return to that state in multiples of d steps, where d ≥ 2 .

As this process is irreducible, all the states have the same period. We can
see that the process is aperiodic because it is possible to ‘loop’ from the 1N
state back to itself in one time step, so this state, and hence all the states,
have a period of 1.

(v) Minimum number of seasons

The table below shows the probabilities of moving from State 1P to State 2R
in different numbers of seasons.

Cumulative
Seasons Path Probability
probability
1 season 1P Æ 2R 0.3 0.3
2 seasons 1P Æ 1N Æ 2R 0.7 ¥ 0.15 = 0.105 0.405
3 seasons 1P Æ 1N Æ 1N Æ 2R 0.7 ¥ 0.85 ¥ 0.15 = 0.08925 0.49425
3 2
4 seasons 1P (Æ 1N ) Æ 2R 0.7 ¥ 0.85 ¥ 0.15 = 0.07586 0.57011
5 seasons 1P (Æ 1N )4 Æ 2R 0.7 ¥ 0.853 ¥ 0.15 = 0.06448 0.63460

So we see that it will take 5 seasons for the probability to exceed 60%.

An alternative answer with 6 states (Top of Division 1, Bottom of Division 1,

Elsewhere in Division 1, Top of Division 2, Bottom of Division 2, Elsewhere
in Division 2) scored 0 for part (i), but could score full credit for other parts if
followed through correctly.

© IFE: 2019 Examinations Page 143

 Batch3

27 Subject CT4 September 2014 Question 6

(i)(a) Number of states

A chain is Markov if the current state fully determines the probabilities for
future movements between states.

With this NCD system, if policyholders on Level 2 make a claim, the state
they move to will depend on whether or not they made a claim the previous
year. If they made no claim the previous year, they move down to Level 1,
but if they made at least one claim in the previous year, they move down to
Level 0. So, to obtain a Markov model, we need to split Level 2 into two
states, eg 2+ for those who didn’t make a claim the previous year and 2– for
those who did.

The resulting model will have 5 states.

(i)(b) Transition graph

The transition graph looks like this:

1–p 1–p 1–p 1–p

0 1 2+ 3

p p p 1–p

p
p 2-

(ii) Long-term proportion at the 25% level

The one-step probability transition matrix P for the process is:

0 1 2+ 2- 3
0 Èp 1- p 0 0 0˘
Í ˙
1 Íp 0 1- p 0 0 ˙
2+ Í0 p 0 0 1- p ˙
Í ˙
2- Í p 0 0 0 1- p ˙
3 Í0 0 0 p 1 - p ˙˚
Î

Page 144 © IFE: 2019 Examinations

 Batch3

The stationary distribution is the set of probabilities p i that satisfy the matrix
equation p = p P with the additional condition Âpi = 1.
i

Written out in full, this matrix equation is:

p0 = pp 0 + pp 1 + pp 2 - ... (1)
p1 = (1 - p )p 0 + pp 2 + ... (2)
p 2+ = (1 - p )p 1 ... (3)
p 2- = pp 3 ... (4)
p3 = (1 - p )p 2 + + (1 - p )p 2 - + (1 - p )p 3 ... (5)

We can try to express all the probabilities in terms of p 3 (say).

Equation (4) already tells us that:

p 2 - = pp 3

Equation (5) then tells us that:

p 3 = (1 - p )p 2 + + (1 - p )( pp 3 ) + (1 - p )p 3

fi (1 - p )p 2+ = p 3 - (1 - p )pp 3 - (1 - p)p 3 = p2p 3

p2
fi p 2+ = p3 ... (6)
1- p

Equation (3) then tells us that:

p 2 + = (1 - p )p 1
p2
fi p 3 = (1 - p)p 1
1- p
p2
fi p1 = p3 ... (7)
(1 - p )2

© IFE: 2019 Examinations Page 145

 Batch3

However the question tells us that p 3 = 9p 1 , or p 1 = 1p . Comparing this

9 3
with Equation (7), we see that:

p2 1
2
=
(1 - p ) 9

Square-rooting (and remembering that p and 1 - p must be positive) gives:

p 1
=
1- p 3
fi 3p = 1 - p
fi 4p = 1
p= 1
fi 4

Substituting this value of p into Equations (4), (6) and (7), we get:

p 1 = 91 p 3
p 2 - = 41 p 3
p 2+ = 1 p
12 3

Finally, from Equation (1), we have:

p 0 = pp 0 + pp 1 + pp 2 -
ie p 0 = 41 p 0 + 41 p 1 + 41 p 2 -
fi 4p 0 = p 0 + p 1 + p 2 -
fi 3p 0 = p 1 + p 2 -
fi p 0 = 31 (p 1 + p 2 - ) = (
1 1
p
3 9 3 )
+ 41 p 3 = 13
p
108 3

So the vector of stationary probabilities has the form:

(p 0 , p 1, p 2 + , p 2 - , p 3 ) = ( 13
p , 1 p , 1 p , 1 p ,p
108 3 9 3 12 3 4 3 3 )

Page 146 © IFE: 2019 Examinations

 Batch3

Since these probabilities must add up to 1, this tells us that:

13
p
108 3
+ 91 p 3 + 12
1 p + 1p +p = 1
3 4 3 3

169
fi p
108 3
=1
108
fi p3 = 169
= 0.6391

So the stationary distribution is:

(p 0 , p 1, p 2 + , p 2 - , p 3 ) = ( 13 12
, , 9 , 27 , 108
169 169 169 169 169 )
or
(0.0769,0.0710,0.0533,0.1598,0.6391)

The proportion at the 25% discount level in the long run is:

9 27 36
p 2+ + p 2- = 169
+ 169 = 169
= 0.2130

(iii)(a) New state space

Considering each of the discount levels:

 Level 3: Policyholders on Level 3 (40% discount) cannot have made a
claim the previous year. So State 3 is not affected.
 Level 2: We have already split Level 2 (25% discount level) according to
whether or not there was a claim the previous year. So this already
allows us to distinguish the two different probabilities.
 Level 1: Because there will be different probabilities for Level 1 for those
who came up from Level 0 and those who came down from Level 2,
Level 1 will now have to be split as well.
 Level 0: All policyholders on Level 0 must have made a claim the
previous year. So this state is not affected.

So we will need to split Level 1 into two states, eg State 1+ for those who
didn’t make a claim the previous year and State 1– for those who did. So we
now need 6 states in total.

© IFE: 2019 Examinations Page 147

 Batch3

(iii)(b) Transition matrix

Let q denote the probability of a claim if no claim was made the previous
year and r denote the probability of a claim if a claim was made the
previous year.

The transition matrix now looks like this:

0 1+ 1- 2 + 2 - 3
0 È r 1- r 0 0 0 0 ˘
Í ˙
1+ Íq 0 0 1- q 0 0 ˙
1- Í r 0 0 1- r 0 0 ˙
Í ˙
2 + Í0 0 q 0 0 1- q˙
2 - Ír 0 0 0 0 1- r ˙
Í ˙
3 ÎÍ 0 0 0 0 q 1 - q ˚˙

Page 148 © IFE: 2019 Examinations

 Batch3

28 Subject CT4 April 2015 Question 7

(i) Markov chain

A Markov chain is a stochastic process with a discrete state space and a

discrete time set that has the Markov property, ie the future probabilities
depend only on the current state.

(ii) Transition matrix

Since users are equally likely to move to any of the possible states, the
probabilities are as shown below:

1
2

N(ile) B(anana)
1
2

1
1 1 2
2 3
1
3

1
C(heep) H(andbook)
1
3

So the transition matrix is:

N B C H
N È0 1 1 0˘
Í 2 2 ˙
B Í1 0 1
0˙
Í2 2 ˙
C Í1 1 0 1˙
Í3 3 3˙
H ÍÎ 0 0 1 0 ˙˚

© IFE: 2019 Examinations Page 149

 Batch3

(iii) Long-term proportions

The long-term probabilities for the four states can be found by calculating the
stationary distribution, ie the set of probabilities p i that satisfy the matrix
equation p = p P , with the additional condition  pi = 1.
i
Written out in full, this matrix equation is:

pN = 1p + 31 p C
2 B
pB = 1p + 31 p C
2 N
pC = 1p + 21 p B + p H
2 N
1
pH = p
3 C

Equivalently:

6p N = 3p B + 2p C (1)
6p B = 3p N + 2p C (2)
6p C = 3p N + 3p B + 6p H (3)
3p H = p C (4)

Subtracting Equation (2) from Equation (1) tells us that:

6p N - 6p B = 3p B - 3p N
fi 9p N = 9p B
fi pN = pB

Equations (3) and (4) then tell us that:

6p C = 3p N + 3p N + 2p C
fi 4p C = 6p N
3
fi pC = 2
pN

and Equation (4) tells us that:

pH = 1 3
3 2 ( )
pN = 1
p
2 N

Page 150 © IFE: 2019 Examinations

 Batch3

So the vector of stationary probabilities has the form:

(
(p N , p B , p C , p H ) = p N , p N , 32 p N , 21 p N )
Since these probabilities must add up to 1, this tells us that:

p N + p N + 32 p N + 21 p N = 1
fi 4p N = 1
fi pN = 1
4

So: (p N , p B , p C , p H ) = ( 1 1 3 1
, , ,
4 4 8 8 )
The long-term proportions are:

 1 = 25% on the Nile site

4
 1 = 25% on the Banana site
4
3
 8
= 37.5% on the Cheep site
 1 = 12.5% on the Handbook site.
8

29 Subject CT4 September 2015 Question 10

(i) Transition graph

If we denote the state for markers on holiday by H , the transition graph for
this process looks like this:

0.1 0.6

A B

0.8 0.2
0.5 0.5

0.1 0.2
H

© IFE: 2019 Examinations Page 151

 Batch3

(ii) Transition matrix

The transition matrix is:

A B H
A È0.8 0.1 0.1˘
Í ˙
B Í 0.2 0.6 0.2˙
H ÍÎ0.5 0.5 0 ˙˚

(iii) Long-term probabilities

The long-term probabilities for the 3 states can be found by finding the
stationary distribution, ie the set of probabilities p i that satisfy the matrix
equation p = p P with the additional condition Âpi = 1.
i
Written out in full, this matrix equation is:

pA = 0.8p A + 0.2p B + 0.5p H (1)

pB = 0.1p A + 0.6p B + 0.5p H (2)
pH = 0.1p A + 0.2p B (3)

If we subtract Equation (2) from Equation (1), we get:

p A - p B = 0.7p A - 0.4p B
fi 0.3p A = 0.6p B
fi p A = 2p B

Equation (3) then gives:

p H = 0.1(2p B ) + 0.2p B = 0.4p B

So the vector of stationary probabilities has the form:

(p A , p B , p H ) = (2p B , p B ,0.4p B )

Page 152 © IFE: 2019 Examinations

 Batch3

Since these probabilities must add up to 1, this tells us that:

2p B + p B + 0.4p B = 1

fi 3.4p = 1
B

fi pB =
1
3.4
= 0.2941 = ( ) 5
17

and (p A , p B , p H ) = ( 10 5 2
, ,
17 17 17 )
So the long-term proportions marking each subject are:
10
 17
= 58.82% for Subject A
5
 17
= 29.41% for Subject B

(iv) Proportion to be allocated to Subject B

Let x denote the proportion to be allocated to Subject B on returning from

holiday. The probabilities of 0.5 in the model are now replaced by x and
1- x .

0.1 0.6

A B

0.8 0.2
1–x x

0.1 0.2
H

The transition matrix now becomes:

A B H
A È 0.8 0.1 0.1˘
Í ˙
B Í 0.2 0.6 0.2˙
H ÍÎ1 - x x 0 ˙˚

© IFE: 2019 Examinations Page 153

 Batch3

The revised stationary distribution equations are:

p A* = 0.8p A* + 0.2p B* + (1 - x )p H* (1*)

p B* = 0.1p A* + 0.6p B* + xp H* (2*)
p H* = 0.1p A* + 0.2p B* (3*)

If we require p A* = p B* , then Equation (1*) tells us that:

p B* = 0.8p B* + 0.2p B* + (1 - x )p H*

fi 0 = (1 - x )p H*

fi x =1

So the required proportion is 100%, ie all examiners returning to marking

must be allocated to Subject B.

30 Subject CT4 April 2016 Question 10

(i) Probability Yolanda will be covered by Company C for 5 years

Yolanda will be covered by Company C for at least 5 years if she renews her
policy 4 times with Company C. This has probability:

0.4 4 = 0.0256

(ii) Probability Company A does not cover Zachary at the time of the fire

Company A will cover Zachary’s home at the time of the fire if he is insured
with Company A during the third year, which has probability pAA (2) .

We can calculate this probability by considering the possible paths:

pAA (2) = 0.5

¥ 0.5
 + 0.2
¥ 0.2
 + 0.2
¥ 0.3
 + 0.1
 ¥ 0 = 0.35
AÆ AÆ A AÆ B Æ A AÆC Æ A AÆD Æ A

So the probability that Company A does not cover his home at the time of
the fire is:

1 - pAA (2) = 1 - 0.35 = 0.65

Page 154 © IFE: 2019 Examinations

 Batch3

(iii) Expected proportions of homeowners

The long-term probabilities for the four states can be found by finding the
stationary distribution, ie the set of probabilities p i ≥ 0 that satisfy the matrix
equation p = p P with the additional condition Âpi = 1.
i
Written out in full, this matrix equation is:

pA = 0.5p A + 0.2p B + 0.3p C (1)

pB = 0.2p A + 0.6p B + 0.2p C + 0.2p D (2)
pC = 0.2p A + 0.1p B + 0.4p C + 0.2p D (3)
pD = 0.1p A + 0.1p B + 0.1p C + 0.6p D (4)

We can subtract 0.4p B from each side of Equation (2) to make all the
coefficients on the RHS equal to 0.2. Using the fact that
p A + p B + p C + p D = 1 then gives:

0.6p B = 0.2p A + 0.2p B + 0.2p C + 0.2p D = 0.2(p A + p B + p C + p D ) = 0.2

0.2 1
fi pB = = 3
0.6

Similarly, subtracting 0.5p D from each side of Equation (4) gives:

0.5p D = 0.1p A + 0.1p B + 0.1p C + 0.1p D = 0.1(p A + p B + p C + p D ) = 0.1

0.1 1
fi pD = = 5
0.5

Subtracting Equation (3) from Equation (2) tells us that:

p B - p C = 0.5p B - 0.2p C
fi 0.8p C = 0.5p B
5 5 1 5
fi pC = 8
pB = 8
¥ 3
= 24

Finally, we can find p A by subtraction:

5 31
p A = 1 - p B - p C - p D = 1 - 31 - 24
- 1
5
= 120

© IFE: 2019 Examinations Page 155

 Batch3

So: (p A , p B , p C , p D ) = ( 31 1 5 1
, , ,
120 3 24 5 ) = (0.2583, 0.3333, 0.2083, 0.2)
The long-term proportions are:

31
120
= 25.83% for Company A
1 = 33.33% for Company B
3
5
24
= 20.83% for Company C
1 = 20% for Company D.
5

(iv) Transition matrix after the takeover

If Company A is combined with Company D, any transfers to these

companies will now go to the new company. So we need to combine the
columns for A and D:

AD B C
A È0.6 0.2 0.2 ˘
Í ˙
B Í0.3 0.6 0.1˙
C Í0.4 0.2 0.4 ˙
Í ˙
D ÍÎ0.6 0.2 0.2 ˙˚

Since Companies A and D now correspond to a single state, we also need to

combine the rows for A and D:

AD B C
AD È0.6 0.2 0.2 ˘
Í ˙
B Í0.3 0.6 0.1˙
C ÍÎ0.4 0.2 0.4 ˙˚

Here the state labelled AD is for the new combined company (Addda).

(v) Comment

These calculations assume that the underlying probabilities will not be

affected by the merger. This will depend on the reason why Company A is
taking over Company D.

If Company D was similar to Company A in terms of premium rates and

customer satisfaction, then this assumption may be valid.

Page 156 © IFE: 2019 Examinations

 Batch3

However, the transfer probability of zero from Company D to Company A

suggests that customers of Company D tend to avoid Company A.

If one of the two companies is in difficulties or has a gained a bad reputation,

the merger may change the underlying probabilities.

If the merger results in a more efficient combined company, Addda may be

able to offer lower premiums and attract more customers.

31 Subject CT4 September 2016 Question 2

Irreducible?

A Markov chain is irreducible if it is possible to move from any state i to any

state j in a finite number of steps.

Chain 1 is irreducible because we can cycle through all the states following
the circuit 1 Æ 2 Æ 3 Æ 1.

Chain 2 is irreducible because we can cycle through all the states following
the circuit 1 Æ 2 Æ 4 Æ 3 Æ 1 .

Chain 3 is not irreducible because it is not possible to move from State 2 to

State 1.

Periodic?

A Markov chain is periodic (with period p ) if there is a number p > 1 such

that the process can only return to each state in a multiple of p steps.

Chain 1 is aperiodic (not periodic) because we can return to any given state
in 2 or 3 or 4 or 5 etc steps, and these numbers are not restricted to a
multiple of some number greater than 1.

Chain 2 is periodic with period 2 because we can only return to any given
state in an even number of steps.

Chain 3 is not periodic because neither state satisfies the definition of

periodic. From State 1 it is not possible to return to State 1 at all and a
‘return’ to State 2 will occur after just 1 step.

© IFE: 2019 Examinations Page 157

 Batch3

32 Subject CT4 September 2016 Question 11

(i) Transition diagram

The transition graph for this process looks like this:

2
1––  1 – 3

0% 20%

2
 
2
 2
40%

2
1––

(ii) Range of values of b

The transition matrix will be valid if the entries in each row add up to 1 (which
they do) and each entry lies in the range [0,1] .

We can easily see that:

 the b entries require 0 £ b £ 1
 the b 2 entries require -1 £ b £ 1
 the 1 - 3 b entry requires 0 £ b £ 1
3
 the 2 b entry requires 0 £ b £ 1 .
2

In combination, these limit the range to 0 £ b £ 1 .

3

The 1 - b - b 2 entries also require 0 £ 1 - b - b 2 £ 1.

The right-hand part of this inequality is automatically satisfied since we have

already limited b to positive values, so that 1 - b - b 2 must be less than or
equal to 1. So we just need to check when 0 £ 1 - b - b 2 .

Page 158 © IFE: 2019 Examinations

 Batch3

The roots of the quadratic equation 0 = 1 - b - b 2 are:

1 ± ( -1)2 - 4(1)( -1) 1 ± 5

b = = = 0.618 or - 1.618
-2 -2

Since the quadratic coefficient of 1 - b - b 2 is negative (ie a = -1 ), its graph

will have an inverted U shape, which means that it will take positive values
between the two roots, ie for the range -1.618 £ b £ 0.618 . This includes
the whole of the range 0 £ b £ 1 and so doesn’t limit the existing range we
3
have for b any further.

So the range of valid values for b is 0 £ b £ 1 .

3

(iii) Is the chain irreducible or periodic?

We can see from the diagram in part (i) that, in general, it is possible to cycle
through the states in the order 0% Æ 20% Æ 40% Æ 0% . This means that
it is possible to move from any state i to any state j in a finite number of
steps, which is the definition of irreducible.

However, in the special case where b = 0 , the transition rates between the
states become zero and each state becomes an absorbing state. In this
case the chain is not irreducible.

A Markov chain is periodic (with period p ) if there is a number p > 1 such

that the process can only return to each state in a multiple of p steps. This
cannot be the case here because the model will always have ‘loops’,
whatever the value of b . So it will never be periodic.

(iv) Long-term proportions

When b = 0.1 the transition matrix becomes:

Ê 0.89 0.1 0.01ˆ

P = Á 0.1 0.7 0.2 ˜
Á ˜
ÁË 0.01 0.1 0.89˜¯

© IFE: 2019 Examinations Page 159

 Batch3

The long-term probabilities for the 3 states can be found by finding the
stationary distribution, ie the set of probabilities p i that satisfy the matrix
equation p = p P with the additional condition Âpi = 1.
i
Written out in full, this matrix equation is:

p 0 = 0.89p 0 + 0.1p 20 + 0.01p 40

p 20 = 0.1p 0 + 0.7p 20 + 0.1p 40
p 40 = 0.01p 0 + 0.2p 20 + 0.89p 40

To simplify the arithmetic we can multiply these equations through by 100 or

by 10 so that they only involve whole numbers:

100p 0 = 89p 0 + 10p 20 + p 40 (1)

10p 20 = p 0 + 7p 20 + p 40 (2)
100p 40 = p 0 + 20p 20 + 89p 40 (3)

If we subtract Equation (3) from Equation (2) to eliminate p 0 , we get:

10p 20 - 100p 40 = -13p 20 - 88p 40

fi 23p 20 = 12p 40
23
fi p 40 = p
12 20

Equation (1) then gives:

23 143 13
11p 0 = 10p 20 + 12 p 20 = p
12 20
fi p0 = p
12 20

So the vector of stationary probabilities has the form:

(p 0 , p 20 , p 40 ) = ( 13
p , p , 23 p
12 20 20 12 20 )

Page 160 © IFE: 2019 Examinations

 Batch3

Since these probabilities must add up to 1, this tells us that:

13 23
p
12 20
+ p 20 + 12 p 20 = 1
fi 4p 20 = 1
fi p 20 = 1 ( = 0.25)
4

and (p 0 , p 20 , p 40 ) = ( 13 1 23
, ,
48 4 48 ) = (0.271,0.25,0.479)
So the long-term proportions are:

13
 48
= 27.1% with a 0% marginal tax rate
 1 = 25% with a 20% marginal tax rate
4
23
 48
= 47.9% with a 40% marginal tax rate.

(v) Probabilities for Lucy

Since Lucy currently pays 20% tax, the probabilities for Lucy’s tax rate for
2013 can be found from the middle row of entries in the matrix P 2 .

Ê 0.89 0.1 0.01ˆ Ê 0.89 0.1 0.01ˆ Ê • • • ˆ

2Á
P = 0.1 0.7 0.2 ˜ Á 0.1 0.7 0.2 = 0.161 0.52 0.319˜
˜ Á
Á ˜Á ˜ Á ˜
ÁË 0.01 0.1 0.89˜¯ ÁË 0.01 0.1 0.89˜¯ ÁË • • • ˜¯

So the probabilities for Lucy’s tax rate in 2013 are 16.1% for 0% tax, 52% for
20% tax and 31.9% for 40% tax.

33 Subject CT4 April 2017 Question 3

(i) Definition of a Markov chain

A Markov chain is a stochastic process that has a discrete time set and a
discrete state space and satisfies the Markov property, ie the future
probabilities depend only on the state currently occupied.

(ii) Time-homogeneous versus time-inhomogeneous Markov chains

A Markov chain is time-homogeneous if the transition probabilities remain

constant over time and time-inhomogeneous if they do not.

© IFE: 2019 Examinations Page 161

 Batch3

An example of a time-homogeneous Markov chain is a simple random walk,

since the probabilities of moving up or down by 1 unit are always equal to ½.

An example of a time-inhomogeneous Markov chain is where we record the

state of an individual at the end of each year as healthy, sick or dead. The
probabilities of mortality, sickness and recovery will change as the person
gets older.

34 Subject CT4 April 2017 Question 5

(i) Transition matrix

The transition diagram for this model is:

0.5 0.5 0.5

0.3 0.3 0.3 0.3

0.7 0 1 2 3 4 0.8

0.2 0.2 0.2 0.2

and the transition matrix is:

0 1 2 3 4
0 È0.7 0.3 0 0 0 ˘
Í ˙
1 Í 0.2 0.5 0.3 0 0 ˙
2 Í 0 0.2 0.5 0.3 0 ˙
Í ˙
3 Í 0 0 0.2 0.5 0.3 ˙
4 ÍÎ 0 0 0 0.2 0.8 ˙˚

(ii) Will they increase the size of the rack?

The long-run probabilities for the five states are the entries in the stationary
distribution. These comprise the set of probabilities p i ≥ 0 that satisfy the
matrix equation p = p P , with the additional condition Âpi = 1.
i

Page 162 © IFE: 2019 Examinations

 Batch3

Written out in full, this matrix equation is:

p 0 = 0.7p 0 + 0.2p 1 (1)

p 1 = 0.3p 0 + 0.5p 1 + 0.2p 2 (2)
p 2 = 0.3p 1 + 0.5p 2 + 0.2p 3 (3)
p 3 = 0.3p 2 + 0.5p 3 + 0.2p 4 (4)
p 4 = 0.3p 3 + 0.8p 4 (5)

Rearranging Equation (1) gives:

0.3p 0 = 0.2p 1

0.3
fi p1 = p = 1.5p 0
0.2 0

Equation (2) then gives:

1.5p 0 = 0.3p 0 + 0.5 (1.5p 0 ) + 0.2p 2

fi 0.45p 0 = 0.2p 2

0.45
fi p2 = p = 2.25p 0
0.2 0

Equation (3) then gives:

2.25p 0 = 0.3 (1.5p 0 ) + 0.5 (2.25p 0 ) + 0.2p 3

fi 0.675p 0 = 0.2p 3

0.675
fi p3 = p = 3.375p 0
0.2 0

Equation (4) then gives:

3.375p 0 = 0.3 (2.25p 0 ) + 0.5 (3.375p 0 ) + 0.2p 4

fi 1.0125p 0 = 0.2p 4

1.0125
fi p4 = p 0 = 5.0625p 0
0.2

© IFE: 2019 Examinations Page 163

 Batch3

Now, using the fact that p 0 + p 1 + p 2 + p 3 + p 4 = 1 , we have:

p 0 + 1.5p 0 + 2.25p 0 + 3.375p 0 + 5.0625p 0 = 1

fi 13.1875p 0 = 1

1 5.0625
fi p0 = = 0.0758 and p 4 = = 0.3839
13.1875 13.1875

So the long-term probability that the rack is completely full or empty is:

p 0 + p 4 = 0.0758 + 0.3839 = 0.4597

Since this is greater than 35%, the city will increase the size of the rack.

(iii) Comment

If the size of the rack is increased, there will be more states for the process
to occupy, so we would expect the probabilities for each state to decrease.

However, we cannot be sure what will happen because the model makes
some big assumptions, such as assuming fixed values for the probabilities
0.3 and 0.2. In reality, these will vary depending on the time of day. Also,
increasing the size of the rack may encourage more people to come to work
by bike.

35 Subject CT4 April 2017 Question 9

(i) Transition matrix

The transition matrix P is:

A B C
A È 0.6 0.2 0.2 ˘
Í ˙
B Í 0.25 0.6 0.15 ˙
C ÍÎ 0.1 0.4 0.5 ˙˚

Page 164 © IFE: 2019 Examinations

 Batch3

(ii) Probabilities for the third day

The probabilities of being in each location on the third day are given by the
row for A in the matrix P 2 .

È 0.6 0.2 0.2 ˘ È 0.6 0.2 0.2 ˘ È 0.43 0.32 0.25 ˘

Í ˙Í ˙ Í ˙
P 2 = Í0.25 0.6 0.15 ˙ Í0.25 0.6 0.15 ˙ = Í ∑ ∑ ∑ ˙
ÎÍ 0.1 0.4 0.5 ˚˙ ÎÍ 0.1 0.4 0.5 ˚˙ ÎÍ ∑ ∑ ∑ ˚˙

So the probabilities are: Atlantis = 43%, Beachy = 32% and Coral = 25%

(iii) Graph

Using the probabilities we have already calculated and the long-term

probabilities stated, we can sketch the following graph:

Probability of being at each location

1 A
0.9
0.8
0.7 A
Probability

0.6
0.5 A
0.4 B
B
0.3 A
B
C
0.2
C
0.1 B C
0 C
1 2 3 ... ∞
Day

(iv) Proportion of days involving a flight

The long-term probability of taking a flight on a given day is:

P(flight ) = p A ( pAB + pAC ) + p B ( pBA + pBC ) + p C ( pCA + pCB )

= 0.326 ¥ (0.2 + 0.2) + 0.419 ¥ (0.25 + 0.15) + 0.256 ¥ (0.1 + 0.4)
= 0.426

© IFE: 2019 Examinations Page 165

 Batch3

(v) State space for a Markov chain

We will need a 6-state model with the following states (where the bars
indicate the current state):

A Visited location A only

AB Visited locations A and B and currently in A
AB Visited locations A and B and currently in B
AC Visited locations A and C and currently in A
AC Visited locations A and C and currently in C
ABC Visited all three locations A, B and C

(vi) Transition diagram

The transition diagram looks like this:

AB

AB

A ABC

AC

AC

Page 166 © IFE: 2019 Examinations

 Batch3

36 Subject CT4 September 2017 Question 1

(i) Determine the remaining entries

Impossible movements (where there is no arrow on the diagram) have zero

probability. So:

pAC = pBA = pBB = pCB = 0

Also, the entries in each row of the matrix must add up to 1. So:

pAB = 1 - 0.2 - 0 = 0.8 and pCA = 1 - 0 - 0.4 = 0.6

So the complete matrix is:

A Ê 0.2 0.8 0 ˆ
B Á 0 0 1.0 ˜
Á ˜
C ÁË 0.6 0 0.4˜¯

(ii) Valid sample paths

For Path 1 in part (a) the sequence of states occupied at integer time points
is:

C A A A B C C A A B ?

All of the movements in this path have strictly positive probabilities, so this is
a valid sample path.

For Path 2 in part (b) the sequence of states occupied at integer time points
is:

A B B …

The movement from B to B has zero probability, so this is not a valid sample
path.

© IFE: 2019 Examinations Page 167

 Batch3

37 Subject CT4 September 2017 Question 8

(i) Premium

Let P denote the full annual premium.

We can calculate the average (expected) annual premium paid by all

policyholders by applying the proportions and the discount rates to each
level:

Av prem = 4.4% ¥ P + 10.5% ¥ 0.85P + 25.1% ¥ 0.7P + 60.0% ¥ 0.6P

= 0.66895P

The number of claims for each policyholder during the year has a
Poisson(0.35) distribution.

Since claims are paid up to a maximum of 3, the probabilities for the annual
number of claims from each policy are:

P (0 claims) = e -0.35 = 0.70469

P (1 claim) = 0.35e -0.35 = 0.24664
0.352 -0.35
P (2 claims) = e = 0.04316
2!
P (3 claims) = 1 - 0.70469 - 0.24664 - 0.04316 = 0.00551

So the expected number of claims paid per policy is:

0 ¥ 0.70469 + 1 ¥ 0.24664 + 2 ¥ 0.04316 + 3 ¥ 0.00551 = 0.34949

Equating the expected annual income and outgo, then gives:

0.66895P = 0.34949 ¥ £2,500 = £873.73

873.73
fi P= = £1,306.12
0.66895

So the full annual premium is £1,306.12 and a policyholder on the 40% level
will pay:

60% ¥ £1,306.12 = £783.67

Page 168 © IFE: 2019 Examinations

 Batch3

(ii) Transition graph

The transition graph for the protected NCD system looks like this (where pi
is the probability of making exactly i claims during the year).

p0 p0 p0 p0 + p1

0% 15% 30% 40%

1 – p0 1 – p0 – p1 1 – p0 – p1 1 – p0 – p1

p1 p1

(iii) Premium with protected NCD

The long-term probabilities can be found by finding the stationary

distribution, ie the set of probabilities p i that satisfy the matrix equation
p = p P with the additional condition Âpi = 1.
i

The probability transition matrix for the protected process is:

0% 15% 30% 40%

0% Ê 1 - p0 p0 0 0 ˆ
P = 15% Á1 - p0 - p1 p1 p0 0 ˜
Á ˜
30% Á 0 1 - p0 - p1 p1 p0 ˜
Á ˜
40% Ë 0 0 1 - p0 - p1 p0 + p1¯

Here p0 = e -0.35 = 0.70469 and p1 = 0.35e -0.35 = 0.24664 . So:

0% 15% 30% 40%

0% Ê 0.29531 0.70469 0 0 ˆ
P = 15% Á 0.04867 0.24664 0.70469 0 ˜
Á ˜
30% Á 0 0.04867 0.24664 0.70469˜
Á ˜
40% Ë 0 0 0.04867 0.95133¯

© IFE: 2019 Examinations Page 169

 Batch3

Written out in full, this matrix equation is:

p0 = 0.29531p 0 + 0.04867p 15 (1)

p 15 = 0.70469p 0 + 0.24664p 15 + 0.04867p 30 (2)
p 30 = 0.70469p 15 + 0.24664p 30 + 0.04867p 40 (3)
p 40 = 0.70469p 30 + 0.95133p 40 (4)

Rearranging Equation (1) gives:

(1 - 0.29531)p 0
p 15 = = 14.479p 0
0.04867

Equation (2) then gives:

(1 - 0.24664)p 15 - 0.70469p 0
p 30 =
0.04867
0.75336(14.479p 0 ) - 0.70469p 0
=
0.04867
= 209.63p 0

Equation (4) then gives:

0.70469p 30 0.70469(209.63p 0 )
p 40 = = = 3,035.1p 0
1 - 0.95133 1 - 0.95133

So the vector of stationary probabilities has the form:

(p 0 , p 15 , p 30 , p 40 ) = (p 0 ,14.479p 0 ,209.63p 0 ,3035.1p 0 )

Since these probabilities must add up to 1, this tells us that:

p 0 + 14.479p 0 + 209.63p 0 + 3,035.1p 0 = 1

fi 3,260.2p 0 = 1
1
fi (p 0 , p 15 , p 30 , p 40 ) =
3,260.2
(1, 14.479, 209.63, 3035.1)
= (0.00031, 0.00444, 0.06430, 0.93096)

Page 170 © IFE: 2019 Examinations

 Batch3

So the expected annual premium paid by all policyholders is now:

(0.00031 + 0.00444 ¥ 0.85 + 0.06430 ¥ 0.7 + 0.93096 ¥ 0.6)P

= 0.60766P

Assuming that the expected claim payments remain the same, this leads to
the new premium equation:

0.60766P = £873.73

873.73
fi P= = £1, 437.85
0.60766

So the full premium is £1,437.85 and a policyholder on the 40% level will
pay:

60% ¥ £1, 437.85 = £862.71

(iv) Advantages and disadvantages

The protected system may encourage policyholders to stay with the same
insurer, as they start to build up a no claims history, which they do not want
to lose.

It should also discourage policyholders from claiming, which will reduce the
insurer’s costs.

However, it could also lead to bad publicity if some policyholders feel that
they have lost their NCD protection through no fault of their own (eg because
of an accident involving an uninsured driver).

© IFE: 2019 Examinations Page 171

 Batch3

FACTSHEET

This factsheet summarises the main methods, formulae and information

required for tackling questions on the topics in this booklet.

Markov chains

A Markov process with a discrete time set and discrete state space is called
a Markov chain.

Using the Markov property and the law of total probability, the Chapman-
Kolmogorov equation for a time-inhomogeneous process is:

pij (s, t )   pik (s, u )pkj (u, t )

k

Here s  u  t . Choose whatever value of u is convenient for the

calculation.

Time-homogeneous Markov chains

Here the transition probabilities depend only on the time lag t  s and not on
the absolute values of s and t .

The one-step transition probability matrix is:

P   pij (m, m  1)   pij 

The n -step transition probability matrix is:

P n   pij (m, m  n )   pij( n ) 

 

The Chapman-Kolmogorov equation corresponds to the matrix equation:

P t  s  P u  s P t u

Page 172 © IFE: 2019 Examinations

 Batch3

Stationary distributions

A solution,  to the equations:

   P with  i  0 and the normalisation condition i 1

is called a stationary distribution.

Existence of stationary distributions

If the state space is finite (has a finite number of states), then:

 there is at least one stationary distribution
 but the process may not conform to this distribution in the long term.

Irreducible means that each state can ultimately be reached starting from
any other, ie there exists at least one n  0 such that pij ( n )  0 for all states
i and j .

You can check this by ‘following paths’ on a transition graph.

If the chain is finite and irreducible, then:

 there is a unique stationary distribution
 but the process may not conform to this distribution in the long term.

A periodic chain with period d , is one where return to a given state is only
possible in a number of steps that is a multiple of d . NB: Return does not
need to be possible for all multiples of d .

Check this by finding the highest common factor of ‘return times’ for each
state. States in an irreducible chain have the same periodicity, so you only
need to check one state in this case.

If d  1 the chain is aperiodic.

© IFE: 2019 Examinations Page 173

 Batch3

If the chain is finite, irreducible and aperiodic, then:

 there is a unique stationary distribution,  j

 the process will conform to this distribution in the long term,

ie lim pij ( n )   j
n 

Simple random walk

n
This is defined as X n   Yj where Y j are independent with the common
j 1
probability distribution:

P Y j  1  p, P Y j   1  1  p

The n -step transition probabilities are:

 n  u n u
  p 1  p  if 0  n  j  i  2n and n  j  i is even
pij( n )   u 

0 otherwise

Simple random walk with boundaries

Here the state space is 0, 1, 2,  , b and boundary conditions have to be

specified at 0 and b . These will depend on the interpretation given to the
chain. Commonly used boundary conditions at 0 would include:

Reflecting boundary: P  X n 1  1 X n  0   1
 

Absorbing boundary: P  X n 1  0 X n  0   1
 

P  X  0X n  0    ,
  n 1 
Mixed boundary: 
 
P X n 1  1X n  0  1  
  

Page 174 © IFE: 2019 Examinations

 Batch3

A model of accident proneness

For a given driver, any period j is either accident free ( Y j  0 ) or gives rise
to exactly one accident ( Y j  1 ).

The probability of an accident in the next period is estimated using the

driver’s past record as follows (all variables y j are either 0 or 1):

f  y1  y 2  ...  y n 
P Yn 1  1 Y1  y1,Y2  y 2, ... , Yn  y n  
  g n 

where f , g are two given increasing functions satisfying 0  f  m   g  m  .

The complementary probability is:

f  y1  y 2  ...  y n 
P Yn 1  0Y1  y1,Y2  y 2, ... , Yn  y n   1 
  g n 

Y1, Y2, ... , Yn , ... does not have the Markov property, but the cumulative
number of accidents experienced by the driver:

n
Xn   Yj
j 1

is a Markov chain with state space S  0, 1, 2, ... .

Estimating transition probabilities

nij is the observed number of transitions from state i to j

ni is the observed number of transitions from state i .

nij
The best estimate of pij is pˆ ij  .
ni

The conditional distribution of Nij given Ni is Binomial Ni , pij .  

This can be used to construct confidence intervals for pij .

© IFE: 2019 Examinations Page 175

 Batch3

Assessing the fit

If nijk is the number of times t 1  t  N  2  such that xt  i , xt 1  j and

xt  2  k and the Markov property holds we expect:


nijk to be Binomial Nij , p jk 
The chi-square goodness-of-fit test is based on the test statistic:

 nijk  nij pˆ jk 
2
2
m  
i j k nij pˆ jk

Simulation

A time-homogeneous Markov chain can be simulated by using the

conditional distributions of X n (ie the entries given in the rows of the
transition matrix).

A ‘look-up’ table can be constructed for each conditional distribution. Use

these in sequence to generate a sample path.

Two-state Markov model

 key assumptions
Markov property

– h qx +t = h m x + t + o(h ) as h Æ 0

constant transition rate over each year

 likelihood function / fitting the model

L( m ) μ e - mv i m di likelihood for one individual

- mv d
L( m ) μ e m likelihood for whole group

find the MLE of m to fit the model

Page 176 © IFE: 2019 Examinations

 Batch3

 key formulae / results

= exp Ê - Ú m x + s ds ˆ
t
t px probability of staying alive
Ë 0 ¯
D d
m = or mˆ = formulae for MLE of m
V v
Ê m ˆ
m ~ N Á m, asymptotic distribution of MLE of m
Ë E [V ] ˜¯

E [Di - mVi ] = 0 , var[Di - mVi ] = E [Di ] exact results

 exam questions (quite rare)

reproducing one of the algebraic steps

 specific points
this is a ‘warm-up’ model for general Markov jump processes

The steps in deriving the maximum likelihood estimate of m under the

two-state Markov model are:
 Define the notation.
 Start from the 3 assumptions.
 Derive a formula for t px .

 Write down f (d i , v i ) , the joint distribution of (Di ,Vi ) for one individual.
This uses the formula for t px .

 Deduce the form of the likelihood function for the group.

 Find the formula for the MLE of m .

The mean and variance of Di - mVi are then worked out. To do the mean:

 Write down a total probability equation, involving an integral.

 Differentiate the equation with respect to m .

 Identify the resulting equation as E [Di - mVi ] = 0 .

The Core Reading mentions the variance as well, but it is unlikely you’d be
asked to prove this. (If you were, you’d be given appropriate guidance in the
question.)

© IFE: 2019 Examinations Page 177

 Batch3

Alternatively, since this estimation is carried out using maximum likelihood,

we could use the asymptotic properties of maximum likelihood estimators to
say that asymptotically:
 m is normally distributed

 m is an unbiased estimator of m , ie E ( m ) = m

 var ( m ) is given by the Cramér-Rao lower bound (CRLB).

The formula for the CRLB is given on page 23 of the Tables.

Poisson model

 parametric
– parameter is m (combined with E c as mE c )

 key assumptions

Dx ~ Poisson ( mE xc ) distribution of number of deaths

 likelihood function

c
e - mE x ( m E xc )d
L( m ) = P [D = d ] =
d!

 estimating the parameters for the model

D D
m = = formula for MLE of m
V E xc
 
E [ m ] = m , var[  ]   properties of MLE of m
V E xc

 exam questions
– numerical calculations
– writing down the likelihood function
– calculating the MLE

– calculating E c using the exact method

Page 178 © IFE: 2019 Examinations

 Batch3

 specific points

– central exposed-to-risk ( E c ) corresponds to the waiting time V in

the two-state model
– the numerical values of the MLEs are the same as for the
two-state model

© IFE: 2019 Examinations Page 179

 Batch3

NOTES

Page 180 © IFE: 2019 Examinations

 Batch3

NOTES

© IFE: 2019 Examinations Page 181

 Batch3

NOTES

Page 182 © IFE: 2019 Examinations

 Batch3

NOTES

© IFE: 2019 Examinations Page 183

 Batch3

NOTES

Page 184 © IFE: 2019 Examinations