Batch3

Subject CS2
Revision Notes
For the 2019 exams

Stochastic process models

Booklet 1

Covering

Chapter 1 Stochastic processes

The Actuarial Education Company

Batch3
 Batch3

CONTENTS

Contents Page

Links to the Course Notes and Syllabus 2

Overview 3
Core Reading 4
Past Exam Questions 14
Solutions to Past Exam Questions 24
Factsheet 41

Copyright agreement

All of this material is copyright. The copyright belongs to Institute and

Faculty Education Ltd, a subsidiary of the Institute and Faculty of Actuaries.
The material is sold to you for your own exclusive use. You may not hire
out, lend, give, sell, transmit electronically, store electronically or photocopy
any part of it. You must take care of your material to ensure it is not used or
copied by anyone at any time.

Legal action will be taken if these terms are infringed. In addition, we may
seek to take disciplinary action through the profession or through your
employer.

These conditions remain in force after you have finished using the course.

© IFE: 2019 Examinations Page 1

 Batch3

LINKS TO THE COURSE NOTES AND SYLLABUS

Material covered in this booklet

Chapter 1 Stochastic processes

These chapter numbers refer to the 2019 edition of the ActEd course notes.

Syllabus objectives covered in this booklet

The numbering of the syllabus items is the same as that used by the Institute
and Faculty of Actuaries.

3.1 Describe and classify stochastic processes.

3.1.1 Define in general terms a stochastic process and in

particular a counting process.

3.1.2 Classify a stochastic process according to whether it:

(a) operates in continuous or discrete time

(b) has a continuous or a discrete state space

(c) is a mixed type

and give examples of each type of process.

3.1.3 Describe possible applications of mixed processes.

3.1.4 Explain what is meant by the Markov property in the

context of a stochastic process and in terms of filtrations.

Page 2 © IFE: 2019 Examinations

 Batch3

OVERVIEW

This booklet covers Syllabus objective 3.1, which relates to the definition and
properties of stochastic processes.

A stochastic process is a mathematical model of a quantity or state that

varies randomly over time. It is a collection of random variables, X t , one for
each value of t in a set, J , known as the time set. The time set is the set of
points in time at which the value of the process is recorded. This may be
discrete or continuous.

The set of values that the random variables X t can take is called the state
space of the process, and is denoted by S . If the X t are discrete random
variables, then the state space is discrete. If the X t are continuous random
variables, then the state space is continuous.

Labelling the time set and state space as discrete or continuous leads to a
four-way classification of stochastic processes. This has been examined
many times in the past (in Subject CT4).

There are several other items of terminology associated with stochastic

processes, including the concepts of stationarity and the Markov property.
These properties are used repeatedly throughout the course (in the chapters
on Markov chains, Markov jump processes and time series).

Some processes are very complicated, eg not stationary, but have an

incremental process that has a very simple structure, eg the random
variables are independent and identically distributed. So studying
incremental processes is important. This has added importance because
there is also a link between the properties of an incremental process and
whether the process itself satisfies the Markov property.

© IFE: 2019 Examinations Page 3

 Batch3

CORE READING

All of the Core Reading for the topics covered in this booklet is contained in
this section.

We have inserted paragraph numbers in some places, such as 1, 2, 3 …, to

help break up the text. These numbers do not form part of the Core
Reading.

The text given in Arial Bold font is Core Reading.

The text given in Arial Bold Italic font is additional Core Reading that is not
directly related to the topic being discussed.
____________

Chapter 1 – Stochastic processes

Types of stochastic processes

1 A stochastic process is a model for a time-dependent random

phenomenon. So, just as a single random variable describes a static
random phenomenon, a stochastic process is a collection of random
variables X t , one for each time t in some set J .
____________

2 The process is denoted  X t : t  J  .

____________

3 The set of values that the random variables X t are capable of taking is
called the state space of the process, S .
____________

4 The first choice that one faces when selecting a stochastic process to
model a real life situation is that of the nature (discrete or continuous)
of the time set J and of the state space S .
____________

Page 4 © IFE: 2019 Examinations

 Batch3

Discrete state space with discrete time changes

5 A motor insurance company reviews the status of its customers yearly.

Three levels of discount are possible (0, 25%, 40%) depending on the
accident record of the driver. In this case the appropriate state space
is S = {0, 25, 40} and the time set is J = {0, 1, 2, } where each interval
represents a year. This problem is studied in Booklet 2.
____________

Discrete state space with continuous time changes

6 A life insurance company classifies its policyholders as Healthy, Sick

or Dead. Hence the state space S  H , S , D . As for the time set, it is
natural to take J  [0,  ) as illness or death can occur at any time. On
the other hand, it may be sufficient to count time in units of days, thus
using J  0, 1, 2,... . This problem is studied in Booklet 3.
____________

Continuous state space

7 Claims of unpredictable amounts reach an insurance company at

unpredictable times; the company needs to forecast the cumulative
claims over [0, t ] in order to assess the risk that it might not be able to
meet its liabilities. It is standard practice to use [0,  ) both for S
and J in this problem. However, other choices are possible: claims
come in units of a penny and do not really form a continuum. Similarly
the intra-day arrival time of a claim, that is the time at which it arrives
on a particular day, is of little significance, so that 0,1,2,... is a
possible choice for J and/or S .
____________

© IFE: 2019 Examinations Page 5

 Batch3

Processes of mixed type

8 Just because a stochastic process operates in continuous time does

not mean that it cannot also change value at predetermined discrete
instants; such processes are said to be of mixed type. As an example,
consider a pension scheme in which members have the option to retire
on any birthday between ages 60 and 65. The number of people
electing to take retirement at each year of age between 60 and 65
cannot be predicted exactly, nor can the time and number of deaths
among active members. Hence the number of contributors to the
pension scheme can be modelled as a stochastic process of mixed
type with state space S = {1, 2, 3, } and time set or domain J = [0, • ) .
Decrements of random amounts will occur at fixed dates due to
retirement as well as at random dates due to death.
____________

As a rule, one can say that continuous time and continuous state
space stochastic processes, although conceptually more difficult than
discrete ones, are also ultimately more flexible (in the same way as it is
easier to calculate an integral than to sum an infinite series).

It is important to be able to conceptualise the nature of the state space

of any process which is to be analysed, and to establish whether it is
most usefully modelled using a discrete, a continuous, or a mixed time
domain. Usually the choice of state space will be clear from the nature
of the process being studied (as, for example, with the Healthy-Sick-
Dead model), but whether a continuous or discrete time set is used will
often depend on the specific aspects of the process which are of
interest, and upon practical issues like the time points for which data
are available.
____________

9 Counting processes

A counting process is a stochastic process, X , in discrete or

continuous time, whose state space S is the collection of natural
numbers {0, 1, 2, …}, with the property that X (t ) is a non-decreasing
function of t .
____________

Page 6 © IFE: 2019 Examinations

 Batch3

Defining a stochastic process

10 Sample paths

Having selected a time set and a state space, it remains to define the
process  X t : t  J  itself. This amounts to specifying the joint
distribution of X t1 , X t2 , ..., X tn for all t1, t 2 , ..., t n in J and all
integers n . This appears to be a formidable task; in practice this is
almost invariably done indirectly, through some simple intermediary
process.
____________

11 A joint realisation of the random variables X t for all t in J is called a

sample path of the process; this is a function from J to S .
____________

The properties of the sample paths of the process must match those
observed in real life (at least in a statistical sense). If this is the case,
the model is regarded as successful and can be used for prediction
purposes. It is essential that at least the broad features of the real life
problem be reproduced by the model; the most important of these are
discussed in the next subsections.
____________

Stationarity

12 A stochastic process is said to be stationary, or strictly stationary, if

the joint distributions of X t1 , X t2 , ..., X tn and X k +t1 , X k +t2 , ..., X k +tn are
identical for all t1, t2 , ..., t n and k + t1, k + t2 , ... , k + t n in J and all
integers n . This means that the statistical properties of the process
remain unchanged as time elapses.
____________

Recall the example with the three states Healthy, Sick and Dead. One
would certainly not use a strictly stationary process in this situation, as
the probability of being alive in 10 years’ time should depend on the
age of the individual and hence will vary over time.
____________

© IFE: 2019 Examinations Page 7

 Batch3

13 Strict stationarity is a stringent requirement which may be difficult to

test fully in real life.
____________

14 For this reason another condition, known as weak stationarity is also in

use. This requires that the mean of the process m  t   E  X t  is
constant and that the covariance of the process:


cov  X s , X t   E  X s  m  s 
   X t  m t  

depends only on the time difference t  s .

____________

Increments

15 An increment of a process is the amount by which its value changes

over a period of time, eg X t u  X t (where u  0 ).
____________

16 The increments of a process often have simpler properties than the

process itself.
____________

17 Example

Let St denote the price of one share of a specific stock. It might be

considered reasonable to assume that the distribution of the return
St u
over a period of duration u, , depends on u but not on t.
St
Accordingly the log-price process X t  log St would have stationary
increments:

St u
X t u  X t  log
St

even though X t itself is unlikely to be stationary.

____________

Page 8 © IFE: 2019 Examinations

 Batch3

18 A process X t is said to have independent increments if for all t and

every u > 0 the increment X t u  X t is independent of all the past of
the process  X s :0  s  t  .
____________

In the last example it is a form of the efficient market hypothesis to

assume that X t = log St has independent increments.

Many processes are defined through their increments.

____________

The Markov property

19 A major simplification occurs if the future development of a process

can be predicted from its present state alone, without any reference to
its past history. Stated precisely this reads:

P ÎÈ X t Œ A | X s1 = x1, X s2 = x 2 ,  , X sn = x n , X s = x ˚˘ = P ÈÎ X t Œ A | X s = x ˘˚

for all times s1 < s <  < sn < s < t , all states x1, x 2 ,  , x n and x in S
and all subsets A of S . This is called the Markov property.
____________

It can be argued that the model with states healthy, sick and dead can
be a Markov process. If there is full recovery from the sick state to the
healthy state, past sickness history should have no effect on future
health prospects.
____________

20 A process with independent increments has the Markov property.

Proof

P ÈÎ X t Œ A| X s1 = x1, X s2 = x 2 ,  , X sn = x n , X s = x ˘˚

= P ÈÎ X t - X s + x Œ A| X s1 = x1, X s2 = x 2 ,  , X sn = x n , X s = x ˘˚
= P ÈÎ X t - X s + x Œ A| X s = x ˘˚
= P ÈÎ X t Œ A| X s = x ˘˚
____________

© IFE: 2019 Examinations Page 9

 Batch3

Filtrations

21 The following structures underlie any stochastic process X t :

 a sample space W : each outcome w in W determines a sample

path X t (w )

 a set of events F : this is a collection of events, by which is meant

subsets of W , to which a probability can be attached

 for each time t , a smaller collection of events Ft à F : this is the

set of those events whose truth or otherwise are known at time t.
In other words an event A is in Ft if it depends only on X s ,
0<s£t .
____________

22 As t increases, so does Ft : Ft  Fu , t  u . Taken collectively, the

family  Ft t 0 is known as the (natural) filtration associated with the
stochastic process X t , t  0 ; it describes the information gained by
observing the process or the internal history of X t up to time t .
____________

23 The process X t can be said to have the Markov property if:

P  X t  x Fs   P  X t  x X s 

for all t  s  0 .
____________

When a Markov process has a discrete state space and a discrete time
set it is called a Markov chain; Markov chains are studied in Booklet 2.
When the state space is discrete but the time set is continuous, one
uses the term Markov jump process; Markov jump processes are
studied in Booklets 2 and 3.

Using the preliminaries in this section we can now show by a series of

examples how to define a stochastic process.
____________

Page 10 © IFE: 2019 Examinations

 Batch3

Examples

White noise

24 Consider a discrete-time stochastic process consisting of a sequence

of independent random variables X 1,..., X n ,... .

The Markov property holds in a trivial way.

The process is stationary if and only if all the random variables X n

have the same distribution. Such sequences of independent identically
distributed (IID for short) random variables are sometimes described as
a discrete-time white noise.
____________

White noise processes are normally defined as having a mean of zero

at all times: that is m (t ) = E ( X t ) = 0 for all values of t . They may be
defined in either discrete or continuous time. In a white noise process
with a mean of zero, the covariance of the process:

cov(s , t ) = E ÈÎ( X s - m(s ))( X t - m(t ))˘˚

is zero for s π t . The main use of white noise processes is as a

starting point to construct more elaborate processes below.
____________

General random walk

25 Start with a sequence of IID random variables Y1,...,Y j ,... as above and
n
define the process X n   Yj with initial condition X 0  0 .
j 1

This is a process with stationary independent increments, and thus a

discrete-time Markov process. It is known as a general random walk.
The process is not even weakly stationary, as its mean and variance
are both proportional to n .
____________

© IFE: 2019 Examinations Page 11

 Batch3

26 In the special case where the steps Y j of the walk take only the values
1 and 1 , the process is known as a simple random walk.
____________

Poisson process

27 A Poisson process with rate  is a continuous-time integer-valued

process Nt , t  0 with the following properties:

(i) N0  0

(ii) Nt has independent increments

(iii) Nt has Poisson distributed stationary increments:

   t  s   e    t  s 
n
P Nt  Ns  n    , s  t , n  0, 1, ...
n!
____________

28 This is a Markov jump process with state space S  0,1,2,... . It is not

stationary: as in the case for the random walk, both the mean and
variance increase linearly with time.
____________

29 This process is of fundamental importance when counting the

cumulative number of occurrences of some event over 0, t  ,
irrespective of the nature of the event (car accident, claim to insurance
company, arrival of customer at a service point). A detailed study of
this process and its extensions is one of the subjects of Booklet 3.
____________

Page 12 © IFE: 2019 Examinations

 Batch3

Compound Poisson process

30 Start with a Poisson process Nt , t  0 and a sequence of IID random

variables Y j , j  1 . A compound Poisson process is defined by:

Nt
Xt   Yj ,t  0 (1.1)
j 1
____________

31 This process has independent increments and thus the Markov

property holds. It serves as a model for the cumulative claim amount
reaching an insurance company during 0, t  : Nt is the total number of
claims over the period and Z j is the amount of the j-th claim.
____________

The basic problem of classical risk theory consists of estimating the

probability of ruin:

 u  = P u  ct  X t  0 for some t  0 

for a given initial capital u, premium rate c, X t defined as in (1.1), and

some fixed distribution of the claims.

© IFE: 2019 Examinations Page 13

 Batch3

PAST EXAM QUESTIONS

This section contains all the relevant exam questions from 2008 to 2017 that
are related to the topics covered in this booklet.

Solutions are given after the questions. These give enough information for
you to check your answer, including working, and also show you what an
outline examination answer should look like. Further information may be
available in the Examiners’ Report, ASET or Course Notes. (ASET can be
ordered from ActEd.)

We first provide you with a cross-reference grid that indicates the main
subject areas of each exam question. You can use this, if you wish, to
select the questions that relate just to those aspects of the topic that you
may be particularly interested in reviewing.

Alternatively, you can choose to ignore the grid, and attempt each question
without having any clues as to its content.

Page 14 © IFE: 2019 Examinations

 9
8
7
6
5
4
3
2
1

18
17
16
15
14
13
12
11
10
Question
Four-way










classification



Stationarity


Increments
Cross-reference grid

© IFE: 2019 Examinations







 Poisson process
Compound Poisson










process





Counting process
Batch3









Random walk





Markov chain
Markov jump



process


White noise

Question attempted

Page 15
 Batch3

1 Subject CT4 April 2008 Question 3

(i) Define the following stochastic processes:

(a) Poisson process

(b) compound Poisson process. [4]

(ii) Identify the circumstances in which a compound Poisson process is also

a Poisson process. [1]
[Total 5]

2 Subject CT4 September 2008 Question 2

The classification of stochastic models according to:

 discrete or continuous time variable

 discrete or continuous state space

gives rise to a four-way classification.

Give four examples, one of each type, of stochastic models which may be
used to model observed processes, and suggest a practical problem to
which each model may be applied. [4]

3 Subject CT4 April 2009 Question 7

(i) Explain how the classification of stochastic processes according to the

nature of their state space and time space leads to a four way
classification. [2]

(ii) For each of the four types of process:

(a) give an example of a statistical model

(b) write down a problem of relevance to the operation of:

 a food retailer

 a general insurance company. [6]

[Total 8]

Page 16 © IFE: 2019 Examinations

 Batch3

4 Subject CT4 April 2010 Question 3

For each of the following processes:

 counting process
 general random walk
 compound Poisson process
 Poisson process
 Markov jump chain

(a) State whether the state space is discrete, continuous or can be either.

(b) State whether the time set is discrete, continuous, or can be either. [5]

5 Subject CT4 April 2011 Question 7

(i) Define a counting process. [2]

(ii) For each of the following processes:

 simple random walk
 compound Poisson
 Markov chain

(a) state whether each of the state space and the time set is discrete,
continuous or can be either

(b) give an example of an application which may be useful to a

shopkeeper selling dried fruit and nuts loose. [6]
[Total 8]

6 Subject CT4 September 2011 Question 3

Describe how a strictly stationary stochastic process differs from a weakly

stationary stochastic process. [3]

© IFE: 2019 Examinations Page 17

 Batch3

7 Subject CT4 April 2012 Question 1

(i) Define a general random walk. [1]

(ii) State the conditions under which a general random walk would become
a simple random walk. [1]
[Total 2]

8 Subject CT4 April 2013 Question 3

For both of the following sets of four stochastic processes, place each
process in a separate cell of the following table, so that each cell correctly
describes the state space and the time space of the process placed in it.
Within each set, all four processes should be placed in the table:

Time space
Discrete Continuous
State Space

Discrete

Continuous

(a) General random walk, compound Poisson process, counting process,

Poisson process.

(b) Simple random walk, compound Poisson process, counting process,

white noise. [5]

Page 18 © IFE: 2019 Examinations

 Batch3

9 Subject CT4 September 2013 Question 3

(i) Define a Poisson process. [2]

A bus route in a large town has one bus scheduled every 15 minutes. Traffic
conditions in the town are such that the arrival times of buses at a particular
bus stop may be assumed to follow a Poisson process.

Mr Bean arrives at the bus stop at 12 midday to find no bus at the stop. He
intends to get on the first bus to arrive.

(ii) Determine the probability that the first bus will not have arrived by
1:00pm the same day. [2]

The first bus arrived at 1:10pm but was full, so Mr Bean was unable to
board it.

(iii) Explain how much longer Mr Bean can expect to wait for the second bus
to arrive. [1]

(iv) Calculate the probability that at least two more buses will arrive between
1:10pm and 1:20pm. [2]
[Total 7]

10 Subject CT4 September 2014 Question 1

For each of the following processes:

 counting process
 simple random walk
 compound Poisson process
 Markov jump process

(i) State whether the state space is discrete, continuous or can be either.
[2]

(ii) State whether the time set is discrete, continuous or can be either. [2]
[Total 4]

© IFE: 2019 Examinations Page 19

 Batch3

11 Subject CT4 September 2014 Question 7

(i) Define a Poisson process. [2]

(ii) Prove the memoryless property of the exponential distribution. [2]

Suppose there are three independent exponential distributions:

X with parameter x
Y with parameter y
Z with parameter z

(iii) (a) Demonstrate that min( X ,Y , Z ) is also an exponential distribution.

(b) Give the parameter of this exponential distribution. [2]

The arrivals of different types of vehicles at a toll bridge are assumed to

follow Poisson processes whereby:

Type of vehicle Rate

Motorcycle 2 per minute
Car 5 per minute
Goods vehicle 1.5 per minute

The toll for a motorcycle is £1, for a car £2 and for a goods vehicle £5.

(iv) State the name of the stochastic process that describes the total value
of tolls collected. [1]

(v) Calculate the expected value of tolls collected per hour. [1]

On the advice of a structural engineer, no more than two goods vehicles are
allowed across the bridge in any given minute. If more than two goods
vehicles arrive then some goods vehicles have to wait to go across.

(vi) Calculate the probability that more than two goods vehicles arrive in any
given minute. [2]

(vii) Calculate the probability that exactly £4 in tolls is collected in a given

minute. [4]
[Total 14]

Page 20 © IFE: 2019 Examinations

 Batch3

12 Subject CT4 April 2015 Question 1

For a simple random walk:

(i) Define the process. [2]

(ii) Write down the nature of the state space and time space in which it
operates. [1]

(iii) Describe an example of a practical application of the process. [1]

[Total 4]

13 Subject CT4 September 2015 Question 5 (part)

The following diagrams illustrate sample paths for four stochastic processes.

(ii) Identify which sample path is most likely to correspond to a:

● discrete time, discrete state process
● continuous time, discrete state process
● discrete time, continuous state process
● continuous time, continuous state process. [2]

© IFE: 2019 Examinations Page 21

 Batch3

14 Subject CT4 April 2016 Question 5

(i) Define the following types of stochastic process:

(a) a Poisson process
(b) a compound Poisson process [3]

Consider the modelling of the following situations:

A the number of claims for motorcycle accidents received by an insurer’s

telephone claim line

B the number of breakfast bagels sold by a New York bagel bar

C the number of breakdowns of freezers in a large supermarket

D the cost of wasted food caused by breakdowns of freezers in a large

supermarket.

(ii) Comment on which of the following stochastic processes will be most

suitable for modelling each of the four situations above:
 time-homogeneous Poisson process
 time-inhomogeneous Poisson process
 time-homogeneous compound Poisson process
 time-inhomogeneous compound Poisson process [6]
[Total 9]

15 Subject CT4 April 2017 Question 2

(i) Define an increment of a process. [1]

The rate of mortality in a certain population at ages over exact age 30 years,
h(30 + u ) , is described by the process:

h(30 + u ) = B(1 + g ) u u ≥ 0

where B and g are constants.

(ii) Show that the increments of the process log[h(30 + u )] are stationary.
[3]
[Total 4]

Page 22 © IFE: 2019 Examinations

 Batch3

16 Subject CT4 April 2017 Question 4

(i) Describe how a classification based on the nature of the state and time
spaces of stochastic processes leads to a four-way categorisation. [2]

(ii) List FOUR stochastic processes, one for each of the four categories in
your answer to part (i). [2]
[Total 4]

17 Subject CT4 September 2017 Question 2

For each of the following processes:

 general random walk
 Markov jump process
 compound Poisson process
 Markov chain

(a) State whether the state space is discrete, continuous or can be either.

(b) State whether the time set is discrete, continuous, or can be either. [4]

18 Subject CT4 September 2017 Question 3

Calls arrive on Fred’s desk phone according to a Poisson process with

parameter 3, with time measured in hours.

(i) Write down the expected number of phone calls Fred receives each
hour. [1]

Fred has not received a phone call for 15 minutes.

(ii) Give the expected time until Fred next receives a phone call. [1]

Fred goes into a meeting for half an hour.

(iii) Determine the probability that Fred has NOT missed a call when he
returns to his desk. [1]

The average length of a call to Fred is 7 minutes.

(iv) Determine the probability that if a caller phones Fred the line will be
engaged, assuming that Fred is at his desk to receive calls. [2]
[Total 5]

© IFE: 2019 Examinations Page 23

 Batch3

SOLUTIONS TO PAST EXAM QUESTIONS

The solutions presented here are just outline solutions for you to use to
check your answers. See ASET for full solutions.

1 Subject CT4 April 2008 Question 3

(i)(a) Poisson process

A Poisson process with rate l is a continuous-time, integer-valued process

{Nt : t ≥ 0} with the following properties:
1. N0 = 0

2. {Nt : t ≥ 0} has independent increments

3. {Nt : t ≥ 0} has Poisson distributed stationary increments, ie:
n
e - l (t - s ) ÈÎ l (t - s )˘˚
P (Nt - Ns = n ) =
n!

for 0 £ s < t and n = 0,1, 2,... .

(i)(b) Compound Poisson process

A compound Poisson process is a process of the form:

X t = Y1 + Y2 +  + YNt

{
where {Nt : t ≥ 0} is a Poisson process and Y j : j = 1, 2, 3,... is a sequence}
of independent and identically distributed random variables.

We define X t = 0 when Nt = 0 .

Page 24 © IFE: 2019 Examinations

 Batch3

(ii) When is a compound Poisson process also a Poisson process?

If Y j = 1 for all j , then:

X t = Y1 + Y2 +  + YNt = Nt

So X t is also a Poisson process.

2 Subject CT4 September 2008 Question 2

Discrete state space / discrete time

Example: Markov chain process. Process moves between discrete states at

fixed time steps, and follows the Markov property.

Practical problem: modelling no-claims discount levels for motor insurance

premiums. Policyholders can change discount level at each policy renewal,
according to rules defined in terms of how many claims they made over the
preceding year.

Discrete state space / continuous time

Example: Markov jump process. Process moves between discrete states at

any moment in continuous time, and follows the Markov property.

Practical problem: modelling sickness experience for the purpose of pricing

a health insurance contract. Lives are assumed to transfer between discrete
states of healthy, sick and dead (and sometimes with several categories of
‘sick’), at any moment in continuous time.

Continuous state space / discrete time

Example: general random walk process. The value of the process changes
at discrete time steps, by the addition of a random amount that can take any
value over a continuous range. The random amounts would be independent
and identically distributed (IID).

Practical problem: the FTSE100 index could be modelled in, say, daily time
steps, by taking the previous day’s value and adding some random amount,
which is an IID continuous random variable (and includes negative values).
(To be more realistic, the resulting answer could be rounded to the nearer
0.1, so as to have the same degree of accuracy as the index itself.)

© IFE: 2019 Examinations Page 25

 Batch3

Continuous state space / continuous time

Example: compound Poisson process. The value of the process

accumulates by the addition of random amounts, on a continuous state
space involving only positive values, and which can occur at any point in
continuous time.

Practical problem: could be used to model the total claim amounts paid over
a period of time on an insurance portfolio, where each addition represents
the amount paid out on a single claim, claims can occur at any point in
continuous time, and the amounts take a random, continuously distributed
positive value.

3 Subject CT4 April 2009 Question 7

(i) Four-way classification of stochastic processes

Stochastic processes can be classified according to their time set and their
state space. The time set can be discrete or continuous and the state space
can be discrete or continuous. So we obtain a four-way classification as
illustrated below:

Time set
Discrete Continuous
Discrete
State space
Continuous

(ii)(a) Examples of statistical models

Discrete state space, discrete time set

Examples include Markov chains, simple random walks, counting processes

with discrete time sets and white noise processes with discrete state spaces.

Discrete state space, continuous time set

Examples include Markov jump processes (of which the Poisson process is
a special case) and counting processes with continuous time sets (and,
again, the Poisson process is a special case of this).

Continuous state space, discrete time set

Examples include general random walks and time series (eg white noise with
a continuous state space).

Page 26 © IFE: 2019 Examinations

 Batch3

Continuous state space, continuous time set

Examples include Brownian motion, diffusion processes and compound

Poisson processes with continuous state spaces.

(ii)(b) Examples of relevant problems

Food retailer

Discrete state space, discrete time set

Examples include the number of customers each day, the number of items
that become out of date at the end of each day and the number of products
that are out of stock at the end of each day.

Discrete state space, continuous time set

Examples include the number of customers up to time t , the number of

customers in the check-out queue at time t and the number of items sold up
to time t .

Continuous state space, discrete time set

Examples include the value of the goods sold each day, the value of the
goods in stock at the end of each day and the value of goods that become
out of date at the end of each day.

Continuous state space, continuous time set

Examples include the total value of goods sold up to time t and the
company’s share price.

General insurer

Discrete state space, discrete time set

Examples include a no claims discount system, the number of policies sold

each month and the number of claims received each month.

Discrete state space, continuous time set

Examples include the number of policies sold up to time t and the number
of claims received up to time t .

© IFE: 2019 Examinations Page 27

 Batch3

Continuous state space, discrete time set

Examples include the total amount of claims paid each month and the total
amount insured at the end of each month.

Continuous state space, continuous time set

Examples include the total amount of claims paid up to time t and the
company’s share price.

4 Subject CT4 April 2010 Question 3

State spaces and time sets

A counting process has discrete states but its time set can be either discrete
or continuous.

A general random walk may have either discrete or continuous states but its
time set is discrete.

A compound Poisson process may have either discrete or continuous states

but its time set is continuous.

A Poisson process has discrete states and its time set is continuous.

A Markov jump chain has discrete states and its time set is discrete.

5 Subject CT4 April 2011 Question 7

(i) Counting process

A counting process can have either a discrete or a continuous time set. It

has discrete states 0,1, 2, 3,  . Its value is a non-decreasing function of
time, which means that it can never drop back to a lower state.

(ii)(a) State spaces and time sets

A simple random walk has discrete states and a discrete time set.

A compound Poisson process may have either discrete or continuous states

but its time set is continuous.

A Markov chain has discrete states and its time set is discrete.

Page 28 © IFE: 2019 Examinations

 Batch3

(ii)(b) Examples of applications

Examples of a simple random walk include:

 the number of five pound notes in the till each time a five pound note
passes between the shopkeeper and a customer
 the number of days so far this year that the shopkeeper has made a
profit minus the number of days he or she has made a loss.

Examples of a compound Poisson process include:

 the total number of bags used in sales by time t

 the total weight of nuts sold by time t .

Examples of a Markov chain include:

 the number of people in the shop each time the doors are opened
 the no claims discount level on the shopkeeper’s delivery van.

6 Subject CT4 September 2011 Question 3

Strictly stationary versus weakly stationary processes

A stochastic process is stationary if its statistical properties do not change

over time.

Expressed mathematically, a process { X t } is strictly stationary if, for any

positive integer n, the joint distribution of ( X t1 , X t2 ,  , X tn ) and
( X t1 + k , X t2 + k ,  , X tn + k ) is the same for all values of k , ie if all the statistical
properties remain the same when the times involved are all shifted by the
same amount.

This is a more stringent condition than is usually required in practice. More

often, weak stationarity is used, which only assumes that the first two
moments of the process (ie the means, variances and covariances) remain
constant over time.

© IFE: 2019 Examinations Page 29

 Batch3

A process { X t } is weakly stationary if the following two conditions hold:

 E ( X t ) = E ( X t + k ) for all t and k , ie if the mean value is the same at all

times

 cov( X t , X t + k ) depends only on k , ie if the covariances depend only on

the ‘lag’ k .

7 Subject CT4 April 2012 Question 1

(i) Define a general random walk

If Y1,Y2 , are independent and identically distributed random variables,

then the process X n = Y1 + Y2 +  + Yn (with X 0 = 0 ) is a general random
walk.

(ii) Conditions for a simple random walk

If Y1,Y2 , (the ‘steps’ in the walk) can only take the value –1 or 1, it is a
simple random walk.

8 Subject CT4 April 2013 Question 3

Set (a)

The only combination that will work here is:

Time Set
Discrete Continuous
State Space

Discrete Counting process Poisson process

General random Compound Poisson

Continuous
walk process

Page 30 © IFE: 2019 Examinations

 Batch3

The logic here is that we have no choice for where to put the Poisson
process, but there is some flexibility with the other three. The general
random walk and compound Poisson process have discrete and continuous
time sets respectively. We need another discrete time set. So we must
choose the discrete option for the time set of the counting process, which
then forces it to go in the top left box. This then forces us to take the
continuous state space option for the remaining two processes, which go in
the bottom row.

Set (b)

The only combination that will work here is:

Time Set
Discrete Continuous
Simple random
State Space

Discrete Counting process

walk

Compound Poisson
Continuous White noise
process

The logic this time is that we have no choice for where to put the simple
random walk, which must go in the top left box. The counting process has a
discrete state space, so this must go in the top right box. The compound
Poisson process can then only go in the bottom right box (since it has a
continuous time set) and the white noise must go in the bottom left.

9 Subject CT4 September 2013 Question 3

(i) Define a Poisson process

See Core Reading Paragraph 27.

(ii) Probability that the first bus will not have arrived by 1pm

The probability that the first bus will not have arrived by 1pm is the
probability that no buses arrive (ie no events occur) during the 1 hour period
between midday and 1pm.

We are told that the buses are scheduled to arrive once every 15 minutes.
1 (if we work in minutes).
So   15

© IFE: 2019 Examinations Page 31

 Batch3

The number of events in one hour has a Poisson distribution with mean:

 (t  s )  1  60  4
15

So the probability of no events occurring is:

e 4  0.0183

(iii) How long can he expect to wait for the second bus?

Since Poisson processes are memoryless, the expected time till the next
event will always have the same distribution, T ~ Exp( ) . So the time he
1
can expect to wait till the next bus is E [T ]   15 minutes.


(iv) Probability of at least two more buses

The number of buses arriving between times s and t has a

Poisson [ (t  s )] distribution. So the number of buses arriving in a
10-minute period has a Poisson  15 1  10   Poisson  2  distribution. So the
  3
probability of at least two more buses arriving in the next 10 minutes is:

2 2 e 3
2  32
1 e

3  3
 1  35 e  0.1443
no buses 
one bus

10 Subject CT4 September 2014 Question 1

A counting process has discrete states but its time set can be either discrete
or continuous.

A simple random walk has discrete states and its time set is discrete.

A compound Poisson process may have either discrete or continuous states

but its time set is continuous.

A Markov jump process has discrete states and its time set is continuous.

Page 32 © IFE: 2019 Examinations

 Batch3

11 Subject CT4 September 2014 Question 7

(i) Define a Poisson process

A Poisson process with rate l is a continuous-time process Nt with

discrete state space {0,1,2,} for which:

 N0 = 0

 Nt has independent, stationary increments

 Nt - Ns ~ Poisson [ l (t - s )] , t > s .

(ii) Prove the memoryless property

Let T denote the waiting time till the next event in a Poisson process with
rate l . Then T ~ Exp( l ) and the memoryless property states that, for
any s > 0 , P (T > t + s | T > s ) = P (T > t ) . We can prove this as follows:

P (T > t + s,T > s ) P (T > t + s )

P (T > t + s | T > s ) = =
P (T > s ) P (T > s )
e - l (t +s )
= = e - lt
e-ls
= P (T > t )

(iii)(a) min( X ,Y , Z )

We can demonstrate this by considering the distribution function of

min( X ,Y , Z ) :

P ÎÈmin( X ,Y , Z ) £ t ˚˘ = 1 - P ÎÈmin( X ,Y , Z ) > t ˚˘

= 1 - P ÎÈ X > t ,Y > t , Z > t ˚˘
= 1 - P ( X > t )P (Y > t )P (Z > t ) (using independence)
= 1 - e - tx e - ty e -tz
= 1 - e -t ( x + y + z )

This matches the distribution function of an exponential distribution.

© IFE: 2019 Examinations Page 33

 Batch3

(iii)(b) Parameter of the distribution

The parameter for the distribution is x + y + z .

(iv) Name of the stochastic process

The total value of tolls collected form a compound Poisson process.

(v) Expected value of tolls per hour

We expect 2 ¥ 60 = 120 motorcycles per hour, and each will pay £1. Doing
the equivalent calculation for cars and goods vehicles, the expected value of
tolls collected per hour is:

2 ¥ 60 ¥ £1 + 5 ¥ 60 ¥ £2 + 1.5 ¥ 60 ¥ £5 = 60 ¥ (2 ¥ £1 + 5 ¥ £2 + 1.5 ¥ £5)

= £1,170

(vi) Probability of more than 2 goods vehicles in a minute

The number of goods vehicles arriving in a minute will follow a Poisson(1.5)

distribution. So, the probability of more than 2 arriving in a minute is:

P (N > 2) = 1 - {P (N = 0) + P (N = 1) + P (N = 2)}
ÏÔ 1.52 -1.5 ¸Ô
= 1 - Ìe -1.5 + 1.5e -1.5 + e ˝
ÓÔ 2 ˛Ô
= 1 - 3.625e -1.5 = 0.1912

(vii) Probability that exactly £4 is collected in a minute

We can work out this probability by listing the events that will generate
exactly £4 in tolls, together with their respective probabilities. If we write M,
C and G for motorcycle, car and goods vehicle, these are:

24 -2 16 -8.5
4M ,0C,0G Æ e ¥ e -5 ¥ e -1.5 = e = 0.0001356
4! 24
22 -2 51 -5
2M ,1C,0G Æ e ¥ e ¥ e -1.5 = 10e -8.5 = 0.0020347
2! 1!
52 -5 25 -8.5
0M ,2C,0G Æ e -2 ¥ e ¥ e -1.5 = e = 0.0025434
2! 2

The total probability for these is 0.0047137.

Page 34 © IFE: 2019 Examinations

 Batch3

12 Subject CT4 April 2015 Question 1

(i) Simple random walk

A simple random walk is a process X 0 , X1, X 2,  where X 0 = 0 and

X n = Y1 + Y2 +  + Yn for n = 1, 2, 3,  . The Yi ’s are independent identically
distributed random variables with distribution:

Ï +1 with probability p
Yi = Ì
Ó -1 with probability 1 - p

for some fixed probability 0 < p < 1 .

(ii) State space and time set

The state space for a simple random walk is the discrete set
{ , -2, -1, 0,1, 2, } consisting of all the integers (including negative integers
and zero).

The time set is the discrete set of time points t = 0,1, 2,  , ie the
non-negative integers.

(iii) Example

Suppose that a person goes to a casino with £100 and repeatedly bets £1
on red on the roulette wheel. If the ball lands on red they get their stake of
£1 back plus their winnings of £1 (ie £1 profit); if not, they lose their stake
(ie £1 loss). If we record this player’s overall profit after each spin, this will
follow a simple random walk.

This is actually a simple random walk with an absorbing boundary, since the
overall profit will remain on –100 if the player runs out of money. Absorbing
boundaries are discussed in Booklet 2.

© IFE: 2019 Examinations Page 35

 Batch3

13 Subject CT4 September 2015 Question 5 (part)

(ii) Identify the types of process

The most likely correspondence is:

A = continuous time, discrete state space

B = discrete time, continuous state space

C = discrete time, discrete state space

D = continuous time, continuous state space

14 Subject CT4 April 2016 Question 5

(i)(a) Poisson process

See Core Reading Paragraph 27.

(i)(b) Compound Poisson process

See Core Reading Paragraph 30.

(ii) Most suitable process

Situation A – Number of claims from motorcycle accidents

Answer: Time-inhomogeneous Poisson process

Here we are just counting the number of claims, so a Poisson process will be
most appropriate. Motorcycle accidents will be more likely at times of the
year when the weather is bad (eg because of poor visibility or icy roads). So
the claims rate will not be constant and a time-inhomogeneous process will
be most appropriate.

You could also argue that, if accidents can result in more than one claim (or
none, if there are no injuries or damage done), then this would be a time-
inhomogeneous compound Poisson process.

Page 36 © IFE: 2019 Examinations

 Batch3

Situation B – number of breakfast bagels sold by a New York bar

Answer: Time-inhomogeneous compound Poisson process

Here we are counting the number of bagels, but some customers may buy
more than one, so we need to use a compound Poisson process. The
number of customers will vary according to the time of day, so the purchase
rate will not be constant and a time-inhomogeneous process will be most
appropriate.

You could also argue that, if customers just buy one bagel each for their own
breakfast, this would be a time-inhomogeneous Poisson process.

Situation C – Number of breakdowns of freezers

Answer: Time-homogeneous Poisson process

Here we are just counting the number of breakdowns, so a Poisson process

will be most appropriate. If we can assume that the breakdown rate will be
constant, then a time-homogeneous process will be most appropriate.

You could also argue for a time-inhomogeneous process if, for example, you
think the motors would be more likely to burn out during hot weather in the
summer.

Situation D – Cost of wasted food caused by breakdowns of freezers

Answer: Time-homogeneous compound Poisson process

Here we are counting the cost of wastage arising from the breakdowns, so
we need to use a compound Poisson process. Again, if we can assume that
the breakdown rate will be constant, then a time-homogeneous process will
be most appropriate.

As for Situation C, you could also argue for a time-inhomogeneous process

here.

© IFE: 2019 Examinations Page 37

 Batch3

15 Subject CT4 April 2017 Question 2

(i) Definition of an increment

An increment of a stochastic process is the change in the value of the

process over a specified time interval, eg the increment of the process X
over the time interval [s, t ] is X t - X s .

(ii) Show that the increments are stationary

The increment of the process X u = log h(30 + u ) over the period [s, t ] is:

X t - X s = log h(30 + t ) - log h(30 + s )

= log B(1 + g )t - log B(1 + g )s
= { log B + t log(1 + g )} - { log B + s log(1 + g )}
= (t - s )log(1 + g )

So the increments of the process over any interval of length (t - s ) will all
have exactly the same value. As this value depends only on (t - s ) , not on
the particular values of s and t , the increments are stationary.

16 Subject CT4 April 2017 Question 4

(i) Four-way classification

The time set for a stochastic process, ie the times at which the value of the
process is observed, can be either discrete or continuous.

The state space for the process, ie the possible values the process can take,
can also be either discrete or continuous. This leads to a four-way ( = 2 ¥ 2 )
classification of stochastic processes.

(ii) Examples of four processes

Examples of each type are:

 Discrete time, discrete states: Markov chain
 Discrete time, continuous states: General random walk
 Continuous time, discrete states: Markov jump process
 Continuous time, continuous states: Compound Poisson process

Page 38 © IFE: 2019 Examinations

 Batch3

17 Subject CT4 September 2017 Question 2

The table below shows the state spaces and the time sets for these
processes:

Process State space Time set

General random walk CD D
Markov jump process D C
Compound Poisson process CD C
Markov chain D D

D = discrete, C = continuous and CD = either

18 Subject CT4 September 2017 Question 3

(i) Expected number of phone calls

The hourly number of calls has a Poisson( l ) distribution. So the expected

number of calls is l , ie 3.

(ii) Expected time until the next phone call

The distribution of the waiting time until the next call is Exp( l ) , irrespective
of the past call history. So the expected time until the next call is:

1 1
= hours = 20 minutes
l 3

(iii) Probability that Fred has not missed a call

The probability that Fred has not missed a call is the same as the probability
that there were no calls during the 30 minutes he was away. As the number
of calls N during this period has a Poisson ( 1
2 )
¥ 3 = Poisson (1.5)
distribution, the required probability is:

P (N = 0) = exp ( -1.5) = 0.22313

© IFE: 2019 Examinations Page 39

 Batch3

(iv) Probability the line will be engaged

On average Fred will receive 3 calls per hour and each call will take 7
minutes on average. So, on average, he will spend 3 ¥ 7 = 21 minutes on
the phone during each hour.
21 7
This represents a proportion = = 0.35 of his time.
60 20

So the probability that the line will be engaged is 0.35 or 35%.

Page 40 © IFE: 2019 Examinations

 Batch3

FACTSHEET

This factsheet summarises the main methods, formulae and information

required for tackling questions on the topics in this booklet.

Stochastic process definition

A stochastic process is a model for a time-dependent random phenomenon.

It is a collection of ordered random variables { X t : t  J } , one for each
time t in the time set J . The set of values that the X t are capable of
taking is called the state space S.

Stationarity

If a process is stationary its statistical properties do not vary over time.

Strict stationarity requires that the joint distribution of any set of random
variables { X t1 , X t2 ,  , X tn } is the same as the joint distribution
of { X t1 + k , X t2 + k , , X tn + k } , ie when all times are shifted (lagged) by k .

Weak stationarity only requires that the first two moments do not vary over
time, ie E ( X t ) and var( X t ) are constant, and that cov( X t1 , X t2 ) depends
only on the lag t2 - t1 .

Check weak stationarity by checking means, then variances, then

covariances. Stop at the first ‘failure’ or check all conditions to show weak
stationarity.

Independent increments

An increment of a stochastic process (that has a numerical state space) is

the change in the value between two times, ie X t2  X t1 . If this is
independent of the past values of the process up to and including time t1
then the process is said to have independent increments.

© IFE: 2019 Examinations Page 41

 Batch3

Filtration

The filtration Fs of a process at time s is the set of events whose

happening or not can be determined from the information available at
time s .

Introducing this concept is essential for a continuous time process, but it is

also a useful notation for discrete time processes.

Markov property

If the probabilities for the future values of a process are dependent only on
the latest available value, the process has the Markov property.

Stated mathematically, a process { X t } has the Markov property if:

P È X t Œ A | X s1 = x1, X s2 = x2,, X sn = xn , X s = x ˘ = P ÈÎ X t Œ A | X s = x ˘˚
Î ˚

for all times s1 < s <  < sn < s < t in the time set J , all states x1, x2,, xn
and x in the state space S , and all subsets A of S .

Check for the Markov property by:

1. Checking for independent increments. A process with independent

increments is Markov.

2. Checking if the process satisfies the Markov definition (ie the equations
above).

3. Inspecting the structure of the model and deciding that the Markov
property is true.

Page 42 © IFE: 2019 Examinations

 Batch3

Specific processes

White noise

White noise is a sequence of independent and identically distributed (IID)

random variables.

Random walk

A general random walk X n is defined as:

n
Xn   Yj
j 1

where Yi , i  1, 2, ..., n is a sequence of IID random variables and the

process has initial condition, X 0  0 .

In the special case where the Y j only take the values 1 and 1 , the
process is a simple random walk.

Poisson process

A Poisson process with rate  is a continuous-time integer-valued

process Nt , t  0 with the following properties:

1. N0  0

2. Nt has independent increments

3. Nt has Poisson distributed stationary increments where:

  t  s   e   t s 
n

P Nt  Ns  n     , s  t , n  0, 1, ...
n!

© IFE: 2019 Examinations Page 43

 Batch3

Compound Poisson process

A compound Poisson process X t is defined as:

Nt
Xt   Yj , t  0
j 1

where Nt , t  0 is a Poisson process and Y j , j  1 is a sequence of IID

random variables.

Properties of specific processes

Time State
Markov Stationary
domain space
White Discrete or Discrete or
Yes Yes
noise continuous continuous
Simple
random Discrete Discrete Yes No
walk
General
random Discrete Continuous Yes No
walk
Poisson
Continuous Discrete Yes No
process
Compound
Discrete or
Poisson Continuous Yes No
continuous
process

Processes of mixed type

A process of mixed type is one that operates in continuous time but that can
also change value at predetermined discrete instants.

Counting processes

A counting process, X (t ) , is a stochastic process in discrete or continuous

time, whose state space is the set of whole numbers {0, 1, 2, ...} , with the
property that X (t ) is a non-decreasing function of t . A Poisson process is
an example of a counting process.

Page 44 © IFE: 2019 Examinations

 Batch3

NOTES

© IFE: 2019 Examinations Page 45

 Batch3

NOTES

Page 46 © IFE: 2019 Examinations

 Batch3

NOTES

© IFE: 2019 Examinations Page 47

 Batch3

NOTES

Page 48 © IFE: 2019 Examinations

 Batch3

NOTES

© IFE: 2019 Examinations Page 49

 Batch3

NOTES

Page 50 © IFE: 2019 Examinations