ERG2040C: Lecture 7

Examples and Conditional Expectation

Minghua Chen (minghua@ie.cuhk.edu.hk)

Information Engineering
The Chinese University of Hong Kong

Readings: Ch. 2.1-2.4, 2.6 of the textbook.

M. Chen (CUHK) Examples and conditional expectation March 28, 2008 1/9
Review: Discrete random variables
1 A random variable is a deterministic function from sample space to
real numbers.

M. Chen (CUHK) Examples and conditional expectation March 28, 2008 2/9
Review: Discrete random variables
1 A random variable is a deterministic function from sample space to
real numbers.

2 PMF (for discrete random variables):

P
PX (x) = P(X = x),
x pX (x) = 1.

M. Chen (CUHK) Examples and conditional expectation March 28, 2008 2/9
Review: Discrete random variables
1 A random variable is a deterministic function from sample space to
real numbers.

2 PMF (for discrete random variables):

P
PX (x) = P(X = x),
x pX (x) = 1.

3 Define a probability model using a discrete r.v.

Define the function X .
Define PMF for X .
M. Chen (CUHK) Examples and conditional expectation March 28, 2008 2/9
Review: Expectation and variance
1 Expectation: the central gravity of the probability mass
P
E [X ] = xP x · pX (x).
E [g (X )] = x g (x) · pX (x).
E [X ] is a linear operation.
In general E [g (X )] 6= g (E [X ]).

M. Chen (CUHK) Examples and conditional expectation March 28, 2008 3/9
Review: Expectation and variance
1 Expectation: the central gravity of the probability mass
P
E [X ] = xP x · pX (x).
E [g (X )] = x g (x) · pX (x).
E [X ] is a linear operation.
In general E [g (X )] 6= g (E [X ]).
2 Variance: how the probability mass concentrates around E [X ]
  P  
var(X ) = E (X − E [X ])2 = x (x − E [X ])2 pX (x) = E X 2 − E 2 [X ].
var(aX ) =?

M. Chen (CUHK) Examples and conditional expectation March 28, 2008 3/9
Bernoulli (Indicator) random variable

Define a Bernoulli r.v. on a sample space:


1, a particular event A happens;
X =
0, otherwise.

The PMF is

p, if x = 1;
pX (x) =
1 − p, else.

M. Chen (CUHK) Examples and conditional expectation March 28, 2008 4/9
Bernoulli (Indicator) random variable

Define a Bernoulli r.v. on a sample space:


1, a particular event A happens;
X =
0, otherwise.

The PMF is

p, if x = 1;
pX (x) =
1 − p, else.

Bernoulli r.v. is simple and very important. Examples:

A “Mark Six” ticket wins a prize or not.
The preference of a person who is for or against a certain political
candidate.
A homework is selected to be graded by the TA.

M. Chen (CUHK) Examples and conditional expectation March 28, 2008 4/9
Bernoulli (Indicator) random variable

Define a Bernoulli r.v. on a sample space:


1, a particular event A happens;
X =
0, otherwise.

The PMF is

p, if x = 1;
pX (x) =
1 − p, else.

The expectation and variance of a Bernoulli r.v. X :

E [X ] = 1 · p + 0 · (1 − p) = p,
var(X ) = E [X 2 ] − E 2 [X ] = 12 · p − p 2 = p(1 − p).

M. Chen (CUHK) Examples and conditional expectation March 28, 2008 4/9
Binomial random variable

Experiment: n independent coin tosses. Define Bernoulli r.v.s

Xi , (i = 1, 2, . . . , n) be the outcome of the i-th toss:

1, the i-th toss gives HEAD;
Xi =
0, the i-th toss gives TAIL.

The PMF is pXi (1) = p and pXi (0) = 1 − p.

M. Chen (CUHK) Examples and conditional expectation March 28, 2008 5/9
Binomial random variable

Experiment: n independent coin tosses. Define Bernoulli r.v.s

Xi , (i = 1, 2, . . . , n) be the outcome of the i-th toss:

1, the i-th toss gives HEAD;
Xi =
0, the i-th toss gives TAIL.

The PMF is pXi (1) = p and pXi (0) = 1 − p.

Y = ni=1 Xi is a Binomial r.v., following a Binomial PMF, denoted by

P
B(n, p):
n
!
X
pY (k) = P Xi = k
i=1
 
n k
= p (1 − p)n−k .
k

M. Chen (CUHK) Examples and conditional expectation March 28, 2008 5/9
Binomial random variable
Pn
Y = i=1 Xi is a Binomial r.v., following a Binomial distribution B(n, p):
n
!
X
pY (k) = P Xi = k
i=1
 
n k
= p (1 − p)n−k .
k

M. Chen (CUHK) Examples and conditional expectation March 28, 2008 6/9
Binomial random variable
Pn
Y = i=1 Xi is a Binomial r.v., following a Binomial distribution B(n, p):
n
!
X
pY (k) = P Xi = k
i=1
 
n k
= p (1 − p)n−k .
k

Binomial r.v.s model aggregate outcomes of a series of experiments.

Examples:
The number of prizes that 5 random-number tickets win.
The number of citizens in Hong Kong vote for a certain political
candidate.
The number of homework the TA grades in a given week

M. Chen (CUHK) Examples and conditional expectation March 28, 2008 6/9
Binomial random variable
Pn
Y = i=1 Xi is a Binomial r.v., following a Binomial distribution B(n, p):
n
!
X
pY (k) = P Xi = k
i=1
 
n k
= p (1 − p)n−k .
k

The expectation of a Binomial r.v. Y with parameter n and p:

n
X n!
E [Y ] = k p k (1 − p)n−k
(n − k)!k!
k=0
n  
X n − 1 k−1
= np p (1 − p)n−k = np = nE [Xi ]?
k −1
k=0

M. Chen (CUHK) Examples and conditional expectation March 28, 2008 6/9
Conditional PMF and expectation

Conditional PMF: given an event A with P(A) > 0,

pX |A (x) = P(X = x|A).

Conditional expectation:
X
E [X |A] = xpX |A (x).
x

E [X |X ≥ 2] =?

M. Chen (CUHK) Examples and conditional expectation March 28, 2008 7/9
Total expectation theorem: divide and conquer!

Partition sample space into A1 , A2 , . . . , An :

(D. P. Bertsekas & J. N. Tsitsiklis, Introduction to Probability, Athena Scientific Publishers, 2002)

P(B) = P(A1 )P(B|A1 ) + P(A2 )P(B|A2 ) + · · · + P(An )P(B|An )

E [X ] = P(A1 )E [X |A1 ] + P(A2 )E [X |A2 ] + · · · + P(An )E [X |An ]

Example: computing expected average salary of CSE graduates.

M. Chen (CUHK) Examples and conditional expectation March 28, 2008 8/9
Geometric random variable

Experiment: flip a coin until you see a HEAD, assuming P(H) = p.

Define a Geometric r.v Z as the number of tosses needed:

PZ (k) = (1 − p)k−1 p, k = 1, 2, . . .

M. Chen (CUHK) Examples and conditional expectation March 28, 2008 9/9
Geometric random variable

Experiment: flip a coin until you see a HEAD, assuming P(H) = p.

Define a Geometric r.v Z as the number of tosses needed:

PZ (k) = (1 − p)k−1 p, k = 1, 2, . . .

Geometric r.v.s model the time till a particular event happens. Examples:
The life time of a hard disk.
The time to the next packet arrival at a router.
The time to the first time a student’s homework is graded by TAs.

M. Chen (CUHK) Examples and conditional expectation March 28, 2008 9/9
Geometric random variable

Experiment: flip a coin until you see a HEAD, assuming P(H) = p.

Define a Geometric r.v Z as the number of tosses needed:

PZ (k) = (1 − p)k−1 p, k = 1, 2, . . .

Memoryless property: Given Z > m, the r.v. Z − m has the same

geometric PMF as Z : (intepretation?)
(
(1−p)k−1 p k−m−1 p, if k > m;
pZ |Z >m (k) = P(Z = k|Z > m) = (1−p)m = (1 − p)
0, otherwise

M. Chen (CUHK) Examples and conditional expectation March 28, 2008 9/9
Geometric random variable

Experiment: flip a coin until you see a HEAD, assuming P(H) = p.

Define a Geometric r.v Z as the number of tosses needed:

PZ (k) = (1 − p)k−1 p, k = 1, 2, . . .

Memoryless property: Given Z > m, the r.v. Z − m has the same

geometric PMF as Z : (intepretation?)
(
(1−p)k−1 p k−m−1 p, if k > m;
pZ |Z >m (k) = P(Z = k|Z > m) = (1−p)m = (1 − p)
0, otherwise

E [Z − m|Z > m] = E [Z ]?, E [Z |Z > m] = E [Z ]?

M. Chen (CUHK) Examples and conditional expectation March 28, 2008 9/9