Module 15

Moment Generating Function

() Module 15 Moment Generating Function 1 / 16

 X : a discrete or A.C. r.v. with p.m.f./p.d.f fX (·);
Define n   o
AX = t ∈ R : E e tX < ∞ .

Clearly 0 ∈ Ax , and thus Ax 6= φ.

Definition 1: The moment generating function (m.g.f.) of r.v. X is

defined by  
MX (t) = E e tX , t ∈ AX .

() Module 15 Moment Generating Function 1 / 16

Remark 1:
(a)
MX (0) = 1 and MX (t) > 0, ∀ t ∈ AX ;
(b) Let Y = cX + d, for some real constants c 6= 0 and
d ∈ R. Then
 
MY (t) = E e tY
 
= E e t(cX +d)
 
= e td E e ctX
nx o
td
= e MX (ct), t ∈ AY = : x ∈ AX .
c
(c) The name moment generating function to the transform
MX (t), t ∈ AX , is attributed to the fact that MX (t) can be
used to generate moments (µ0r = E (X r ) , r = 1, 2, . . .) of
r.v. X .
() Module 15 Moment Generating Function 2 / 16
Result 1:

Suppose that, for some h > 0, the m.g.f. MX (t) is finite for every
t ∈ (−h, h). Then
(a) MX (t) is differentiable any number of times in (−h, h);
(b) for each r ∈ {1, 2, . . .} , µ0r = E (X r ) is finite and
 
0 (r ) d
µr = MX (0) = MX (t) ;
dt r t=0

(c) for t ∈ (−h, h)

∞ r
X t
MX (t) = µ0r .
r!
r =0

() Module 15 Moment Generating Function 3 / 16

Proof:
(Outline of the proof for A.C. case). Fix r ∈ {1, 2, . . .}.
MX (t) = E (e tX )
Z ∞
= e tx fX (x)dx
−∞
Z ∞
d r MX (t) dr
= e tx fX (x)dx
dt r dt r −∞
Z ∞ r 
d tx
= r
e fX (x)dx
−∞ dt
Z ∞
= x r e tx fX (x)dx
−∞
 r  Z ∞
d MX (t)
= x r fX (x)dx = µ0r ,
dt r t=0 −∞
d r
where passing of derivative dt r under the integral sign can be justified

through advanced mathematical arguments.

() Module 15 Moment Generating Function 4 / 16
Also,
Z ∞
MX (t) = e tx fX (x)dx
−∞
∞
∞ X
(tx)r
Z
= fX (x)dx
−∞ r =0 r!
∞
X tr Z ∞
= x r fX (x)dx
r! −∞
r =0
∞ r
X t
= µ0r , r = 1, 2, . . . ,
r!
r =0

where interchange of integral and summation sign can be justified through

advanced mathematical arguments.

() Module 15 Moment Generating Function 5 / 16

Corollary 1:
Under the notation and assumptions of above result, let
ψX (t) = ln MX (t), t ∈ (−h, h). Then
(1)
µ01 = E (X ) = ψX (0);
(2)
and µ2 = Var(X ) = ψX (0).
Proof. We have, for t ∈ (−h, h),
(1)
(1) MX (t)
ψX (t) =
MX (t)
(1)
(1) MX (0)
ψX (0) = = µ01
MX (0)
(2) (1)
(2) MX (t)MX (t) − (MX (t))2
ψX (t) =
(MX (t))2
(2) (2) (1)
ψX (0) = MX (0) − (MX (0))2
() Module 15 Moment Generating Function 6 / 16
 = µ02 − (µ01 )2 = Var(X ).

Remark 2:
(a) The function ψX (t), t ∈ (−h, h), is called the cumulant
generating function of X ;
(b) For r = 1, 2, . . .

tr
µ0r = coefficient of in Maclaurin’s series expansion
r!
of MX (t).

() Module 15 Moment Generating Function 7 / 16

Example 1:
Let X be a r.v. with p.m.f.
e −λ λx

x! , if x ∈ {0, 1, 2, . . .}
fX (x) = ,
0, otherwise

where λ > 0 is a fixed constant. Then SX = {0, 1, 2, . . .} and

MX (t) = E (e tX )
∞
−λ
X (λe t )x
= e
x!
x=0
−λ λe t
= e e
λ(e t −1)
= e ,

is finite for every t ∈ R. Thus µ0r is finite for every r ∈ {1, 2, . . .}. We
have, for t ∈ R,
() Module 15 Moment Generating Function 8 / 16
 ψX (t) = ln MX (t) = λ(e t − 1), t ∈ R

(1) (2)
ψX (0) = ψX (0) = λ

⇒ E (X ) = Var(X ) = λ.

Alt.
λ2 (e t − 1)2 λ3 (e t − 1)3
MX (t) = 1 + λ(e t − 1) + + + ···
2! 3!

λ2 S 2 λ3 S 3
= 1 + λS + + + ··· ,
2! 3!
where
t2 t3
S = et − 1 = t + + + ··· .
2! 3!
() Module 15 Moment Generating Function 9 / 16
Thus
t2 2λ2 λ3 t3
 
2 λ
MX (t) = 1 + λt + (λ + λ ) + + + + ···
2! 3! (2!)2 3! 3!

E (X ) = Coefficient of t in MX (t) = λ

t2
E (X 2 ) = Coefficient of in MX (t) = λ + λ2
2!

t3
E (X 3 ) = Coefficient of in MX (t) = λ3 + 3λ2 + λ
3!

Var(X ) = E (X 2 ) − (E (X ))2 = λ.

() Module 15 Moment Generating Function 10 / 16

Example 2:
Let X be a r.v. with p.d.f.

λe −λx , if x > 0

fX (x) = ,
0, otherwise

where λ > 0 is a constant. Here SX = [0, ∞). Also

MX (t) = E (e tX )
Z ∞
= e tx fX (x)dx
−∞
Z ∞
= λ e −(λ−t)x dx
0
 t −1
= 1−
λ
is finite for every t < λ. Thus moments of all orders exist.
() Module 15 Moment Generating Function 11 / 16
 t t2 tr
MX (t) = 1 + + 2 + · · · + r + · · · , −λ < t < λ
λ λ λ
tr
µ0r = coefficient of in expansion of MX (t)
r!
r!
= , r = 1, 2, . . .
λr
Alt.

(1) 1 t −2
MX (t) = 1−
λ λ
(2) 2  t −3
MX (t) = 1 −
λ2 λ
(r ) r !  t −(r +1)
MX (t) = 1 − , −λ < t < λ
λr λ
(r ) r!
µ0r = MX (0) = r , r = 1, 2, . . . .
λ
() Module 15 Moment Generating Function 12 / 16
Example 3: Let X be a r.v. with p.d.f.
1 1
fX (x) = . , −∞ < x < ∞.
π 1 + x2
For t > 0

MX (t) = E (e tX )
Z ∞
1 1
= e tx . dx
−∞ π 1 + x2
1 ∞ e tx
Z
≥ dx
π 0 1 + x2
1 ∞ tx
Z
≥ dx (e y ≥ y , ∀y > 0)
π 0 1 + x2
= ∞
⇒ MX (t) = ∞, ∀t > 0,

i.e., MX (t) is not finite on any interval (−h, h), h > 0. Note that here
E (X ) does not exists.
() Module 15 Moment Generating Function 13 / 16
Take Home Problems

1 Let X be a r.v. with p.m.f.

 cp
xp , if x ∈ {1, 2, . . . }
fX (x) = ,
0, otherwise

where p > 1 is a constant and cp is the normalizing constant. Show

that the m.g.f. of X is not finite on any interval (−h, h), h > 0. Do
the moments exist?
2 Let X be a r.v. with m.g.f.
t t
1 e2 e− 3
MX (t) = + + , t ∈ R.
2 6 3
Find P(|X | > 0) and the p.m.f. of Y = X 2 .

() Module 15 Moment Generating Function 14 / 16

Abstract of Next Module

Two r.v.s X and Y are said to have the same distribution (written as
d
X = Y ) if they have the same d.f.;
d
X = Y does not imply that X = Y ;
We will study properties of r.v.s having the same distribution.

() Module 15 Moment Generating Function 15 / 16

 Thank you for your patience

() Module 15 Moment Generating Function 16 / 16