Mad 1284 – Statistics

Module -IV : Generating Functions

J. Ravinder
Assistant Professor
Department of Mathematics and Actuarial Science
 MAD 1284 : Statistics Module – IV : Generating Functions

Let X be a random variable. The mean and the variance of X are also known as the 1st and 2nd moments of
X , respectively.
Moment Generating Functions (MGF) are just another way of describing probability distributions.
The nth Moment of a random variable X is denoted by n
The Moment Generating Function of a random variable X is denoted by M X ( t )

S.No. Discrete Random Variable Continuous Random Variable


1. nth Moment about the Origin n  E[ X n ]   x n P ( X  x ) n  E[ X n ]   x n f ( x ) dx



nth Moment about the Mean n  E[( X  X ) ]   ( x  X ) P ( X  x )  n  E[( X  X ) ]   ( x  X ) f ( x ) dx
n n n n
2.
(Central Moment) 


Moment Generating Function M X ( t )  E[e t x ]   e t x P ( X  x ) M X ( t )  E[e t x ]   e t x f ( x ) dx
3.
(MGF) 

Note : i ) X  E[ X ]  1
ii) The Moment about the Mean is also known as Central Moment.
M4-01_2
 MAD 1284 : Statistics Module – IV : Generating Functions

Example 1. Find the m.g.f of a random variable ‘X’ whose probability function is
1
P ( x )  x , x  1, 2, 3, . . .
3
Solution 1. Consider the given probability mass function (pmf)
1
P( x)  , x  1, 2, 3, . . . . . . (1)
3x
Moment Generating Function :

1
et 
2
 et   et   et 
3

M X ( t )  E[e ]   e P ( X  x )
xt xt
 1          . . . 
3 3 3 3 

 1   equation (1)
M X ( t )  E [e x t ]   e x t  x 
3   
 e x t e t
 1 
  x  x  1,2,3, . . .   t 
 1 
 1  x  x 2  x 3  . . .
x 1 3 3  e  
  
1 1 x
 3 
e1t e 2t e 3t e4t
 1  2  3  4  ... et  3 
3 3 3 3   t 
1 2 3 4
3 3  e 
 et   et   et   et 
             ... et
3 3 3 3  M X (t ) 
3  et
M4-01_3
 MAD 1284 : Statistics Module – IV : Generating Functions

 2 e 2 x if x  0
Example 2. A random variable has the pdf given by f ( x)  
0 otherwise
Find (i) Mean
(ii) Variance
(iii) Standard Deviation
and (iv) the moment generating function.
Formulas : If X is a Continuous Random Variable, then
 
i ) n  E[ X n ]   x n f ( x ) dx for n  1, 1   x f ( x ) dx  E[ X ]  Mean  X
 

 
ii ) n  E[ X n ]   x n f ( x ) dx for n  2, 2   x 2 f ( x ) dx  E[ X 2 ]
 

 
2
iii ) Var[ X ]  E[ X 2 ]   E[ X ]  2  1
2

iv ) SD    Var[ X ]

v ) M X ( t )  E[e ]   e x t f ( x ) dx
xt


M4-01_4
 MAD 1284 : Statistics Module – IV : Generating Functions

Solution 2. Consider the given probability density function (pdf)

 2 e 2 x if x  0
f ( x)   . . . (1)
0 otherwise
(i) Mean : (Mean is a 1st Moment about the Origin)
x 

 
   e 2 x   e 2 x  
1  E[ X ]   x f ( x ) dx    E[ X ]   x f ( x ) dx 
1 n n

 1  2  x   1  2 
 (0)v2  ...
  2 
n

 ( 2) 

  
for n  1, 1  E[ X ]  1 
 x 0

1
0  x 
  x f ( x ) dx   x f ( x ) dx  xe 2 x e 2 x 
 2  
4  x  0
 0
 2
0 
 equation (1)
  x (0) dx   x (2e 2 x ) dx
e 2(  ) (0)e 2(0)  e 
2(0)
 ( )e 2(  )
  2   
4 
0
  2 4 2
 2  x e 2 x dx
1
0  2   e   0, e 0  1
4
  u dv  uv  uv1  uv 2  uv 3  . . . 
 2 x 
 where u  x , dv  e dx  1
 e 2 x   1  E[ X ]  . . . (2)
 u  1, v   dv   e 2 x dx   2
 2 
 e 2 x e 2 x 
 u  0, v1   v   dx  2 

 2  2  
M4-01_5
 MAD 1284 : Statistics Module – IV : Generating Functions

(ii) Variance : (Variance is a 2nd Central Moment)

Consider  2  e 2 x   e 2 x 

 2  2  x    (2 x )  2 

 

2  E[ X 2 ]   x 2 f ( x ) dx     x f ( x ) dx   2   ( 2) 
1 n n
 n E [ X ] 

   x 
 for n  2, 21  E[ X 2 ]  2  
  e 2 x 
 (2)   
0
  x 2 f ( x ) dx   x 2 f ( x ) dx 3 
(0)v ... 
 ( 2) 
3
 0  x0
0 
 equation (1)
 x 2 e 2 x xe 2 x e 2 x 
x 
  x 2 (0) dx   x 2 (2e 2 x ) dx
 2   
4  x  0
 0

 2 2
 2  x 2 e 2 x dx
 ( )2 e 2(  ) ( )e 2(  ) e 2(  )
 0   2   
   2 2 4
 
          
(0)e 2(0)  e 
u dv uv u v u v u v . . . 2(0)

1 2 3
 (0)2 e 2(0)
 
 where u  x 2 , dv  e 2 x dx  2 2 4 
 e 2 x 
 u  2 x , v   dv   e 2 x dx  
 2  1
 e 2 x e 2 x   2   e   0, e 0  1
 u  2, v1   v   dx   4
2  2 
2
 
1

 e 2 x
e 2 x 

 2  E[ X 2 ]  . . . (3)
u  0, v 2   v1   2 dx  2
  2   2  
3

M4-01_6
 MAD 1284 : Statistics Module – IV : Generating Functions

Since the Variance is the 2nd Central Moment, (iv) Moment Generating Function :

 Var[ X ]  2 M X ( t )  E[e ]   e x t f ( x ) dx
xt


 E[ X 2 ]   E[ X ]
2
0 
  e x t f ( x ) dx   e x t f ( x ) dx
 
2
 2  1  equation (2) & (3)
 0
0 
 equation (1)
1 1
2
  e x t (0) dx   e x t (2e 2 x ) dx
    0
2  2  
2 x
1 1   2e e xt
dx  2  e x t  2 x dx
  0 0
2 4 
1  2  e  ( 2 t ) x dx
 Var[ X ]  . . . (4) 0
4  e ax 
 e dx 
ax
x  
 e  ( 2 t ) x   a 
(iii) Standard Deviation (SD) :  2 
 (2  t )  x  0
We know that SD    Var[ X ]
 e  ( 2 t )(  ) e  ( 2 t )(0) 
 equation (4)  2   e   0, e 0  1
 (2  t ) 
1
4  (2  t )
1 2
 SD   M X (t ) 
2 2t
M4-01_7
 MAD 1284 : Statistics Module – IV : Generating Functions

2 at x  1
 3
Exercise 1. Find the mgf for the distribution given by 
f ( x)   1 at x  2
3



0 otherwise

 2( x  1) if 1  x  2
Exercise 2. A random variable has the pdf given by f ( x)  
 0 otherwise
Find (i) Mean
(ii) Variance
(iii) Standard Deviation
and (iv) the moment generating function.

Answer 1. M X ( t )  E[e t x ]  1  2e t  e 2 t 
3

2 
e  e 2t  t e 2t 
2 t
2. 1  5 ; 2  17 ; Var[ X ]  1 ; SD  1 ; M X (t ) 
3 6 18 3 2 t
M4-01_8
MAD 1284 : Statistics Module – IV : Generating Functions
Properties :

1. The “zeroth central moment is 1”. i .e., 0  1.

2. The “first central moment is 0”. i .e., 1  0.

 
2
3. The “second central moment is called Variance”. i .e., 2  Var[ X ]  E[ X 2 ]   E[ X ]  2  1
2

 
3
4. The “third central moment is called Skewness”.  3  3  32 1  2 1

    4     6       3    
2 4
5. The “fourth central moment is called Kurtosis”.  4 4 2 1 2 1 1

6. M X Y ( t )  M X ( t ). MY ( t )
9. n ( X )  E  X  E[ X ] 
n
 
d  10. n ( X  c )  n ( X )
7.  M X ( t )   1  E[ X ]
 dt t 0
11. n (aX )  a n ( X )
n

 d2 
8.  2 M X ( t )   2  E[ X 2 ] 12. n ( X  Y )  n ( X )  n (Y )
 dt t 0
13. n (aX  bY )  a n ( X )  b n (Y )
n n

M4-01_9
 MAD 1284 : Statistics Module – IV : Generating Functions

Example 1. Prove that the moment generating function of the sum of a number of independent random
variable is equal to the product of their respective moment generating functions.
Proof 1. Claim : M X Y ( t )  M X ( t ). MY ( t ) . . . (1)
(i.e., the moment generating function of the sum of a number of independent random variable
is equal to the product of their respective moment generating functions.)
Consider the LHS of equation (1),
LHS  M X Y ( t )  M X ( t )  E[e xt ]

 E  e ( x  y ) t 

 E  e xt  yt 

 E  e xt .e yt 

 E  e xt  E  e yt 

 M X ( t ) MY ( t )
 RHS
Hence the proof.
M4-01_10
 MAD 1284 : Statistics Module – IV : Generating Functions
3
Example 2. If a random variable X has the moment generating function M X ( t )  ,
3t
find (i) mean, (ii) variance and (iii) standard deviation of X.
Solution 2. Consider the given Moment Generating Function
3
M X (t )  . . . (1)
3t
d
M X (t ) 
d  3  d2
M X (t ) 
d d  (i) Since Mean  1
   M X (t ) 
dt dt  3  t  dt 2
dt  dt   d 
  M X (t ) 
d  1  d  3   dt t 0
 3     2 
dt  3  t  dt  (3  t )   3 
  2 
 1 d   2 d   (3  t )  t  0
 3 (3  t )   3 (3  t ) 
 (3  t ) dt  (3  t ) dt 3
2 3
  
(3  0)2
 1   2 
 3 (  1)   3 (  1)  3
 (3  t )
2
  (3  t )
3
 
9
d 3 d2 6
M X (t )  M X (t )  Mean 
1
 1
dt (3  t )2 dt 2
(3  t )3 3
M4-01_11
 MAD 1284 : Statistics Module – IV : Generating Functions

 d2  (iii)
(ii) Since 2   2 M X ( t ) 
 dt t 0 We know that SD    Var[ X ] ( ii ) Var[2 X  3]
 E[(2 X  3)2 ]   E[2 X  3]
2
 6   1
  3  9
 (3  t )  t  0  E[(2 X )2  (3)2  2(2 X )(3)]
1

6  SD    2 E[ X ]  3 
2

(3  0)3 3
6  E[4 X 2  9  12 X ]
 * * *
27   (2 E[ X ])2  32  2(2 E[ X ])(3) 
2 Example 3. Compute
2 
9 ( i ) E[2 X  3]  E[4 X 2 ]  E[9]  E[12 X ]
 Var[ X ]  2
( ii ) Var[2 X  3]
  4( E[ X ]) 2
 9  12 E[ X ]
 
2
 2  1 Solution 3.  4 E[ X 2 ]  9  12 E[ X ]
2
2 1  2  1 ( i ) E[2 X  3]  E[2 X ]  E[3]
    4( E[ X ])2  9  12 E[ X ]
9  3 9 9  2 E[ X ]  E[3]
1  2 E[ X ]  3  4  E[ X 2 ]  ( E[ X ])2 
Var[ X ]  2 
9 E[2 X  3]  2 E[ X ]  3 Var[2 X  3]  4 Var[ X ]
M4-01_12