IIT JAM MS Probability Cheat Sheet

Statistics and Analytics Department

\ I
Author: Pravat Ku mar Hati
 Probability Theory Law of Total Probability
n n
Definitions
p (BJ = ~ p (Bl.A;] p [.A;J fl= LJA.
• Sample space !l i=l t=l
• Outcome (paint or element) w e !l BAYES' T REOREM
• Event A !;; fl n
p (B !.A;J p (A;J
• u -algebra. A
p IA, IB( = Ej=l p (BI A;Jp (A;] n = LJA.
0e A
1.

3. A e A
=
2. A,,A,, ... ,eA
= e LJ;:
-.A A
1 .A;eA Incluoion-Exolusion Principle
i= l

• Probability Distribution P
1. P (A[ ~ 0 'v'A
2. P [!l[ =1
3. p [QA;] =t. p ~[
Random Variables
Random Variable (RY)
• Probability space (fl, A, P) X :!l ➔ R

Properties Probability Maas Function (PMF)

• PIil = 0 fx(x) = P (X = "'J = P ({we !l: X(w) =:r}J

• B=llnB=(AU-.A)nB=(AnB)U(-.AnB) Probability Density Function (PDF)
• P(-.A] = 1 - P (A]
•
•
p (Bl= p [An B] +P (-.A nB]
P(!l] = 1 P (0[ =0
P (a 5 X 5 b] = J.• /(z) ,h
• ~ (Un A.)= n. -.An , (nnAn) = Un -.A. DEMORGAN
Cumulative Distribution Function (CDF)
• P IU. A.[ = 1 - P In. -.A.(
• P(AUB]= P (A]+P(B]-P(AnB] Fx : R ➔ (0, l J Fx(::i:) =PIX 5 :r(
= P(AUB] :, P (A] +P(B]
• P(AUB] = P (An~J +P(-.AnBJ + P (AnBJ
1. Nondecreasing: :r1 < :r2 =
F(::c,) 5 F(xo)
2. Normalized: lim.-.- = 0 and lim«-.+- = 1
• p [An ,BJ= p [AJ - P [An BJ 3. Right.Continuous: lim,,~F(11) =F(:r)
Continuity of Probabilitiea

• A, cA2 c ...
• A1:::>A2:::>, ..
=
= lim,,...._P[A.J=P[AJ
lim..-.-P (A.(=P(A]
wbereA=LJ;:,A.
wbereA=n:,A.
P (a 5Y 5 blX = x] = !. /yJX(lll!E)d11 a5b

/(:r, 11)
Independence lL

Conditional Probability
AlLB = P (AnBJ =P[AJ P(BJ
ln<rependence
frix(Y I:r) = /x(::i:)

P (AnB[ =
1. P (X 5 x, Y 5 yJ P (X 5 z] P [Y 5 11]
P(AIBJ = P(B( P (Bl > 0 2. fx,y(x,11) = /x(:r)fy(y)
 Distribution Overview
Discrete Distribut ions
Notation1 Fx(z) Jx (x) 1B [X] V[X] Mx(• )

Uniform Uni!{o, . . . , b} {~::'°'

1
::::Sb
x> b
I (a < "' < b)
b - a +I
a+b
2
(b - o +J) 2 - l
12
eru _ e - (b+l)•

s(b - a)

Bemoulli B,.-n(p) (I - p)' - • ,,. (1 - p)'- • p p( l - p)

Binomw (:)p' (l - p)•-• np np(l - p)

Multioomw Mult(n,p)
nl St SJ,
•
L x, = n np, np,(1 - p,)
:rd ... :tA:1
1 .. 'Pitt
,_,
x - np )
(';)(;:':;) nm nm(N - n )(N - m)
Hypergeomclric Byp (N, m, n)
"'<I> ( Jnp(l - p) (~) N N2(N - 1)
1- p 1- p
Negative Binominl NBin (r ,p) (•+r - l)p'(l - p)" r- r-
r- 1 p p•
1 1- p
Geometric Ceo(p) 1 - (1-p)' •EN+ p(l - p)•-t :c EN+-
p ,;;- 1 - (1 - p)..
>..ll!e- "'
PoiBSOlll Po(>.) -.- ,- >. >.

Ull--~1

§
..... .. .•
..._ ..... p.♦l

hl!l. 11•♦.I
-
a .
l •; l
•• ... ••. l•
! I . •
~ ! \ .
!
! . \•,
\ •"
I

!
•.I
................ ................................... . . ~------------~
• • » » ., • • • •
 Continuous Distributions
Notation Fx(z) /x(z) EIXI VIX) Afx(•l

Uniform Unif (o,b) {~--

= 1
z <B
0<2:<b
z>b
I(<> < < < b)
b-o
<>+ b
2
(b- <>)'
12
e•b - e 1•
s(b- o)

Normal N (µ,u 2
) ~(z) = [_ ¢(t)dt ¢(,:) = _1_ e,p { µ)'} (z- µ •
CT {
exp µa+-- u•••}
u..ni 2u2 2

Log-No,mal btN(µ ,u') ½+ ~ <rf [bt:ii' l .,J~c,2-{ (lo;,;;µ)'} eP+<i',/'J (e0

>
- l)e'I<+•
>

Multivariate Normal MVN(µ,I:) (2,r)-•1•11::1-1/>,- ! (•-P) ri,-• (•-p) µ I: exp {µ.T•+ ½aTEa}
r ( ~) ( .,• )-<•+1>1•
S~udant's t Studeot(v) (i, i)
I, Jinn'(,) 1 + -;;- 0 0

Chi-square •
X• 1 (k "')
r(k/2) 1 2' 2
1
~t•r(k/2) 2
il/'J-1
•
-•/'J I;; 2k (1 - 2si-•1• • < 1/2
(4 12)"1";,
(d1 d1) (412+1',)•1+4' do 2d~(d1 + do - 2)
F F(d1,do) J cl1~'i,,1; 2 ~2 ,-B(!IJ, t ) do -2 d1(do - 2)2 (do - 4)
1 -r/p 1
Expooeatial Exp(/1) 1
-•-~//J
-•
II
II 11• l-/1• (s < 1/fJ)
-y(c,, •/II) 1 o-1 -•/fl 2
Gamma Gamma. (a,/1) r(a) /J•"' • o/1 o/1 (1!11.) • C• < 1/11)
r(o)
0
r(o,!) fl _p_ o > 1 p•
lo.veme Gamma lnvGIIDllDA (o, fJ)
r (o) r(o)"'
-a-1 -fl/'z
• o-1 (o - 1)2 (o - 2)2
o> 2 2(1c~)•l'l Ko ( J-4.B•)

Dir;dilet Dir (c,)

r (E'
' •-1 o), IT
•
s~-1
o, E 1x,J (1- E 1x,))
II:-1 r (o,; c-1 t:'-1 o, t:'-, o,+ 1
r (o + fJ) o-1 (1 _ ),_, 0 o/1 - c-1 + ) •
Beta Beta(o,/1) J,(a,/1) r (o) r(fJ)" r o+/1 (o + /1)2 (o +II+ 1) l+~ }]a:P:r :,
1 k (a=) •-1 - ( •/>.)•
Wmbull WeibuD(A,k) -•-(:1:/>.)• r r • Ar(1+ ~) • ( 1+ :i;2) -µ •
Ar I; ~
"-o nJ
r(1+ ~)
k

Pareto Pareto ( Zm, o) 1- rmr

7 X~:t:m 0
.,::_
x~x'" ~ o>l •• 2 o(-xma) 0 r(-o, -:£:ma) 8 < 0
:ca+l o-1 (o - l)~o - 2) " >
 \/11. . . l!IOft-lJ

:
- - •.• o,a>.-.:
...o.,I.. '
··""·'
•••?,.... Ol ' r,
...,_...,
- ..........,
. . .....,
11•?.•' •l

"""""'
11•-.,,,c,°• I
• -· -
..•.....,
h t

:
ll• l t2lG1• 1
.
:
:
t ~ .• I t:
/
~-------
... .
• I

--·J.
' : •
. L_ .
• ., •
:
•• .. .. , •
-·
4 u u u u 4
' '
,.
........, - ...
e.,,en.....
: - ,, ' ...
..,. :.,.....,, = - ,.1 :
..,...u
- . .. 1,11 .. 1
•·· • - :tll• l

.. ... = .........1
~.. ,111, .... ,
'•'-• ..•......
. .. ,.... 1

\
..... ,...... ,a:,

~· \
~ , •
\ =

\
: :
~ ~~ ~
:
;
.
;
,
-. ;
:
\__
: g : :
, ,
• • •
- • • • u a

,.......... ..............
- ,. ,, ..•....
- ... ,..,, ,., -· "
.......
.....:.,.,..
~ ~ - l•l,••O..S &o•'-""' '
,. ,
• J . toU
•• , , •• 1
:. :
l • 1, 1 •'
lol, lo U
lol, 1 ..s
&o•l.••l
&o•'-••·

~
= -
. = •
t t, t t·

' '

. ~
• , ,
:
:
•• .. •• .. .. u
2

g
.. ., -.. u u u
. _J
•
~

, ,
 Transformations • E (XY] = / :i:11/x,y(x,y) dFx(z) dFy(y)
lx,Y
'Iransfmmation function
o E (<p(Y)] j <p(E (Xl) (cL J ENSEN inequality)
Z = <p(X)
= =
Disc.rete

/z(z) = l' [<p(X) = z] = P [{x: <p(:i:) = z}] = P [X E <p- 1 (z)] = L /(:i:)

• E (X]
-
o P [X ;>: Y[ = 0

= LP (X ;:: z]
E (X] ;>: E (Y[ AP [X = Y] = 1 E (X] = E (Y]

:a:eip-t(•)
Sample mean
Continuous

Fz(z) = P [<p(X) 5 z] = f f(x) dz with A, = {a:: <p(:i:) 5 z}

IA. Conditione.l expec~a.tion

Specie.I case if 'I' strictly mollOtone • E(YIX =x] = J 11/(y lz)dy

/z(z) = /x(<p-l(z)) l!'l'-'(z)I = /x(x) 1:1 = /x(z)l~I

• E(X] = E (E (XIY]l
• E(<p(X,Y)IX =x[ = 1-: <p(,:,y).fy1x(11l:i:)d:i:
The Rule of the Lazy Statistician

E [Z] = j <p(x) dFx(,:) • E(<p(Y,Z)IX = :i:[ = 1-:

• E(Y +z IX] =E(Y IX) +IE (Z IX)
<p(y,z)/(Y,Z)jX(ll,z l:i:)dydz

IE (IA (z)] = f IA(x) dFx(x) = 1. dFx(x) = II' (X t A]

• E [<p(X)Y IX]= <p(X)E (Y IX]
• E(YIX)=c =
Cov(X,Y)=O
Convolution

1- zy~[ /x,Y(x,z-x)dx
• Z :=X+Y Jz(z) = fx,Y(x,z-x)dx =- 5 Variance
Definition IWd properties
• Z :=IX-YI fz(z) = 2 f
- -

/x,y(x,z +z)dx
0

• V(X) =•J = E [(X -E(Xl)'] =E [X']- E (X) 2

• Z :=; /z(z) =L lxlfx,y(x,zz)dx ,M- L xf,(x)fx(x)jy(xz)dx • V [t

i=l
X;] = t
e.=l
V [X;[ + 2 I: Cov (X;, Y;[
i#J

4 Expectation • v[t.x;] =t.V(X;) iffX;lLX;

Definition and properties
Stando.rd deviation
Lxfx(x) X mscrete sd(X) = yV(X] = ux

• EIX)=µx= j xdFx(x)= •
CoV8li&DCe
• Cov (X, Y] = E [(X -E (Xl)(Y -E [Yl)] = E (XY] -E (X]E (Y)
( j xfx(x) X continuous
• Cov (X,a[ = 0
• P[X=c]=l
oE[cX[=cE(X)
=
E[c] =c •
•
Cov(X,X[ =V[X[
Cov (X, Y) = Cov [Y.X]
• E IX +Y) =E(X) +E(Y] • Cov (aX,bY) = abCov (X, Y]
 • Cov[X+a,Y+b]=Cov[X,YJ • Jim.,....-Bin(n,p) = Po(np) (n la.rge, p sme.11)
• Jim.,..._ Bin (n,p) = N (np, np(l - p)) (n large, p far from O and 1)
• Cov [t, x; ,t.Y;] = t.t.Cov[X,,Y;I Negative Bioomie.J
• X ~NBin(l,p)=Geo(p)
Correlation • X ~NBin(r,p) = I:;~1 Geo(p)
x YI= Oov [X,YJ • X, ~NBin(r,,p) ==> I;X, ~NBin(Er.,p)
I
P ' Jv [XIV JY] • X ~NBin(r,p). Y ~ Bin(s+ r,p) ==> P[X 5 •I= P[Y;,: rJ
Independence
Poisson
XlLY ==> p[X,YJ=O-=- Cov[X,YJ=O -=- E JXYJ =EJX] E[Y]
Sample variance
.x, ~ Po(A,) II X, lLX; ==> t,x, ~ Po (t. A;)
1 ~
S
2
n- 1 i:=l
- 2
= -L...J(X;-Xn) • x,~Po(A,)11X,1LX; ==> x, I:x;
n
j= l
~Bin
• (I:x;,E~
" A·
j=l
A;)
J=l J

Conditional varia.oce Exponentie.J

n
• v 1Y JX] = E [(Y -E 1Y 1xn• IX]= E [Y• IX] -E JY 1x1• • X, ~Exp(P) IIX, lLX; ==> I:x, ~Gamma(n,P)
• V[YJ =E[V[YJX]I +V JE[Y JX]l i=l
• Memoryless property: PIX > x + 11 IX > yJ = P JX > z]
6 Inequalities Norme.J
• X ~N(µ,u") ==> (x,7 ) ~/1/(0, 1)
CAUCHY-SCHWARZ
2
E [XYJ 5 E [X ] E [Y 2 2
] • X~N(p,u')11Z=aX+b ==> Z~N(aµ+b ,a2u2)
MARKOV
• X ~N (p, , crf) II Y ~N (J,2,u~) ==> X + Y ~N (}', +µ2,uf + ~)
p [cp(X) ;,: ti 5 E [cpiX)I • x, ~N (µ;,un ==> E,X, ~N (E,µ;, I;; ul)
• P[a<X 5b[=~(~)-~(7)
CBEBYSHEV
P IIX - E [XII f
;:: ti 5 v 1 • ~(-z) = 1 - ~(z) ,/1(:i:) = - z,/>(z)
• Upper quantile of N (0, 1): z 0 = ~-1(1 -
,/1'(:i:) = (z' - 1),/>(z)
o)
CHERNOFF Garruna
• X ~Garruna(at,P) -=- X/P~ Gamm&(o,l)
JENSEN
• Gamma ("'•P) ~ I:~=• Exp (P)
• X; ~ Gamma (o.,P) II X, lLX; ==> I:;,X, ~ Gamma(E,o.,P)
E [cp(X)J ;,: cp(E [XI) cp convex
r(at)
• --
Aa
-1- :t
o
a-1 - l , ,_
e u.:i;

7 D istribution Relationships Beta

Bioomie.J • 1 a-•(i - )fl- 1 = r(o+ P) a-•(i - )P- •
n
B( "'· p)"' "' r( at)rtp)"' "'
• X; ~ Bem(p) ==> LX; ~ Bin (n,p) • E[x•] = B(o+k,P) o+k-1 E[x•- •1
i=l
B(at, P) o+P+k - 1
• X ~ Bin (n,p),Y ~Bin(m,p) ==> X +Y ~Bin(n+m,p) • Beta(l, 1) ~ Unif(O,l)
 Som e special transformatio ns :
• If;'( ~ B cta l (,n , 11 ) then ( 1 - X ) ~ B cta l (n , 111.)
~
I JX Bcta2(·,n . 11 ) itlwn l ~X ~
13ctal (u1,11)
• If X Bctu .1 (111.11) then 1 :._'\ , rv Beh12(1n,11)
rv

• If X ~ ~
Garn1na(n., 8) and Y Ga1111na (rn, 8)
and are iuclcpendcnt then
U = ;~ ~Befo2 (11. 111 }. V = x '~ 1, ~Befa l (11, ·11i) and U,V are a lso indcpc11dent

• S11,ving X ~ Chi square ( n) is eq11ivtdent to 1-1Hy ing X ~ Gan1111a. ( " I)

2' 2 .
• If X ~ U11ifon11 (0, 1) lhc11 - 2l11 (X ) ~ C hi Sq11n.rc· (2) .
• If)( ~ Dctn l(n, 1) then - 211/11 (X ) ~ C h i square (2)
• If X ~ Exponeul'ial (111can = ,\) tbcn \I: ~ C hi Sq1111re (2)
~
• If X I ,. t.hcu )( ~
2
F1.,.
• If X ~ F,,, ·" then IU +"r,t.,x ~ Beta 1( •.-~•. '2' }
• If X ~ ~
F.,,., , ,, then ...r1x Bctit2(!!.!.
2 . !.!.)
i
• If )(, Y a rc iid U nifonn (0, 1) ra n<lu1n variables ,
Let U = J - 2lH( ..Y).'li11 (21rY) nn<l \/ = J - 2l11 (X )cu.. (21rY ).
Then U, \I fo llows Non11al(O, l ) aud 1-1rc indcpcndc11t .
 Probability a nd ~Ion1ent Generating Functions
• Gx(t) == B [tX] j,J < l

u., rt)= Gx(c' ) = E [· Xf] = B

•A [
(~t)']= E~ E (X•]
00

ET &!
-
•e
,=O •=0
• P (X = 0) == Gx (O)
• P (X == 1) = Gx (
0)
~ . c~1(o)
• P (.X = 1] = .,
=
• E ~\: ] C~'C(l-)
••
=
• E [-\ •) J1f) (0)
XI ] (t) _
• E [ (X _ k)! = G:c (1 )
• V(X) = G~-H•-) +G~1C11- ) -(G'x(1-))2
O
• Gx (t) = Gy lt) ==- X Y

Convergence
Let { X 1, X2 , •.• } be r~sequence of R\r's and let X be nnotber RV . Let F',. dBtote
the CDP of X n and let F dmote the CDF of X .

Types of oom-ergeuce

1. In distribution (wenkJ1~, in law): X,. ~ X

-~ lim F.,. (t) = F (t)

!.. In probability: X .. ~ X
Vt where F continuous

3. Alrrust surely (strongly): Xn ➔ X

P [ ~ X .. =X] =P (w E fJ : ~ Xn (w) =X(<..~)] = l

 4. In quadratic mean (Lo): Xn ~ X CLT notations

Relationships
.....
•Xn ➔ X = p
Xfl ➔ X = Xn ➔ X
0

eX0 ~X = Xfl ➔ X
eX0 ~X11 (3ceR)P (X=c]= l X 0 4X =
•X0 4XII Y. ➔ Y
•Xn ➔ XAYn ~Y ~ Xn+Yn~X+Y
=Xn+Y0 ➔ X+Y

•Xn2+XAYn4Y ~ X,,Yn.!+XY
• Xn-.!+ X
• Xn Ei X
=
= \O(Xn)-.!+ \O(X)
\O(Xn) Ei \O(X)
Continuity correction

• Xn ~ b ~ lim......-E (Xn] = b II l ~ - V (Xn( = 0

• X1, ... ,X0 UDII E(X] =µ /IV (X] < 00 ~ Xn :!;µ
SLOTZKY'S THEOREM

• X 0 EiX andY0 4 c =
= X 0 +Yn EiX +c
Delta method

• X 0 Ei X and Y 0 4 c X 0 Y 0 Ei cX
• In general: X • .E; X !Llld Yn .E, Y =,!> Xn + Yn ~ X +Y
Yft =N (µ. :) = IO(Yft) =N (\0(11).(,p'(µ))' : )

10.1 Law of Large Numbers (LLN)

Let {X1 , ..• ,X.} l>e o. sequeaoe of IID RV's, E (X1] = µ , and V (X1] < oo.
Weak(WLLN)
- p
Xn ➔ µ, n ➔ oo

Stroag (WLLN)

10.2 Central Limit Theorem (CLT)

Let {X1, ••. ,X.} l>e a sequeaoe of IID RV's, E (X1] = µ , and V (X1] = o-2.
 -.•. 0
I .:
i.
•

"
"ff"
•
•
f

.
8

l '·-.
, I .,
'
'
I

·~
0 •
II I

.~ I
•• •
0 •
~

I j 11i
f
. ..
0

• I

I •
8~
C
,
I i
z