Module 17

INEQUALITIES

() Module 17 INEQUALITIES 1 / 17
X : a given r.v. with d.f. FX (·) and p.m.f/p.d.f. fX (·);
g : R → R: a given function;
Inequalities provide useful estimates of probabilities (or moments)
when they can not be evaluated precisely;
In this module we will derive some useful inequalities.

() Module 17 INEQUALITIES 1 / 17
Result 1:
Let g : R → [0, ∞) be a non-negative function such that E (g (X )) < ∞.
Then, for any c > 0,
E (g (X ))
P({g (X ) > c}) ≤ .
c

Proof: (For A.C. Case.) Let A = {x ∈ R : g (x) > c}. Then

Z ∞
E (g (X )) = g (x)fX (x)dx
−∞
Z Z
= g (x)fX (x)dx + g (x)fX (x)dx
ZA Ac

≥ g (x)fX (x)dx (g (·) ≥ 0, fX (·) ≥ 0)

ZA∞
= g (x)IA (x)fX (x)dx
−∞

() Module 17 INEQUALITIES 2 / 17
 Z
≥ c fX (x)dx (g (x)IA (x) ≥ cIA (x), ∀ x ∈ R)
A

= c P(A)

= c P({g (X ) > c})

E (g (X ))
⇒ P ({g (X ) > c}) ≤ .
c
Notation: For any set B ⊆ R and any integrable function h(·)
Z Z ∞
h(x)dx = h(x)IB (x)dx.
B −∞

() Module 17 INEQUALITIES 3 / 17
Corollary 1:
Let g : [0, ∞) → [0, ∞) be a non-negative and strictly ↑ function such
that E (g (|X |)) < ∞. Then for any c > 0 such that g (c) > 0
E (g (|X |))
P({|X | > c}) ≤ .
g (c)

Proof:
P({|X | > c}) = P({g (|X |) > g (c)})
E (g (|X |))
≤ . (using Result 1)
g (c)

Corollary 2: Let r > 0 and c > 0. Then

E (|X |r )
P({|X | > c}) ≤ ,
cr
provided E (|X |r ) < ∞.
() Module 17 INEQUALITIES 4 / 17
Corollary 3 (Markov Inequality): Suppose that E (|X |) < ∞. Then
E (|X |)
P({|X | > c}) ≤ .
c
Proof: Take r = 1 in Corollary 2.

Result 2 (Chebyshev Inequality): Let X be a r.v. with finite mean

µ = E (X ) and finite variance σ 2 = E ((X − µ)2 ). Then for any  > 0
1
P ({|X − µ| > σ}) ≤
2
or equivalently
1
P ({|X − µ| ≤ σ}) ≥ 1 − .
2
Proof: Using Corollary 2 for r = 2, we have
E (|X − µ|2 )
P ({|X − µ| > σ}) ≤
2 σ 2
Var (X ) 1
= 2 2
= 2.
 σ 
() Module 17 INEQUALITIES 5 / 17
Remark 1:

For any probability distribution

1
P ({µ − 2σ ≤ X ≤ µ + 2σ}) ≥ 1 − = 0.75;
22
1
P ({µ − 3σ ≤ X ≤ µ + 3σ}) ≥ 1 − ≥ 0.88.
32

() Module 17 INEQUALITIES 6 / 17
Example 1 (Chebyshev’s bound is sharp):
Let X be a r.v. with p.m.f.
1


 8, if x ∈ {−1, 1}



3
fX (x) = 4, if x = 0 .




0, otherwise


Then
1 1 3
µ = E (X ) = × −1 + × 1 + × 0 = 0;
8 8 4
1 1 3 1
σ 2 = Var (x) = E (X 2 ) = × (−1)2 + × 12 + × 0 = .
8 8 4 4
199
For  = 100 , Chebyshev inequality gives the following inequality
1002
P ({|X − µ| > σ}) ≤ ≈ 0.25252.
1992
() Module 17 INEQUALITIES 7 / 17
Actual probability is
 
199
P ({|X − µ| > σ}) = P |X | >
200

= P ({X ∈ {−1, 1}})

1
= = 0.25.
4

() Module 17 INEQUALITIES 8 / 17
Example 2:
Let X be a r.v. with p.d.f.
 1
√ √
 √
2 3
, if − 3<x < 3
fX (x) = .

0, otherwise

Then √
Z 3
x
µ = E (X ) = √ √ dx = 0;
− 3 2 3
√
Z 3
2 2 1
σ = E (X ) = √ x 2 × √ dx = 1.
− 3 2 3
3
Chebyshev inequality for  = gives
2
 
3 4
P |X | > ≤ = 0.444 · · · .
2 9

() Module 17 INEQUALITIES 9 / 17
Actual probability is
   
3 3 3
P |X | > = 1−P − ≤X ≤
2 2 2
Z 3
2 1
= 1− √ dx
−3 2 3
√2
3
= 1− = 0.134 · · ·
2
Here the Chebyshev bound is not that sharp.

() Module 17 INEQUALITIES 10 / 17
Definition 1: Let −∞ ≤ a < b ≤ ∞. A function φ : (a, b) → R is said to
be convex (concave) on (a,b) if

φ(αx + (1 − α)y ) ≤ (≥) αφ(x) + (1 − α)φ(y ), ∀ x, y ∈ (a, b), α ∈ (0, 1).

The function φ is said to be strictly convex (strictly concave) if the above

inequality is strict.
We state the following standard result without providing its proof.
Result 3: Let φ : (a, b) → R be a given function.
(a) Then, φ is convex ⇔ −φ is concave;
(b) Let φ be differentiable on (a,b) and let φ0 denote the
derivative function . Then φ is convex (concave) on (a, b) iff
φ0 is ↑ (↓) on (a, b);
(c) Let φ be twice differentiable on (a, b) and let φ00 denote the
second derivative function. Then φ is convex (concave) on
(a, b) iff φ00 (x) ≥ (≤ ) 0, ∀ x ∈ (a, b).

() Module 17 INEQUALITIES 11 / 17
Result 4 (Jensen Inequality):
Let X be a r.v. with support SX ⊆ (a, b) and let φ : (a, b) → R be a
convex (concave) function; here −∞ ≤ a < b ≤ ∞. Then
E (φ(X )) ≥ (≤) φ(E (X )),
provided the expectations exist.

Proof: For simplicity assume that φ is differentiable on (a, b). Then φ0 ↑

on (a, b) and for x ∈ SX (note that µ = E (X ) ∈ (a, b))
φ(x) = φ(µ) + (x − µ)φ0 (ξ),
for some ξ between x and µ. Since φ0 ↑ it follows that
(x − µ)φ0 (ξ) ≥ (x − µ)φ0 (µ). Therefore
φ(x) ≥ φ(µ) + (x − µ)φ0 (µ), ∀ x ∈ (a, b)
⇒ φ(X ) ≥ φ(µ) + (X − µ)φ0 (µ)
⇒ E [φ(X )] ≥ φ(µ) = φ(E (X )).
() Module 17 INEQUALITIES 12 / 17
Example 3

(a) E (|X |) ≥ |E (X )|, (φ(x) = |x| is convex on R);

(b) E (X 2 ) ≥ (E (X ))2 , (φ(x) = x 2 is convex on R);
(c) If P(X ≥ 0) = 1 and r > 1, then E (X r ) ≥ (E (X ))r (for
r > 1, φ(x) = x r is convex on [0, ∞));
(d) If P(X > 0) = 1 and r < 1, then E (X r ) ≤ (E (X ))r (for
r < 1, φ(x) = x r is concave on (0, ∞));
(e) If P(X > 0) = 1, then E (ln X ) ≤ ln E (X ) (φ(x) = ln x is
concave on (0, ∞));
(f) E (e X ) ≥ e E (X ) ; (φ(x) = e x is convex on R).

() Module 17 INEQUALITIES 13 / 17
Example 4:

For 0 < p < q < ∞, show that

1 1
(E (|X |p )) p ≤ (E (|X |q )) q .

Solution: Using Example 3 (c)

q q
E (|Y | p ) ≥ (E (|Y |)) p
q
⇒ (E (|X |q )) ≥ (E (|X |p )) p (taking Y = X p )
1 1
⇒ (E (|X |q )) q ≥ (E (|X |p )) p .

Remark 2: If E (|X |q ) < ∞ for some q > 0, then

E (|X |p ) < ∞, ∀ 0 < p < q.

() Module 17 INEQUALITIES 14 / 17
Take Home Problems

(1) Suppose that P(X > 0) = 1. Show that

1
E (X )E ( ) ≥ 1.
X
(2) Let ωi > 0, ai > 0, i = 1, . . . , n and let ni=1 ωi = 1. Show
P
that
n n
X Y 1
ai ωi ≥ aiωi ≥ Pn ωi .
i=1 i=1 i=1 ai

(AM ≥ GM ≥ HM).

() Module 17 INEQUALITIES 15 / 17
Abstract of Next Module

We will introduce a set of numerical measures that provide a

summary of prominent features of a probability distribution.

() Module 17 INEQUALITIES 16 / 17
 Thank you for your patience

() Module 17 INEQUALITIES 17 / 17