Chapter 2

Elementary Probability Theory

2/29/2024 1
 Sample Space and Events
An Experiment: is some procedure (or process) that
we do and it results in an outcome.
A random experiment: is an experiment we do not
know its exact outcome in advance but we know the
set of all possible outcomes.
 The set of all possible outcomes of a statistical
experiment is called the sample space and is denoted
by S.
 Each outcome (element or member) of the sample
space S is called a sample point.
2/29/2024 2
 Events
An event A is a subset of the sample space S.
That is AS.
• We say that an event A occurs if the outcome
(the result) of the experiment is an element of
A.
 S is an event ( is called the impossible
event)
 SS is an event (S is called the sure
event)
2/29/2024 3
 Examples
 Experiment: Selecting a ball from a box containing 6
balls numbered 1,2,3,4,5 and 6. (or tossing a die)
• This experiment has 6 possible outcomes
 The sample space is S={1,2,3,4,5,6}.
 Consider the following events:
E1= getting an even number ={2,4,6}S
E2= getting a number less than 4={1,2,3}S
E3 = getting 1 or 3={1,3}S
E4 = getting an odd number={1,3,5}S
E5 = getting a negative number={ }= S
E6 = getting a number less than 10 =
{1,2,3,4,5,6}=SS
2/29/2024 4
 Cont’d
Notation:
n(S)= no. of outcomes (elements) in S.
n(E)= no. of outcomes (elements) in the event E.
Example:
• Experiment: Selecting 3 items from manufacturing
process; each item is inspected and classified as
defective (D) or non-defective (N).
• This experiment has 8 possible outcomes
S={DDD,DDN,DND,DNN,NDD,NDN,NND,NNN}
2/29/2024 5
 Cont’d

Consider the following events:

A={at least 2 defectives}= {DDD,DDN,DND,NDD}S
B={at most one defective}={DNN,NDN,NND,NNN}S
C={3 defectives}={DDD}S
2/29/2024 6
 Some Operations on Events
 Let A and B be two events defined on the sample
space S. S

Complement of The Event A:

Ac or A‘: Ac = {x S: xA }
 Ac consists of all points of S
that are not in A. Ac occurs if A does not.
S

• Intersection:
 AB =AB={x S: xA and xB}
 Consists of all points in both A and B.
 Occurs if both A and B occur together.
2/29/2024 7
Mutually Exclusive (Disjoint) Events
Two events A and B are mutually exclusive (or
disjoint) if and only if AB=; that is, A and B have
no common elements (they do not occur together).

AB   AB = 
A and B are not A and B are
mutually mutually
exclusive exclusive
(disjoint)

2/29/2024 8
 Cont’d
• Union: S

AB = {x S: xA or xB }

Consists of all outcomes in
A or in B or in both A and B.
 Occurs if A occurs,
or B occurs, or both A and B occur.
That is AB Occurs if at least one of
A and B occurs.

2/29/2024 9
 Counting Sample points
There are many counting techniques which can be
used to count the number points in the sample space
(or in some events) without listing each element.
 In many cases, we can compute the probability of
an event by using the counting techniques.
1. Combinations:
• In many problems, we are interested in the number
of ways of selecting r objects from n objects without
regard to order. These selections are called
combinations.
2/29/2024 10
 Counting Sample points

Notation:
n factorial is denoted by n! and is defined by:
n! n  n  1 n  2  2 1 for n  1, 2, 
0! 1
Example:

Theorem :
5! = 5 ∗ 4 ∗ 3 ∗ 2 ∗ 1 = 120
The number of combinations of n distinct objects taken r at a time is denoted by
and is given by:
n
 
r
n n!
   ; r  0, 1, 2, , n
 r  r ! n  r !
2/29/2024 11
 Cont’d
• Example:
If we have 10 equal–priority operations and only 4
operating rooms are available, in how many ways can
we choose the 4 patients to be operated on first?
• Solution: n = 10 r=4
The number of different ways for selecting 4
patients from 10 patients is
10  10! 10! 10  9  8  7  6  5  4  3  2  1
    
 4  4! 10  4! 4!  6! 4  3  2  1  6  5  4  3  2  1
 210 (different ways)

2/29/2024 12
Probability of an Event
• To every point (outcome) in the sample space of an
experiment S, we assign a weight (or probability),
ranging from 0 to 1, such that the sum of all weights
(probabilities) equals 1.
The weight (or probability) of an outcome measures
its likelihood (chance) of occurrence.
To find the probability of an event A, we sum all
probabilities of the sample points in A. This sum is
called the probability of the event A and is denoted
by P(A).
2/29/2024 13
 Cont’d
 Definition:
The probability of an event A is the sum of the weights
(probabilities) of all sample points in A. Therefore,
1. 0 ≤ 𝑃 𝐴 ≤ 1.
2. 𝑃 𝑆 = 1.
3. 𝑃 ∅ = 0.
• Example :
A balanced coin is tossed twice. What is the
probability that at least one head occurs?
• Solution: S = {HH, HT, TH, TT}
2/29/2024 14
 Cont’d
• A = {at least one head occurs}= {HH, HT, TH}
Since the coin is balanced, the outcomes are equally
likely; i.e., all outcomes have the same weight or
probability.

2/29/2024 15
 Cont’d
The probability that at least one head occurs is:
P(A) = P({at least one head occurs})
= P(HH) + P(HT) + P(TH)
= 0.25+0.25+0.25
= 0.75
• Theorem:
If an experiment has n(S)=N equally likely different
outcomes, then the probability of the event A is:
n( A) n( A) no. of outcomes in A
P( A)   
n( S ) N no. of outcomes in S
2/29/2024 16
 Cont’d
• Example:
A mixture of candies consists of 6 mints, 4 toffees, and
3 chocolates. If a person makes a random selection of
one of these candies, find the probability of getting:
(a) a mint?
(b) a toffee or chocolate?
• Solution: Define the following events:
M = {getting a mint}
T = {getting a toffee}
C = {getting a chocolate}
2/29/2024 17
 Cont’d
• Experiment: selecting a candy at random from 13 candies,
n(S) = no. of outcomes of the experiment of selecting a candy.
= no. of different ways of selecting a candy from 13 candies.
13 
    13
1
• The outcomes of the experiment are equally likely because
the selection is made at random.
(a) M = {getting a mint}
n(M) = no. of different ways of selecting a mint candy
from 6 mint candies
 6
    6
1

2/29/2024 18
 Cont’d
n M  6
P(M )= P({getting a mint})= 
nS  13
(b) TC = {getting a toffee or chocolate}
n(TC) = no. of different ways of selecting a toffee
or a chocolate candy
= no. of different ways of selecting a toffee candy
+ no. of different ways of selecting a chocolate candy
 4  3
 
1     4  3  7
  1
= no.of different ways of selecting a candy from 7 candies
7

1
7
 
P(TC )= P({getting a toffee or chocolate})=

2/29/2024 19
 Cont’d
• Example :
In a poker hand consisting of 5 cards, find the probability of
holding 2 aces and 3 jacks.
• Solution:
• Experiment: selecting 5 cards from 52 cards.
n(S) = no. of outcomes of the experiment of selecting 5 cards
from 52 cards.
 52  52!
 
 5  5!  47!  2598960

 

• The outcomes of the experiment are equally likely because

the selection is made at random.
2/29/2024 20
 Cont’d
• Define the event A = {holding 2 aces and 3 jacks}
n(A) = no. of ways of selecting 2 aces and 3 jacks
= (no. of ways of selecting 2 aces)  (no. of
ways of selecting 3 jacks)
= (no. of ways of selecting 2 aces from 4 aces)  (no.
of ways of selecting 3 jacks from 4 jacks)

n A  24
   0.000009
P(A ) n S 2598960

2/29/2024 21
 Conditional probability
Definition:
Conditional probability is, the probability that an event will
occur based on the condition that another event has occurred.
 The conditional probability of B given A is the probability
that event B occurs, given that event A has already occurred.
• If A and B are two events from a sample space with P(A) ≠
0, then the conditional probability of B given A,
denoted by , has two equivalent expressions:

2/29/2024 22
 Cont’d
𝑛𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑜𝑢𝑡𝑐𝑜𝑚𝑒𝑠 𝑖𝑛(𝐴 𝑎𝑛𝑑 𝐵)
𝑃 𝐵𝐴 =
𝑛𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑜𝑢𝑡𝑐𝑜𝑚𝑒𝑠 𝑖𝑛 𝐴
 This uses subsets of the sample space and,
𝑃(𝐴 𝑎𝑛𝑑 𝐵)
𝑃 𝐵𝐴 =
𝑃(𝐴)
 This uses the calculated probabilities P A and B and P A .
• The second formula can be rewritten as
𝑃 𝐴 𝑎𝑛𝑑 𝐵 = 𝑃 𝐴 ∗ 𝑃(𝐵)
𝑃 𝐵 𝐴 is read as “the probability of B given A.”
• Using set notation, conditional probability is written like
this:
2/29/2024 23
 Cont’d
𝑃(𝐴 ∩ 𝐵)
𝑃 𝐵𝐴 =
𝑃(𝐴)
The “conditional probability of B given A” only has
meaning if event A has occurred. That is why the formula
for 𝑃 𝐵 𝐴 has the requirement that P(A) ≠ 0.
• Example:
The academy awards is soon to be shown.
For a specific married couple the probability that the husband
watches the show is 80%, the probability that his wife watches
the show is 65%, while the probability that they both watch
the show is 60%.

2/29/2024 24
 Cont’d
• If the husband is watching the show, what is the probability
that his wife is also watching the show
• Solution:
The academy awards is soon to be shown.
Let B = the event that the husband watches the show
P[B]= 0.80
Let A = the event that his wife watches the show
P[A]= 0.65 and P[A ∩ B]= 0.60
P  A  B 0.60
P  A B     0.75
P  B 0.80
2/29/2024 25
 Independent Events
• Definition:
Two events A and B are called independent if

P  A  B  P  A P  B
if P  B  0 and P  A  0 then
Note:

P  A  B P  A P  B 
P  A B     P  A
P  B P  B
P  A  B P  A P  B 
and P  B A    P  B
P  A P  A

2/29/2024 26
 Cont’d
Thus in the case of independence the conditional probability
of an event is not affected by the knowledge of the other event.
The multiplicative rule of probability
 P  A P  B A if P  A  0
P  A  B  
 P  B  P  A B  if P  B   0
and
P  A  B  P  A P  B if A and B are independent.

2/29/2024 27
 Example
 A coin is flipped and a die is rolled. Find the probability of
getting a head on the coin and a 4 on the die.
Answer: 𝑃(𝐻) = 1/2, 𝑃 4 = 1/6
P(H and 4)  P(H )  P(4)
1 1 1
P( H and 4)   
2 6 12
Exercise!
1.A box contains 3 red balls, 2 blue balls, and 5 white balls.
A ball is selected and its color noted. Then it is replaced. A
2nd ball is selected and its color noted. Find the probability of
a. Selecting 2 blue balls
b. Selecting a blue ball and then a white ball
c. 2/29/2024
Selecting a red ball and then a blue ball 28
 Cont’d
2. A poll found that 46% of Americans say they suffer from
stress. If 3 people are selected at random, find the probability
that all three will say they suffer from stress.
3. A card is drawn from a deck and replaced; then a 2nd card
is drawn. Find the probability of getting a queen and then an
ace.

2/29/2024 29
Dependent Events
 When the outcome or occurrence of the first event affects
the outcome or occurrence of the second event in such a
way that the probability is changed.
Examples of Dependent Events
1. Draw a card from a deck. Do not replace it and draw
another card.
2. Having high grades and getting a scholarship
3. Parking in a no parking zone and getting a ticket
 When 2 events are dependent, the probability of both
occurring is
P( A and B)  P( A)  P( BlA)
2/29/2024 30
 Cont’d
 The slash reads:
“The probability that B occurs given that A has already
occurred.”
Example 1:
53% of residents had homeowner’s insurance. Of these,
27% also had car insurance. If a resident is selected at
random, find the prob. That the resident has both
homeowner’s and car insurance.
Answer:
P(H and C)  P(H )  P(ClH )
P(H and C)  (.53)(.27)  .1431

2/29/2024 31
 Cont’d
Example2:
3 cards are drawn from a deck and NOT replaced. Find the
following probabilities.
a. Getting 3 jacks
b. Getting an ace, king, and queen
c. Getting a club, spade, and heart
d. Getting 3 clubs.
Answer:
a. Getting 3 jacks
4 3 2 1
P( J and J and J )      .000181
52 51 50 5525

2/29/2024 32
 Cont’d
b. Getting an ace, king, queen
4 4 4 8
P( A and K and Q)      .000483
52 51 50 16575

c. Getting a club, spade, and heart

13 13 13 169
P(C and S and H )      .017
52 51 50 10200

d. Getting 3 clubs
13 12 11 11
P(C and C and C )     or .013
52 51 50 850
2/29/2024 33
 Cont’d
Exercise!
(1)There are 6 black socks and 4 white socks in a
drawer. If one sock is taken out without looking and
then a second one is taken out, what is the probability
that they both will be black?
(2) A card is drawn from a deck of 10 cards numbered
1 through 10 and a number cube is rolled. Find the
probability of each below.
a) P(10 and 3)
b) P(two even numbers)
c) P(two numbers less than 4)
2/29/2024 34
 Random Variables and Expectations
A random variable x takes on a defined set of
values with different probabilities.
• Example1: if you roll a die, the outcome is
random (not fixed) and there are 6 possible
outcomes, each of which occur with probability
one-sixth.
• Example2: if you poll people about their
voting preferences, the percentage of the
sample that responds “Yes on Proposition 100”
is a also a random variable (the percentage will
be slightly differently every time you poll).

2/29/2024 35
 Random Variables
 Random variables can be either discrete or continuous
• Discrete random variables have a countable
number of outcomes
Examples: Dead/alive, treatment/placebo, dice,
counts, etc.
• Continuous random variables have an infinite
continuum of possible values.
Examples: blood pressure, weight, the speed of a
car, the real numbers from 1 to 6.

2/29/2024 36
 Probability functions
A probability function maps the possible values of x
against their respective probabilities of occurrence,
p(x)
• p(x) is a number from 0 to 1.0.
• The area under a probability function is always 1.
• Discrete example: roll of a die

 P(x)  1
all x

That is,
2/29/2024 37
 Probability mass functions (pmf)
x p(x)
1 p(x=1)=1/6

2 p(x=2)=1/6

3 p(x=3)=1/6

4 p(x=4)=1/6

5 p(x=5)=1/6

6 p(x=6)=1/6
2/29/2024 38
1.0
 Cumulative distribution function (CDF)

P(x)
1.0
5/6
2/3
1/2
1/3

1/6

1 2 3 4 5 6 x

2/29/2024 39
 Cumulative distribution function (CDF)

x P(x≤A)
1 P(x≤1)=1/6

2 P(x≤2)=2/6

3 P(x≤3)=3/6

4 P(x≤4)=4/6

5 P(x≤5)=5/6

6 P(x≤6)=6/6
2/29/2024 40
 Example
The number of patients seen in the ER in any given hour is a
random variable represented by x. The probability
distribution for x is:
x 10 11 12 13 14
P(x) .4 .2 .2 .1 .1

Find the probability that in a given hour:

a. exactly 14 patients arrive
b. At least 12 patients arrive
c. At most 11 patients arrive

2/29/2024 41
 Exercises
1. If you toss a die, what’s the probability that you roll a 3 or
less?
2. Two dice are rolled and the sum of the face values is six?
What is the probability that at least one of the dice came
up a 3?
3. Two dice are rolled and the sum of the face values is six.
What is the probability that at least one of the dice came
up a 3?

2/29/2024 42
 Continuous case
The probability function that accompanies a
continuous random variable is a continuous
mathematical function that integrates to 1.
 For example, recall the negative exponential function
(in probability, this is called an “exponential
distribution”): f ( x)  e x
 This function integrates to 1:
 


0
e  x  e  x
0
 0 1 1

2/29/2024 43
 probability density function “(pdf)”
 The following is the graph of the function:

p(x)=e-x

 The probability that x is any exact particular value (such as

1.9976) is 0; we can only assign probabilities to possible
ranges
2/29/2024 of x. 44
 Example1
• Clinical example: Survival times after lung transplant
may roughly follow an exponential function.
• Then, the probability that a patient will die in the
second year after surgery (between years 1 and 2) is
23%. -x p(x)=e

x
1 2
2 2


x x
P(1  x  2)  e  e  e 2  e 1  .135  .368  .23
1
1
2/29/2024 45
Example 2: Uniform distribution
 The uniform distribution: all values are equally likely.
f(x)= 1 , for 1 x 0.
p(x)

x
1

 We can see it’s a probability distribution because it

integrates to 1 (the area under the curve is 1):
1 1

1  x
0
0
1 0 1

2/29/2024 46
 Expected Value and Variance
 All probability distributions are characterized by an
expected value (mean) and a variance (standard deviation
squared).
• Expected value is just the average or mean (µ) of random
variable x.
• It’s sometimes called a “weighted average” because more
frequent values of X are weighted more highly in the
average.
• It’s also how we expect X to behave on-average over the
long run (“frequentist” view again).

2/29/2024 47
 Expected value, formally
Discrete case:

E( X )   x p(x )
all x
i i

Continuous case:

E( X )  
all x
xi p(x i )dx

2/29/2024 48
 Symbol Interlude
E(X) = µ
these symbols are used interchangeably
Example: expected value
Recall the following probability distribution of ER
arrivals:
x 10 11 12 13 14
P(x) .4 .2 .2 .1 .1

x
i 1
i p ( x)  10(.4)  11(.2)  12(.2)  13(.1)  14(.1)  11.3
2/29/2024 49
 Cont’d
Sample Mean is a special case of Expected Value.
Sample mean, for a sample of n subjects:
n

x i n


i 1 1
X  xi ( )
n i 1 n
The probability (frequency) of each person in the sample is
1/n.
 Expected value is an extremely useful concept for good
decision-making!

2/29/2024 50
 Variance
2 = 𝑉𝑎𝑟(𝑥) = 𝐸(𝑥 − 𝜇)2
“The expected (or average) squared distance (or
deviation) from the mean”
  Var ( x)  E[( x   ) ] 
2 2
 (x
all x
i   ) p(x i )
2

 Discrete case:
Var ( X )   (x
all x
i   ) p(x i )
2

 Continuous case:
Var ( X )   ( xi   ) p(x i )dx
2

2/29/2024 all x 51