Understanding Conditional Probability,

Total Probability, and Bayes’ Theorem

Abstract

This project explores three fundamental concepts in probability theory: conditional

probability, the theorem of total probability, and Bayes' theorem. The paper aims to explain
these concepts in simple terms, illustrate them with real-life examples, and demonstrate
their applications in various fields like medicine, business, and machine learning. By
understanding these concepts, one can gain a deeper understanding of how probability
theory applies to decision-making and prediction in uncertain situations.

Introduction to Probability and Its Concepts

Probability is the branch of mathematics that deals with the likelihood or chance of
different outcomes. You encounter it every day, from predicting the weather to rolling dice.
In this paper, we focus on three major probability concepts that help us deal with
uncertainty:
1. Conditional Probability
2. Theorem of Total Probability
3. Bayes’ Theorem

Conditional Probability - Definition and Concept

Conditional probability refers to the probability of an event occurring, but only when we
know that another event has already occurred. It's like changing the rules of a game because
of new information. For example, imagine you’re at a party, and you see a group of people
laughing. If I tell you that this group of people is a group of comedians, then your
expectation of the probability that they’re laughing increases a lot, right? You’d think there’s
a much higher chance that they’re laughing because the...

Mathematically, the formula for conditional probability looks like this:

P(A|B) = P(A ∩ B) / P(B)

Here:
- P(A|B) is the probability of event A happening, given that event B has already occurred.
- P(A ∩ B) is the probability of both A and B happening together.
- P(B) is the probability that event B happens.

The key takeaway here is that the probability of A changes depending on the knowledge of
B. This idea is super useful when you're analyzing situations where the occurrence of one
event affects the likelihood of another.

Understanding Conditional Probability with an Example

Example: Exam Performance Based on Study Time

Consider the following situation:

- Event A: Scoring above 80% on an exam.
- Event B: Studying for more than 10 hours.

We know that the probability of scoring above 80% (event A) is likely higher if you studied
for more than 10 hours (event B). So, in this case, we can say that the chances of event A
happening depend on whether event B (studying for 10+ hours) has occurred. By using
conditional probability, we can express how event A’s chances change given event B.

If we didn’t know about the study time, we might assume a lower chance of scoring well, but
once we know that someone studied for a long time, our estimate of the chances of them
scoring well increases. Conditional probability helps us adjust our expectations with new
information.

Properties of Conditional Probability

1. Multiplication Rule for Probability

The multiplication rule is a key property of conditional probability. It helps you calculate
the probability of two events happening at the same time. The formula for it is:

P(A ∩ B) = P(A|B) × P(B)

In simple terms, this means that the probability of both events happening is the probability
of B happening, multiplied by the probability of A happening given B. This rule is super
important because it lets us calculate joint probabilities when we have conditional
probabilities.
2. Law of Total Probability

Another important rule in probability theory is the law of total probability, which we'll get
into in the next section. It allows you to calculate the overall probability of an event by
breaking it down into simpler, conditional probabilities. This is incredibly helpful in
situations where you know there are multiple possible ways an event can happen, and you
want to combine all the possible scenarios to get a total probability.

Theorem of Total Probability - Definition

The theorem of total probability helps us find the probability of an event by considering all
the possible ways that event can happen. It's like solving a puzzle where each piece
represents a different way the event can occur. To calculate the total probability, we
consider all these pieces and add them up.

The formula looks like this:

P(A) = Σ P(A|B_i) P(B_i)

Here:
- P(A|B_i) is the conditional probability of event A, given event B_i.
- P(B_i) is the probability of event B_i happening.

This is really useful when you know that an event can happen in multiple ways (for
example, different age groups, different weather conditions, etc.), and you need to combine
the probabilities of all those ways to find the total chance of event A happening.

Applying the Theorem of Total Probability

Example: Customer Demographics and Purchase Behavior

Let’s take a closer look at how the theorem of total probability works in the real world.
Imagine you work at a store and want to know the probability of a customer buying
something. But customers are different: some are young, some are middle-aged, and some
are older. Each group is more or less likely to make a purchase, so the probability isn’t the
same for everyone.

Here’s the setup:

- Event A: A customer makes a purchase.
- B1: A customer is young.
- B2: A customer is middle-aged.
- B3: A customer is elderly.

We know:
- The probability of a young customer making a purchase is 40% (P(A|B_1) = 0.4).
- The probability of a middle-aged customer making a purchase is 50% (P(A|B_2) = 0.5).
- The probability of an elderly customer making a purchase is 30% (P(A|B_3) = 0.3).

Also, the chances of a customer belonging to each group are:

- P(B_1) = 0.3 (30% of customers are young)
- P(B_2) = 0.4 (40% are middle-aged)
- P(B_3) = 0.3 (30% are elderly)

To find the total probability of a customer making a purchase, we use the formula for total
probability:

P(A) = (0.4 × 0.3) + (0.5 × 0.4) + (0.3 × 0.3) = 0.12 + 0.20 + 0.09 = 0.41

So, the total probability that any given customer will make a purchase is 41%. This shows
how we break down the problem into smaller pieces (different age groups), and then
combine those pieces to find the total probability.

Bayes’ Theorem - Introduction

Bayes’ theorem allows us to update the probability of an event as we get new evidence. It is
an essential tool when we have prior knowledge and need to adjust our beliefs based on
new data. The formula for Bayes’ theorem is:

P(A|B) = P(B|A) P(A) / P(B)

Where:
- P(A|B) is the updated probability of event A happening, given the new evidence B.
- P(B|A) is the likelihood: the probability of getting evidence B, given that event A is true.
- P(A) is your prior belief about the probability of A (before you knew about B).
- P(B) is the total probability of getting evidence B.

Deriving Bayes’ Theorem

Bayes' theorem is based on the idea of conditional probability. If you know how two events
are related (through conditional probability), you can rearrange the formulas to find the
reverse relationship. This allows you to update your beliefs when new evidence comes in.

Starting with the definition of conditional probability, we know:

P(A|B) = P(A ∩ B) / P(B)

Now, we can break down the intersection P(A ∩ B) using the multiplication rule:

P(A ∩ B) = P(B|A)P(A)

Substituting this back into the original equation, we get:

P(A|B) = P(B|A)P(A) / P(B)

This gives us Bayes’ theorem, which lets us calculate the updated probability of something
happening, given new information. It’s like using new facts to adjust your old estimates.

Example of Bayes’ Theorem in Action

Example: Medical Diagnosis

Let’s say you take a medical test for a rare disease. The test has:
- A 99% chance of being correct if you have the disease.
- A 95% chance of being correct if you don’t have the disease.
- The disease is rare, so only 1% of people have it.

You get a positive result. What’s the chance you actually have the disease?

Using Bayes’ theorem, we can calculate the updated probability of having the disease. With
the data above, we can apply the formula to adjust the probability of having the disease
based on the test result.

Conclusion

This project has provided a detailed examination of three essential concepts in probability:
Conditional Probability, the Theorem of Total Probability, and Bayes’ Theorem. These
concepts are powerful tools used to make informed decisions in situations involving
uncertainty. From predicting weather patterns to medical diagnoses, these concepts are
fundamental in applying probability theory to real-world problems.
Bibliography

1. Feller, W. (1971). An Introduction to Probability Theory and Its Applications.

2. Ross, S. M. (2010). Introduction to Probability and Statistics for Engineers and Scientists.
3. Bishop, C. M. (2006). Pattern Recognition and Machine Learning.