Random Signal Analysis

ECC504
Dr. Ranjan bala jain
Professor
Department of EXTC
VESIT
W hen you
E nter this class
L earning is Fun And
C ooperation is Expected
O ur Positive Attitude and
M utual respect are part of
E verything We do and Say....

In the world of Random Signal Analysis !!!

Module 1: Basic Concepts in
Probability
1. Definitions of probability
2. Joint, conditional, and total probability,
Bayes’ theorem
3. Independence of events
4. Binary symmetric communication
channel analysis using Bayes’ theorem.
Introduction :
Continuous Iteration between Model & System

Deterministic &
probabilistic models
Mathematical Models
● To study some observational phenomena, a
mathematical model is required.
● Mathematical models can be divided into two

types:
(i) Deterministic model, and
(ii) Probabilistic model.
Introduction : Deterministic model
● A deterministic model can be used for a physical quantity and
the process generating it provided sufficient information is
available about the initial state and the dynamics of the process
generating the physical quantity.
For example,
● We can determine the position of a particle moving under a
constant force if we know the initial position of the particle and
the magnitude and the direction of the force.
● We can determine the current in a circuit consisting of
resistance, inductance and capacitance for a known voltage
source applying Kirchoff's laws.
Introduction :

Noise Variables
affect
Transformation
Of inputs to
Outputs.
Introduction :

Deviation in the Model

 On observing a cloudy sky we say that ● Introduction :
there is a chance of rain. At the end of the
day, it may rain or may not. Probabilistic
model
In a cricket match between two teams, we ● That are very
often say that the chance of winning for
one team is more than that of the other much close to
team
chance are
The total number of marks you score in probably,
the University examination is a random
variable until the results are declared. likely, etc.

The age of a friend whose birth date

you do not know is random.
That are very much close to chance are probably,
likely
 Introduction : Probabilistic model
• On observing a cloudy sky we say that there is a chance of rain. At
the end of the day, it may rain or may not.
• In a cricket match between two teams, we often say that the chance
of winning for one team is more than that of the other team.
• The total number of marks you score in the University
examination is a random variable until the results are declared.
• The age of a friend whose birth date you do not know is random.
That are very much close to chance are probably, likely,
etc.
Introduction : Probabilistic model
● Many of the physical quantities are random in the sense that
these quantities cannot be predicted with certainty and can be
described in terms of probabilistic models only. For example,
● The outcome of the tossing of a coin cannot be predicted with
certainty. Thus the outcome of tossing a coin is random.
● The number of ones and zeros in a packet of binary data
arriving through a communication channel cannot be precisely
predicted is random.
● The ubiquitous noise corrupting the signal during acquisition,
storage and transmission can be modelled only through
statistical analysis.
Introduction : Probabilistic model
● The number of particles emitted by a radioactive material
cannot be predicted precisely during a specified time interval
even if we know the exact shape, dimensions and chemical
composition of the material.
● That is, there is no mathematical relation that gives a direct
relationship between the number of particles emitted and
characteristics of radioactive material. Hence, there is no
certainty in predicting the emitted number of particles. Hence,
we call the event as random. In this case, we must consider a
probabilistic model.
● The output of a solar detector which depends on weather
 •• is one that may be represented by
Deterministic parameter values, such as a sinusoid,
signal
which may be perfectly reconstructed
given an amplitude, frequency, and
phase .

•• Such as noise, do not have this property.

Stochastic/ While they may be approximately
represented by several parameters,
Random stochastic signals have an element of
signals randomness that prevents them from
being perfectly reconstructed from a
past history.
Introduction : Probabilistic model
● A deterministic signal is one that may be represented
by parameter values, such as a sinusoid, which may
be perfectly reconstructed given an amplitude,
frequency, and phase.
● Stochastic/random signals, such as noise, do not have
this property. While they may be approximately
represented by several parameters, stochastic signals
have an element of randomness that prevents them
from being perfectly reconstructed from a past
history.
How to Interpret Probability
• The word probability literally means chance, a very
commonly used word in day-to-day conversation.
• These terms are used in the context when there is
uncertainty in the outcome of an event.
• Probability is a mathematical measure for uncertainty,
likelihood or chance.
• A mean of evaluating the uncertainty, likelihood and
chance of outcome resulting from a statistical
experiment is called theory of probability.
How to Interpret Probability
● Mathematically, the probability that an event will occur is
expressed as a number between 0 and 1.
● Notationally, the probability of event A is represented by P (A).
● If P (A) equals zero, event A will almost definitely not occur.
● If P (A) is close to zero, there is only a small chance that event A
will occur.
● If P (A) equals 0.5, there is a 50-50 chance that event A will occur.
● If P(A) is close to one, there is a strong chance that event A will
occur.
● If P(A) equals one, event A will almost definitely occur.
How to Interpret Probability
● In a statistical experiment, the sum of
probabilities for all possible outcomes is
equal to one.
● This means, for example, that if an
experiment can have three possible outcomes
(A, B, and C), then
● P(A) + P(B) + P(C) = 1.
Quiz
● The probability of any event is ???
● Mathematically, the probability that an event will occur is expressed as
a number between 0 and 1.
Quiz
Which of the following uses Probability theory ?
● Electron emission, Telephone calls,
● Radar detection, Quality control,
● Select a ball from an urn containing balls numbered 1
to 50. Note the number of the ball.
● System failure, Games of chance,
● Statistical mechanics, Turbulence, Noise,
● Birth and Death rates, and Queueing theory,
The theory of probability deals with averages of mass
phenomena occurring sequentially or simultaneously:
Related Terms
● Random Signal: A random signal is a signal characterized by uncertainty
before its actual occurrence.
● System: A system is one which operates on an input signal and produce
another signal as output.
● Experiment: An experiment is an act which can be repeated under
identical or uniform conditions.
● Random Experiment: A random experiment is an experiment that is
repeated under same conditions, any number of times, for which the
outcome is any one of several possible outcomes that cannot be
predicted in advance
Related Terms
● Trial: Any particular performance of a random experiment is called a trial.
● A random experiment is specified by stating an experimental procedure and
a set of one or more measurements or observations.
You can not change your Future,
but you can change your habits
and sure your habits will change
your future !
-Dr. A P J Abdul Kalam
Experiments Examples:
● The specification of a random experiment must include an
unambiguous statement of exactly what is measured or observed.

● For example, random experiments may consist of the same

procedure but differ in the observations made.
● Experiment E1: Select a ball from an urn containing balls numbered 1
to 50. Note the number of the ball.
● Experiment E2: Select a ball from an urn containing balls numbered 1
to 4. Suppose that balls 1 and 2 are black and that balls 3 and 4 are
white. Note the number and color of the ball you select.
Experiments
● Experiment E3: Toss a coin three times and note the sequence of heads and
tails.
● Experiment E4: Toss a coin three times and note the number of heads.
● Experiment E5: Count the number of voice packets containing only silence
produced from a group of N speakers in a 10-ms period.
● Experiment E6: A block of information is transmitted repeatedly over a noisy
channel until an error-free block arrives at the receiver. Count the number
of transmissions required.
● Experiment E7: Pick a number at random between zero and one.
Experiments
● Experiment E8: Measure the time between page requests in a Web server.
● Experiment E9: Measure the lifetime of a given computer memory chip in a
specified environment.
● Experiment E10: Determine the value of an audio signal at time t1
● Experiment E11: Determine the values of an audio signal at times t1 & t2
● Experiment E12: Pick two numbers at random between zero and one.
● Experiment E13: Pick a number X at random between zero and one, then pick
a number Y at random between zero and X.
● Experiment E14: A system component is installed at time t=0, For t ≥ 0, let
X(t)=1as long as the component is functioning, and let X(t)=0 after the
component fails.
Observations
● A random experiments may consist of the same procedure but differ in the
observations made, as illustrated by E3 &E4.
● A random experiment may involve more than one measurement or
observation, as illustrated by E2, E3, E11,E12 & E13.
● A random experiment may even involve a continuum of measurements, as
shown by E14.
● Experiments E3, E4, E5,E6 & E12 and E13 are examples of sequential
experiments that can be viewed as consisting of a sequence of simple sub
experiments.
● Can you identify the sub experiments in each of these? Note that in E13 second
sub experiment depends on the outcome of the first sub experiment.
The Sample Space
● Since random experiments do not consistently yield the same
result, it is necessary to determine the set of possible results.
● The set of all possible outcomes of a random experiment is called
sample space which is denoted by S.
● In tossing a coin experiment, the sample space is S = {H, T}
● The possible outcomes of a sample space are known as sample
points represented as ξ. Where ξ is an element or point in S.
● A sample space can be classified as (i) discrete sample space, and
(ii) uncountable sample space.
The Sample Space
● Each performance of a random experiment can then be viewed as
the selection at random of a single point (outcome) from S.
● In other words, when we perform a random experiment, one and
only one outcome occurs out of these.
● The sample space S can be specified compactly by using set notation.
It can be visualized by drawing tables, diagrams, intervals of the
real line, or regions of the plane. There are two basic ways to
specify a set:
● List all the elements, separated by commas, inside a pair of braces:
A ={0,1,2,3}, Give a property that specifies the elements of the set:
A= {x: x is an integer such that 0≤x ≤ 3}
The
sample
Spaces
corresponding
to the
experiments
1-14
The Sample Space
● A sample space can be finite, countably infinite, or uncountably
infinite.
● A finite or countably infinite sample space is called a discrete
sample space.
● An uncountable sample space is called a continuous sample space
● We call S a discrete sample space if S is countable; that is, its
outcomes can be put into one-to-one correspondence with the
positive integers.
● We call S a continuous sample space if S is not countable.
Types of sample Space

Finite
S={HH, HT,TH,TT}

Discrete
Countable Infinite
Sample S={0.0,0.1, 0.2,0.3,0.4}

Space
Uncountable
Continuous
HW
● Determine, which type of sample spaces they are
?Discrete / continuous ?
● Experiments E1-E5 have finite discrete sample spaces.
● Experiment E6 has a countably infinite discrete sample
space.
● Experiments E7-E13 have continuous sample spaces.
Events
● We are usually not interested in the occurrence of specific outcomes, but
rather in the occurrence of some event (i.e., whether the outcome
satisfies certain conditions).
● This requires that we consider subsets of S. We say that A is a subset of B if
every element of A also belongs to B.
● For example, in Experiment which involves the measurement of a voltage,
we might be interested in the event “signal voltage is negative.”
● The conditions of interest define a subset of the sample space, namely,
the set of points ξ from S that satisfy the given conditions.
● For example, “voltage is negative” corresponds to the set {ξ: -∞<ξ<0}.
● The event occurs if and only if the outcome of the experiment is in this
subset. For this reason events correspond to subsets of S.
Events
● An event is simply a set of possible outcomes. To be more specific, an
event is a subset A of the sample space S.

● Two events of special interest are the certain event, S, which consists of
all outcomes and hence always occurs,

● impossible or null event, which contains no outcomes and hence

never occurs.
Events : Example
Experiment E1: Select a ball from an urn containing balls numbered 1 to 50. Note the number of the ball.
E1: “An even-numbered ball is selected: A1 = {2,4,6,8…50}

Experiment E2: Select a ball from an urn containing balls numbered 1 to 4. Suppose that balls 1 and 2 are
black and that balls 3 and 4 are white. Note the number and color of the ball you select.
E2: “The ball is white and even-numbered: A2 = {(2,w)}

Experiment E3: Toss a coin three times and note the sequence of heads and tails.
E3: “The three tosses give the same outcome: A3={(HHH), (TTT)}

Experiment E4: Toss a coin three times and note the number of heads.
E4: “The number of heads equals the number of tails: A4= {Ø}

Experiment E5: Count the number of voice packets containing only silence produced from a group of N
speakers in a 10-ms period.
E5: “No active packets are produced,” A5 = (0)
Events : HW
● E6: “Fewer than 10 transmissions are required:
● E7: “The number selected is nonnegative:
● E8: “Less than seconds elapse between page requests:
● E9: “The chip lasts more than 1000 hours but fewer than 1500 hours:
● E10:“The absolute value of the voltage is less than 1 volt:
● E11:“The two voltages have opposite polarities:
● E12:“The two numbers differ by less than 1/10:
● E13:“The two numbers differ by less than 1/10:
● E14: “The system is functioning at time t1:
Events
• An event may consist of a single outcome, as in A2 and A5 event from a
discrete sample space that consists of a single outcome is called an elementary
event.

• An event may also consist of the entire sample space as in A7.

• The null event Ø, arises when none of the outcomes satisfy the
conditions that specify a given event, as in A4
Identities
Probability Measure
● An assignment of real numbers to the events defined in a sample
space S is known as the probability measure.
● Consider a random experiment with a sample space S, and let A be a
particular event defined in S.
● Probability can be defi ned in several ways. There are three types of
definitions.
Classical definition

Relative-frequency definition

Axiomatic definition
Probability: Classical Definition
The probability P(A) of an event A is the ratio of the
number of outcomes n(A) of an experiment that are
favourable to A to the total number of possible outcomes
of the experiment.
Note that the total number of possible outcomes is equal
to the sample space n(S).
Therefore, the probability, P(A) = n(A) / n(S)
Example: Classical Definition
● The probability of getting an even no in die tossing experiment ?
● Exp: Die tossing
● Event : to get even no in a particular toss.
● S= (1,2,3,4,5,6)
● A= (2,4,6)
● P(A) = n(A)/ n(S) = 3/6 = ½ =0.5
● This definition of probability is also known as apriori or
mathematical.
Examples
● A fair die is rolled once. What is the probability of getting a 6‘ ?:
P=1/6
● A fair coin is tossed twice. What is the probability of getting two heads'?:
p=1/4
Example
● A standard deck of playing cards has 52 cards that can be divided in several manners.
There are four suits (spades, hearts, diamonds, and clubs), each of which has 13 cards
(ace, 2, 3, 4, . . . , 10, jack, queen, king). There are two red suits (hearts and diamonds)
and two black suits (spades and clubs). Also, the jacks, queens, and kings are referred
to as face cards, while the others are number cards. Suppose the cards are sufficiently
shuffled (randomized) and one card is drawn from the deck. each atomic outcome has
a probability of 1/52.
● Define the events: A = {red card selected}, B = {number card selected}, and C = {heart
selected}.
● Pr(A) = 26/52 = 1/2. Likewise, Pr(B) = 40/52 = 10/13 and Pr(C) = 13/52 = 1/4.
● Events A and B have 20 outcomes in common,
● hence Pr(A, B) = 20/52 = 5/13. Likewise, Pr(A,C) = 13/52 = 1/4 and Pr(B,C) = 10/52 = 5/26.
● It is interesting to note that Pr(A,C) = Pr(C). This is because C ⊂ A and as a result A ∩ C = C.
Probability : Relative Frequency Definition
Consider a random experiment that is performed n times.

If an event occurs n(A) times then the probability of event

A; P(A) is defined as,

The ratio n(A)/n is the relative frequency for this event.

Example : Relative Definition
● To find the probability that spare part produced by a machine is
defective.
● We have to check the record of defective items produced over a
considerable period of time.
● If out of 10000 items, 500 are found to be defective then
● P(A) = 500/ 10000 = 0.05
● This definition is called as statistical or aposteriori definition of
probability.
Axiomatic Definition
In this approach, the fundamentals of set theory is used to define probability. Consider
a random experiment with sample space S. For each event, we assign a non-negative
number called probability of the event A denoted as P (A).
The probability P(A) satisfies three axioms:
Axiom 1: 0 ≤ P (A) ≤ 1, which means that the probability of A is a non-negative number
between 0 and 1 including 0 and 1.
Axiom 2: P (S) = 1. The probability of sample space is one.
Axiom 3: For any sequence of mutually exclusive events,
P(A1 U A2 U A3 U An) = P(A1) + P(A2) + P(A3) +……+ P(An)
Mutually Exclusive Events
Two events A and B are said to be mutually exclusive if the occurrence of any
one prevents the outcome of another and vice versa.

That is, both the events cannot occur simultaneously, i.e if A occurs, B can
not occur and vice versa.

If the sample space S is not finite, then axiom 3 must be modified as follows:
Axiom 3: If A1, A2, . . . is an infinite sequence of mutually exclusive events in S (A i
& Aj = 0 for i ≠ j), then,
Ex 1
Ex 2
Ex 3
Ex 4
Ex 5
EX6
Ex 7
Quick Revision : Q.1
Which of the following statement is false ?
• We say that A is a subset of B if every element of A also belongs to B.
• The event occurs if and only if the outcome of the experiment is in this
subset
• events correspond to subsets of S.
• All the above
• None of these
Quick Revision : Q.2
Identify the type of event, if
● P(A) = 1
● n(B) = 0
● n(A) = S
● n(C)= 1
Quick Revision : Q.3
Classical definition of probability is related to
● No. of outcomes of a random experiment
● No. of outcomes of an event related to that experiment
● Both
● Can not say.
Both, P(A) = n(A)/ n(S)
Quick Revision : Q.4
Classical definition of probability is also known as
● Mathematical definition
● Apriori Probability
● Both
● Can not say.
Both, P(A) = n(A)/ n(S)
Quick Revision : Q.4
Relative definition of probability is related to
● No. of times a random experiment is experiment is performed
● No. of times that an event occurs
● Both
● Can not say.
Both,
Quick Revision : Q.5
Relative definition of probability is used
● When No. of outcomes of a random experiment are known in advance
● When No. of outcomes of a random experiment are not known in
advance
● Both
● Can not say.
When No. of outcomes of a random experiment are not known in advance
Quick Revision : Q.6
Classical definition of probability is used
● When No. of outcomes of a random experiment are known in advance
● When No. of outcomes of a random experiment are not known in
advance
● Both
● Can not say.
When No. of outcomes of a random experiment are known in advance
P(A) = n(A)/ n(S)
Quick Revision : Q.7
Classical definition of probability is also known as,
● Mathematical/ Priori definition
● Aposteriori
● Both
● Axiomatic
Mathematical/ Priori definition
P(A) = n(A)/ n(S)
Quick Revision : Q.8
Relative definition of probability is also known as,
● Mathematical/ Priori definition
● Statistical/Aposteriori
● Both
● Axiomatic
Statistical/Aposteriori
Quick Revision : Q.9
Which of the following is not true ?
1.0 ≤ P (A) ≤ 1, which means that the probability of A is a non-negative number
between 0 and 1 including 0 and 1.
2.P (S) = 1. The probability of sample space is one.
3.For any sequence of mutually exclusive events,
P(A1 U A2 U A3 U An) = P(A1) + P(A2) + P(A3) +……+ P(An)
4. For any sequence of mutually exclusive events,
P(A1 U A2 U A3 U An) ǂ P(A1) + P(A2) + P(A3) +……+ P(An)
P(A1 U A2 U A3 U An) ǂ P(A1) + P(A2) + P(A3) +……+ P(An)
Elementary properties of probability
Revision Quiz
● Suppose, we toss a coin 150 times and we get head for 102 times. The
probability of getting a tail is, which definition of probability you will
apply ?
● A coin is tossed 100 times out of which head comes 12 times. Find the
experimental probability of getting a head ?
● On February 1, 2003, the Space Shuttle Columbia exploded. This was the
second disaster in 113 shuttle missions for NASA. Now, the probability
that a future mission is successfully completed will be ?
● The probability of getting an even integer from the list 1, 2, 3, 4, .... will be
?
Knowledge is Power
But
Practice is the Key !!!
Theorem 3
Ex 8
B
● From this we can find that A and are two mutually exclusive events.
Also,

A'B
Prove that
Elementary properties of probability
Equally Likely Events
Equally likely events :
Ex 9
Ex 10
Ex 11
Ex 12
Ex
13
Quick Revision
Which of the following is not equally likely event ?
● Tossing a dice
● Tossing a coin
● Picking a ball containing many balls of different colors
● All of the above
Picking a ball containing many balls of different colors
Which of the following is not correct ?

None of these
Quick Revision
If A is subset of B, then which of the following is true ?
● Probability of A is equal to probability of B
● Probability of A is greater than probability of B
● Probability of A is less than probability of B
● None of the above
Probability of A is less than probability of B
Quick Revision
● which of the following is true ?
Basic Principle of Counting
● Consider two experiments are to be performed. If one experiment can
result in any of m outcomes and if another experiment results in any of n
possible outcomes then there are mn possible outcomes of the two
experiments.
● This can be extended to k experiments in which the first one may result
in any of n1 possible outcomes. For each of the possible outcomes, there
are n2 possible outcomes of the second experiment; and for each of the
possible outcomes, of the first two experiments, there are n 3 possible
outcomes of the third experiment, and so on. Then for k experiments,
there are a total of n1n2 … nk outcomes.
Permutations
● Consider the letters a, b and c. These letters can be arranged in 6
different ways as shown below: abc, acb, bac, bca, cab, cba
● Each arrangement is known as permutation. The first letter can be
chosen from any three letters. Once we choose the first letter, the
second letter can be chosen from any two letters, and the third letter can
be chosen from the remaining 1.
● Therefore, the number of permutations is 3 *2 *1 = 6.
● This can be expanded to n objects. The different permutations of the n
objects is n(n – 1) (n – 2) … 3.2.1 = n!
Permutations
● For example, in a cricket team with 11 players, the different batting
orders are 11! = 39916800
● In n objects of which n1 are alike, n2 are alike … nr are alike, the number
of permutations is

● The number of permutations of n objects taking r at time is,

Combinations
● For three letters a, b and c, the number of permutations is six. For
permutations, the order in which the letters are taken is important.
Therefore, abc and acb be counted as two different permutations.
● However, in combination, the order in which the letters are taken is not
considered. Therefore, the number of combinations with letters a, b and
c is only one. The number of two-letter combinations that can be formed
from a, b and c are ab, bc, ca. The number of combinations of n things
taken r at time is denoted by,
Combinations
● If n1 + n2 + … + nr = n then the number of possible division of n distinct
objects from r distinct groups of respective sizes n1, n 2, …, n r is ,
Ex 10:
Addition Theorem
HW : Check with Ven Diagram-
Ex of Addition Theorem
Ex of Addition Theorem
Ex
Hw
Conditional Probability: Ex
● Consider a box containing two white balls and one black ball.
● Assume that we have already drawn a ball from the box and the ball is not replaced.
● Now if we draw a ball from the box then the probability of getting a white ball from
the box depends on the outcome of the first draw.
● If we had drawn a white ball on the first draw, the probability of getting a white ball on
the second draw is ½ since the box is left with one white ball and one black ball after
the first draw.
● Suppose if we had drawn a black ball on the first draw, the probability of getting a
white ball on the second draw is 1. That is, the probability of occurrence of second
event depends on the outcome of the first event.
● If we denote the first event as A and the second event as B then the probability of the
event B, given that A is known to have occurred, is denoted by the conditional
probability denoted by P(B | A).
Conditional Probability: Ex
● Consider an experiment of tossing three coins.
● The sample space of the experiment is
● S = {TTT, TTH, THT, HTT, HHT, HTH, THH, HHH}
● The probability of any event is 1/8.
● Let A be the event in which at least two tails appear, A = {TTT, TTH, THT,
HTT}, P(A) = 4/8 =1/2
● Let B be the event in which the first coin shows a head. B={HTT, HHT,
HTH, HHH}, P(B)= ½
● A ∩B = {HTT}, P(A∩B) = 1/8
Conditional Probability: Ex contd…
● Now if we want to find the ● Therefore, conditional probability of A
probability of Event A with the given that B has occurred is defined as
occurrence of Event B then the
sample space reduces from the
set S to its subset B.
● So now sample space
S= {HTT, HHT, HTH, HHH}
● And now A = {HTT}
● P(A/B)= ¼
● Note that the elements of B
which favour the event A are the
common elements of A and B,
that is the elements of A ∩ B.
Properties of Conditional probability
Ex:
● Solution: Let A be the event of getting a prime number and B be the
event of getting an odd number.
● The universal sample space : S = {1, 2, 3, 4, 5, 6}
● A = {2, 3, 5}
● B = {1, 3, 5}
● A ∩ B = {3, 5}
● P(A)= ½, P(B)=1/2, P(A∩B)=2/6 =1/3
● P(A/B)= ? P(A∩B) / P(B) = 1/3 / ½ = 2/3
 Do it now !
Delegate it !
Defer it
Dump it !
Do Exercise !
Quick Revision
1. If E and F are two events associated with the same sample
space of a random experiment then P (E|F) is given by
_________
a) P(E∩F) / P(F), provided P(F) ≠ 0
b) P(E∩F) / P(F), provided P(F) = 0
c) P(E∩F) / P(F)
d) P(E∩F) / P(E
P(E∩F) / P(F), provided P(F) ≠ 0
Quick Revision
2. Let E and F be events of a sample space S of an experiment, if
P(S|F) = P(F|F), then value of P(F|F)is
a) 0
b) -1
c) 1
d) 2
c)1 We know that P(S|F) = P(S∩F) / P(F). (By formula for conditional
probability)
Which is equivalent to P(F|F) = P(F) / P(F) = 1, hence the value of P(F|F) = 1.
Quick Revision
3. If P(A) = 1/5, P(B) = 0, then what will be the value of P(A|B)?
a) 0
b) 1
c) Not defined
d) 1/5
c)Not defined
Explanation: We know that P(A|B) = P(A∩B) / P(B). (By formula for conditional
probability)
The value of P(B) = 0 in the given question. As the value of denominator
becomes 0, the value of P(A|B) becomes un-defined.
Quick Revision
4. Which notation represents the probability that a student is
over 21 given that they live on campus?
a) P(over 21/ lives on campus)
b) P(under 21/ lives on campus)
c) P(lives on campus / over 21)
d) P(lives on campus / under 21)

P(over 21/ lives on campus)

Independent Events
Two events A and B are independent if and only if
● P(A ∩ B) = P(A) P(B)

In case of three events, the condition for independence are

● P(A1 ∩ A2) = P(A1) P(A2)
● P(A1 ∩ A3) = P(A1) P(A3)
● P(A2 ∩ A3) = P(A2) P(A3)
● P(A1 ∩ A2 ∩ A3) = P(A1) P(A2) P(A3)
If all the above conditions are satisfied then the entire set of events are
mutually independent.
Independent Events
● If A and B are independent events then

P(A UB) = P(A) + P(B) – P(A) P(B)

● If A1, A2 and A3 are independent,

P[A1 ∩ (A2 ∩ A3)] = P(A1) P(A2 ∩ A3)

Multiplication Theorem on Probability
If A and B are two events associated with a random experiment
then,
● P(A ∩ B) = P(A) P(B | A) if P(A) ǂ 0
● P(A ∩ B) = P(B) P(A | B) if P(B) ǂ 0
Ex
● Contd..
● Solution :
HW
P(atleast one odd) = 1 - P(all even)
= 1 - p(even 1st) p(even 2nd)p(even 3rd)= 1 - (3/6)(3/6)(3/6)=1- (1/2)^3= 1-1/8 =
7/8`

Solution 1.25: P(at least one odd) = 1 - P(all even)

= 1 - p(even 1st) p(even 2nd)p(even 3rd)= 1 - (3/6)(3/6)(3/6)=1- (1/2)^3= 1-1/8 =
7/8`
answer
HW: 1.25
● Probability of getting an odd number in a single throw of a dice =3/6=1/2
Similarly, probability of getting an even number in a single throw of a
dice =3/6=1/2
● Probability of getting an even number three times = 1/2 × 1/2 × 1/2 = 1/8
● Therefore, probability of getting an odd number at least once :
=1− Probability of getting an odd number in none of the throws
=1− Probability of getting an even number thrice
=1−1/8
=7/8
Joint Probability

The joint probability of two events A and B is,

Total Probability Theorem
A decomposition of a set S into 2 or
The total probability more disjoint nonempty subsets is
theorem states that if an called a partition of S

event A can occur in N

mutually exclusive ways, E1 A

E1 or E2 or …or EN then
E4
the probability of E2 E3

occurrence of A is given EN

by,
P(A) = P(E1) P(A| E 1) + P(E2) P(A |E 2) +……. + P(EN) P(A | EN)
Total Probability Theorem
● Proof: Let S be the sample space, E1, E2, …, EN are N mutually exclusive and
exhaustive events.
● These events satisfy Ei ∩ Ej = Ø, i ǂ j = 1, 2, …, N
● The sample space, S = E1 U E2 U E3 U … U EN

A∩E2 A∩E3
A = (A ∩ E1)U(A ∩ E2) U (A ∩ E3) U … U (A ∩ EN)

E1 A
A∩E1

P(A )= P(A ∩ E1)+P(A ∩ E2) +P (A ∩ E3) + … +P (A ∩ EN) E4

E2 E3
Since P(A/E1 )= P(A ∩ E1)/P( E1) EN

P(A ) = P(A/ E1)P(E1) + P(A /E2)P(E2) + P(A /E3)P(E3) + … + P(A /EN)P(EN)

Total Probability Theorem
● P(A)

E1 A
The theorem of total probability
can be used to determine the E2
E4
E3
probability of a complex event
EN
in terms of related simpler
events.
This result will be used in Bays'
theorem.
Quick revision
Identify the correct statement
● The theorem of total probability can be used to determine the probability
of a complex event in terms of related simpler events.
● P(A) = P(E1) P(A| E1) + P(E2) P(A |E2) +……. + P(EN) P(A | E N)
● This result will be used in Bays' theorem.
All the above
None of these
Total Probability Theorem
This can be better understood with the help of the below three simple examples.
● The doctor reaches the patient for treatment, takes different modes of
transportation, and his probability of arriving on time is equal to the
summation of his probability of arriving on time, from different modes of
transportation.
● A student is to represent the school in an external competition. The probability
of selecting the student for the competition is equal to the summation of the
probability of selecting this student from the different classes in the school.
● The probability of finding a defective mango is the summation of the
probability of finding this defective mango from the different boxes of
mangoes.
● E1 : Selection of bag 1
E1
● E2 : Selection of bag 2
● A : from the selected bag pick a
ball that should be black. E2

● P(A) = ?
A
● P(E1) = P(E2) =1/2
● P(A) = P(E1) P(A/E1) + P(E2) P(A/ ● Substitute all values in the
E2) formula,
● P(A/E1)= 4/9 ● P(A)= (½ *4/9) + (½*5/8) = 77/144
● P(A/E2)= 5/8
● E1 : Boys student
● E2 : Girls student E1
● A : from the total students,
selected one will get distinction E2
● P(A) = ?
● P(E1) =0.6 , P(E2) =0.4 A

● P(A) = P(E1) P(A/E1) + P(E2) P(A/ ● Substitute all values in the

E2) formula,
● P(A/E1)= 0.3 ● P(A)= (0.6 *0.3) + (0.4*0.35) = 0.32
● P(A/E2)= 0.35
Ex
In an electronics laboratory, there are identically looking capacitors of three
makes A, B & C are in the ratio 2:3:4. It is known that 1% of A , 1.5% of B
and 2% of C are defective.
● What percentages of capacitors in the laboratory are defective? P(D)=?
● If a capacitor picked at defective is found to be defective, what is the
probability it is of make C. P(C/D) =?
● P(A)= 2/9, P(B)=3/9,P(C)= 4/9
B
● P(D/A)=0.01, P(D/B)=0.015, P(D/C)=0.02 A C
B
D

P(D) = P(A) P(D/A) + P(B) P(D/B)+ + P(C) P(D/C)=0.0167

Baye’s Theorem
● Let E 1, E2, …, EN be N mutually
exclusive and exhaustive events
associated with a random
E1 A
experiment.
● If A is any event which occurs E4
with E1 or E 2 or … or EN then… E2 E3
● If P( E 1/ A) = ? P( E 2/ A) = ?........ EN
● In general.. P( E i / A) = ??

P(A)
Baye’s Theorem
● Proof: Let S be the sample space, E1, E2, …, EN are N mutually exclusive and
exhaustive events.
● These events satisfy Ei ∩ Ej = Ø, i ǂ j = 1, 2, …, N
● The sample space, S = E1 U E2 U E3 U … U EN
A∩E1 A∩E1
A = (A ∩ E 1)U(A ∩ E2) U (A ∩ E3) U … U (A ∩ EN)

P(A )= P(A ∩ E 1)+P(A ∩ E2) +P (A ∩ E3) + … +P (A ∩ E N) E1 A

A∩E1 E4
Since P(A/E1 )= P(A ∩ E1)/P( E1) E2 E3
EN

P(A ) = P(A/ E1)P(E1) + P(A /E2)P(E2) + P(A /E3)P(E3) + … + P(A /EN)P(EN)

Baye’s Theorem E1 A

E4
● Using the multiplication theorem of probability, E2 E3
P(A ∩ Ei) = P(A) P(E i | A) = P(Ei) P(A/E i) EN

● P(E i | A) = P(A ∩ E i ) / P(A)

● P(A) can be written from total probability theorem.

● So P(E i | A) = P(A ∩ Ei) / P(A/ E1)P(E1) + P(A /E2)P(E2) + P(A /E3)P(E3) + … +

P(A /EN)P(EN), i=1...N

● P(E i | A) = P(Ei) P(A/Ei) / P(A/ E1)P(E1) + P(A /E2)P(E2) + P(A /E3)P(E3) + … +

P(A /EN)P(EN), i=1...N
Baye’s Theorem
● The probability P(Ei)is called the a priori probability and
● The probability P(Ei/A)is called the a posteriori
probability.
● Thus the Bays' theorem enables us to determine the a
posteriori probability from the observation that A has
occurred.
● This result is of practical importance and is the heart of
Baysean classification, Baysean estimation
● E1 : letter from TATA NAGAR
● E2 : Letter from CALCUTTA E1
● A : Two consecutive letter TA on envelope
● P(E2/A)= ? Baye’s theorem
● P(A) = ? Total probability theorem E2
● P(E1) =1/2 , P(E2) =1/2
● P(A) = P(E1) (PA/E1) + P(E2) P(A/E2)
● Now we take TA as a single letter and fi A
nd the probability of Event A given the
Event E1. ● P(E2/A)= P(E2) P(A/E2) / P(A)
1 23 4 5 6 78 1 2 3 4 5 6 7 8
TA T A N A G A R T A TA N A G A R ● P(E2/A) = ½ * 1/7 / (0.5 *2/8) +
● P(A/E1)= 2/8 (0.5*1/7)
● P(A/E2)= 1/7 (In Calcutta, TA is taken a ● P(E2/A)= 4/11
single letter, total no. of letters 7 )
● Substitute all values in the formula,
● P(A)= (0.5 *2/8) + (0.5*1/7)
Ex : In a binary communication system a zero and a one is transmitted with
probability 0.6 and 0.4 respectively. Due to error in the communication
system a zero becomes a one with a probability 0.1 and a one becomes a
zero with a probability 0.08.
Determine the probability (i) of receiving a one and (ii) that a one was
transmitted when the received message is one.
Let T 0 : Event associated with transmission of 0
T 1: Event associated with transmission of 1
R0: Event associated with reception of 0
T0 R
R1: Event associated with reception of 1
P(T0) = 0.6, P(T 1)=0.4,
P(R1/T 0) = 0.1, P(R0/ T 0) = 0.9 T1
P(R0/T 1) =0.08, P(R1/T1)= 0.92
P(R1) = ?? = P(R1/T 0 ) P(T 0) + P(R1/T 1 ) P(T 1) = 0.428
Ex : In a binary communication system a zero and a one is transmitted with
probability 0.6 and 0.4 respectively. Due to error in the communication system a
zero becomes a one with a probability 0.1 and a one becomes a zero with a
probability 0.08.
Determine the probability (i) of receiving a one and (ii) that a one was transmitted
when the received message is one.

P(R1) = ?? = P(R1 /T 0) P(T 0 ) + P(R1 /T 1) P(T 1 ) = 0.428

P(T1/R1) = P(R1 /T 1) P(T 1 ) / P(R1)
= 0.4*0.92 / 0.428 T0 R
= 0.859

T1
Determine the probability that a zero was transmitted
when the received message is one. P(T0/R1) = ??? 0.14
Hw : In a binary communication system a zero and a one is transmitted with
probability 0.6 and 0.4 respectively. Due to error in the communication system a
zero becomes a one with a probability 0.1 and a one becomes a zero with a
probability 0.08.
Determine the probability (i) of receiving a zero and (ii) that a one was transmitted
when the received message is zero.
Let T0 : Event associated with transmission of 0
T1 : Event associated with transmission of 1
R 0 : Event associated with reception of 0
R 1 : Event associated with reception of 1 T0 R
P(T 0 ) = 0.6, P(T1 )=0.4,
P(R1 /T 0 ) = 0.1, P(R0 / T 0 ) = 0.9
P(R0 /T 1 ) =0.08, P(R1/T1)= 0.92 T1
P(R0 ) = ?? = P(R0 /T 0) P(T 0 ) + P(R0 /T 1) P(T 1 ) =0.9*0.6 +0.08*0.4=0.86
P(R1 ) = ?? = P(R1 /T 0) P(T 0 ) + P(R1 /T 1) P(T 1 ) = ??? (HW)
Hw : In a binary communication system a zero and a one is transmitted with
probability 0.6 and 0.4 respectively. Due to error in the communication
system a zero becomes a one with a probability 0.1 and a one becomes a
zero with a probability 0.08.
Determine the probability (i) of receiving a zero and (ii) that a one was
transmitted when the received message is zero.

P(R0) = ?? = P(R0/T 0 ) P(T 0) + P(R0/T 1 ) P(T 1)

= 0.9*0.6+0.08*0.4 = 0.86 T0 R

P(T1/R0) = ?? =P(R0/T 1 ) P(T 1) / P(R0)= 0.32/ 0.86 T1

P(T0/R0) = P(R0/T 0 ) P(T 0) / P(R0) = ??
HW:With reference to previous problem, solve the following:
1) Determine the probability of receiving a zero: P(R0 )
Determine the probability that a zero was transmitted when the received
message is zero. P(T0/R0)
Determine the probability that a one was transmitted when the received
message is zero. P(T1/R0)
2) Determine the probability of receiving a one: P(R 1)
Determine the probability that a zero was transmitted when the received
message is one. P(T 0/R1)
Determine the probability that a one was transmitted when the received
message is one. P(T 1/R1)
3) Determine the probability of error of transmission : P(T1/R0) + P(T 0/R1)
HW
● A single card is drawn from a standard deck of playing cards. What is the
probability that the card is a face card provided that a queen is drawn
from the deck of cards? P(Q/F) = ? 1/3

J
Q

F
K
Simulation of coin flipping and die tossing.
● coin_flip=round(rand(1)) % Simulate flip of a coin.
● die_toss=ceil(6*rand(1)) % Simulate toss of one die. dice_toss=ceil(6*rand(1,2))
% Simulate toss of two dice
● The command rand(m,n) creates a matrix of m rows and n columns, where
each element of the matrix is a randomly selected number equally likely to fall
anywhere in the interval (0,1).
● By rounding this number to the nearest integer, we can create a randomly
selected number equally likely to be 0 or 1
● Similarly, if we multiply rand(1) by 6 and round up to the nearest integer, we
will get one of the numbers 1, 2, ... , 6 with equal probability. This can be used
to simulate the rolling of a die.
Relative frequency approach to assign the probability of the event,

● A = {sum of two dice = 5}.

● % Simulation code for dice tossing experiment.
● n=1000; % Number of times to toss the dice.
● die1=ceil(6*rand(1,n)); % Toss first die n times.
● die2=ceil(6*rand(1,n)); % Toss second die n times.
● dice_sum=die1+die2; % Compute sum of two tosses.
● nA=sum(dice_sum==5); % Count number of times sum = 5.
● pA=nA/n % Display relative frequency
● Change the value of n up to 10000, in increment of 1000, plot the graph
between relative frequency vs n
● Give the interpretation of results.
 मु कु राने के मकसद ना ढूं ढो...
वरना जदगी यूं ही कट जाएगी,
कभी बेवजह भी मु कु रा के दे खो...
आपके साथ साथ
जदगी भी मु कु राएगी...!!

Thank You !