15-Apr-19 Dr.
RAVILLA DILLI, ECE, MIT, Manipal, India
DIGITAL COMMUNICATIONS
Topics to be Discussed in this Section
Information Theory
15-Apr-19 Dr. RAVILLA DILLI, ECE, MIT, Manipal, India 2
Digital Communications
Figure: Block diagram of a Digital communication system
15-Apr-19 Dr. RAVILLA DILLI, ECE, MIT, Manipal, India 3
Digital Communications
Source Coding: To minimize the no. of bits per unit time required to
represent the source output. Also known as “data compression”.
Ex: Huffman coding.
Encryption: To make source bits transmission secure.
It is conversion of source bits (message text) into a source stream that looks
like meaningless random bits of data (cipher text) is known as encryption.
Ex: Data Encryption Standard (DES), RSA system.
Channel Coding: To correct transmission errors introduced by the channel.
It is a process of introducing redundant bits to a sequence of information
bits in a controlled manner to correct transmission errors.
Ex: Repetition code, Reed-Solomon codes, CRC codes.
The encoded sequence that is the output of the channel encoder is referred
to as codeword.
15-Apr-19 Dr. RAVILLA DILLI, ECE, MIT, Manipal, India 4
Digital Communications
Modulation: To map the codewords into waveforms which are then Txed
over the physical medium known as the channel.
Ex: Phase shift keying (PSK), quadrature amplitude modulation (QAM).
Channel: The physical transmission medium; it can be wireless or wireline.
It corrupts transmitted waveforms due to various effects such as noise,
interference, fading, and multipath transmission.
Ex: Additive white Gaussian noise (AWGN) channel.
15-Apr-19 Dr. RAVILLA DILLI, ECE, MIT, Manipal, India 5
Digital Communications
Demodulation: To convert received noisy waveform to a sequence of bits,
which is an estimate of the transmitted data bits.
Channel Decoding: To estimate the information bits, and correct the
transmission errors.
The performance of the channel decoder is measured by the bit error rate
(BER) or the frame error rate (FER) of the decoded information sequence.
BER is defined as the expected number of information bit decoding errors
per decoded information bit.
Decryption: To recover the plain text from the cipher text using a key.
Source Decoding: To reconstruct the original source bits from the decoded
information sequence.
Due to channel errors, the final reconstructed signal may be distorted.
15-Apr-19 Dr. RAVILLA DILLI, ECE, MIT, Manipal, India 6
Digital Communications
Uncertainty, Information, and Entropy:
15-Apr-19 Dr. RAVILLA DILLI, ECE, MIT, Manipal, India 7
Digital Communications
Uncertainty, Information, and Entropy:
15-Apr-19 Dr. RAVILLA DILLI, ECE, MIT, Manipal, India 8
Digital Communications
Properties of Information:
15-Apr-19 Dr. RAVILLA DILLI, ECE, MIT, Manipal, India 9
Digital Communications
Properties of Information:
15-Apr-19 Dr. RAVILLA DILLI, ECE, MIT, Manipal, India 10
Digital Communications
Uncertainty, Information, and Entropy:
15-Apr-19 Dr. RAVILLA DILLI, ECE, MIT, Manipal, India 11
Digital Communications
Uncertainty, Information, and Entropy:
15-Apr-19 Dr. RAVILLA DILLI, ECE, MIT, Manipal, India 12
Digital Communications
Uncertainty, Information, and Entropy:
15-Apr-19 Dr. RAVILLA DILLI, ECE, MIT, Manipal, India 13
Digital Communications
Uncertainty, Information, and Entropy:
15-Apr-19 Dr. RAVILLA DILLI, ECE, MIT, Manipal, India 14
Digital Communications
Shannon-Fano Algorithm:
15-Apr-19 Dr. RAVILLA DILLI, ECE, MIT, Manipal, India 15
Digital Communications
Shannon-Fano Algorithm: Example 1
15-Apr-19 Dr. RAVILLA DILLI, ECE, MIT, Manipal, India 16
Digital Communications
Shannon-Fano Algorithm: Example 1
15-Apr-19 Dr. RAVILLA DILLI, ECE, MIT, Manipal, India 17
Digital Communications
Shannon-Fano Algorithm: Example 1
15-Apr-19 Dr. RAVILLA DILLI, ECE, MIT, Manipal, India 18
Digital Communications
Shannon-Fano Algorithm: Example 2
15-Apr-19 Dr. RAVILLA DILLI, ECE, MIT, Manipal, India 19
Digital Communications
Shannon-Fano Algorithm: Example 2
15-Apr-19 Dr. RAVILLA DILLI, ECE, MIT, Manipal, India 20
Digital Communications
Shannon-Fano Algorithm: Example 2
15-Apr-19 Dr. RAVILLA DILLI, ECE, MIT, Manipal, India 21
Digital Communications
Shannon-Fano Algorithm: Example 2
15-Apr-19 Dr. RAVILLA DILLI, ECE, MIT, Manipal, India 22
Digital Communications
Shannon-Fano Algorithm:
15-Apr-19 Dr. RAVILLA DILLI, ECE, MIT, Manipal, India 23
Digital Communications
Shannon-Fano Algorithm:
15-Apr-19 Dr. RAVILLA DILLI, ECE, MIT, Manipal, India 24
Digital Communications
Shannon-Fano Algorithm:
15-Apr-19 Dr. RAVILLA DILLI, ECE, MIT, Manipal, India 25
Digital Communications
Huffman Coding:
Example: Perform Huffman Encoding for the following probabilities.
15-Apr-19 Dr. RAVILLA DILLI, ECE, MIT, Manipal, India 26
Digital Communications
Huffman Coding:
Example: Perform Huffman Encoding for the following probabilities.
15-Apr-19 Dr. RAVILLA DILLI, ECE, MIT, Manipal, India 27
Digital Communications
Huffman Coding:
Example: Perform Huffman Encoding for the following probabilities.
15-Apr-19 Dr. RAVILLA DILLI, ECE, MIT, Manipal, India 28
Digital Communications
Huffman Coding:
Example: Perform Huffman Encoding for the following probabilities.
15-Apr-19 Dr. RAVILLA DILLI, ECE, MIT, Manipal, India 29
Digital Communications
Huffman Coding:
Example: Perform Huffman Encoding for the following probabilities.
15-Apr-19 Dr. RAVILLA DILLI, ECE, MIT, Manipal, India 30
Digital Communications
Huffman Coding:
Example: Perform Huffman Encoding for the
following probabilities.
15-Apr-19 Dr. RAVILLA DILLI, ECE, MIT, Manipal, India 31
Digital Communications
Huffman Coding:
15-Apr-19 Dr. RAVILLA DILLI, ECE, MIT, Manipal, India 32
Digital Communications
Discrete Memoryless Channels:
Fig: A Simple Communication System
15-Apr-19 Dr. RAVILLA DILLI, ECE, MIT, Manipal, India 33
Digital Communications
Discrete Memoryless Channels:
15-Apr-19 Dr. RAVILLA DILLI, ECE, MIT, Manipal, India 34
Digital Communications
Discrete Memoryless Channels:
H(X): Average information per character or symbol transmitted by the
source or the entropy of the source.
H(Y): Average information received per character at the receiver or the
entropy of the receiver.
H(X, Y): Average information per pair of transmitted and received characters
or the average uncertainty of the communication system as a whole.
15-Apr-19 Dr. RAVILLA DILLI, ECE, MIT, Manipal, India 35
Digital Communications
Discrete Memoryless Channels:
H (X|Y): A specific character yj being received.
This may be the result of the transmission of one of the xk with a given
probability.
The average value of the Entropy associated with this scheme when yj covers
all the received symbols
i.e., E {H (X|yj)} is the entropy H (X|Y), called the ‘Equivocation’, a measure
of information about the source when it is known that Y is received.
15-Apr-19 Dr. RAVILLA DILLI, ECE, MIT, Manipal, India 36
Digital Communications
Discrete Memoryless Channels:
H (Y|X) : This is a measure of information about the receiver.
The marginal Entropies H(X) and H(Y) give indications of the probabilistic
nature of the transmitter and receiver respectively.
H (Y|X) indicates a measure of the ‘noise’ or ‘error’ in the channel.
H(X |Y) tells about the ability of recovery or reconstruction of the
transmitted symbols from the observed output symbols.
15-Apr-19 Dr. RAVILLA DILLI, ECE, MIT, Manipal, India 37
Digital Communications
Joint Entropies:
We define the following entropies, which can be directly computed from the
JPM.
15-Apr-19 Dr. RAVILLA DILLI, ECE, MIT, Manipal, India 38
Digital Communications
Joint Entropies:
15-Apr-19 Dr. RAVILLA DILLI, ECE, MIT, Manipal, India 39
Digital Communications
Conditional Entropies:
from the definition of the conditional probability we have:
15-Apr-19 Dr. RAVILLA DILLI, ECE, MIT, Manipal, India 40
Digital Communications
Conditional Entropies:
Taking the average of the above entropy function for all admissible
characters received, we have the average “conditional Entropy” or
“Equivocation”:
“Equivocation” specifies the average amount of information needed to
specify an input character provided we are allowed to make an observation
of the output produced by that input.
15-Apr-19 Dr. RAVILLA DILLI, ECE, MIT, Manipal, India 41
Digital Communications
Conditional Entropies:
All the five entropies so defined are all inter-related as:
15-Apr-19 Dr. RAVILLA DILLI, ECE, MIT, Manipal, India 42
Digital Communications
Joint and Conditional Entropies:
Example 1: Determine different entropies for the JPM given below and
verify their relationships.
15-Apr-19 Dr. RAVILLA DILLI, ECE, MIT, Manipal, India 43
Digital Communications
Joint and Conditional Entropies:
Example 1: Determine different entropies for the JPM given below and
verify their relationships.
Solution:
15-Apr-19 Dr. RAVILLA DILLI, ECE, MIT, Manipal, India 44
Digital Communications
Joint and Conditional Entropies:
Solution:
15-Apr-19 Dr. RAVILLA DILLI, ECE, MIT, Manipal, India 45
Digital Communications
Joint and Conditional Entropies:
Solution:
15-Apr-19 Dr. RAVILLA DILLI, ECE, MIT, Manipal, India 46
Digital Communications
Joint and Conditional Entropies:
Solution:
15-Apr-19 Dr. RAVILLA DILLI, ECE, MIT, Manipal, India 47
Digital Communications
Joint and Conditional Entropies:
Solution:
15-Apr-19 Dr. RAVILLA DILLI, ECE, MIT, Manipal, India 48
Digital Communications
Joint and Conditional Entropies:
Solution:
15-Apr-19 Dr. RAVILLA DILLI, ECE, MIT, Manipal, India 49
Digital Communications
Joint and Conditional Entropies:
Example 2: The input source to a noisy communication channel is a random
variable X over the four symbols a, b, c, d. The output from this channel is a
random variable Y over these same four symbols. The joint distribution of these
two random variables is as follows:
15-Apr-19 Dr. RAVILLA DILLI, ECE, MIT, Manipal, India 50
Digital Communications
Joint and Conditional Entropies:
15-Apr-19 Dr. RAVILLA DILLI, ECE, MIT, Manipal, India 51
Digital Communications
Joint and Conditional Entropies:
Solution:
15-Apr-19 Dr. RAVILLA DILLI, ECE, MIT, Manipal, India 52
Digital Communications
Joint and Conditional Entropies:
Solution:
15-Apr-19 Dr. RAVILLA DILLI, ECE, MIT, Manipal, India 53
Digital Communications
Source Coding Theorem:
15-Apr-19 Dr. RAVILLA DILLI, ECE, MIT, Manipal, India 54
Digital Communications
Discrete Memoryless Channel:
15-Apr-19 Dr. RAVILLA DILLI, ECE, MIT, Manipal, India 55
Digital Communications
Discrete Memoryless Channel:
15-Apr-19 Dr. RAVILLA DILLI, ECE, MIT, Manipal, India 56
Digital Communications
Mutual Information:
15-Apr-19 Dr. RAVILLA DILLI, ECE, MIT, Manipal, India 57
Digital Communications
Mutual Information:
15-Apr-19 Dr. RAVILLA DILLI, ECE, MIT, Manipal, India 58
Digital Communications
Channel Capacity Theorem:
15-Apr-19 Dr. RAVILLA DILLI, ECE, MIT, Manipal, India 59
Digital Communications
Channel Capacity Theorem:
15-Apr-19 Dr. RAVILLA DILLI, ECE, MIT, Manipal, India 60
Digital Communications
Channel Capacity Theorem:
15-Apr-19 Dr. RAVILLA DILLI, ECE, MIT, Manipal, India 61
Digital Communications
Channel Capacity Theorem:
15-Apr-19 Dr. RAVILLA DILLI, ECE, MIT, Manipal, India 62
Digital Communications
Channel Capacity Theorem:
15-Apr-19 Dr. RAVILLA DILLI, ECE, MIT, Manipal, India 63
Digital Communications
Convolutional Codes:
15-Apr-19 Dr. RAVILLA DILLI, ECE, MIT, Manipal, India 64
Digital Communications
Convolutional Codes:
15-Apr-19 Dr. RAVILLA DILLI, ECE, MIT, Manipal, India 65
Digital Communications
Convolutional Codes:
15-Apr-19 Dr. RAVILLA DILLI, ECE, MIT, Manipal, India 66
Digital Communications
Convolutional Codes:
15-Apr-19 Dr. RAVILLA DILLI, ECE, MIT, Manipal, India 67
Digital Communications
Convolutional Codes:
15-Apr-19 Dr. RAVILLA DILLI, ECE, MIT, Manipal, India 68
Digital Communications
Convolutional Codes:
15-Apr-19 Dr. RAVILLA DILLI, ECE, MIT, Manipal, India 69
Digital Communications
Convolutional Codes:
15-Apr-19 Dr. RAVILLA DILLI, ECE, MIT, Manipal, India 70
Digital Communications
Convolutional Codes:
15-Apr-19 Dr. RAVILLA DILLI, ECE, MIT, Manipal, India 71
Digital Communications
Convolutional Codes:
15-Apr-19 Dr. RAVILLA DILLI, ECE, MIT, Manipal, India 72
Digital Communications
Convolutional Codes:
15-Apr-19 Dr. RAVILLA DILLI, ECE, MIT, Manipal, India 73
Digital Communications
Convolutional Codes:
15-Apr-19 Dr. RAVILLA DILLI, ECE, MIT, Manipal, India 74
Digital Communications
Code Tree for Convolutional Codes:
15-Apr-19 Dr. RAVILLA DILLI, ECE, MIT, Manipal, India 75
Digital Communications
Spread Spectrum:
15-Apr-19 Dr. RAVILLA DILLI, ECE, MIT, Manipal, India 76
Digital Communications
Spread Spectrum:
15-Apr-19 Dr. RAVILLA DILLI, ECE, MIT, Manipal, India 77
Digital Communications
Spread Spectrum:
15-Apr-19 Dr. RAVILLA DILLI, ECE, MIT, Manipal, India 78
Digital Communications
Spread Spectrum:
15-Apr-19 Dr. RAVILLA DILLI, ECE, MIT, Manipal, India 79
Digital Communications
Spread Spectrum:
15-Apr-19 Dr. RAVILLA DILLI, ECE, MIT, Manipal, India 80
Digital Communications
Spread Spectrum:
15-Apr-19 Dr. RAVILLA DILLI, ECE, MIT, Manipal, India 81
Digital Communications
Spread Spectrum:
15-Apr-19 Dr. RAVILLA DILLI, ECE, MIT, Manipal, India 82
Digital Communications
Spread Spectrum:
15-Apr-19 Dr. RAVILLA DILLI, ECE, MIT, Manipal, India 83
Digital Communications
Spread Spectrum:
15-Apr-19 Dr. RAVILLA DILLI, ECE, MIT, Manipal, India 84
Digital Communications
Spread Spectrum: Freq. Hopping
The spectrum of the Txed signal is spread sequentially rather than
instantaneously (i.e. freq. hopping is ordered by pseudo-random sequence).
Freq. Hopping spread spectrum: The carrier hops randomly from one freq.
to another freq. as per the pseudo random sequence.
15-Apr-19 Dr. RAVILLA DILLI, ECE, MIT, Manipal, India 85
Digital Communications
Spread Spectrum: Freq. Hopping
The incoming binary data are applied to an M-ary FSK modulator.
The modulated signal and the o/p of digital freq. synthesizer are applied to a
mixer.
15-Apr-19 Dr. RAVILLA DILLI, ECE, MIT, Manipal, India 86
Digital Communications
Spread Spectrum: Freq. Hopping
The filter is designed to select the sum freq. component resulting from the
multiplication process as the Txed signal.
Successive k-bit segments of a PN sequence drive the freq. synthesizer that
enables the carrier freq. hop over 2k distinct values.
In single hop, the BW of the Txed signal is same as the resulting from the use
of a conventional MFSK.
15-Apr-19 Dr. RAVILLA DILLI, ECE, MIT, Manipal, India 87
Digital Communications
Spread Spectrum: Freq. Hopping
15-Apr-19 Dr. RAVILLA DILLI, ECE, MIT, Manipal, India 88
Digital Communications
Spread Spectrum: Freq. Hopping
15-Apr-19 Dr. RAVILLA DILLI, ECE, MIT, Manipal, India 89
Digital Communications
Spread Spectrum: Freq. Hopping
15-Apr-19 Dr. RAVILLA DILLI, ECE, MIT, Manipal, India 90
Digital Communications
Spread Spectrum: Freq. Hopping
15-Apr-19 Dr. RAVILLA DILLI, ECE, MIT, Manipal, India 91
Digital Communications
Spread Spectrum: Freq. Hopping
Fast-freq. Hopping:
jammer, that involves two functions:
1. Measurement of the spectral content of the Txed signal.
2. Retuning of the interfering signal to that portion of the freq. band.
To defeat the jammer, the Txed signal must be hopped to a new carrier freq.
Before the jammer is able to complete the above two processes.
15-Apr-19 Dr. RAVILLA DILLI, ECE, MIT, Manipal, India 92
Digital Communications
Spread Spectrum: Freq. Hopping
Fast-freq. Hopping:
Non coherent detection includes:
1. Foe each FH/MFSK symbol, separate decisions are made on the K-freq.-
hop chips received, based on majority the MFSK symbols is dehopped.
2. For each FH/MFSK symbol, likelihood functions are computed as
functions of the total signal Rxed over K chips, the larger one is selected.
15-Apr-19 Dr. RAVILLA DILLI, ECE, MIT, Manipal, India 93
Digital Communications
Spread Spectrum: CDMA
The design of codes should have:
1. Each code is approximately orthogonal( i.e. low cross-correlation) with
all other codes.
2. CDMA system operates asynchronously.
3. CDMA does not require an external synchronization network.
4. CDMA degrades in its performance as the no. od users increases.
5. CDMA is immune to external interference(i.e. multipath rejection or
deliberate jamming)
15-Apr-19 Dr. RAVILLA DILLI, ECE, MIT, Manipal, India 94
QUERIES ?
15-Apr-19 Dr. RAVILLA DILLI, ECE, MIT, Manipal, India
15-Apr-19 Dr. RAVILLA DILLI, ECE, MIT, Manipal, India
