ToPIC 5:
TNFORMATIDN THEORY
THfOPMATION ( ) :
present in an, qen event cdeperos on
intormaton
The ocCunce of tat event.
probability af o aall
occorence d event is veny
probabiliby ot
Re crtent in tRat evnt will be
inho ostton
-hen, -1Re
’ BITS and
's guen by
TN FO UNITS
T- loq ( )cets)
ENTPOPY (H): in a discee
ingor rnatn prsent
aver age amount of
The
Sounce is called as
N
N (OP)
CONDITION FOR MAX\ MIZING ENTPOP 4:
Discre te Srce :.S PCS)P
S Ps):!-p SoUYce
Hcs)=
d H)
dP
lag,
log, (i-p)
2P =|
P
matiMom ohen all A.
a soUTCe fs
wh egual pmbabilib
KoUTCe OccUT
has 'MPvenk 4Ren,
* eoTCe
logMMbilyrl
* DATA TRANSM SSION RATE
gven as poduct of soU1ce
It is
Synrtad and entrpy
Rs XH] bitsec
Q Consider a binany eou
Detrnine
otth prbailtty yand 3l4 nepectdy .
HC) -[4ct co 4
SoU1ce.
") what is he entropy of second order
HCs)=? # (s) = hH(S)
S2 SI
S S>
4.0462 +04$2 to4 66
H(S)- 1. 62 bts| anol.
HCs): 2HCa)
ENTPOPY CoNbITIO
NAL AND TOLNT B[N PELATON
M
ENTROPY: loN CoNDIT
AL
H[93:-}Z
oP[xi,
ENTROPY: M TopNT
- LY] H
ENTROP|: TYPES
OF
ayrnbd and madmoro
higheat hasing bits having ofSsyrkeol
mos
probablity Lract
ae colirg tongth
Gcheres Conapt trtpy
able Vavi in" e
V2
P(xs):S%
:-[-395+ (-or) 4coqo]
[-os+eo sa) +(-0 s13)].
H[x93: log()+ log ()+log ()+
Cos0)]
bits syrnbol 264
H):[xN]-H[
264 -|S4
2-49-|29
|:38 bits syrnbol.
MUTUAL
TNFOPMATION (1)
b<n tx and Rx in
*
wpnsents data tronte
I
itssymbd
I[x:]: HLJ- H ]bit |aymtd
(OP)'
# CHANNEL CAPACITY
marinigd Value of tro mabun
Channel apacby is,
-tsanssered ln.Tx and Rx..
UNITS: blts| symba.
(x:y))
c = MAx [1
clog M- H[
(op)
C
log
CHANNEL EFFICIENcy
C
CHAN NEL REDUNDANCY
q: Detzroine intoatirn -transkr betueen x and Y e hence
deduce channl Gapacity fo the yolloaing Comoni catn
channal.
r)
POX)- o4
P[x]
P[x] 0.6
Plx»]-o4 X
* H[9]=- fo.loy, (oe)+oyly, (o )
*
P]-0.6 P
H:-|ovalog, (o3)+ o>lag.(o2) to sloy
944356= 0 Bybikmb.
=0.
C #[3-[]
MAx
*y=o 1344
(o3) loq, o-38 4(o'62) log, 6)1+(o C
o.8
.o
3X|o302.
O3
channel. tllosoing the capacty
of Det1ntne
tie tg'
%
[Jmax
-H H -Bsc
sitslsyrnb =Ilog² H[ymax
X
P
probailbtes ve
BSc. ascalled equal
fa
CionNNEL sVMMIPIC BINAPY
CAPACITY THEOREM SHANNON
* CHAN NEL
a dizrete memny-lees RoUrce
Ihe Capasby of
qven y bits
c- Blog. (iw)
shee
"Pbb
Sipad to
TRADE OFF B/N
BW
Coni deu an mdog ignd band limitbed to 4rH
-tansnitkd tinunua 'hansd ving 3-bib PCM ysten
higher than-ae
The' sampling ate ysed is as
vale.The syclero uUses hannshose'Bw is tokte
has sign to moice sato ef, 20dB.
betexnone tie tollbing?
) strate o the PeM Syeten.
then
) Dekrmine tie Coposty a dnned hence hed hethey
dta can be tx on channl
C
30d8 to.
C- lox o lo toi)
NoTE:
2
lk BW Constnt or eroy-tree, Tx
keeping
he mfin. BWgue d by the ohanne by trmpng
BW
mine sodB -for dtrtsn lecs Tx?
fu) Deter
constant at
SAP
BWmin ?
:30xo
BWonin lo9, (1+1o0)
ML DECODING SCH CMES IN DETERMINNG
MAf
* USE OF
m
Tx ( )
() 2()
P()P(m)
P()
P()
PC) mi P()
MAP RULE 2
Yo, M, is Tx
P(m) F()4> P(*.)r C );
ML RULE: P(m) = P(ni)
Fnd the Pe
Anany
MAP and Ml decodng algo thms
P(r)=o
P(m o"3 ,
Sol:
MAP:
P(mo) ( P(m) P)
(0)0 1)
stx
Pe -(o")(o-)+(o.3) (o-4)
ML:
(1o)s0 (h-o)ao+
* TOPC 6: CHANNE L
TyPES oF CObING
SoWCe cong
2 channad cading
SoURCE coDING (x)
In sounte oing Vaiable leng codng
hemes an
per symbd tHence
uced to wduce , Re onoof bite duces
bandu ( BW)
bit vate () &
Eq: ttausman coding
shannon- faro Cocing
CHeN NEL CON 6?
In hanel coing enha Gits (paity
bits) ae added at the end of data als e doi,
he Tx.
.This is dore to enhance the dota
" The bit vate M6) BN requies
Ez* Block Codes
Convolotn Codes.
* LIN EAR BLOCK CD DES
preserted as (o)
nhere, k ’ no-of data bits
n no-f bit în code oox
- paiby bits exta Gb nondant bike
n6
n -k -3
5 y MATIC
t
ae oddet
odded ether ot trhe end af
parity bite
-the the data bil 4hen
tavig
its o1) at he
s said to
to be
be sycleratbc
ie code added in betuen -he data bits,
bits avo
is
eaid to be'
ode
hen -the
P
P d P
CoDE:
LINEAR
X
Scheme s ead to bi lintan, SUr
aesulk în a coee coond. of t6e
two Code oonde again
MODULO-2 AbbitioN
Same
ui(ni) 3)
P did
P =d ds
d P2 Hbls t dn
d
2 2
W
3 4
2
CG
3 3
L
C3
* HAMMING WElGHT :
's psent in 4 qien
he no of
Called as
* HAMM|N6 DISTANCE:
-he mo.of locatons in which e
This fndicaks
Succecsive Code C0ords diter
* MINIMUM 4AM MING <TANCE (dmin)
bao euCes`ive
The smallect harnring drtance oln any hoo
Code oode.
detect e c Comctin
uooud. Capulbitba
of agueri Code
.he smalletb hareing: weight ercept tor a
Code od is
harring didance (din).
* ERROR DETECTIGN E COPPECTION CAPABILITIES
Anw ln.) brer bloct code ohidh Capable of
Gondtar.
Coechng i shaold Sabty -ie fulloing
.2t|,
ti|
Ahy buk). inear dock Kode, Capatle ot dekcg
should catsby cordi tbon.
dntn ttl|
HAMMING
40v)) bnear bloct code Te cad to
L hannino
Code 1 -ntl
(G;3) hartsrg cnde.
(34) 141
Atttamm.rg
anming Code.
CHECK
CYCLIC PEDUN DAN CY (co)
Ths ts only an eror debçtin algrithm.
Enoy tomectirn ts not
poctible nith, this alqoithrn
i< gven by g x"+X tl, deterine -tse codeiey
Tk toro th arnel o the obove data
Sol?
o at the end.of data a ceovdina to -the
Add noof
ovder of glx
T 10 o )4too000o(oDObIolo
. Bnal
0o000 rem ainderis
o0000
o0||
Oo 00 0 PAPITY BIts
CHE=E SUM BITS
Px: |0o) ooI o11||1o00otn
I0011
oo0oI
oo
i. inal Check cum panity bte aome oo
:, No ERPOR OCUPED.sr
Dischd'pans ty bits and take data bik
are nonzeo
0snd.
Bx code ovd +ty Cude
Ast for etrans is $ln’ AR
3thutornahc Repeat
