Llnear 8egresslon wlLh
muluple varlables
Muluple feaLures
Machlne Learnlng
Andrew ng
!"#$ &'$$(
)
*
+,"-$ &./000*
2104 460
1416 232
1334 313
832 178
. .
123453$ '$6(2,$7 &86,"693$7*:
Andrew ng
!"#$ &'$$(
)
*
;2<9$, ='
9$>,==<7
;2<9$, ='
?==,7
@A$ =' B=<$
&C$6,7*
+,"-$ &./000*
2104 3 1 43 460
1416 3 2 40 232
1334 3 2 30 313
832 2 1 36 178
. . . . .
123453$ '$6(2,$7 &86,"693$7*:
noLauon:
= number of feaLures
= lnpuL (feaLures) of Lralnlng example.
= value of feaLure ln Lralnlng example.
Andrew ng
PypoLhesls:
revlously:
Andrew ng
lor convenlence of noLauon, dene .
MuluvarlaLe llnear regresslon.
Llnear 8egresslon wlLh
muluple varlables
CradlenL descenL for
muluple varlables
Machlne Learnlng
Andrew ng
PypoLhesls:
CosL funcuon:
arameLers:
(slmulLaneously updaLe for every )
8epeaL
CradlenL descenL:
Andrew ng
(slmulLaneously updaLe )
D,6>"$E( F$7-$E(
8epeaL
revlously (n=1):
new algorlLhm :
8epeaL
(slmulLaneously updaLe for
)
Llnear 8egresslon wlLh
muluple varlables
CradlenL descenL ln
pracuce l: leaLure Scallng
Machlne Learnlng
Andrew ng
L.g. = slze (0-2000 feeL
2
)
= number of bedrooms (1-3)
G$6(2,$ !-63"EA
ldea: Make sure feaLures are on a slmllar scale.
slze (feeL
2
)
number of bedrooms
Andrew ng
G$6(2,$ !-63"EA
CeL every feaLure lnLo approxlmaLely a range.
Andrew ng
8eplace wlLh Lo make feaLures have approxlmaLely zero mean
(uo noL apply Lo ).
1$6E E=,<63"#64=E
L.g.
Llnear 8egresslon wlLh
muluple varlables
CradlenL descenL ln
pracuce ll: Learnlng raLe
Machlne Learnlng
Andrew ng
D,6>"$E( >$7-$E(
- uebugglng": Pow Lo make sure gradlenL
descenL ls worklng correcLly.
- Pow Lo choose learnlng raLe .
Andrew ng
Lxample auLomauc
convergence LesL:
ueclare convergence lf
decreases by less Lhan
ln one lLerauon.
0 100 200 300 400
no. of lLerauons
16H"EA 72,$ A,6>"$E( >$7-$E( "7 I=,H"EA -=,,$-(3C:
Andrew ng
16H"EA 72,$ A,6>"$E( >$7-$E( "7 I=,H"EA -=,,$-(3C:
CradlenL descenL noL worklng.
use smaller .
no. of lLerauons
no. of lLerauons no. of lLerauons
- lor sumclenLly small , should decrease on every lLerauon.
- 8uL lf ls Loo small, gradlenL descenL can be slow Lo converge.
Andrew ng
!2<<6,CJ
- lf ls Loo small: slow convergence.
- lf ls Loo large: may noL decrease on
every lLerauon, may noL converge.
1o choose , Lry
Llnear 8egresslon wlLh
muluple varlables
leaLures and
polynomlal regresslon
Machlne Learnlng
Andrew ng
K=27"EA 5,"-$7 5,$>"-4=E
Andrew ng
+=3CE=<"63 ,$A,$77"=E
rlce
(y)
Slze (x)
Andrew ng
LB="-$ =' '$6(2,$7
rlce
(y)
Slze (x)
Llnear 8egresslon wlLh
muluple varlables
normal equauon
Machlne Learnlng
Andrew ng
CradlenL uescenL
normal equauon: MeLhod Lo solve for
analyucally.
Andrew ng
lnLuluon: lf 1u
Solve for
(for every )
Andrew ng
!"#$ &'$$(
)
*
;2<9$, ='
9$>,==<7
;2<9$, ='
?==,7
@A$ =' B=<$
&C$6,7*
+,"-$ &./000*
1 2104 3 1 43 460
1 1416 3 2 40 232
1 1334 3 2 30 313
1 832 2 1 36 178
!"#$ &'$$(
)
*
;2<9$, ='
9$>,==<7
;2<9$, ='
?==,7
@A$ =' B=<$
&C$6,7*
+,"-$ &./000*
2104 3 1 43 460
1416 3 2 40 232
1334 3 2 30 313
832 2 1 36 178
Lxamples:
Andrew ng
$M6<53$7 N '$6(2,$7:
L.g. lf
Andrew ng
ls lnverse of maLrlx .
CcLave: pinv(X*X)*X*y
Andrew ng
(,6"E"EA $M6<53$7O '$6(2,$7:
CradlenL uescenL normal Lquauon
no need Lo choose .
uon'L need Lo lLeraLe.
need Lo choose .
needs many lLerauons.
Works well even
when ls large.
need Lo compuLe
Slow lf ls very large.
Llnear 8egresslon wlLh
muluple varlables
normal equauon
and non-lnverublllLy
(opuonal)
Machlne Learnlng
Andrew ng
normal equauon
- WhaL lf ls non-lnveruble? (slngular/
degeneraLe)
- CcLave: pinv(X*X)*X*y
Andrew ng
WhaL lf ls non-lnveruble?
8edundanL feaLures (llnearly dependenL).
L.g. slze ln feeL
2
slze ln m
2
1oo many feaLures (e.g. ).
- ueleLe some feaLures, or use regularlzauon.
