EECE 7370 Advanced Computer Vision

Lecture 5
Layers, CNNs, Architectures
Next Class
Training NN

Announcements
First paper review has been posted
Groups need to be formed by Thursday Sep 28th

1
 Robust
Systems

Linear Classi cation Lab

Northeastern 2
UNIVERSITY Advanced Computer Vision
fi
 Robust
Systems

Linear Classi er Limitations Lab

Only one template per class.

Data might not be linearly separable.

Northeastern 3
UNIVERSITY Advanced Computer Vision
fi
 Robust
Systems

Feature Transformations Lab

Original Space
<latexit sha1_base64="5n3ZJc68LBmweYlVkQsqcGz72Lc=">AAAB/XicbVDJSgNBEO1xjXEbl5uXxiAIQpgJQb0IQS8eI5gFkkno6fQkTXoWu2vEcQj+ihcPinj1P7z5N3aSOWjig4LHe1VU1XMjwRVY1rexsLi0vLKaW8uvb2xubZs7u3UVxpKyGg1FKJsuUUzwgNWAg2DNSDLiu4I13OHV2G/cM6l4GNxCEjHHJ/2Ae5wS0FLX3Jf4ArfVnYT0oVPCJzjplEZds2AVrQnwPLEzUkAZql3zq90LaeyzAKggSrVsKwInJRI4FWyUb8eKRYQOSZ+1NA2Iz5STTq4f4SOt9LAXSl0B4In6eyIlvlKJ7+pOn8BAzXpj8T+vFYN37qQ8iGJgAZ0u8mKBIcTjKHCPS0ZBJJoQKrm+FdMBkYSCDiyvQ7BnX54n9VLRPi2Wb8qFymUWRw4doEN0jGx0hiroGlVRDVH0iJ7RK3oznowX4934mLYuGNnMHvoD4/MHkIaUBQ==</latexit>

p
r = x2 +⇣ y 2⌘
1 y
<latexit sha1_base64="WwJYM7ko+jY2fc/b2sN1vHG/Cn8=">AAACE3icbVA9SwNBEN3z2/gVtbRZDIIKhjsJaiMEbSwVjAnkYtjbzCVL9vaO3TkxHPcfbPwrNhaK2NrY+W/cxBR+PRh4vDfDzLwgkcKg6344E5NT0zOzc/OFhcWl5ZXi6tqViVPNocZjGetGwAxIoaCGAiU0Eg0sCiTUg/7p0K/fgDYiVpc4SKAVsa4SoeAMrdQu7vrYA2T0mPrI1HW25+W+hBC3/VAzng3y7Db3tej2cKddLLlldwT6l3hjUiJjnLeL734n5mkECrlkxjQ9N8FWxjQKLiEv+KmBhPE+60LTUsUiMK1s9FNOt6zSoWGsbSmkI/X7RMYiYwZRYDsjhj3z2xuK/3nNFMOjViZUkiIo/rUoTCXFmA4Doh2hgaMcWMK4FvZWynvMhoE2xoINwfv98l9ytV/2DsqVi0qpejKOY45skE2yTTxySKrkjJyTGuHkjjyQJ/Ls3DuPzovz+tU64Yxn1skPOG+f/SieRQ==</latexit>

✓ = tan
x

Feature Space

Data might not be linearly separable.

Northeastern 4
UNIVERSITY Advanced Computer Vision
 Robust
Systems

Feature Transformations Lab

Original Space
<latexit sha1_base64="5n3ZJc68LBmweYlVkQsqcGz72Lc=">AAAB/XicbVDJSgNBEO1xjXEbl5uXxiAIQpgJQb0IQS8eI5gFkkno6fQkTXoWu2vEcQj+ihcPinj1P7z5N3aSOWjig4LHe1VU1XMjwRVY1rexsLi0vLKaW8uvb2xubZs7u3UVxpKyGg1FKJsuUUzwgNWAg2DNSDLiu4I13OHV2G/cM6l4GNxCEjHHJ/2Ae5wS0FLX3Jf4ArfVnYT0oVPCJzjplEZds2AVrQnwPLEzUkAZql3zq90LaeyzAKggSrVsKwInJRI4FWyUb8eKRYQOSZ+1NA2Iz5STTq4f4SOt9LAXSl0B4In6eyIlvlKJ7+pOn8BAzXpj8T+vFYN37qQ8iGJgAZ0u8mKBIcTjKHCPS0ZBJJoQKrm+FdMBkYSCDiyvQ7BnX54n9VLRPi2Wb8qFymUWRw4doEN0jGx0hiroGlVRDVH0iJ7RK3oznowX4934mLYuGNnMHvoD4/MHkIaUBQ==</latexit>

p
r = x2 +⇣ y 2⌘
1 y
<latexit sha1_base64="WwJYM7ko+jY2fc/b2sN1vHG/Cn8=">AAACE3icbVA9SwNBEN3z2/gVtbRZDIIKhjsJaiMEbSwVjAnkYtjbzCVL9vaO3TkxHPcfbPwrNhaK2NrY+W/cxBR+PRh4vDfDzLwgkcKg6344E5NT0zOzc/OFhcWl5ZXi6tqViVPNocZjGetGwAxIoaCGAiU0Eg0sCiTUg/7p0K/fgDYiVpc4SKAVsa4SoeAMrdQu7vrYA2T0mPrI1HW25+W+hBC3/VAzng3y7Db3tej2cKddLLlldwT6l3hjUiJjnLeL734n5mkECrlkxjQ9N8FWxjQKLiEv+KmBhPE+60LTUsUiMK1s9FNOt6zSoWGsbSmkI/X7RMYiYwZRYDsjhj3z2xuK/3nNFMOjViZUkiIo/rUoTCXFmA4Doh2hgaMcWMK4FvZWynvMhoE2xoINwfv98l9ytV/2DsqVi0qpejKOY45skE2yTTxySKrkjJyTGuHkjjyQJ/Ls3DuPzovz+tU64Yxn1skPOG+f/SieRQ==</latexit>

✓ = tan
x

Feature Space

Data might not be linearly separable.

Northeastern 5
UNIVERSITY Advanced Computer Vision
 Building A Complicated Function
Given a library of simple functions

Idea 2: Compositions
Compose into a
• Deep Learning

complicate function • Grammar models

• Scattering transforms…

(C) Dhruv Batra Slide Credit: Marc'Aurelio Ranzato, Yann LeCun 6

 Robust
Systems

Neural Networks: Use Multiple Layers Lab

Wx
<latexit sha1_base64="(null)">(null)</latexit>

Northeastern 7
UNIVERSITY Advanced Computer Vision
 Robust
Systems

Neural Networks: Use Multiple Layers Lab

Wx
<latexit sha1_base64="(null)">(null)</latexit>

Linear score: f (x, W ) = W x

<latexit sha1_base64="(null)">(null)</latexit>

Concatenated score: f (x, W ) = W2 (W1 x)

<latexit sha1_base64="(null)">(null)</latexit>

Still linear score!: f (x, W ) = W2 (W1 x) = (W1 W2 )x = W x

<latexit sha1_base64="(null)">(null)</latexit>

Northeastern 8
UNIVERSITY Advanced Computer Vision
 Robust
Systems

Neural Networks: Use Multiple Layers Lab

Pooling

Wx
<latexit sha1_base64="(null)">(null)</latexit>

Non-linear activations
Northeastern 9
UNIVERSITY Advanced Computer Vision
 Robust
Systems

Neural Networks: Use Multiple Layers Lab

Wx
<latexit sha1_base64="(null)">(null)</latexit>

Linear score: f (x, W ) = W x

<latexit sha1_base64="(null)">(null)</latexit>

max(., .) Concatenated score:

<latexit sha1_base64="(null)">(null)</latexit>

Linear Classi er

f (x, W ) = W2 (max(0, W1 x))

<latexit sha1_base64="(null)">(null)</latexit>

Feature Extraction

Linear Classi er
Feature Extraction

2-layer Neural Net (1-hidden layer NN)

Northeastern 10
UNIVERSITY Advanced Computer Vision
fi
fi
 Robust
Systems

Neural Networks: Use Multiple Layers Lab

Wx
<latexit sha1_base64="(null)">(null)</latexit>

Linear score: f (x, W ) = W x

<latexit sha1_base64="(null)">(null)</latexit>

max(., .) Concatenated score:

<latexit sha1_base64="(null)">(null)</latexit>

f (x, W ) = W2 (max(0, W1 x))

<latexit sha1_base64="(null)">(null)</latexit>

Multiple templates per class

Northeastern 11
UNIVERSITY Advanced Computer Vision
 Robust
Systems

Neural Networks: Use Multiple Layers Lab

Wx
<latexit sha1_base64="(null)">(null)</latexit>

Linear score: f (x, W ) = W x

<latexit sha1_base64="(null)">(null)</latexit>

max(., .) Concatenated score:

<latexit sha1_base64="(null)">(null)</latexit>

f (x, W ) = W2 (max(0, W1 x))

<latexit sha1_base64="(null)">(null)</latexit>

Linear combination of templates

Northeastern 12
UNIVERSITY Advanced Computer Vision
 Robust
Systems

Neural Networks: Use Multiple Layers Lab

Wx
<latexit sha1_base64="(null)">(null)</latexit>

Linear score: f (x, W ) = W x

<latexit sha1_base64="(null)">(null)</latexit>

max(., .) Concatenated score:

<latexit sha1_base64="(null)">(null)</latexit>

f (x, W ) = W3 (max(0, W2 .(max(0, W1 x)))

<latexit sha1_base64="(null)">(null)</latexit>

Fully connected NN
Multi-Layer Perceptron (MLP)

Northeastern 13
UNIVERSITY Advanced Computer Vision
 Robust
Systems

Fully Connected Neural Net

Lab

200 x 200 pixels: 40K inputs

Suppose 40K hidden inputs

1600 M ~2B parameters!

14
Northeastern
UNIVERSITY Advanced Computer Vision
 Robust
Systems

Locally Connected Net

Lab

200 x 200 pixels: 40K inputs

lter size: 10x10

4M parameters!

Note: this parameterization is good when the input image is

registered (e.g., face recognition)
15
Northeastern
UNIVERSITY Advanced Computer Vision
fi
 Robust
Systems

Locally Connected Net

Lab

Stationarity: statistics are similar at different locations

200 x 200 pixels: 40K inputs

lter size: 10x10

Note: this parameterization is good when the input image is

registered (e.g., face recognition)
16
Northeastern
UNIVERSITY Advanced Computer Vision
fi
 Robust
Systems

Convolutional Layer
Lab

Stationarity: statistics are similar at different locations

200 x 200 pixels: 40K inputs

lter size: 10x10

Convolutions with learned kernels: share the same parameters

across different locations (assuming input is stationary)
17
Northeastern
UNIVERSITY Advanced Computer Vision
fi
 Convolutional Layer

(C) Dhruv Batra Slide Credit: Marc'Aurelio Ranzato 18

 Convolutional Layer

(C) Dhruv Batra Slide Credit: Marc'Aurelio Ranzato 19

 Convolutional Layer

(C) Dhruv Batra Slide Credit: Marc'Aurelio Ranzato 20

 Convolutional Layer

(C) Dhruv Batra Slide Credit: Marc'Aurelio Ranzato 21

 Convolutional Layer

(C) Dhruv Batra Slide Credit: Marc'Aurelio Ranzato 22

 Convolutional Layer

(C) Dhruv Batra Slide Credit: Marc'Aurelio Ranzato 23

 Convolutional Layer

(C) Dhruv Batra Slide Credit: Marc'Aurelio Ranzato 24

 Convolutional Layer

(C) Dhruv Batra Slide Credit: Marc'Aurelio Ranzato 25

 Convolutional Layer

(C) Dhruv Batra Slide Credit: Marc'Aurelio Ranzato 26

 Convolutional Layer

(C) Dhruv Batra Slide Credit: Marc'Aurelio Ranzato 27

 Convolutional Layer

(C) Dhruv Batra Slide Credit: Marc'Aurelio Ranzato 28

 Convolutional Layer

(C) Dhruv Batra Slide Credit: Marc'Aurelio Ranzato 29

 Convolutional Layer

(C) Dhruv Batra Slide Credit: Marc'Aurelio Ranzato 30

 Convolutional Layer

(C) Dhruv Batra Slide Credit: Marc'Aurelio Ranzato 31

 Convolutional Layer

(C) Dhruv Batra Slide Credit: Marc'Aurelio Ranzato 32

 Convolutional Layer

Mathieu et al. “Fast training of CNNs through FFTs” ICLR 2014

(C) Dhruv Batra Slide Credit: Marc'Aurelio Ranzato 33
From Filtering to CNNs

34
 From Filtering to CNNs
If the lter is [-1 1]

35
fi
 From Filtering to CNNs
If the lter is [-1 1] Vertical Edges

36
fi
From Filtering to CNNs
How about having lots of lters?
vertical edges,
horizontal edges,
corners,
dots,
etc

A Filter Bank!

37
fi
Convolutional Layer: Depth Dimensions

38
Convolutional Layer: Depth Dimensions
All have same depth.
(3)

The depth of the output is the number of lters

(4)

39
fi
 Robust
Systems

Convolutional Layer Lab

Northeastern 40
UNIVERSITY Advanced Computer Vision
 Robust
Systems

Convolutional Layer Lab

Northeastern 41
UNIVERSITY Advanced Computer Vision
 Robust
Systems

Convolutional Layer Lab

Northeastern 42
UNIVERSITY Advanced Computer Vision
 Robust
Systems

Convolutional Layer Lab

Northeastern 43
UNIVERSITY Advanced Computer Vision
 Robust
Systems

Convolutional Layer Lab

Northeastern 44
UNIVERSITY Advanced Computer Vision
 Robust
Systems

Convolutional Layer: Spatial Dimensions Lab

Northeastern 45
UNIVERSITY Advanced Computer Vision
 Robust
Systems

Convolutional Layer: Spatial Dimensions Lab

Northeastern 46
UNIVERSITY Advanced Computer Vision
 Robust
Systems

Convolutional Layer: Spatial Dimensions Lab

Northeastern 47
UNIVERSITY Advanced Computer Vision
 Robust
Systems

Convolutional Layer: Spatial Dimensions Lab

Northeastern 48
UNIVERSITY Advanced Computer Vision
 Robust
Systems

Convolutional Layer: Spatial Dimensions Lab

Northeastern 49
UNIVERSITY Advanced Computer Vision
 Robust
Systems

Convolutional Layer: Spatial Dimensions Lab

Northeastern 50
UNIVERSITY Advanced Computer Vision
 Robust
Systems

Convolutional Layer: Spatial Dimensions Lab

N=7, F=3

Output = N - (F-1)/2 - (F-1)/2 = N - F + 1 = 7 - 3 + 1 = 5

Northeastern 51
UNIVERSITY Advanced Computer Vision
 Robust
Systems

Convolutional Layer: Spatial Dimensions Lab

Northeastern 52
UNIVERSITY Advanced Computer Vision
 Robust
Systems

Convolutional Layer: Spatial Dimensions Lab

Northeastern 53
UNIVERSITY Advanced Computer Vision
 Robust
Systems

Convolutional Layer: Spatial Dimensions Lab

Northeastern 54
UNIVERSITY Advanced Computer Vision
 Robust
Systems

Convolutional Layer: Spatial Dimensions Lab

(N-F)/S + 1 = (7-3)/2 + 1 = 3

Northeastern 55
UNIVERSITY Advanced Computer Vision
 Robust
Systems

Convolutional Layer: Spatial Dimensions Lab

(N-F)/S + 1 = (7-3)/3 + 1 = 2.33 Does not t!

Northeastern 56
UNIVERSITY Advanced Computer Vision
fi
 Robust
Systems

Convolutional Layer: Spatial Dimensions Lab

Northeastern 57
UNIVERSITY Advanced Computer Vision
 Robust
Systems

Convolutional Layer: Spatial Dimensions Lab

Northeastern 58
UNIVERSITY Advanced Computer Vision
 Robust
Systems

Using Stride > 1

Lab

output: (N-F+2P)/S + 1

N=5,F=3,P=1,S=1 N=5,F=3,P=1,S=2
(5-3+2)/1 + 1 = 5 (5-3+2)/2 + 1 = 3

59
Northeastern
UNIVERSITY Advanced Computer Vision
 Robust
Systems

Without Padding …
Lab

Spatial dimensions keep shrinking

60
Northeastern
UNIVERSITY Advanced Computer Vision
 Robust
Systems

Example Lab

output: (N-F+2P)/S + 1

Output:
Depth:

Spatial dimensions:

Number of parameters:

Northeastern 61
UNIVERSITY Advanced Computer Vision
 Robust
Systems

Example Lab

output: (N-F+2P)/S + 1

Output:
Depth: 10

Spatial dimensions:

Number of parameters:

Northeastern 62
UNIVERSITY Advanced Computer Vision
 Robust
Systems

Example Lab

output: (N-F+2P)/S + 1

Output:
Depth: 10

Spatial dimensions: (32 + 2x2 - 5)/1 + 1 = 32

Number of parameters:

Northeastern 63
UNIVERSITY Advanced Computer Vision
 Robust
Systems

Example Lab

output: (N-F+2P)/S + 1

Output:
Depth: 10

Spatial dimensions: (32 + 2x2 - 5)/1 + 1 = 32

Number of parameters: 10 x (5 x 5 x 3 + 1) = 760

Northeastern 64
UNIVERSITY Advanced Computer Vision
 Robust
Systems

Example Lab

Northeastern 65
UNIVERSITY Advanced Computer Vision
 Robust
Systems

Example Lab

Northeastern 66
UNIVERSITY Advanced Computer Vision
 Robust
Systems

Example Lab

(stride = 1)

Northeastern 67
UNIVERSITY Advanced Computer Vision
 Robust
Systems

Receptive Fields Lab

Northeastern 68
UNIVERSITY Advanced Computer Vision
 Robust
Systems

Receptive Fields Lab

For large images, it might take many layers to “see” the whole image.

Northeastern 69
UNIVERSITY Advanced Computer Vision
 Robust
Systems

Other Types of Layer Lab

Northeastern 70
UNIVERSITY Advanced Computer Vision
 Robust
Systems

Pooling Layer Lab

Northeastern 71
UNIVERSITY Advanced Computer Vision
 Robust
Systems

Max pooling Lab

Brings invariance to small shifts

Northeastern 72
UNIVERSITY Advanced Computer Vision
 Robust
Systems

Max Pooling Lab

Northeastern 73
UNIVERSITY Advanced Computer Vision
 Robust
Systems

Pooling Layer Examples

Lab

Max-pooling:

Average-pooling:

L2-pooling:

L2-pooling over features:

Slide Credit: Marc'Aurelio Ranzato

74
Northeastern
UNIVERSITY Advanced Computer Vision
 Robust
Systems
Lab

Northeastern
UNIVERSITY
 Robust
Systems

Karpathy’s Live Demo Lab

https://cs.stanford.edu/people/karpathy/convnetjs/demo/cifar10.html

Northeastern 76
UNIVERSITY Advanced Computer Vision
 Robust
Systems
Lab

CNN Architectures

Northeastern 77
UNIVERSITY Advanced Computer Vision
 Robust
Systems

LeNet-5 [LeCun et al, 1998]

Lab

Northeastern 78
UNIVERSITY Advanced Computer Vision
2012
 Robust
Systems

AlexNet Lab

Output volume:

Northeastern 81
UNIVERSITY Advanced Computer Vision
 Robust
Systems

AlexNet Lab

Output volume: 96 x [(227 - 11)/4 +1] x [(227 - 11)/4 +1]

96 x 55 x 55

Northeastern 82
UNIVERSITY Advanced Computer Vision
 Robust
Systems

AlexNet Lab

Output volume: 96 x [(227 - 11)/4 +1] x [(227 - 11)/4 +1]

96 x 55 x 55

Number of Parameters in the layer:

Northeastern 83
UNIVERSITY Advanced Computer Vision
 Robust
Systems

AlexNet Lab

Output volume: 96 x [(227 - 11)/4 +1] x [(227 - 11)/4 +1]

96 x 55 x 55

Number of Parameters in the layer: 96 x (11 x 11 x 3) = 35K

Northeastern 84
UNIVERSITY Advanced Computer Vision
 Robust
Systems

AlexNet Lab

Output volume:

Northeastern 85
UNIVERSITY Advanced Computer Vision
 Robust
Systems

AlexNet Lab

Output volume: 96 x [(55 - 3)/2 +1] x [(55 - 3)/2 +1]

96 x 27 x 27

Number of Parameters in the layer:

Northeastern 86
UNIVERSITY Advanced Computer Vision
 Robust
Systems

AlexNet Lab

Output volume: 96 x [(55 - 3)/2 +1] x [(55 - 3)/2 +1]

96 x 27 x 27

Number of Parameters in the layer: 0

Northeastern 87
UNIVERSITY Advanced Computer Vision
 Robust
Systems

AlexNet Lab

Northeastern 89
UNIVERSITY Advanced Computer Vision
 Robust
Systems

AlexNet Lab

Northeastern 90
UNIVERSITY Advanced Computer Vision
 Robust
Systems

AlexNet Lab

Northeastern 91
UNIVERSITY Advanced Computer Vision
 Robust
Systems

ImageNet Challenge Winners Lab

Northeastern 92
UNIVERSITY Advanced Computer Vision
 Robust
Systems

ImageNet Challenge Winners Lab

Northeastern 93
UNIVERSITY Advanced Computer Vision
 2012 Performance
A. Krizhevsky, I. Sutskever, and G. E. Hinton won the challenge.

94
 Robust
Systems

ImageNet Challenge Winners Lab

Northeastern 95
UNIVERSITY Advanced Computer Vision
 Robust
Systems

ZFNet [Zeller and Fergus, 2013]

Lab

Northeastern 96
UNIVERSITY Advanced Computer Vision
 Robust
Systems

ImageNet Challenge Winners Lab

Northeastern 97
UNIVERSITY Advanced Computer Vision
 Robust
Systems

VGGNet [Simonyan and Zisserman, 2014]

Lab

Northeastern 98
UNIVERSITY Advanced Computer Vision
 Robust
Systems

VGGNet [Simonyan and Zisserman, 2014]

Lab

Northeastern 99
UNIVERSITY Advanced Computer Vision
 Robust
Systems

VGGNet [Simonyan and Zisserman, 2014]

Lab

Northeastern 100
UNIVERSITY Advanced Computer Vision
 Robust
Systems

VGGNet [Simonyan and Zisserman, 2014]

Lab

Northeastern 101
UNIVERSITY Advanced Computer Vision
 Robust
Systems

VGGNet [Simonyan and Zisserman, 2014]

Lab

Northeastern 102
UNIVERSITY Advanced Computer Vision
 Robust
Systems

VGGNet [Simonyan and Zisserman, 2014]

Lab

Northeastern 103
UNIVERSITY Advanced Computer Vision
 Robust
Systems

VGGNet [Simonyan and Zisserman, 2014]

Lab

Northeastern 104
UNIVERSITY Advanced Computer Vision
 Robust
Systems

VGGNet [Simonyan and Zisserman, 2014]

Lab

Northeastern 105
UNIVERSITY Advanced Computer Vision
 Robust
Systems

VGGNet [Simonyan and Zisserman, 2014]

Lab

Northeastern 106
UNIVERSITY Advanced Computer Vision
 Robust
Systems

VGGNet [Simonyan and Zisserman, 2014]

Lab

Northeastern 107
UNIVERSITY Advanced Computer Vision
 Robust
Systems

VGGNet Lab

Northeastern 108
UNIVERSITY Advanced Computer Vision
 Robust
Systems

ImageNet Challenge Winners Lab

Northeastern 109
UNIVERSITY Advanced Computer Vision
 Robust
Systems

GoogLeNet [Szegedy et al, 2014]

Lab

Northeastern 110
UNIVERSITY Advanced Computer Vision
 Robust
Systems

GoogLeNet [Szegedy et al, 2014]

Lab

Northeastern 111
UNIVERSITY Advanced Computer Vision
 Robust
Systems

GoogLeNet [Szegedy et al, 2014]

Lab

Northeastern 112
UNIVERSITY Advanced Computer Vision
 Robust
Systems

GoogLeNet [Szegedy et al, 2014]

Lab

Northeastern 113
UNIVERSITY Advanced Computer Vision
 Robust
Systems

GoogLeNet [Szegedy et al, 2014]

Lab

Northeastern 114
UNIVERSITY Advanced Computer Vision
 Robust
Systems

GoogLeNet [Szegedy et al, 2014]

Lab

Northeastern 115
UNIVERSITY Advanced Computer Vision
 Robust
Systems

GoogLeNet [Szegedy et al, 2014]

Lab

Northeastern 116
UNIVERSITY Advanced Computer Vision
 Robust
Systems

GoogLeNet [Szegedy et al, 2014]

Lab

Northeastern 117
UNIVERSITY Advanced Computer Vision
 Robust
Systems

GoogLeNet [Szegedy et al, 2014]

Lab

Northeastern 118
UNIVERSITY Advanced Computer Vision
 Robust
Systems

GoogLeNet [Szegedy et al, 2014]

Lab

Northeastern 119
UNIVERSITY Advanced Computer Vision
 Robust
Systems

GoogLeNet [Szegedy et al, 2014]

Lab

Northeastern 120
UNIVERSITY Advanced Computer Vision
 Robust
Systems

GoogLeNet [Szegedy et al, 2014]

Lab

Northeastern 121
UNIVERSITY Advanced Computer Vision
 Robust
Systems

1x1 Conv Lab

Northeastern 122
UNIVERSITY Advanced Computer Vision
 Robust
Systems

GoogLeNet [Szegedy et al, 2014]

Lab

Northeastern 123
UNIVERSITY Advanced Computer Vision
 Robust
Systems

GoogLeNet [Szegedy et al, 2014]

Lab

Northeastern 124
UNIVERSITY Advanced Computer Vision
 Robust
Systems

GoogLeNet [Szegedy et al, 2014]

Lab

Northeastern 125
UNIVERSITY Advanced Computer Vision
 Robust
Systems

GoogLeNet [Szegedy et al, 2014]

Lab

Northeastern 126
UNIVERSITY Advanced Computer Vision
 Robust
Systems

GoogLeNet [Szegedy et al, 2014]

Lab

Northeastern 127
UNIVERSITY Advanced Computer Vision
 Robust
Systems

GoogLeNet [Szegedy et al, 2014]

Lab

Northeastern 128
UNIVERSITY Advanced Computer Vision
 Robust
Systems

GoogLeNet [Szegedy et al, 2014]

Lab

Northeastern 129
UNIVERSITY Advanced Computer Vision
 Robust
Systems

GoogLeNet [Szegedy et al, 2014]

Lab

Northeastern 130
UNIVERSITY Advanced Computer Vision
 Robust
Systems

GoogLeNet [Szegedy et al, 2014]

Lab

Northeastern 131
UNIVERSITY Advanced Computer Vision
 Robust
Systems

ImageNet Challenge Winners Lab

Northeastern 132
UNIVERSITY Advanced Computer Vision
 Robust
Systems

ResNet [He et al, 2015]

Lab

Northeastern 133
UNIVERSITY Advanced Computer Vision
 Robust
Systems

ResNet [He et al, 2015]

Lab

Is deeper better? How deep should we go?

The deeper net is doing worse, but it is not due to over tting.

Northeastern 134
UNIVERSITY Advanced Computer Vision
fi
 Robust
Systems

ResNet [He et al, 2015]

Lab

Why deeper is not doing better?

It should at least to be able to achieve the same performance!

Why?

Hypothesis: It is an optimization problem.

Northeastern 135
UNIVERSITY Advanced Computer Vision
 Robust
Systems

ResNet [He et al, 2015]

Lab

Solution: Help the net learning this solution and improve from
there.

The layers learn the “residual” F(x) = H(x) - x instead of H(x)

Northeastern 136
UNIVERSITY Advanced Computer Vision
 Robust
Systems

ResNet [He et al, 2015]

Lab

ResNet stacks residual blocks. Each block has 3x3 conv.

Northeastern 137
UNIVERSITY Advanced Computer Vision
 Robust
Systems

ResNet [He et al, 2015]

Lab

Northeastern 138
UNIVERSITY Advanced Computer Vision
 Robust
Systems

ResNet [He et al, 2015]

Lab

Northeastern 139
UNIVERSITY Advanced Computer Vision
 Robust
Systems

ResNet [He et al, 2015]

Lab

Northeastern 140
UNIVERSITY Advanced Computer Vision
 Robust
Systems

ResNet [He et al, 2015]

Lab

Northeastern 141
UNIVERSITY Advanced Computer Vision
 Robust
Systems

ResNet [He et al, 2015]

Lab

Northeastern 142
UNIVERSITY Advanced Computer Vision
 Robust
Systems

ResNet [He et al, 2015]

Lab

Northeastern 143
UNIVERSITY Advanced Computer Vision
 Robust
Systems

Training ResNet Lab

Northeastern 144
UNIVERSITY Advanced Computer Vision
 Robust
Systems

ResNet [He et al, 2015]

Lab

Northeastern 145
UNIVERSITY Advanced Computer Vision
 Robust
Systems
Lab

Northeastern 146
UNIVERSITY Advanced Computer Vision
 Robust
Systems
Lab

Northeastern 147
UNIVERSITY Advanced Computer Vision
 Robust
Systems
Lab

Northeastern 148
UNIVERSITY Advanced Computer Vision
 Robust
Systems
Lab

Northeastern 149
UNIVERSITY Advanced Computer Vision
 Robust
Systems
Lab

Northeastern 150
UNIVERSITY Advanced Computer Vision
 Robust
Systems
Lab

Northeastern 151
UNIVERSITY Advanced Computer Vision