ECE 4554 / ECE 5554

Computer Vision

Edge Detection
Virginia Tech
Fall 2023

[Credit to Prof. Creed Jones for several of these slides]

1
What do you
see here?

2
What do you
see here?

3
 original

regions edges

4
 Feature extraction

• How can we begin to extract useful information from an image?

• Some common feature types are related to image regions and intensity edges

• Edge: a sudden change in pixel values (within the 2D image space)

– Many methods have been developed
– Most can be characterized as high-pass filters
– Main topic for today!

• Region: a connected portion of an image

– Pixels within a region are considered to be similar to one another,
according to some measure
– Regions are often produced by image segmentation procedures, a topic for later
– For the previous example, simple thresholding was applied

5
 Thresholding

• Compare every pixel value with a given constant T, called the threshold value, and
assign an output value based on the comparison
• Example:

1 if I ( r , c ) ≥ T
I NEW ( r , c ) = 
0 if I ( r , c ) < T

• This is an example of binarization

• In some cases, the comparisons are reversed
• In some cases, different output values are used
• Output values may be associated with “foreground” and “background”

6
 7

T = 213
T = 137
Examples of thresholding

T = 159
T = 124
EDGE DETECTION

8
An edge is an abrupt change in image intensity

9
 What can cause an intensity edge?

Sudden change
Depth in reflectance:
discontinuity here, paint on
object’s surface

Sudden change in Sudden change

3D surface orientation in illumination:
here, harsh shadows
10
• Consider a "profile" plot of intensity
values, taken approximately from the
row indicated

• This information is 1-dimensional

• We can let x represent the horizontal

direction, and let f(x) represent the
image intensity

• How do you normally detect change

in some function f(x)?

11
 Using the first derivative to detect edges

• Ideal case:
intensity function
𝑑𝑑𝑑𝑑 𝑓𝑓 𝑥𝑥 + ∆𝑥𝑥 − 𝑓𝑓 𝑥𝑥 image (along horizontal scanline) first derivative
= lim
𝑑𝑑𝑑𝑑 Δ𝑥𝑥→0 ∆𝑥𝑥

• Discrete approximation:

𝑑𝑑𝑑𝑑 𝑓𝑓 𝑥𝑥 + ∆𝑥𝑥 − 𝑓𝑓 𝑥𝑥
≈
𝑑𝑑𝑑𝑑 ∆𝑥𝑥

edges correspond to
extrema of derivative

12
 Now extend this idea to a 2-dimensional image I(x, y)

𝜕𝜕𝐼𝐼 𝐼𝐼 𝑥𝑥 + ∆𝑥𝑥, 𝑦𝑦 − 𝐼𝐼 𝑥𝑥, 𝑦𝑦

≈
𝜕𝜕𝜕𝜕 ∆𝑥𝑥

𝜕𝜕𝜕𝜕 𝐼𝐼 𝑥𝑥, 𝑦𝑦 + ∆𝑦𝑦 − 𝐼𝐼 𝑥𝑥, 𝑦𝑦

≈
𝜕𝜕𝑦𝑦 ∆𝑦𝑦

• ∆𝑥𝑥 and ∆𝑦𝑦 may be defined to be one pixel, or some other value that we choose

• These are the x- and y-components of the rate of change in intensity

13
Mandrill: “a large fierce gregarious baboon of western Africa”

14
Convert to gray-scale; compute 𝐼𝐼𝑥𝑥 𝑟𝑟, 𝑐𝑐 = 𝐼𝐼 𝑟𝑟, 𝑐𝑐 + 1 − 𝐼𝐼 𝑟𝑟, 𝑐𝑐 ;
take the absolute value; and then threshold using T = 20

15
The derivative approximation 𝐼𝐼𝑥𝑥 𝑟𝑟, 𝑐𝑐 = 𝐼𝐼 𝑟𝑟, 𝑐𝑐 + 1 − 𝐼𝐼 𝑟𝑟, 𝑐𝑐 can be
represented using a 2D kernel

• Here is an equivalent 3x3 kernel (assuming cross-correlation)

0 0 0
0 -1 1
0 0 0

• This isn’t much of a kernel – but it illustrates the fact that we can represent
derivative operations using our linear-filtering concepts

16
 Consider some alternatives

0 0 0 0 0 0 -1 0 1 -1 0 1
1 1
0 -1 1 -1 0 1 -1 0 1 ×
3
-2 0 2 ×
4
0 0 0 0 0 0 -1 0 1 -1 0 1

Recall our starting point:

𝜕𝜕𝐼𝐼 𝐼𝐼 𝑥𝑥 + ∆𝑥𝑥, 𝑦𝑦 − 𝐼𝐼 𝑥𝑥, 𝑦𝑦
≈
𝜕𝜕𝜕𝜕 ∆𝑥𝑥

17
 Now consider the vertical direction

0 1 0 0 1 0 1 1 1 1 2 1
1 1
0 -1 0 0 0 0 0 0 0 ×
3
0 0 0 ×
4
0 0 0 0 -1 0 -1 -1 -1 -1 -2 -1

Recall our starting point:

𝜕𝜕𝜕𝜕 𝐼𝐼 𝑥𝑥, 𝑦𝑦 + ∆𝑦𝑦 − 𝐼𝐼 𝑥𝑥, 𝑦𝑦
≈
𝜕𝜕𝜕𝜕 ∆𝑦𝑦

18
 Some well-known templates for edge detection

 − 1 0 1 1 2 1
 − 2 0 2 0 
Sobel    0 0 
 − 1 0 1  − 1 − 2 − 1

− 1 0 1 1 1 1
Prewitt − 1 0 1 0 0 0
   
− 1 0 1 − 1 − 1 − 1

Roberts 1 0  0 − 1
0 −1 1 0 
   

img0 = ...
sobelx = cv2.Sobel(img0, cv2.CV_64F, 1, 0, ksize=3) # x
sobely = cv2.Sobel(img0, cv2.CV_64F, 0, 1, ksize=3) # y
https://www.bogotobogo.com/python/OpenCV_Python/python_opencv3_Image_Gradient_Sobel_Laplacian_Derivatives_Edge_Detection.php

19
 -1 0 1 1 2 1
-2 0 2 0 0 0
Ix -1 0 1 Iy -1 -2 -1

20
 We can combine
Original the 2 results:
image: I | Ix | + | I y |

21
Q: How should the partial derivatives be combined?
A: Consider the gradient vector:
I x 
∇I =  
I y 
2 2
Magnitude of gradient: ∇I= Ix + I y

−1  Iy

Direction of gradient: 
θ = tan  
 Ix 

22
 SOBEL
Magnitude

23
 We can also examine the direction component of the Sobel gradient.
It naturally ranges from −𝜋𝜋 to 𝜋𝜋 radians, so we can scale it the range 0 to
255 for display; but remember, angle is a circular quantity!

24
 We can also examine the direction component of the Sobel gradient.
It naturally ranges from −𝜋𝜋 to 𝜋𝜋 radians, so we can scale it the range 0 to
255 for display; but remember, angle is a circular quantity!

25
 When is the angle component ever used?
Example: if you want an image consisting of strong edges that are
diagonal, then you could multiply the Sobel magnitude times the
difference of the angle from 45 degrees

26
CANNY EDGE DETECTION

27
 A widely used method for edge detection
was developed by Canny in the mid-1980’s

http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.420.3300&rep=rep1&type=pdf
28
 Canny’s idea was to find edge pixels that satisfy a few criteria
on what a “good” representation of an edge would look like
– assuming a noisy image

• Low error rate

– Don’t mark any non-edge pixels as edges
– Don’t miss marking any edge pixels
• Good localization
– The detected edge pixels should correspond well to the actual edges in the image
• Avoid multiple responses
– We want each edge to be 1 pixel in width – not a “thick” outline

29
 The Canny procedure uses both an estimate of the gradient
and local neighborhood information to produce a skeleton representation

There are five steps to the Canny edge-detection process:

• Smooth the image using a Gaussian filter
• Approximate the gradient (usually with Sobel kernels)
• Perform non-maximum suppression to thin the edge and localize to its max value
• Binarize the image, using a dual-threshold procedure
• Optionally, perform edge tracking to join shorter edge elements into longer
curves

30
Here’s an example from
Canny’s paper

31
 Some parameters must be set:
width of Gaussian, and upper and lower thresholds for the edge strength

32
 First step: the image is smoothed using a Gaussian filter
– but what is the right 𝜎𝜎 to use?

• If 𝜎𝜎 is small, then the edges will be more sensitive to noise

– but edges will be located more accurately
• If 𝜎𝜎 is large, then image noise will be suppressed more
– but edge localization may be less accurate
2 4 5 4 2
1
• A popular value is 𝜎𝜎 = 1.4 pixels × 4 9 12 9 4
115
5 12 15 12 5
4 9 12 9 4
2 4 5 4 2

33
Canny step 1: smooth the original image using a Gaussian kernel
– in this case, 𝜎𝜎 = 1.4

34
Canny step 2: approximate the gradient (usually with Sobel kernels);
both magnitude and direction are extracted

35
Canny step 3: compare the Sobel magnitude (edge strength) to each pixel
in the neighborhood in the direction of max gradient
and suppress (set to zero) all edge pixels that are not the maximum

36
 Canny step 4: binarize the image, using a dual-threshold procedure

• First, threshold the “edge image” with an initial, higher threshold – any pixel with
edge intensity above this threshold is marked as an edge pixel
• Then, re-examine the edge image: any pixel with edge intensity above a second,
lower threshold that is a neighbor of a pixel that is above the first threshold is
also marked as an edge pixel
• What does it mean for 2 pixels to be “neighbors”?
For a 3x3 neighborhood, there are two kinds of connectivity (neighbors)
– usually, 8-connectivity is used

8-neighbors of pixel A 4-neighbors of pixel A

(8-connectivity) A (4-connectivity) A

37
 Canny step 5: optionally, perform edge tracking
to join shorter edge elements into longer curves

• Sometimes, the edge is tracked (via a contour-following process)

either to remove some of the pixels below the high threshold,
or to fill in gaps below a certain size
• This step is not always done, as it relies on assumptions about the content of the
image

38
Example of Canny edge detection

39
40
Original image Canny example

41
 Canny example

σ =1 σ =2
σ =3 σ =4

42
 Some notes about the Canny edge detector

• The parameters, especially sigma, can affect the results dramatically

• The output is typically a binary image indicating edge points
• Many methods make use of the Canny detector to identify edge locations, and
then use other procedures to get useful information related to those edges
– Sobel magnitude for edge strength
– Sobel angle for edge direction
– Original image for object intensity
• Edge polarity is lost: no distinction between light-to-dark and dark-to-light
transitions
– although the method can be modified to filter on polarity

43
MORE EDGE DETECTORS

44
People have considered improvements to the Canny procedure

• Deriche (1987) replaced the Sobel with a different edge operator, designed to
optimize the Canny criteria
– He replaced a FIR (Finite Impulse Response) filter with an IIR (Infinite Impulse
Response) filter
– This has computational advantages but is less well-tempered on edges of all directions

• Others have proposed enhancements for better noise rejection

– Wang and He 2004 - http://en.cnki.com.cn/Article_en/CJFDTOTAL-ZGTB200408010.htm
– Rong, Li and Zhang 2014 - http://ieeexplore.ieee.org/abstract/document/6885761/

45
 “Compass” edge detection

 − 1 0 1 0 1 2 1 2 1 2 1 0
 − 2 0 2  − 1 0 1 0 0 0  1 0 − 1 
       
 − 1 0 1  − 2 − 1 0 − 1 − 2 − 1 0 − 1 − 2
0° 45° 90° 135°
• At each pixel location, compute responses to several
templates
• The largest magnitude determines the “winner”
(so, this is a nonlinear procedure)
• These are sometimes called “generalized Sobel masks”

46
 Another “compass” example:
Nevatia-Babu masks
− 100 − 100 0 100 100 − 100 32 100 100 100  100 100 100 100 100
− 100 − 100 0 100 100 − 100 − 78 92 100 100
 − 32
 78 100 100 100
 
− 100 − 100 0 100 100 − 100 − 100 0 100 100 − 100 − 92 0 92 100
     
− 100 − 100 0 100 100 − 100 − 100 − 92 78 100 − 100 − 100 − 100 − 78 32 
− 100 − 100 0 100 100 − 100 − 100 − 100 − 32 100 − 100 − 100 − 100 − 100 100

0° 30° 60°
 100 100 100 100 100   100 100 100 100 100  100 100 100 32 − 100
 100
 100 100 100 100 

 100
 100 100 78 − 32 

100
 100 92 − 78 − 100
 0 0 0 0 0   100 92 0 − 92 − 100 100 100 0 − 100 − 100
     
− 100 − 100 − 100 − 100 − 100  32 − 78 − 100 − 100 − 100 100 78 − 92 − 100 − 100
− 100 − 100 − 100 − 100 − 100 − 100 − 100 − 100 − 100 − 100 100 − 32 − 100 − 100 − 100

90° 120° 150°

47
USING 2ND DERIVATIVES

48
 We have looked at 1st-derivative operators
What about using 2nd derivatives?

• The simplest nondirectional 2nd-derivative operator

is the Laplacian:
2 2
∂ I ∂ I
L ( x, y ) = ∇ I ( x, y ) = 2 + 2
2

∂x ∂y

• Here are common approximations to the Laplacian:

0 1 0 1 4 1
1 −4 1 4 −20 4
0 1 0 1 4 1

49
50
Result of Laplacian

51
 The result of the Laplacian operator can be added back
to the original image to enhance the edges in the image

52
In 1979, Marr, Hildreth, and Poggio caused a lot of excitement when they
suggested the combination of Laplacian and Gaussian filtering

• Differentiation and convolution are both associative and commutative

𝐼𝐼𝑛𝑛𝑛𝑛𝑛𝑛 𝑥𝑥, 𝑦𝑦 = 𝛻𝛻 2 𝐼𝐼 𝑥𝑥, 𝑦𝑦 ∗ 𝐺𝐺 𝑥𝑥, 𝑦𝑦

• Do this, then detect zero-crossings in the result

• Implement as:
𝐼𝐼𝑛𝑛𝑛𝑛𝑛𝑛 𝑥𝑥, 𝑦𝑦 = 𝛻𝛻 2 𝐼𝐼 𝑥𝑥, 𝑦𝑦 ∗ 𝐺𝐺 𝑥𝑥, 𝑦𝑦 = 𝛻𝛻 2 𝐼𝐼 𝑥𝑥, 𝑦𝑦 ∗ 𝐺𝐺 𝑥𝑥, 𝑦𝑦 = 𝐼𝐼 𝑥𝑥, 𝑦𝑦 ∗ 𝛻𝛻 2 𝐺𝐺 𝑥𝑥, 𝑦𝑦

• The Laplacian-of-Gaussian (LoG) operator: 𝛻𝛻 2 𝐺𝐺 𝑥𝑥, 𝑦𝑦

53
2D edge detection filters

Laplacian of Gaussian

Gaussian derivative of Gaussian

𝐺𝐺 𝜎𝜎 𝐺𝐺 𝜎𝜎 𝐺𝐺 𝜎𝜎

• is the Laplacian operator:

54
 𝛻𝛻 2 𝐺𝐺, shown upside down

This operator has several names:

• LoG operator
• Marr-Hildreth operator
• “Mexican hat” operator
• “Sombrero” function

By carefully selecting the width of the

Gaussian, we can control the level of detail in
the result
Each choice of sigma selects a different “scale”

55
 𝜕𝜕2
Laplacian of Gaussian: 𝐺𝐺 ∗ 𝑓𝑓
𝜕𝜕𝑥𝑥 2

Consider

𝐺𝐺 Laplacian of Gaussian
operator

𝐺𝐺 *

Where is the edge? Zero-crossings of bottom graph

Slide credit: Seitz 56

 Effect of σ on derivatives

σ = 1 pixel σ = 3 pixels

The apparent structures differ, depending on σ

Larger values: coarser-scale features detected

Smaller values: finer-scale features detected

Slide credit: Grauman (adapted) 57

LoG example

σ =1 σ =2

σ =3 σ =4

58
59
Janusz Starzyk, Univ of Colorado
60
LGN: Lateral Geniculate Nucleus

61
62
 Summary: Edge Detection

• Edges vs. regions

• Thresholding

• What causes an intensity edge?

• Common edge-detection kernels
– Sobel, Prewitt, Roberts
• Canny edge detection
• Compass edge detection
• Laplacian of Gaussian (LoG) operator

63