Computer
Vision
Recording
and Displaying
of Images
Computer
Vision
Acquisition and display of images
A readers digest :
1. displays
2. cameras
3. illumination
Computer
Vision
displays
Computer
Vision
Cathode Ray Tubes
Computer
Vision
The video standard
viewing quality psychophysics
discrete character, both spatial and temporal,
should be invisible
video standard defines :
1. number of lines and columns
2. aspect ratio
3. scanning frequency
Computer
Vision
TV : number of lines
Aspect ratio of pixels = 4 /3
Europe : CCIR : 625 lines, 575 visible
USA : RETMA : 525 lines, 484 visible
(other lines lost during flyback)
Computer
Vision
TV : sampling frequency
flickerfree images : at least 60 Hz
halved by interlacing : odd and even lines
separately :
1st field
2nd
field
1
2
3
4
...
5
...
E
T
IN
Computer
Vision
G
N
I
C
A
RL
Europe : repetition of 25Hz (50/2)
USA : 30Hz (60/2)
LCD display
Liquid Crystal Display
polarizers
passive pixel
active pixel
No light generation use backlighting
Computer
Vision
LCD display
Liquid Crystal Display
polarizers
passive pixel
Electrical field
active pixel
No light generation use backlighting
Computer
Vision
cameras
Computer
Vision
Optics for image formation
the pinhole model :
X i Yi
f
= =
= m
X o Yo Z o
Computer
Vision
(m = linear magnification)
Camera obscura + lens
Computer
Vision
The thin-lens equation
lens to capture enough light :
1
1 1
=
ZO Zi f
PO
Computer
Vision
assuming
spherical lens surfaces
incoming light parallel to axis
thickness << radii
same refractive index on both sides
The depth-of-field
Z 0 = Z 0 Z 0 =
Z 0 (Z 0 f )
Z0 + f d / b f
decreases with d, increases with Z0
strike a balance between incoming light and
large depth range
Computer
Vision
Deviations from the lens model
3 assumptions :
1. all rays from a point are focused onto 1 image point
2. all image points in a single plane
3. magnification is constant
deviations from this ideal are aberrations
Computer
Vision
Aberrations
2 types :
1. geometrical
2. chromatic
geometrical : small for paraxial rays
chromatic : refractive index function of
wavelength
Computer
Vision
Geometrical aberrations
spherical aberration
astigmatism
the most important type
radial distortion
coma
aberrations are reduced by combining lenses
Computer
Vision
Radial Distortion
magnification different
for different angles of inclination
Can be corrected if parameters are known
Computer
Vision
Chromatic aberration
rays of different wavelengths focused
in different planes
cannot be removed completely
sometimes achromatization is achieved for
more than 2 wavelengths
Computer
Vision
Cameras
we consider 2 types :
1. CCD
2. CMOS
10
Computer
Vision
Cameras
Computer
Vision
Cameras
11
Computer
Vision
Cameras
Computer
Vision
The CCD camera
12
Computer
Vision
CMOS
Same sensor elements as CCD
Each photo sensor has its own amplifier
More noise (reduced by subtracting black image)
Lower sensitivity (lower fill rate)
Uses standard CMOS technology
Allows to put other components on chip
Smart pixels
Foveon
4k x 4k sensor
0.18 process
70M transistors
Computer
Vision
CCD vs. CMOS
Mature technology
Specific technology
High production cost
High power consumption
Higher fill rate
Blooming
Sequential readout
Recent technology
Standard IC technology
Cheap
Low power
Less sensitive
Per pixel amplification
Random pixel access
Smart pixels
On chip integration
with other components
2006 was year of sales cross-over
13
Computer
Vision
Colour cameras
We consider 3 concepts:
1. Prism (with 3 sensors)
2. Filter mosaic
3. Filter wheel
Computer
Vision
Prism colour camera
Separate light in 3 beams using dichroic prism
Requires 3 sensors & precise alignment
Good color separation
14
Computer
Vision
Prism colour camera
Computer
Vision
Filter mosaic
Coat filter directly on sensor
Demosaicing (obtain full colour & full resolution image)
15
Computer
Vision
Filter wheel
Rotate multiple filters in front of lens
Allows more than 3 colour bands
Only suitable for static scenes
Computer
Vision
Prism vs. mosaic vs. wheel
approach
# sensors
Separation
Cost
Framerate
Artefacts
Bands
Prism
3
High
High
High
Low
3
Mosaic
1
Average
Low
High
Aliasing
3
Wheel
1
Good
Average
Low
Motion
3 or more
High-end
cameras
Low-end
cameras
Scientific
applications
16
Computer
Vision
Upcoming X3 technology of Foveon
Exploits the wavelength
dependent depth to
which a photon
penetrates silicon
And splits colors without
the use of any filters
creates a stack of pixels at one place
new CMOS technology
Computer
Vision
Geometric camera model
perspective projection
(Man Drawing a Lute, woodcut, 1525, Albrecht Drer)
17
Computer
Vision
Models for camera projection
the pinhole model revisited :
center of the lens = center of projection
notice the virtual image plane
this is called perspective projection
Computer
Vision
Perspective projection
Zc
u
v
Xc
Yc
origin lies at the center of projection
Y
X
theuZ =
axisf coincides withvthe
=optical
f axis
X -axis to image
Z rows, Y -axis Zto columns
c
18
Computer
Vision
Pseudo-orthographic projection
u= f
X
Z
v= f
Y
Z
If Z is constant x= kX and y = kY,
where k=f/Z
i.e. orthographic projection + a scaling
Good approximation if /Z constant, i.e. if objects
are small compared to their distance from the camera
Computer
Vision
Pictoral comparison
Pseudo orthographic
Perspective
19
Computer
Vision
Projection matrices
the perspective projection model is incomplete :
what if :
1. 3D coordinates are specified in a
world coordinate frame
2. Image coordinates are expressed as
row and column numbers
We will not consider additional refinements,
such as radial distortions,...
Computer
Vision
(X,Y,Z)
Projection
matrices
(u,v)
u= f
r2
v= f
u
X
r3
r1
r1,P C
r3,P C
r2 ,P C
r3,P C
u= f
r11 ( X C1 ) + r12 (Y C2 ) + r13 ( Z C3 )
r31 ( X C1 ) + r32 (Y C2 ) + r33 ( Z C3 )
v= f
r21 ( X C1 ) + r22 (Y C2 ) + r23 ( Z C3 )
r31 ( X C1 ) + r32 (Y C2 ) + r33 ( Z C3 )
20
Computer
Vision
Projection matrices
Image coordinates are to be expressed as
pixel coordinates
x
012
0
1
2
3
x = k x u + s v + x0
k y v + y0
y =
with :
Computer
Vision
(x0, y0) the pixel coordinates of the principal point
NB7
kx the knumber
ofand
pixels
percalled
unit
length
horizontally
means
internally
NB6
NB4
they
are
are
known,
the
the
camera
camera
is
NB3
,s,x
ythe
are
internal
camera
xy,k
0 and
0known,
NB1::::fully
often
only
integer
pixel
coordinates
matter
NB2
kwhen
/kyxcalibrated
isthese
called
aspect
ratio
NB5
vector
C
matrix
R
SO
(3)and
areis
the
externally
kparameters
of pixels per unit length vertically
y the number
calibrated
externally
internally
calibrated
external camera parameters
s indicates the skew ; typically s = 0
Projection matrices
Exploiting homogeneous coordinates :
xu k x fs r11x0
yv= =0 kf y r21y0
1
1 0 0r311
fr12f 0f r013r11Xr12 C
r131 X C1
0 ff r023r21Y r22 Cr23
Y
C
fr22
2
2
0 0 1 r r r Z C
r3331Z 32 C333
3
r32
We
have
Wealso
define
the calibration matrix :
k xx s x0k xsf
K = 0 y k y= y00 k0y
1 0
0 0 10
0x0 0u f k x f s x0
fy0 0v= 0 f k y y0
01 11 0
0 1
21
Projection matrices
Computer
Vision
We define
x
p = y ;
1
X
P = Y ,
Z
X
~ Y
P =
Z
1
yielding
p = KR t ( P C )
or,
or,
for some non-zero
p = K (R t | R t C )P
~
p = ( M | t ) P with rank M = 3
~
Computer
Vision
From object radiance to pixel grey levels
After the geometric camera model...
a
camera model
2 steps:
1. from object radiance to image irradiance
2. from image irradiance to pixel grey level
22
Computer
Vision
Image irradiance and object radiance
we look at the irradiance that an object patch
will cause in the image
assumptions :
radiance R assumed known and
object at large distance compared to the focal length
Is image irradiance directly related to the radiance
of the image patch?
Computer
Vision
The viewing conditions
I=
AA
F
= R 0 2l cos 3 cos
Ai
Ai Z
=R
Al
cos4
2
f
23
Computer
Vision
The cos4 law contd
Especially strong effects
for wide-angle and
fisheye lenses
Computer
Vision
From irradiance to gray levels
f = g I + d
Gain
gamma
Dark reference
24
Computer
Vision
illumination
Computer
Vision
Illumination
Well-designed illumination often is key in
visual inspection
25
Computer
Vision
Illumination techniques
Simplify the image processing by controlling
the environment
An overview of illumination techniques:
1. back-lighting
2. directional-lighting
3. diffuse-lighting
4. polarized-lighting
5. coloured-lighting
6. structured-lighting
7. stroboscopic lighting
Computer
Vision
Back-lighting
lamps placed behind a transmitting diffuser plate,
light source behind the object
generates high-contrast silhouette images,
easy to handle with binary vision
often used in inspection
26
Computer
Vision
Example backlighting
Computer
Vision
Directional and diffuse lighting
Directional-lighting
generate sharp shadows
generation of specular reflection
(e.g. crack detection)
shadows and shading yield information about
shape
Diffuse-lighting
illuminates uniformly from all directions
prevents sharp shadows and large intensity
variations over glossy surfaces
27
Computer
Vision
Crack detection
Computer
Vision
Example directional lighting
28
Computer
Vision
Polarized lighting
2 uses:
1. to improve contrast between Lambertian and
specular reflections
2. to improve contrasts between dielectrics and
metals
Computer
Vision
Polarized lighting
specular reflection keeps polarisation :
diffuse reflection depolarises
suppression of specular reflection :
polarizer/analyzer crossed
prevents the large dynamic range caused by glare
29
Computer
Vision
Example pol. lighting (pol./an.crossed)
Computer
Vision
Polarised lighting
distinction between specular reflection from
dielectrics and metals;
works under the Brewster angle for the dielectric
dielectric has no parallel comp. ; metal does
suppression of specular reflection from dielectrics :
polarizer/analyzer aligned
distinguished metals and dielectrics
30
Computer
Vision
Example pol. lighting (pol./an. aligned)
Computer
Vision
Coloured lighting
highlight regions of a similar colour
with band-pass filter: only light from projected pattern
(e.g. monochromatic light from a laser)
differentiation between specular and diffuse reflection
comparing colours ; same spectral composition of
sources!
spectral sensitivity function of the sensors!
31
Computer
Vision
Example coloured lighting
Computer
Vision
Structured and stroboscopic lighting
spatially or temporally modulated light pattern
Structured lighting
e.g. : 3D shape : objects distort the projected
pattern
(more on this later)
Stroboscopic lighting
high intensity light flash
to eliminate motion blur
32
Computer
Vision
Stroboscopic lighting
33
