Lecturer 7:
Camera Calibration
Source credit : Svetlana Lazebnik
All Slides Credit to Svetlana Lazebnik
Dandhi Kuswardhana, Ph.D: EL462/ Visi Komputer
Our goal: Recovery of 3D structure
J. Vermeer, Music Lesson, 1662
A. Criminisi, M. Kemp, and A. Zisserman,Bringing Pictorial Space to Life: computer techniques for the
analysis of paintings, Proc. Computers and the History of Art, 2002
Things aren’t always as they appear…
http://en.wikipedia.org/wiki/Ames_room
Single-view ambiguity
X?
X?
X?
x
Single-view ambiguity
Single-view ambiguity
Rashad Alakbarov shadow sculptures
Anamorphic perspective
Image source
Anamorphic perspective
H. Holbein The Younger, The Ambassadors, 1533
https://en.wikipedia.org/wiki/Anamorphosis
Our goal: Recovery of 3D structure
• When certain assumptions • In general, we need
hold, we can recover multi-view geometry
structure from a single view
Image
source
• But first, we need to understand the geometry of a
single camera…
Camera calibration
world coordinate system
• Normalized (camera) coordinate system: camera
center is at the origin, the principal axis is the z-axis,
x and y axes of the image plane are parallel to x and y
axes of the world
• Camera calibration: figuring out transformation from
world coordinate system to image coordinate system
Review: Pinhole camera model
( X ,Y , Z ) ( f X / Z , f Y / Z )
X X
f X f 0
Y Y x = PX
Z fY = f 0
Z
Z 1 0
1
1
Principal point
• Principal point (p): point where principal axis intersects the
image plane
• Normalized coordinate system: origin of the image is at the
principal point
• Image coordinate system: origin is in the corner
Principal point offset
We want the principal
py point to map to (px, py)
instead of (0,0)
px
( X , Y , Z ) ( f X / Z + px , f Y / Z + p y )
X X
f X + Z px f px 0
Y Y
Z f Y + Z py = f py 0
Z
Z
1 0
1
1
Principal point offset
py principal point: ( px , p y )
px
X
f px 0
Y
= f py
0
Z
1 0
1
Principal point offset
py principal point: ( px , p y )
px
X X
px f p x 1 0 f px 0
Y Y
py = f py 1 0 = f py 0
Z Z
1 1 0 1 0
1
calibration matrix projection matrix
1
K [I | 0]
P = KI | 0
Pixel coordinates
1 1
Pixel size:
mx m y
mx pixels per meter in horizontal direction,
my pixels per meter in vertical direction
mx f p x x x
K = my
f p y = y y
1 1 1
pixels/m m pixels
Camera rotation and translation
• In general, the camera
coordinate frame will be
related to the world
coordinate frame by a
camera rotation and a translation
coordinate system world coordinate
system
• Conversion from world to camera coordinate system
(in non-homogeneous coordinates):
~
(
~ ~
X cam = R X − C )
coords. of point coords. of camera center
in camera frame in world frame
coords. of a point
in world frame
Camera rotation and translation
~
( ~ ~
X cam = R X − C )
3D transformation
matrix (4 x 4)
Camera rotation and translation
~
( ~ ~
X cam = R X − C )
3D transformation
matrix (4 x 4)
Camera rotation and translation
2D 3D transformation
transformation matrix (4 x 4)
perspective
matrix (3 x 3) projection
matrix (3 x 4)
Camera rotation and translation
Camera rotation and translation
~
t = − RC
Camera parameters P = K R t
• Intrinsic parameters
• Principal point coordinates
mx f p x x x
• Focal length K = my f p y = y y
• Pixel magnification factors 1 1 1
• Skew (non-rectangular pixels)
• Radial distortion
Camera parameters P = K R t
• Intrinsic parameters
• Principal point coordinates
• Focal length
• Pixel magnification factors
• Skew (non-rectangular pixels)
• Radial distortion
• Extrinsic parameters
~
• Rotation and translation relative
to world coordinate system
P = K R − RC
• What is the projection of the coords. of
camera center
camera center?
in world frame
~
~ C
PC = K R − RC = 0 The camera center is the
null space of the
1 projection matrix!
Camera calibration
x = K R t X
X
x * * * * Y
y = * * * *
Z
* * * *
1
Source: D. Hoiem
Camera calibration
• Given n points with known 3D coordinates Xi
and known image projections xi, estimate the
camera parameters
Xi
xi
P?
Camera calibration: Linear method
xi P1 X i
T
y P T X = 0
x i = PX i x i PX i = 0 i 2 i
1 P3T X i
0 − XTi yi XTi P1
T
X T
i 0 − xi X i P2 = 0
− yi XTi
T
xi X i 0 P3
Two linearly independent equations
Camera calibration: Linear method
0T XT
1 − y1X T
1
T
X1 0T
− x1X P1
T
1
P2 = 0 Ap = 0
T T
0 XTn − yn X n P3
XT 0T − xn XTn
n
• P has 11 degrees of freedom
• One 2D/3D correspondence gives us two linearly
independent equations
• 6 correspondences needed for a minimal solution
• Homogeneous least squares: find p minimizing ||Ap||2
• Solution given by eigenvector of ATA with smallest eigenvalue
Camera calibration: Linear method
0T XT
1 − y1X
T
1
T
X1 0T
− x1X P1
T
1
P2 = 0 Ap = 0
T T
0 XTn − yn X n P3
XT 0T − xn XTn
n
• Note: for coplanar points that satisfy ΠTX=0,
we will get degenerate solutions (Π,0,0),
(0,Π,0), or (0,0,Π)
Camera calibration: Linear vs. nonlinear
• Linear calibration is easy to formulate and solve,
but it doesn’t directly tell us the camera parameters
X
x * * * *
x = K R t X
Y
y = * * * *
Z vs.
* * * *
1
• In practice, non-linear methods are preferred
• Write down objective function in terms of intrinsic and extrinsic
parameters
• Define error as sum of squared distances between measured 2D
points and estimated projections of 3D points
• Minimize error using Newton’s method or other non-linear
optimization
• Can model radial distortion and impose constraints such as known
focal length and orthogonality
A taste of multi-view geometry: Triangulation
• Given projections of a 3D point in two or more
images (with known camera matrices), find
the coordinates of the point
Triangulation
• Given projections of a 3D point in two or more
images (with known camera matrices), find
the coordinates of the point
X?
x2
x1
O1 O2
Triangulation
• We want to intersect the two visual rays
corresponding to x1 and x2, but because of
noise and numerical errors, they don’t meet
exactly
X?
x2
x1
O1 O2
Triangulation: Geometric approach
• Find shortest segment connecting the two
viewing rays and let X be the midpoint of that
segment
x2
x1
O1 O2
Triangulation: Nonlinear approach
Find X that minimizes
d (x1 , P1 X) + d (x 2 , P2 X)
2 2
X?
P1X
x2
x1
P2X
O1 O2
Triangulation: Linear approach
1 x1 = P1 X x1 P1 X = 0 [x1 ]P1 X = 0
2 x 2 = P2 X x 2 P2 X = 0 [x 2 ]P2 X = 0
Cross product as matrix multiplication:
0 − az a y bx
a b = az 0 − a x by = [a ]b
− a y ax 0 bz
Triangulation: Linear approach
1 x1 = P1 X x1 P1 X = 0 [x1 ]P1 X = 0
2 x 2 = P2 X x 2 P2 X = 0 [x 2 ]P2 X = 0
Two independent equations each in terms of
three unknown entries of X
Preview: Structure from motion
Figure credit:
Noah Snavely
Camera 1 Camera 3
Camera 2
R1,t1 R2,t2 R3,t3
• Given 2D point correspondences between multiple images, compute
the camera parameters and the 3D points
Preview: Structure from motion
?
Camera 1 Camera 3
Camera 2
R1,t1 R2,t2 R3,t3
• Structure: Given known cameras and projections of the same 3D
point in two or more images, compute the 3D coordinates of that point
• Triangulation!
Preview: Structure from motion
Camera 1 Camera 3
R1,t1 ? Camera 2
R2,t2 ? ? R3,t3
• Motion: Given a set of known 3D points seen by a camera, compute
the camera parameters
• Calibration!
Useful reference
