3D reconstruction
From Wikipedia, the free encyclopedia
Jump to navigation Jump to search
For 3D reconstruction in medical imaging, see Iterative reconstruction.
3D reconstruction of the general anatomy of the right side view of a small marine
slug Pseudunela viatoris.
In computer vision and computer graphics, 3D reconstruction is the process of
capturing the shape and appearance of real objects. This process can be
accomplished either by active or passive methods.[1] If the model is allowed to
change its shape in time, this is referred to as non-rigid or spatio-temporal
reconstruction.[2]
Contents
1 Motivation and applications
2 Active methods
3 Passive methods
3.1 Monocular cues methods
3.2 Binocular stereo vision
3.2.1 Problem statement and basics
3.2.2 Image acquisition
3.2.3 Camera calibration
3.2.4 Feature extraction
3.2.5 Stereo correspondence
3.2.6 Restoration
3.2.7 3D Reconstruction of Medical Images
4 See also
5 References
6 External links
7 External links
Motivation and applications[edit]
The research of 3D reconstruction has always been a difficult goal. Using 3D
reconstruction one can determine any object�s 3D profile, as well as knowing the 3D
coordinate of any point on the profile.The 3D reconstruction of objects is a
generally scientific problem and core technology of a wide variety of fields, such
as Computer Aided Geometric Design (CAGD), computer graphics, computer animation,
computer vision, medical imaging, computational science, virtual reality, digital
media, etc. For instance, the lesion information of the patients can be presented
in 3D on the computer, which offers a new and accurate approach in diagnosis and
thus has vital clinical value.[3] Digital elevation models can be reconstructed
using methods such as airborne laser altimetry[4] or synthetic aperture radar.[5]
Active methods[edit]
Active methods, i.e. range data methods, given the depth map, reconstruct the 3D
profile by numerical approximation approach and build the object in scenario based
on model. These methods actively interfere with the reconstructed object, either
mechanically or radiometrically using rangefinders, in order to acquire the depth
map, e.g. structured light, laser range finder and other active sensing techniques.
A simple example of a mechanical method would use a depth gauge to measure a
distance to a rotating object put on a turntable. More applicable radiometric
methods emit radiance towards the object and then measure its reflected part.
Examples range from moving light sources, colored visible light, time-of-flight
lasers to microwaves or ultrasound. See 3D scanning for more details.
Passive methods[edit]
Passive methods of 3D reconstruction do not interfere with the reconstructed
object; they only use a sensor to measure the radiance reflected or emitted by the
object's surface to infer its 3D structure through image understanding.[6]
Typically, the sensor is an image sensor in a camera sensitive to visible light and
the input to the method is a set of digital images (one, two or more) or video. In
this case we talk about image-based reconstruction and the output is a 3D model. By
comparison to active methods, passive methods can be applied to a wider range of
situations.
Monocular cues methods[edit]
Monocular cues methods refer to use image (one, two or more) from one viewpoint
(camera) to proceed 3D construction. It makes use of 2D characteristics(e.g.
Silhouettes, shading and texture) to measure 3D shape, and that�s why it is also
named Shape-From-X, where X can be silhouettes, shading, texture etc. 3D
reconstruction through monocular cues is simple and quick, and only one appropriate
digital image is needed thus only one camera is adequate. Technically, it avoids
stereo correspondence, which is fairly complex.
Generating and reconstructing 3D shapes from single or multi-view depth maps or
silhouettes [7]
Shape-from-shading Due to the analysis of the shade information in the image, by
using Lambertian reflectance, the depth of normal information of the object surface
is restored to reconstruct.[8]
Photometric Stereo This approach is more sophisticated than the shape-of-shading
method. Images taken in different lighting conditions are used to solve the depth
information. It is worth mentioning that more than one image is required by this
approach.[9]
Shape-from-texture Suppose such an object with smooth surface covered by replicated
texture units, and its projection from 3D to 2D causes distortion and perspective.
Distortion and perspective measured in 2D images provide the hint for inversely
solving depth of normal information of the object surface.[10]
Binocular stereo vision[edit]
Binocular Stereo Vision obtains the 3-dimensional geometric information of an
object from multiple images based on the research of human visual system.[11] The
results are presented in form of depth maps. Images of an object acquired by two
cameras simultaneously in different viewing angles, or by one single camera at
different time in different viewing angles, are used to restore its 3D geometric
information and reconstruct its 3D profile and location. This is more direct than
Monocular methods such as shape-from-shading.
Binocular stereo vision method requires two identical cameras with parallel optical
axis to observe one same object, acquiring two images from different points of
view. In terms of trigonometry relations, depth information can be calculated from
disparity. Binocular stereo vision method is well developed and stably contributes
to favorable 3D reconstruction, leading to a better performance when compared to
other 3D construction. Unfortunately, it is computationally intensive, besides it
performs rather poorly when baseline distance is large.
Problem statement and basics[edit]
The approach of using Binocular stereo vision to acquire object�s 3D geometric
information is on the basis of visual disparity.[12] The following picture provides
a simple schematic diagram of horizontally sighted Binocular Stereo Vision, where b
is the baseline between projective centers of two cameras.
Geometry of a stereoscopic system
The origin of the camera�s coordinate system is at the optical center of the
camera�s lens as shown in the figure. Actually, the camera�s image plane is behind
the optical center of the camera�s lens. However, to simplify the calculation,
images are drawn in front of the optical center of the lens by f. The u-axis and v-
axis of the image�s coordinate system O1uv are in the same direction with x-axis
and y-axis of the camera�s coordinate system respectively. The origin of the
image�s coordinate system is located on the intersection of imaging plane and the
optical axis. Suppose such world point P whose corresponding image points are
P1(u1,v1) and P2(u2,v2) respectively on the left and right image plane. Assume two
cameras are in the same plane, then y-coordinates of P1 and P2 are identical,
i.e.,v1=v2. According to trigonometry relations,
u_1=f\frac{x_p}{z_p}
u_2=f\frac{x_p-b}{z_p}
v_1=v_2=f\frac{y_p}{z_p}
where(xp, yp, zp) are coordinates of P in the left camera�s coordinate system, f is
focal length of the camera. Visual disparity is defined as the difference in image
point location of a certain world point acquired by two cameras,
{\displaystyle d=u_{1}-u_{2}=f{\frac {b}{z_{p}}}}
based on which the coordinates of P can be worked out.
Therefore, once the coordinates of image points is known, besides the parameters of
two cameras, the 3D coordinate of the point can be determined.
x_p=\frac{bu_1}{d}
y_p=\frac{bv_1}{d}
z_p=\frac{bf}{d}
The 3D reconstruction consists of the following sections:
Image acquisition[edit]
2D digital image acquisition is the information source of 3D reconstruction.
Commonly used 3D reconstruction is based on two or more images, although it may
employ only one image in some cases. There are various types of methods for image
acquisition that depends on the occasions and purposes of the specific application.
Not only the requirements of the application must be met, but also the visual
disparity, illumination, performance of camera and the feature of scenario should
be considered.
Camera calibration[edit]
Camera calibration in Binocular Stereo Vision refers to the determination of the
mapping relationship between the image points P1(u1,v1) and P2(u2,v2), and space
coordinate P(xp, yp, zp) in the 3D scenario. Camera calibration is a basic and
essential part in 3D reconstruction via Binocular Stereo Vision.
Feature extraction[edit]
Main article: Feature extraction
The aim of feature extraction is to gain the characteristics of the images, through
which the stereo correspondence processes. As a result, the characteristics of the
images closely link to the choice of matching methods. There is no such universally
applicable theory for features extraction, leading to a great diversity of stereo
correspondence in Binocular Stereo Vision research.
Stereo correspondence[edit]
Main article: Image registration
Stereo correspondence is to establish the correspondence between primitive factors
in images, i.e. to match P1(u1,v1) and P2(u2,v2) from two images. Certain
interference factors in the scenario should be noticed, e.g. illumination, noise,
surface physical characteristic and etc.
Restoration[edit]
According to precise correspondence, combined with camera location parameters, 3D
geometric information can be recovered without difficulties. Due to the fact that
accuracy of 3D reconstruction depends on the precision of correspondence, error of
camera location parameters and so on, the previous procedures must be done
carefully to achieve relatively accurate 3D reconstruction.
3D Reconstruction of Medical Images[edit]
Clinical routine of diagnosis, patient follow-up, computer assisted surgery,
surgical planning etc. are facilitated by accurate 3D models of the desired part of
human anatomy. Main motivation behind 3D reconstruction includes
Improved accuracy due to multi view aggregation.
Detailed surface estimates.
Can be used to plan, simulate, guide, or otherwise assist a surgeon in performing a
medical procedure.
The precise position and orientation of the patient�s anatomy can be determined.
Helps in a number of clinical areas, such as radiotherapy planning and treatment
verification, spinal surgery, hip replacement, neurointerventions and aortic
stenting.
Applications:
3D reconstruction has applications in many fields. They are:
Medicine[3]
Free-viewpoint video reconstruction[13]
Robotic mapping[14]
City planning
Gaming[15]
Virtual environments and virtual tourism[15]
Earth observation
Archaeology[16]
Augmented reality[17]
Reverse engineering[citation needed]
Motion capture[18]
Gesture recognition and hand tracking[19]
Problem Statement:
Mostly algorithms available for 3D reconstruction are extremely slow and cannot be
used in real-time. Though the algorithms presented are still in infancy but they
have the potential for fast computation.
Existing Approaches:
Delaunay Triangulation (25 Points)
Delaunay and alpha-shapes
Delaunay method involves extraction of tetrahedron surfaces from initial point
cloud. The idea of �shape� for a set of points in space is given by concept of
alpha-shapes. Given a finite point set S, and the real parameter alpha, the alpha-
shape of S is a polytope (the generalization to any dimension of a two dimensional
polygon and a three-dimensional polyhedron) which is neither convex nor necessarily
connected.[20] For a large value, the alpha-shape is identical to the convex-hull
of S. The algorithm proposed by Edelsbrunner and Mucke[21] eliminates all
tetrahedrons which are delimited by a surrounding sphere smaller than a. The
surface is then obtained with the external triangles from the resulting
tetrahedron.[21]
Another algorithm called Tight Cocone[22] labels the initial tetrahedrons as
interior and exterior. The triangles found in and out generate the resulting
surface.
Both methods have been recently extended for reconstructing point clouds with
noise.[22] In this method the quality of points determines the feasibility of the
method. For precise triangulation since we are using the whole point cloud set, the
points on the surface with the error above the threshold will be explicitly
represented on reconstructed geometry.[20]
Marching Cubes
Zero set Methods
Reconstruction of the surface is performed using a distance function which assigns
to each point in the space a signed distance to the surface S. A contour algorithm
is used to extracting a zero-set which is used to obtain polygonal representation
of the object. Thus, the problem of reconstructing a surface from a disorganized
point cloud is reduced to the definition of the appropriate function f with a zero
value for the sampled points and different to zero value for the rest. An algorithm
called marching cubes established the use of such methods.[23] There are different
variants for given algorithm, some use a discrete function f, while other use a
polyharmonic radial basis function is used to adjust the initial point set.[24][25]
Functions like Moving Least Squares, basic functions with local support,[26] based
on the Poisson equation have also been used. Loss of the geometry precision in
areas with extreme curvature, i.e., corners, edges is one of the main issues
encountered. Furthermore, pretreatment of information, by applying some kind of
filtering technique, also affects the definition of the corners by softening them.
There are several studies related to post-processing techniques used in the
reconstruction for the detection and refinement of corners but these methods
increase the complexity of the solution.[27]
Solid geometry with volume rendering Image courtesy of Patrick Chris Fragile Ph.D.,
UC Santa Barbara
