12/4/2012
MATLAB Image Processing
By Bruce Tannenbaum IEEE Boston Meeting, Dec 3, 2012
2012 The MathWorks, Inc. 1
Who uses MATLAB?
12/4/2012
Key Industries
Aerospace and defense Automotive Biotech and pharmaceutical Communications Computers Education Electronics and semiconductors Energy production Industrial automation and machinery Medical devices
Key Capabilities Drive MathWorks Business
Verification, Validation, and Test
Test and
measurement
Model
checking
Code verification Certification kits MATLAB
Coder
Automatic Code Generation
Rapid
prototyping and HIL
Embedded
software
DSP software
VHDL/Verilog PLC
generation
code
System Modeling and Simulation
Discrete-event Simulink DSP designs State charts Physical modeling simulation Communications
systems
Computer System
vision objects
Video processing Computational
biology
Data Analysis and Algorithm Development
Control design Optimization Image processing Signal processing Statistics
Computational
finance
Technical Computing
MATLAB
1985 1990
Application
deployment
1995
Student version Instrument and
2000
Parallel computing
MATLAB Mobile
for iPhone/iPad
2010
database connectivity
2005
Founded in 1984
12/4/2012
Image Processing Toolbox
Perform image processing, analysis, visualization, and algorithm development
Image analysis Image enhancement Spatial transformation Image registration Morphological operations ROI-based processing Image display and exploration
Imaging-related MATLAB Toolboxes
Image Acquisition Toolbox Image Processing Toolbox Computer Vision System Toolbox
Also supports video processing
Parallel Computing Toolbox
For multicore computers, GPUs, and computer clusters
Statistics Toolbox
Clustering, statistics, and machine learning
12/4/2012
Agenda
Working with large data sets
Block processing Parallel processing
Image registration
Intensity-based Feature-based
Video processing
Examples of Large Image Data Sets
Satellite imagery Aerial surveys Super-resolution Image sequences and stacks Volumetric data Multispectral and hyperspectral Mosaics and panoramic imagery
12/4/2012
Common Large Image Challenges
Out of memory errors Slow processing
blockproc and rsetwrite
Block processing
Automatically divides an image into blocks for processing Reduces memory usage Processes arbitrarily large images
Reduced resolution data set
Avoids memory demands in visualizing large images
10
12/4/2012
Improving Performance
MATLAB
Preallocate space for variables Identify bottlenecks with Profiler Vectorize code
Image Processing Toolbox
Many toolbox functions are faster as of R2010a Implicit use of multicore processors
Parallel Computing Toolbox
Explicitly use multicore processors, clusters, and GPUs
11
Example of IPT Performance Improvements
(for imresize)
12
12/4/2012
Parallel Computing with MATLAB
Global Optimization Toolbox Model-Based Calibration Toolbox
13 14
Simulink Design Optimization
Image Processing Toolbox
Optimization Toolbox
High level parallel functions
Low level parallel functions Built on industry standard libraries
MATLAB and Parallel Computing Tools
parfor
System Test
matlabpool jobs, tasks
Industry Libraries Message Passing Interface (MPI) ScaLAPACK
Parallelized Image Segmentation with parfor
Bioinformatics Toolbox
Built in parallel functionality within specific toolboxes
batch
12/4/2012
Scale Up to Clusters, Grids and Clouds
Computer Cluster Desktop Computer Parallel Computing Toolbox MATLAB Distributed Computing Server
Scheduler
15
GPU Support with Parallel Computing Toolbox
Run supported MATLAB code on the GPU
Includes functions such as conv2, filter2
Create kernels from existing CUDA code and PTX files Run kernels on the GPU from MATLAB
16
12/4/2012
Agenda
Working with large data sets
Block processing Parallel processing
Image registration
Intensity-based Feature-based
Video processing
17
Automatic Image Registration
Intensity-based Feature-based
Two new registration types supported:
Intensity-based (Image Processing Toolbox) Feature-based (Computer Vision System Toolbox)
18
12/4/2012
Demo: Intensity-based Image Registration
Goal: Detect febrile symptoms off an image
19
Demo: Intensity-based Image Registration
Webcam image Thermal IR image Registered image
Step 1: Register the image pair to detect the eyes Step 2: Read temperature near eyes & issue alert in case of fever
20
10
12/4/2012
Feature-based Registration
Aligning data from multiple sources Applications
Stabilization Mosaicking, panoramas Stereo vision Data fusion
Key capabilities
Feature detection and extraction Feature matching RANSAC for estimating geometric transformation or fundamental matrix
21
Demo: Feature-based Image Registration
Workflow
Feature detection and extraction Feature matching Geometric transformation estimation with RANSAC
22
11
12/4/2012
RANSAC for Estimation
Random Sample Consensus
Iterative estimation of parameters to a mathematical model from a set of observed data that contains outliers
Our uses of RANSAC
Estimate Geometric Transformation Estimate Fundamental Matrix (useful for stereo vision)
23
Agenda
Working with large data sets
Block processing Parallel processing
Image registration
Intensity-based Feature-based
Video processing
24
12
12/4/2012
Video Processing in MATLAB
myVid = VideoReader(myvideofile.avi); numFrames = myVid.NumberOfFrames; numIter = 10; opticalFlowIn = zeros([size(currentFrame) 5]); opticalFlowOutput = zeros([size(currentFrame) numFrames]); i = 1; Explicit state management while i <= numFrames opticalFlowIn(:,:,2:end) = opticalFlowIn(:,:,1:end-1); opticalFlowIn(:,:,1) = read(myVid,i); flow = opticalFlow(opticalFlowIn(:,:,1),opticalFlowIn(:,:,5), horn-schunck,numIter,magitude-squared); opticalFlowOutput(:,:,i) = flow; i = i+1; end implay(opticalFlowOutput,30)
Explicit indexing Need to maintain buffer
25
Video Processing with System Objects
reader = vision.VideoFileReader reader.Filename = myvideofile.avi; viewer = vision.DeployableVideoPlayer (framerate,30); optical = vision.OpticalFlow optical.Method = horn-schunck; optical.OutputValue = Magitude-squared; optical.ReferenceFrameDelay = 3; optical.MaximumIterationCount = 10;
Initialize objects
while ~isDone(reader) currentFrame = step(reader); OF = step(optical, currentFrame); step(viewer, OF); end
In-the-loop code is much simpler Implicit states, buffering, and indexing Video player runs in-the-loop
26
13
12/4/2012
Demo: Car Detection Using Optical Flow Use optical flow to detect and count moving vehicles on a road
27
Image Acquisition Toolbox Hardware Support
Manufacturers include:
Allied Vision Technologies Basler Baumer DALSA FLIR Hamamatsu Lumenera Matrox Imaging National Instruments PixeLINK Point Grey Qimaging Sony And many more
See Supported Hardware Pages for more information
28
14
12/4/2012
Why use MATLAB for Image and Video Processing?
Read and write many image file formats Visualize and explore images interactively Connect directly to cameras and frame grabbers Use a large library of built-in functions Quickly build custom image processing algorithms Block-process large images to avoid memory issues Process images faster with multiple cores and clusters
29
Additional Resources
MATLAB Central Blog: Steve on Image Processing http://blogs.mathworks.com/steve/
30
15
12/4/2012
Additional Resources
Digital Image Processing Using MATLAB
Gonzalez, Woods, and Eddins Gatesmark
31
Additional Resources
MATLAB Central File Exchange http://www.mathworks.com/matlabcentral/fileexchange/
32
16
12/4/2012
Questions?
2012 The MathWorks, Inc. 33
If Time Permits.. Find the Circles
Goal: Find circles in truecolor image Draw edge lines around the circles
34
17
