LECTURE 1

Introduction, image representation and image

analysis
Computer Vision
• Vision – allows us to perceive and understand the world around us

• What is computer vision?

• Duplicate human vision by a machine (computer)
• Electronic perception and understanding of an image

• Giving computers the ability to see - Computer Vision

• Computer vision – simulating human vision or mimicking human
vision?
Major challenge in Computer Vision
• Our world is 3D
• What about visual sensors (eg. Cameras)?
• Usually give 2D images
• Projection of 3D world to 2D images - loss of information
• Advanced 3D visual sensors available
• Eg. TeraHertz scans
• Analyzing 3D images is more complicated than 2D
• Motion makes CV more complex
• Moving object or moving camera
Model development
• Model has to accommodate multiple variations

• Multiple models to learn variations

• Typical operations in model development

• Image capture
• Pre-processing
• Segmentation
• Model fitting
• Motion prediction
• Qualitative/quantitative conclusion
• Different algorithms available for each operation
Why is CV difficult?
• 3D à 2D loss of information
• Guessing the actual size of object requires additional yardstick
• Interpretation – use of previous knowledge/experience to understand
an image
Image interpretation: image data à model
• Noise – inherently present in real world measurements
• Noise adds uncertainty
• Tools required to deal wilth uncertainty – probabilistic tools
• Too much data – Images/video data is big
• Processor and memory requirements
• Real time performance is a challenge
Why is CV difficult? (…contd)
• Brightness – complex image formation physics
• Reconstructing surface orientation from intensity variations is ill-posed
• Local window vs need for global view
• Local elements (like pixel) analyzed by image analysis algorithm
• Global understanding is difficult with local observations

Local observations Global view

Image representation and image analysis
tasks
• Image understanding
Input image à Model of the image
• Image model
• Reduces information in the image
• Retains only “relevant” inofrmation based on application
• Raw image à … intermediate representations … à image
interpretation
• Intermediate representations – designed by CV
• Establish and maintain relations between entities within and between layers
Image representation
• Four levels of image representation
• From raw image to image with features
• Understanding objexts from features

• Input image – little or no abstraction

• Image with features – highly abstract
description
Image representation
• Broad categories of image representation
• Low level image processing
• High level image understanding
Low level processing
• Knowledge of image content is not used
• Low level processing involves procedures like –
• Image compression
• Noise filtering
• Edge extraction
• Image sharpening etc.
• Image representation on a computer
• Digitized representation in the form of a matrix
• RGB channels for color representation
• A set of matrices (images) for video data
High level processing
AI methods come
under HLP
• For understanding image content
• Make decisions based on understanding of image content

HLP
Input image World model

Compare
Low level Image with Perceived If Differences
processing features / reality appear
Image Model
Feedback to update
image model
Image representation - example

Acutal image Brightness representation of image

Machine only sees an array of numbers

 Image representation - example
• Machine only sees an array of numbers
• To understand these array of numbers
• General knowledge
• Domain specific knowledge
• Information extracted from image
• Etc. are required
• asf

Brightness representation of image

LLP vs HLP
Low Level Processing High Level Processing
Image content is not used Image content is used – object size,
shape, mutial relations between objects
etc. are used
Low level data from original image is Features relevant to the end goal are
used extracted and used
Overlaps with image processing Focusses on image understanding and
techniques – digitizations, noise filtering, decision making
edge extraction, segmentation etc.
Summary
• CV is the science to mimick biological vision systems
• What routine tasks performed by biological vision systems would be
good to accomplish by machines?
• Image representation, LLP and HLP algorithms used depend on the
application (end goal) – eg. Autonomous vehicle navigaton or object
tracking, medical diagnosis etc.
• Modeling human visual system requires understanding our own brain.
CV tasks

Algorithmic
components