Lecture Slides for
Machine Learning
CS 4552
Instructor: Dr. Olfa MZOUGHI
Reference book
◼ "Introduction to Machine Learning" by ,Ethem Alpaydin
https://www.cmpe.boun.edu.tr/~ethem/i2ml/
2
Outline
◼ Introduction
◼ Supervised learning
The K-Nearest-Neighbour (KNN) algorithm
The decision Tree
Bayes classifier
Neural Networks??
◼ Unsupervised learning
K-means
◼ Evaluation metrics
◼ Overfitting
3
CHAPTER 1:
Introduction
Data is Everywhere!
◼ Data is cheap and
abundant (data Cyber Security
E-Commerce
warehouses, data marts…)
◼ Knowledge is expensive
and scarce.
Traffic Patterns Social Networking: Twitter
5
Why “Learn” ?
◼ Machine learning is programming computers to
optimize a performance criterion using example
data or past experience.
6
When we talk about learning
◼ There is no need to “learn” to calculate payroll
◼ Learning is used when:
Human expertise does not exist (navigating on Mars),-
(exploring new relationships between data)
Difficulty to extract human expertise (medical image
processing)
When you need to deal with huge number options and
factors (e.g, Media sites rely on machine learning to shift
through millions of options to give you song or movie
recommendations.)
7
Machine learning: a new programming paradigm
Data Classical
Answers
Rules programming
Machine learning is a type of artificial intelligence (AI) that provides computers with the ability
to learn without being explicitly programed
Data
Machine learning Rules
Answers programming
8
Machine learning Tasks
◼ Supervised Learning
Classification
Regression
◼ Unsupervised Learning
Clustering
Association
◼ Reinforcement Learning
9
Classification(1)
◼ Example: Credit
scoring
◼ Differentiating
between low-risk
and high-risk
customers from
their income and
savings
Discriminant: IF income > θ1 AND savings > θ2
THEN low-risk ELSE high-risk
10
Classification(2)
11
Classification (3)
◼ Find a model for class attribute as a function of
the values of other attributes
# years at
Level of Credit
Tid Employed present
Education Worthy
address
1 Yes Undergrad 7 ?
# years at 2 No Graduate 3 ?
Level of Credit
Tid Employed present 3 Yes High School 2 ?
Education Worthy
address
… … … … …
1 Yes Graduate 5 Yes 10
2 Yes High School 2 No
3 No Undergrad 1 No
4 Yes High School 10 Yes
… … … … … Test
Set
Training
Learn
Model12
Set Classifier
Classification: Applications
◼ Aka Pattern recognition
◼ Face recognition: Pose, lighting, occlusion (glasses,
beard), make-up, hair style
◼ Character recognition: Different handwriting styles.
◼ Speech recognition: Temporal dependency.
Use of a dictionary or the syntax of the language.
Sensor fusion: Combine multiple modalities; eg, visual (lip
image) and acoustic for speech
◼ Medical diagnosis: From symptoms to illnesses
◼ ...
13
Face Recognition
Training examples of a person
Test images
AT&T Laboratories, Cambridge UK
http://www.uk.research.att.com/facedatabase.html
14
Regression Applications
◼ Predict a value of a given continuous valued
variable based on the values of other variables,
assuming a linear or nonlinear model of
dependency.
15
Regression (Example of linear
dependency)
◼ Example: Price of a
used car
◼ x : car attributes y = wx+w0
y : price
◼ Predict wi ?
16
Regression
◼ Examples:
Predicting sales amounts of new product based
on advetising expenditure.
Predicting wind velocities as a function of
temperature, humidity, air pressure, etc.
17
Supervised Learning: Uses
◼ Prediction of future cases: Use the rule to predict
the output for future inputs
◼ Knowledge extraction: The rule is easy to
understand
◼ Compression: The rule is simpler than the data it
explains
◼ Outlier detection: Exceptions that are not covered
by the rule, e.g., fraud
18
Unsupervised Learning
◼ No need to supervise the model.
◼ The model work on its own to discover patterns
and information that was previously undetected.
◼ It mainly deals with the unlabelled data.
19
Clustering
◼ Finding groups of objects such that the objects in a
group will be similar (or related) to one another and
different from (or unrelated to) the objects in other
groups
Inter-cluster
Intra-cluster distances are
distances are maximized
minimized
20
Clustering examples
◼ Understanding
Customer segmentation for targeted marketing
◼ Summarization
Image compression: Color quantization
◼ Anomaly Detection
Detecting changes in the global forest cover.
21
Learning Associations
◼ Basket analysis:
P (Y | X ) probability that somebody who buys X also
buys Y where X and Y are products/services.
Example: P ( chips | milk ) = 0.7
◼ Example of applications:
Market-basket analysis: Rules could be used for sales
promotion, shelf management, and inventory
management
22
Reinforcement Learning
◼ Learning a policy: A sequence of outputs
◼ No supervised output but delayed reward
◼ An agent interacting with the world makes
observations, takes actions, and is rewarded or
punished; it should learn to choose actions in such
a way as to obtain a lot of reward
◼ Examples:
Game playing
Robot in a maze
...
23
Learning
Supervised Unsupervised Reinforcement
classification regression Clustering Association
24
Classification vs. clustering
Classification Clustering
Similarity They both classify data using Features X
Differences Supervised Unsupervised learning
learning
Classes Y are Classes Y are unknown
known
Train a Based on Intra-class
classifier model similarity (or inter-class
f based on diversity) within the feature
training space X.
examples (X,Y) 25
Classification vs. regression
Classification Regression
Similarity They both use supervised learning task that
use training example (X,Y) where X represent
input data features and Y is the output
Differences Y is a class Y is a continous value
label (name)
26
Summary
27
Resources: Datasets
◼ UCI Repository:
http://www.ics.uci.edu/~mlearn/MLRepository.html
◼ UCI KDD Archive:
http://kdd.ics.uci.edu/summary.data.application.html
◼ Statlib: http://lib.stat.cmu.edu/
◼ Delve: http://www.cs.utoronto.ca/~delve/
28
