Gajjala Ashok’s Unit -1 : Machine Learning

Unit I:

Introduction- Artificial Intelligence, Machine Learning, Deep learning, Types of Machine

Learning Systems, Main Challenges of Machine Learning.

Statistical Learning: Introduction, Supervised and Unsupervised Learning, Training and

Test Loss, Tradeoffs in Statistical Learning, Estimating Risk Statistics, Sampling
distribution of an estimator, Empirical Risk Minimization.

Dept. of AID,CSD, ST.Mary’s Women’s Engineering College. Pg.No: 1

Gajjala Ashok’s Unit -1 : Machine Learning

1.1 Artificial Intelligence

Dept. of AID,CSD, ST.Mary’s Women’s Engineering College. Pg.No: 2

Gajjala Ashok’s Unit -1 : Machine Learning

1.1.2 Machine Learning

Dept. of AID,CSD, ST.Mary’s Women’s Engineering College. Pg.No: 3

Gajjala Ashok’s Unit -1 : Machine Learning

Why Use Machine Learning?

Consider how you would write a spam filter using traditional programming techniques (Figure 1-1):

Dept. of AID,CSD, ST.Mary’s Women’s Engineering College. Pg.No: 4

Gajjala Ashok’s Unit -1 : Machine Learning

Dept. of AID,CSD, ST.Mary’s Women’s Engineering College. Pg.No: 5

Gajjala Ashok’s Unit -1 : Machine Learning

1.1.3 Deep Learning

Deep learning is a machine learning technique that teaches computers to do what comes naturally
to humans: learn by example. Deep learning is a key technology behind driverless cars, enabling
them to recognize a stop sign, or to distinguish a pedestrian from a lamppost. It is the key to voice
control in consumer devices like phones, tablets, TVs, and hands-free speakers. Deep learning is
getting lots of attention lately and for good reason. It’s achieving results that were not possible
before.

In deep learning, a computer model learns to perform classification tasks directly from images,
text, or sound. Deep learning models can achieve state-of-the-art accuracy, sometimes exceeding
human-level performance. Models are trained by using a large set of labeled data and neural
network architectures that contain many layers.

Why Deep Learning Matters?

How does deep learning attain such impressive results?

In a word, accuracy. Deep learning achieves recognition accuracy at higher levels than ever before.
This helps consumer electronics meet user expectations, and it is crucial for safety-critical
applications like driverless cars. Recent advances in deep learning have improved to the point
where deep learning outperforms humans in some tasks like classifying objects in images.

While deep learning was first theorized in the 1980s, there are two main reasons it has only recently
become useful:

1. Deep learning requires large amounts of labeled data. For example, driverless car
development requires millions of images and thousands of hours of video.
2. Deep learning requires substantial computing power. High-performance GPUs have a parallel
architecture that is efficient for deep learning. When combined with clusters or cloud

Dept. of AID,CSD, ST.Mary’s Women’s Engineering College. Pg.No: 6

Gajjala Ashok’s Unit -1 : Machine Learning

computing, this enables development teams to reduce training time for a deep learning network
from weeks to hours or less.
Examples of Deep Learning at Work?

Deep learning applications are used in industries from automated driving to medical devices.

Automated Driving: Automotive researchers are using deep learning to automatically detect
objects such as stop signs and traffic lights. In addition, deep learning is used to detect pedestrians,
which helps decrease accidents.

Aerospace and Defense: Deep learning is used to identify objects from satellites that locate areas
of interest, and identify safe or unsafe zones for troops.

Medical Research: Cancer researchers are using deep learning to automatically detect cancer
cells. Teams at UCLA built an advanced microscope that yields a high-dimensional data set used
to train a deep learning application to accurately identify cancer cells.

Industrial Automation: Deep learning is helping to improve worker safety around heavy
machinery by automatically detecting when people or objects are within an unsafe distance of
machines.

Electronics: Deep learning is being used in automated hearing and speech translation. For
example, home assistance devices that respond to your voice and know your preferences are
powered by deep learning applications.

How Deep Learning Works?

Most deep learning methods use neural network architectures, which is why deep learning
models are often referred to as deep neural networks.

The term “deep” usually refers to the number of hidden layers in the neural network. Traditional
neural networks only contain 2-3 hidden layers, while deep networks can have as many as 150.

Deep learning models are trained by using large sets of labeled data and neural network
architectures that learn features directly from the data without the need for manual feature
extraction.

Dept. of AID,CSD, ST.Mary’s Women’s Engineering College. Pg.No: 7

Gajjala Ashok’s Unit -1 : Machine Learning

Figure 1: Neural networks, which are organized in layers consisting of a set of interconnected
nodes. Networks can have tens or hundreds of hidden layers.

One of the most popular types of deep neural networks is known as convolutional neural
networks (CNN or ConvNet). A CNN convolves learned features with input data, and uses 2D
convolutional layers, making this architecture well suited to processing 2D data, such as images.

CNNs eliminate the need for manual feature extraction, so you do not need to identify features
used to classify images. The CNN works by extracting features directly from images. The relevant
features are not pretrained; they are learned while the network trains on a collection of images.
This automated feature extraction makes deep learning models highly accurate for computer
vision tasks such as object classification.

CNNs learn to detect different features of an image using tens or hundreds of hidden layers. Every
hidden layer increases the complexity of the learned image features. For example, the first hidden
layer could learn how to detect edges, and the last learns how to detect more complex shapes
specifically catered to the shape of the object we are trying to recognize.

What's the Difference Between Machine Learning and Deep Learning?

Deep learning is a specialized form of machine learning. A machine learning workflow

starts with relevant features being manually extracted from images. The features are then used to
create a model that categorizes the objects in the image. With a deep learning workflow, relevant
features are automatically extracted from images. In addition, deep learning performs “end-to-end
learning” – where a network is given raw data and a task to perform, such as classification, and it
learns how to do this automatically.

Dept. of AID,CSD, ST.Mary’s Women’s Engineering College. Pg.No: 8

Gajjala Ashok’s Unit -1 : Machine Learning

Another key difference is deep learning algorithms scale with data, whereas shallow learning
converges. Shallow learning refers to machine learning methods that plateau at a certain level of
performance when you add more examples and training data to the network.

1.1.4 Types of Machine Learning Systems

Dept. of AID,CSD, ST.Mary’s Women’s Engineering College. Pg.No: 9

Gajjala Ashok’s Unit -1 : Machine Learning

Dept. of AID,CSD, ST.Mary’s Women’s Engineering College. Pg.No: 10

Gajjala Ashok’s Unit -1 : Machine Learning

Dept. of AID,CSD, ST.Mary’s Women’s Engineering College. Pg.No: 11

Gajjala Ashok’s Unit -1 : Machine Learning

Dept. of AID,CSD, ST.Mary’s Women’s Engineering College. Pg.No: 12

Gajjala Ashok’s Unit -1 : Machine Learning

Dept. of AID,CSD, ST.Mary’s Women’s Engineering College. Pg.No: 13

Gajjala Ashok’s Unit -1 : Machine Learning

Dept. of AID,CSD, ST.Mary’s Women’s Engineering College. Pg.No: 14

Gajjala Ashok’s Unit -1 : Machine Learning

Dept. of AID,CSD, ST.Mary’s Women’s Engineering College. Pg.No: 15

Gajjala Ashok’s Unit -1 : Machine Learning

Dept. of AID,CSD, ST.Mary’s Women’s Engineering College. Pg.No: 16

Gajjala Ashok’s Unit -1 : Machine Learning

Dept. of AID,CSD, ST.Mary’s Women’s Engineering College. Pg.No: 17

Gajjala Ashok’s Unit -1 : Machine Learning

Dept. of AID,CSD, ST.Mary’s Women’s Engineering College. Pg.No: 18

Gajjala Ashok’s Unit -1 : Machine Learning

Dept. of AID,CSD, ST.Mary’s Women’s Engineering College. Pg.No: 19

Gajjala Ashok’s Unit -1 : Machine Learning

Dept. of AID,CSD, ST.Mary’s Women’s Engineering College. Pg.No: 20

Gajjala Ashok’s Unit -1 : Machine Learning

1.1.5 Main Challenges of Machine Learning.

Dept. of AID,CSD, ST.Mary’s Women’s Engineering College. Pg.No: 21

Gajjala Ashok’s Unit -1 : Machine Learning

Dept. of AID,CSD, ST.Mary’s Women’s Engineering College. Pg.No: 22

Gajjala Ashok’s Unit -1 : Machine Learning

Dept. of AID,CSD, ST.Mary’s Women’s Engineering College. Pg.No: 23

Gajjala Ashok’s Unit -1 : Machine Learning

Dept. of AID,CSD, ST.Mary’s Women’s Engineering College. Pg.No: 24

Gajjala Ashok’s Unit -1 : Machine Learning

Dept. of AID,CSD, ST.Mary’s Women’s Engineering College. Pg.No: 25

Gajjala Ashok’s Unit -1 : Machine Learning

Dept. of AID,CSD, ST.Mary’s Women’s Engineering College. Pg.No: 26

Gajjala Ashok’s Unit -1 : Machine Learning

1.2 Statistical Learning:

1.2.1 Introduction

Dept. of AID,CSD, ST.Mary’s Women’s Engineering College. Pg.No: 27

Gajjala Ashok’s Unit -1 : Machine Learning

1.2.2 Supervised and Unsupervised Learning

Dept. of AID,CSD, ST.Mary’s Women’s Engineering College. Pg.No: 28

Gajjala Ashok’s Unit -1 : Machine Learning

Dept. of AID,CSD, ST.Mary’s Women’s Engineering College. Pg.No: 29

Gajjala Ashok’s Unit -1 : Machine Learning

Dept. of AID,CSD, ST.Mary’s Women’s Engineering College. Pg.No: 30

Gajjala Ashok’s Unit -1 : Machine Learning

1.2.2 Training and Test Loss

Dept. of AID,CSD, ST.Mary’s Women’s Engineering College. Pg.No: 31

Gajjala Ashok’s Unit -1 : Machine Learning

Dept. of AID,CSD, ST.Mary’s Women’s Engineering College. Pg.No: 32

Gajjala Ashok’s Unit -1 : Machine Learning

Dept. of AID,CSD, ST.Mary’s Women’s Engineering College. Pg.No: 33

Gajjala Ashok’s Unit -1 : Machine Learning

1.2.3 Tradeoffs in Statistical Learning

Dept. of AID,CSD, ST.Mary’s Women’s Engineering College. Pg.No: 34

Gajjala Ashok’s Unit -1 : Machine Learning

Dept. of AID,CSD, ST.Mary’s Women’s Engineering College. Pg.No: 35

Gajjala Ashok’s Unit -1 : Machine Learning

1.2.4 Estimating Risk

1.2.4.1 In-Sample Risk

Dept. of AID,CSD, ST.Mary’s Women’s Engineering College. Pg.No: 36

Gajjala Ashok’s Unit -1 : Machine Learning

Dept. of AID,CSD, ST.Mary’s Women’s Engineering College. Pg.No: 37

Gajjala Ashok’s Unit -1 : Machine Learning

1.2.4.2 Cross-Validation

Dept. of AID,CSD, ST.Mary’s Women’s Engineering College. Pg.No: 38

Gajjala Ashok’s Unit -1 : Machine Learning

1.2.4.3 Sampling distribution of an estimator

Dept. of AID,CSD, ST.Mary’s Women’s Engineering College. Pg.No: 39

Gajjala Ashok’s Unit -1 : Machine Learning

Bootstrap

Dept. of AID,CSD, ST.Mary’s Women’s Engineering College. Pg.No: 40

Gajjala Ashok’s Unit -1 : Machine Learning

Large sample theory for the MLE *

Dept. of AID,CSD, ST.Mary’s Women’s Engineering College. Pg.No: 41

Gajjala Ashok’s Unit -1 : Machine Learning

1.2.4.5 Empirical risk minimization

Dept. of AID,CSD, ST.Mary’s Women’s Engineering College. Pg.No: 42

Gajjala Ashok’s Unit -1 : Machine Learning

Regularized risk minimization

Structural risk minimization

Dept. of AID,CSD, ST.Mary’s Women’s Engineering College. Pg.No: 43

Gajjala Ashok’s Unit -1 : Machine Learning

Estimating the risk using cross validation

Dept. of AID,CSD, ST.Mary’s Women’s Engineering College. Pg.No: 44

Gajjala Ashok’s Unit -1 : Machine Learning

***********

Dept. of AID,CSD, ST.Mary’s Women’s Engineering College. Pg.No: 45