Neural

Networks
Lecture (3) _ Learning Schemes

Prof. Walaa Gabr

Prof. of Electrical Engineering & Department Chair
Faculty of Engineering at Benha
 Neural Networks

Learning Schemes
Learning is the process by which a neural system acquires ability to carry out certain
tasks by adjusting its internal parameters according to some learning scheme.
Depending on the particular neural architecture considered, learning can be supervised,
unsupervised or reinforced.

Learning Schemes
2

Supervised Unsupervised Reinforced

2
Neural Networks

Learning Schemes

3
 Neural Networks

Learning Schemes
In a Neural Network, the learning (or training) process is initiated by dividing the data into three
different sets:
• Training dataset – This dataset allows the Neural Network to understand the weights between
nodes.
• Validation dataset – This dataset is used for fine-tuning the performance of the Neural Network.
• Test dataset – This dataset is used to determine the accuracy and margin of error of the Neural
Network.
4

Once the data is segmented into these three parts, Neural Network algorithms are applied to them for
training the Neural Network. The procedure used for facilitating the training process in a Neural
Network is known as the optimization, and the algorithm used is called the optimizer. There are
different types of optimization algorithms, each with their unique characteristics and aspects such as
memory requirements, numerical precision, and processing speed.

4
 Neural Networks

Learning Schemes
Supervised Learning
As the name suggests, this type of learning is done under the supervision of a teacher.
This learning process is dependent.
With supervised learning, the algorithm is given a set of particular targets to aim for.
Supervised learning uses labeled data set, one that contains matched sets of observed
inputs, X’s, and the associated outputs, Y’s. The algorithm is “trained,” i.e., the machine
learning algorithm is applied to this data set to infer the patterns between the inputs and
5
outputs. In other words, the algorithm works out fin the equation:
Y=f (X )
The dependent variable (Y ) is the target, while the independent variables (X’s) are
known as features.

5
 Neural Networks

Learning Schemes
Supervised Learning
During the training of ANN under supervised learning,
1. The input vector is presented to the network, which will give an output vector.
2. This output vector is compared with the desired output vector.
3. An error signal is generated, if there is a difference between the actual output and the
desired output vector.
6

4. Depending on this error signal, the weights are adjusted until the actual output is
matched with the desired output.

6
 Neural Networks

Learning Schemes
Supervised Learning
Types of Supervised Learning
Supervised learning can be categorized into two problems, depending on the nature of
the target (Y) variable. These include classification and regression.
Classification focuses on sorting observations into distinct categories. It is when the
target (Y) is a category, such as “blue” or “red” or “disease” and “no disease.” An
example of a classification problem is a spam filter that classifies emails into two
7

categories, ‘safe’ or ‘suspicious.’

Regression is based on making predictions of continuous target variables. Examples of
regression problems include using historical stock market returns to forecast stock price
performance or predicting the number of goals scored by a team.
7
 Neural Networks

Learning Schemes
Supervised Learning

Types of Supervised Learning

Supervised learning is classified into two categories of algorithms:
• Regression: A regression problem is when the output variable is a real value, such as “dollars” or
“weight”.
• Classification: A classification problem is when the output variable is a category, such as “Red” or
“blue” , “disease” or “no disease”.
8

Supervised learning deals with or learns with “labeled” data. This implies that some data is already tagged
with the correct answer.

8
Neural Networks

Learning Schemes
Supervised Learning

9
Neural Networks

Learning Schemes
Supervised Learning
1- Regression 2- Classification
Regression is a type of supervised learning that is used Classification is a type of supervised learning that is used
to predict continuous values, such as house prices, to predict categorical values, such as whether a customer
stock prices, or customer churn. Regression algorithms will churn or not, whether an email is spam or not, or
learn a function that maps from the input features to the whether a medical image shows a tumor or not.
output value. Classification algorithms learn a function that maps from
Some common regression algorithms include: the input features to a probability distribution over the
output classes.
• Linear Regression 10

Some common classification algorithms include:

• Polynomial Regression
• Logistic Regression
• Support Vector Machine Regression
• Support Vector Machines
• Decision Tree Regression
• Decision Trees
• Random Forest Regression
• Random Forests
• Naive Baye
10
 Neural Networks

Learning Schemes
Supervised Learning
Evaluating Supervised Learning Models
Evaluating supervised learning models is an important step in ensuring that the model is accurate and
generalizable. There are a number of different metrics that can be used to evaluate supervised learning
models, but some of the most common ones include:
For Regression
• Mean Squared Error (MSE): MSE measures the average squared difference between the predicted
values and the actual values. Lower MSE values indicate better model performance.
• Root Mean Squared Error (RMSE): RMSE is the square root of MSE, representing the standard
11
deviation of the prediction errors. Similar to MSE, lower RMSE values indicate better model
performance.
• Mean Absolute Error (MAE): MAE measures the average absolute difference between the predicted
values and the actual values. It is less sensitive to outliers compared to MSE or RMSE.
• R-squared (Coefficient of Determination): R-squared measures the proportion of the variance in the
target variable that is explained by the model. Higher R-squared values indicate better model fit.
11
 Neural Networks

Learning Schemes
Supervised Learning
For Classification
• Accuracy: Accuracy is the percentage of predictions that the model makes correctly. It is calculated by
dividing the number of correct predictions by the total number of predictions.
• Precision: Precision is the percentage of positive predictions that the model makes that are actually
correct. It is calculated by dividing the number of true positives by the total number of positive
predictions.
• Recall: Recall is the percentage of all positive examples that the model correctly identifies. It is
calculated by dividing the number of true positives
12
by the total number of positive examples.
• F1 score: The F1 score is a weighted average of precision and recall. It is calculated by taking the
harmonic mean of precision and recall.
• Confusion matrix: A confusion matrix is a table that shows the number of predictions for each
class, along with the actual class labels. It can be used to visualize the performance of the model and
identify areas where the model is struggling.

12
 Neural Networks

Learning Schemes
Supervised Learning
Applications of Supervised learning
Supervised learning can be used to solve a wide variety of problems, including:
• Spam filtering: Supervised learning algorithms can be trained to identify and classify spam emails based on their
content, helping users avoid unwanted messages.
• Image classification: Supervised learning can automatically classify images into different categories, such as animals,
objects, or scenes, facilitating tasks like image search, content moderation, and image-based product
recommendations.
• Medical diagnosis: Supervised learning can assist in medical diagnosis by analyzing patient data, such as medical
images, test results, and patient history, to identify patterns that suggest specific diseases or conditions.
• Fraud detection: Supervised learning models can analyze financial transactions and identify patterns that indicate
fraudulent activity, helping financial institutions prevent fraud and protect their customers.
• Natural language processing (NLP): Supervised learning plays a crucial role in NLP tasks, including sentiment analysis,
machine translation, and text summarization, enabling machines to understand and process human language
effectively.

13
 Neural Networks

Learning Schemes
Supervised Learning

Advantages of Supervised learning Disadvantages of Supervised learning

• Supervised learning allows collecting data and • Classifying big data can be challenging.
produces data output from previous experiences. • Training for supervised learning needs a lot
• Helps to optimize performance criteria with the of computation time. So, it requires a lot of
help of experience. time.
• Supervised machine learning helps to solve various • Supervised learning cannot handle all
types of real-world computation problems. complex tasks in Machine Learning.
• It performs classification and regression tasks. • Computation time is vast for supervised
• It allows estimating or mapping the result to a new learning.
sample. • It requires a labelled data set.
• We have complete control over choosing the It requires a training process.
number of classes we want in the training data.
14
 Neural Networks

Learning Schemes
Unsupervised Learning
What is Unsupervised learning?
Unsupervised learning is a type of machine learning that learns from unlabeled data. This means that the
data does not have any pre-existing labels or categories. The goal of unsupervised learning is to discover
patterns and relationships in the data without any explicit guidance.
Unsupervised learning is the training of a machine using information that is neither classified nor labeled
and allowing the algorithm to act on that information without guidance. Here the task of the machine is
to group unsorted information according to similarities, patterns, and differences without any prior
training of data.
Unlike supervised learning, no teacher is provided that means no training will be given to the machine.
Therefore, the machine is restricted to find the hidden structure in unlabeled data by itself.
You can use unsupervised learning to examine the animal data that has been gathered and distinguish
between several groups according to the traits and actions of the animals. These groupings might
correspond to various animal species, providing you to categorize the creatures without depending on
labels that already exist. 15
 Neural Networks

Learning Schemes
Unsupervised Learning
In this paradigm the neural network is only given a set of inputs and it’s the neural network’s
responsibility to find some kind of pattern within the inputs provided without any external aid.
During the training of ANN under unsupervised learning,
1. The input vectors of similar type are combined to form clusters.
2. When a new input pattern is applied, then the neural network gives an output response indicating the
class to which the input pattern belongs.
Hence, in this type of learning, the network itself must discover the patterns and features from the input
data, and the relation for the input data over the output.
This type of learning paradigm is often used in data mining and is also used by many recommendation
algorithms due to their ability to predict a user's preferences based on the preferences of other similar
users it has grouped together.

16
 Neural Networks

Learning Schemes
Unsupervised Learning
Types of Unsupervised Learning
Two essential types of unsupervised learning are dimension reduction and clustering.
Dimension reduction refers to reducing the number of inputs (features) while retaining
variation across observations to maintain structure and usefulness of the information contained
in the variation. For example, data scientists reduce the number of dimensions in an extensive
data set to simplify modeling and reduce file size.

17
 Neural Networks
Learning Schemes
Unsupervised Learning
Types of Unsupervised Learning
On the other hand, clustering is a type of unsupervised learning that is used to group similar
data points together. Clustering algorithms work by iteratively moving data points closer to their
cluster centers and further away from data points in other clusters.
1.Exclusive (partitioning)
2.Agglomerative Clustering Types:-
1.Hierarchical clustering
3.Overlapping 2.K-means clustering
4.Probabilistic 3.Principal Component Analysis
4.Singular Value Decomposition
5.Independent Component Analysis
6.Gaussian Mixture Models (GMMs)
7.Density-Based Spatial Clustering of Applications with
Noise (DBSCAN)

18
 Neural Networks
Learning Schemes
Unsupervised Learning
Key Points
• Unsupervised learning allows the model to discover patterns and relationships in
unlabeled data.
• Clustering algorithms group similar data points together based on their inherent
characteristics.
• Feature extraction captures essential information from the data, enabling the model to
make meaningful distinctions.
• Label association assigns categories to the clusters based on the extracted patterns and
characteristics.

19
Neural Networks

Learning Schemes
Unsupervised Learning

20

Clustering

20
 Neural Networks
Learning Schemes
Unsupervised Learning
Evaluating Non-Supervised Learning Models
Evaluating non-supervised learning models is an important step in ensuring that the model is effective and useful. However, it can be
more challenging than evaluating supervised learning models, as there is no ground truth data to compare the model’s predictions to.
There are a number of different metrics that can be used to evaluate non-supervised learning models, but some of the most common
ones include:
•Silhouette score: The silhouette score measures how well each data point is clustered with its own cluster members and separated
from other clusters. It ranges from -1 to 1, with higher scores indicating better clustering.
•Calinski-Harabasz score: The Calinski-Harabasz score measures the ratio between the variance between clusters and the variance
within clusters. It ranges from 0 to infinity, with higher scores indicating better clustering.
•Adjusted Rand index: The adjusted Rand index measures the similarity between two clusterings. It ranges from -1 to 1, with
higher scores indicating more similar clusterings.
•Davies-Bouldin index: The Davies-Bouldin index measures the average similarity between clusters. It ranges from 0 to
infinity, with lower scores indicating better clustering.
•F1 score: The F1 score is a weighted average of precision and recall, which are two metrics that are commonly used in supervised
learning to evaluate classification models. However, the F1 score can also be used to evaluate non-supervised learning models, such
as clustering models.

21
 Neural Networks
Learning Schemes
Unsupervised Learning
Application of Unsupervised learning
Non-supervised learning can be used to solve a wide variety of problems, including:
•Anomaly detection: Unsupervised learning can identify unusual patterns or deviations from
normal behavior in data, enabling the detection of fraud, intrusion, or system failures.
•Scientific discovery: Unsupervised learning can uncover hidden relationships and patterns in
scientific data, leading to new hypotheses and insights in various scientific fields.
•Recommendation systems: Unsupervised learning can identify patterns and similarities in user
behavior and preferences to recommend products, movies, or music that align with their interests.
•Customer segmentation: Unsupervised learning can identify groups of customers with similar
characteristics, allowing businesses to target marketing campaigns and improve customer service
more effectively.
•Image analysis: Unsupervised learning can group images based on their content, facilitating tasks
such as image classification, object detection, and image retrieval.
22
 Neural Networks
Learning Schemes
Unsupervised Learning

Advantages of Unsupervised learning Disadvantages of Unsupervised learning

• It does not require training data to be labeled. •Difficult to measure accuracy or effectiveness due to
•Dimensionality reduction can be easily lack of predefined answers during training.
accomplished using unsupervised learning. •The results often have lesser accuracy.
•Capable of finding previously unknown patterns in •The user needs to spend time interpreting and label
data. the classes which follow that classification.
•Unsupervised learning can help you gain insights •Unsupervised learning can be sensitive to data quality,
from unlabeled data that you might not have been including missing values, outliers, and noisy data.
able to get otherwise. •Without labeled data, it can be difficult to evaluate the
•Unsupervised learning is good at finding patterns performance of unsupervised learning models, making
and relationships in data without being told what it challenging to assess their effectiveness.
to look for. This can help you learn new things
about your data.

23
Neural Networks
Learning Schemes
Supervised vs Unsupervised Learning

24
Neural Networks
Learning Schemes
Supervised vs Unsupervised Learning

25
Neural Networks
Learning Schemes
Supervised vs Unsupervised Learning
Parameters Supervised machine learning Unsupervised machine
learning
Algorithms are trained using labeled Algorithms are used against data
Input Data
data. that is not labeled
Computational Complexity Simpler method Computationally complex
Accuracy Highly accurate Less accurate
No. of classes No. of classes is known No. of classes is not known
Data Analysis Uses offline analysis Uses real-time analysis of data
Linear and Logistics regression,
KNN Random forest, multi-class
K-Means clustering, Hierarchical
Algorithms used classification, decision tree, Support
clustering, Apriori algorithm, etc.
Vector Machine, Neural Network,
etc.

Output Desired output is given. Desired output is not given.

26
Neural Networks
Learning Schemes
Supervised vs Unsupervised Learning
Parameters Supervised machine learning Unsupervised machine
learning
Training data Use training data to infer model. No training data is used.
It is not possible to learn larger and It is possible to learn larger and
Complex model more complex models than with more complex models with
supervised learning. unsupervised learning.
Model We can test our model. We can not test our model.
Supervised learning is also called Unsupervised learning is also called
Called as
classification. clustering.
Example: Optical character
Example Example: Find a face in an image.
recognition.
Unsupervised learning does not
supervised learning needs
Supervision need any supervision to train the
supervision to train the model.
model.

27
 Neural Networks
Learning Schemes
Deep Learning and Reinforcement Learning
Deep learning and reinforcement learning functions enable a computer to develop rules
on its own to solve problems.
Deep learning is a self-teaching system in which the existing data is used to train
algorithms to establish patterns and then use that to make predictions about new data.
Highly complex tasks, such as image classification, face recognition, speech
recognition, and natural language processing, are addressed by sophisticated algorithms.

28
 Neural Networks
Learning Schemes
Reinforcement Learning
Reinforcement Learning (RL) is a branch of machine learning focused on making decisions to
maximize cumulative rewards in a given situation. Unlike supervised learning, which relies on a
training dataset with predefined answers, RL involves learning through experience. In RL, an agent
learns to achieve a goal in an uncertain, potentially complex environment by performing actions
and receiving feedback through rewards or penalties.
Key Concepts of Reinforcement Learning
• Agent: The learner or decision-maker.
• Environment: Everything the agent interacts with.
• State: A specific situation in which the agent finds itself.
• Action: All possible moves the agent can make.
• Reward: Feedback from the environment based on the action taken.

29
 Neural Networks

Learning Schemes
Reinforcement Learning
Reinforcement learning is similar to supervised learning in that some feedback is given,
however instead of providing a target output a reward is given based on how well the
system performed.
The aim of reinforcement learning is to maximize the reward the system receives
through trial-and-error.
This paradigm relates strongly with how learning works in nature, for example an
animal might remember the actions it’s previously taken which helped it to find food
(the reward).
In reinforcement learning, a computer learns from trial and error. It learns dynamically
by adjusting actions based on continuous feedback to maximize an outcome.

30
 Neural Networks
Learning Schemes
Reinforcement Learning

How Reinforcement Learning Works

RL operates on the principle of learning optimal behavior through trial and error. The agent
takes actions within the environment, receives rewards or penalties, and adjusts its behavior to
maximize the cumulative reward. This learning process is characterized by the following
elements:
• Policy: A strategy used by the agent to determine the next action based on the current state.
• Reward Function: A function that provides a scalar feedback signal based on the state and
action.
• Value Function: A function that estimates the expected cumulative reward from a given state.
• Model of the Environment: A representation of the environment that helps in planning by
predicting future states and rewards.
31
Neural Networks
Learning Schemes
Reinforcement Learning
How Reinforcement Learning Works
In a way, Reinforcement Learning is the science
of making optimal decisions using experiences.
Breaking it down, the process of Reinforcement
Learning involves these simple steps:
1.Observation of the environment
2.Deciding how to act using some strategy
3.Acting accordingly
4.Receiving a reward or penalty
5.Learning from the experiences and refining
our strategy
6.Iterate until an optimal strategy is found
32
Neural Networks
Learning Schemes
Reinforcement Learning
Example: Navigating a Maze
The problem is as follows: We have an agent and a reward,
with many hurdles in between. The agent is supposed to find
the best possible path to reach the reward. The following
problem explains the problem more easily.
The adjacent image shows the robot, diamond, and fire. The
goal of the robot is to get the reward that is the diamond and
avoid the hurdles that are fired. The robot learns by trying
all the possible paths and then choosing the path which gives
him the reward with the least hurdles. Each right step will
give the robot a reward and each wrong step will subtract
the reward of the robot. The total reward will be calculated
when it reaches the final reward that is the diamond.
33
Neural Networks
Learning Schemes
Reinforcement Learning
Main points in Reinforcement learning –
• Input: The input should be an initial state from which the model will start
• Output: There are many possible outputs as there are a variety of solutions to a
particular problem
• Training: The training is based upon the input, The model will return a state and the
user will decide to reward or punish the model based on its output.
• The model keeps continues to learn.
• The best solution is decided based on the maximum reward.

34
Neural Networks
Learning Schemes
Reinforcement Learning
Difference between Reinforcement learning and Supervised learning:

Reinforcement learning Supervised learning

Reinforcement learning is all about making decisions
sequentially. In simple words, we can say that the In Supervised learning, the decision is
output depends on the state of the current input and made on the initial input or the input
the next input depends on the output of the previous given at the start
input
In supervised learning the decisions
In Reinforcement learning decision is dependent, So we
are independent of each other so
give labels to sequences of dependent decisions
labels are given to each decision.
Example: Object recognition,spam
Example: Chess game,text summarization
detetction
35
Neural Networks
Learning Schemes
Reinforcement Learning
Types of Reinforcement:
1. Positive: Positive Reinforcement is defined as when an event, occurs due to a particular behavior, increases
the strength and the frequency of the behavior. In other words, it has a positive effect on behavior.
Advantages of +ve reinforcement learning are:
o Maximizes Performance
o Sustain Change for a long period of time
o Too much Reinforcement can lead to an overload of states which can diminish the results
2. Negative: Negative Reinforcement is defined as strengthening of behavior because a negative condition is
stopped or avoided.
Advantages of –ve reinforcement learning:
o Increases Behavior
o Provide defiance to a minimum standard of performance
o It Only provides enough to meet up the minimum behavior
36
Neural Networks
Learning Schemes
Reinforcement Learning
Elements of Reinforcement Learning
i) Policy: Defines the agent’s behavior at a given time.
ii) Reward Function: Defines the goal of the RL problem by providing feedback.
iii) Value Function: Estimates long-term rewards from a state.
iv) Model of the Environment: Helps in predicting future states and rewards for planning.

Application of Reinforcement Learnings

i) Robotics: Automating tasks in structured environments like manufacturing.
ii) Game Playing: Developing strategies in complex games like chess.
iii) Industrial Control: Real-time adjustments in operations like refinery controls.
iv) Personalized Training Systems: Customizing instruction based on individual needs.
37
Neural Networks
Learning Schemes
Reinforcement Learning
Advantages and Disadvantages of Reinforcement Learning
- Advantages:
1. Reinforcement learning can be used to solve very complex problems that cannot be solved by
conventional techniques.
2. The model can correct the errors that occurred during the training process.
3. In RL, training data is obtained via the direct interaction of the agent with the environment
4. Reinforcement learning can handle environments that are non-deterministic, meaning that the
outcomes of actions are not always predictable. This is useful in real-world applications where
the environment may change over time or is uncertain.
5. Reinforcement learning can be used to solve a wide range of problems, including those that
involve decision making, control, and optimization.
6. Reinforcement learning is a flexible approach that can be combined with other machine
learning techniques, such as deep learning, to improve performance. 38
Neural Networks
Learning Schemes
Reinforcement Learning
Advantages and Disadvantages of Reinforcement Learning
- Disadvantages:
1. Reinforcement learning is not preferable to use for solving simple problems.
2. Reinforcement learning needs a lot of data and a lot of computation
3. Reinforcement learning is highly dependent on the quality of the reward function. If the
reward function is poorly designed, the agent may not learn the desired behavior.
4. Reinforcement learning can be difficult to debug and interpret. It is not always clear why the
agent is behaving in a certain way, which can make it difficult to diagnose and fix problems.

39
Neural Networks

Neural Y (Actual output)

X (input)
Network

Supervised Error signal (D-Y)

Error Signal
D (Desired output)
Generator
Learning Schemes

Neural
X (input) Y (Actual output)
Network

Unsupervised

40

Neural Y (Actual output)

X (input)
Network

Reinforced Error signal

Error Signal
R (Reinforcement signal)
Generator
40
Neural Networks

41

41
Neural Networks

42