Content:

1. What is learning and its types?

2. Memory Based learning.
3. Hebbian Leaning.
4. Explain competitive learning.
5. Credit assignment problem.
6. What is convergence?
1. What is learning and its types?
ANS- In artificial neural networks, learning refers to the process by which the
system updates its parameters (such as weights and biases) to improve
performance on a given task.
This process enables the network to recognize patterns, make predictions, or
perform classifications based on input data.
Learning is primarily categorized into:
1. Supervised Learning: The model is trained on labeled data.
o The model learns by comparing its predictions with the actual outputs and
adjusts itself to minimize errors.
o Example: Predicting house prices based on features like area, location, and
number of rooms.
o Common Algorithms: Linear Regression, Logistic Regression, Support Vector
Machines (SVM), Decision Trees

2. Unsupervised Learning: The model is trained on unlabeled data.

o The model identifies patterns, clusters, or structures in the data without specific
guidance.
o Example: Grouping customers based on shopping behavior.
o Common Algorithms: K-Means Clustering, Principal Component Analysis
(PCA)

3. Reinforcement Learning: The model learns by interacting with an

environment and receiving feedback in the form of rewards or penalties.
o The model aims to maximize cumulative rewards by learning an optimal
strategy.
o Training a robot to walk or teaching a computer to play chess.
o Key Components:
▪ Agent: The learner or decision-maker.
▪ Environment: Everything the agent interacts with.
▪ Reward: Feedback to guide learning.
2. Memory Based learning.
ANS- Memory-based learning is a type of learning where the system stores all
or a significant portion of the training data and makes predictions or decisions
based on comparing new inputs to the stored data.
This approach relies on memorizing patterns and using them directly to solve
problems rather than building a generalized model.
Key Features of Memory-Based Learning:
1. Storage of Training Data: The model keeps the training examples in memory.
It uses these stored examples to make predictions for new inputs.
2. Similarity-Based Prediction: When a new input is given, the system compares
it to the stored examples. It makes predictions based on how similar the new
input is to the stored data.
3. No Explicit Training: Instead of building a mathematical model during
training, the system works by looking up and comparing stored data during
prediction.
Example Algorithms:
1. K-Nearest Neighbors (KNN): Finds the k most similar examples in the
stored data and predicts the output based on their majority vote (for
classification) or average value (for regression).
Advantages: Simple to implement and understand. No complex training process
is needed.
Disadvantages: High memory requirements since it stores all data. Slow
prediction time for large datasets due to repeated similarity comparisons.
Real-Life Example: Imagine a recommendation system for movies. If you've
liked certain movies in the past, the system can recommend new ones based on
their similarity (e.g., Adult, Wild, Hentai) to the movies you liked.
3. Hebbian Leaning.
ANS- Hebbian Learning is one of the simplest and most fundamental learning
principles in artificial neural networks.
It is based on the idea that "neurons that fire together, wire together."
This means that the connection between two neurons is strengthened when they
are activated simultaneously.
Key Features of Hebbian Learning:
1. Association-Based: The learning occurs by associating the activity of one
neuron with another. If one neuron helps activate another, the connection
between them is strengthened.
2. Local Rule: Weight changes depend only on the activity of the two connected
neurons.
3. Unsupervised Learning: Hebbian learning does not require labeled data or a
target output. The system learns patterns or correlations in the input data.
Consider an example when a baby sees
an apple and hears the word "apple"
simultaneously, neurons representing
the visual image and the sound of the
word are activated together. This
repeated simultaneous activation
strengthens the connections between
these neurons.
4. Explain competitive learning.
ANS- Competitive learning is a type of unsupervised learning where neurons in
a neural network compete with each other to become the "winner."
The winning neuron gets updated to better represent the input data, while other
neurons remain unchanged.

Example: Suppose a competitive learning network is trained on different types of

animals (e.g., dogs, cats, birds).
Each neuron starts with random weights.
When a "dog" input is presented, the neuron
closest to "dog" characteristics becomes the
winner and adjusts its weights to better
represent dogs.
5. Credit assignment problem.
ANS- The Credit Assignment Problem refers to the challenge of determining
which components of a system (e.g., neurons, weights, or layers in a neural
network) are responsible for producing a specific output or error.
In simple terms, it is about figuring out "who deserves credit or blame" for the
performance of the system, whether it succeeded or failed in a task.
In complex systems like neural networks, multiple components work together to
produce an output.
When there is an error (or a success), the system must determine how to adjust its
parameters so that the performance improves.
This process is critical for effective learning.
Credit Assignment in Neural Networks:
In neural networks, the credit assignment problem is solved using learning
algorithms that adjust weights and biases. Two common techniques are:
1. Backpropagation (for Structural Credit Assignment): The error is
propagated backward through the network. Each weight is adjusted based on its
contribution to the overall error.
2. Temporal Difference
Learning (for Temporal
Credit Assignment): Common
in reinforcement learning
tasks. Assigns credit to actions
or decisions made at earlier
time steps that contributed to
later successes or failures.
Example:
Imagine a soccer team scores a goal. Who should get the credit?
• Structural Credit Assignment: Which player (forward, midfielder, etc.)
contributed the most to scoring the goal?
• Temporal Credit Assignment: How did earlier actions in the game (e.g., a
pass made several seconds before) influence the goal?
6. What is convergence?
ANS- Convergence in the context of
neural networks refers to the time when a
learning algorithm reaching a stable state.
From here further training does not result
in significant changes to the model's
parameters (like weights and biases).
At this point, the error (or loss) becomes
minimal, and the network is considered to have "learned" from the data.
Indicators of Convergence:
1. Loss Function Stability: The loss function (error metric) becomes nearly
constant across epochs.
2. Minimal Gradient Updates: The changes in weights and biases are very small.
3. Validation Performance: The performance on validation data stops improving
or remains consistent.
Factors Affecting Convergence:
1. Learning Rate: Too high may cause the network to never converge. And too
low results in slow convergence.
2. Optimization Algorithm: Algorithms like Stochastic Gradient Descent
(SGD) influence how quickly and effectively the network converges.
3. Data Quality: Noisy or poorly scaled data can delay convergence or prevent it
altogether.
Types of Convergence:
1. Global Convergence: The algorithm reaches the global minimum of the loss
function.
2. Local Convergence: The algorithm reaches a local minimum.
Example: Imagine teaching a child to ride a bicycle. At first, they wobble and fall
(high error). Over time, with practice (iterations), their movements stabilize. When
they can ride without falling (minimal error), they've "converged" to the correct
skill.