• Formula: σ(x) = 1 / (1 + e^(-x))
• Output Range: (0, 1)
• Commonly used in binary classification
• Limitation: Saturates for large inputs, leading to vanishing gradients

• Formula: tanh(x) = (e^x - e^(-x)) / (e^x + e^(-x))
• Output Range: (-1, 1)
• Zero-centered activation, better than sigmoid

• Formula: f(x) = max(0, x)
• Fast and efficient
• Limitation: Dying ReLU problem when neurons get stuck

• Converts raw scores into probability distribution
• Used in the output layer for multi-class classification

1. Dropout – Randomly drops neurons during training to prevent co-adaptation
2. L2 Regularization – Penalizes large weights by adding them to the loss function
3. Early Stopping – Stops training when validation error starts increasing
4. Data Augmentation – Expands dataset with transformations (flip, rotate, etc.)
5. Cross-Validation – Ensures model generalization on unseen data

• Uses a pre-trained model (e.g., on ImageNet) for a new task
• Helps reduce training time and data requirements
• Retains useful low-level features (edges, corners)
• Common in computer vision and NLP
• Only top layers are retrained for new task

• A Siamese Network has two or more identical subnetworks with shared weights
• Measures similarity between two inputs
• Outputs feature embeddings which are compared using a distance metric
• Applications: Face verification, signature matching, one-shot learning
• Learns to tell whether two inputs are similar or different

• Selects the maximum value from a patch of the feature map.
• Helps retain the most dominant features.

• Takes the average of values in the patch.
• Used when feature intensity is less important.

• Averages entire feature maps to produce a single number per map.
• Reduces overfitting and used before dense layers.

Input layer: 6 → Hidden Layer: 4 → Hidden Layer: 2 → Output layer (binary) (4 Point)

1. Input to Hidden1 (6 → 4):
   ( (6 \times 4) + 4 = 28 )

2. Hidden1 to Hidden2 (4 → 2):
   ( (4 \times 2) + 2 = 10 )

3. Hidden2 to Output (2 → 1):
   ( (2 \times 1) + 1 = 3 )

✅ Total parameters = 28 + 10 + 3 = 41

• In deep networks, during backpropagation, gradients become very small in early layers.
• Leads to very slow or no learning in initial layers.

• Use ReLU instead of sigmoid/tanh
• Use Batch Normalization
• Use Residual connections (as in ResNet)
• Use architectures like LSTM/GRU for sequential tasks

• IoU (Intersection over Union)
• mAP (mean Average Precision)
• Precision & Recall
• F1 Score

• Measures overlap between predicted and ground truth boxes.
• IoU > threshold (usually 0.5) → correct detection.
• Helps evaluate localization accuracy.

• Algorithm to compute gradients for updating weights using chain rule

1. Input layer receives raw data.
2. Data flows through hidden layers, where weights and biases are applied.
3. Activation functions add non-linearity.
4. Output layer makes prediction.
5. Loss is computed using a loss function.
6. Backpropagation adjusts weights to minimize the loss.
7. Process repeats for multiple epochs.

• Controls the size of weight updates during training.
• Too small → slow learning; too large → overshooting minima.

• Adds a fraction of previous update to the current one.
• Helps accelerate convergence and escape local minima.

• Technique to normalize the inputs to a layer for each mini-batch.
• Helps maintain a mean of 0 and standard deviation of 1.

• Speeds up training.
• Reduces internal covariate shift.
• Allows higher learning rates.
• Acts as a regularizer, sometimes reducing the need for dropout.

• Optimization algorithm to minimize loss by updating weights in the direction of negative gradient.

1. Batch Gradient Descent

   • Uses entire dataset for each update.
   • Stable but slow and memory intensive.

2. Stochastic Gradient Descent (SGD)

   • Uses one data point per update.
   • Faster, but has high variance.

3. Mini-Batch Gradient Descent

   • Uses small batches of data.
   • Combines speed and stability.

• Introduce non-linearity to the model.
• Enable the network to learn complex patterns and relationships.

• Formula: f(x) = max(0, x)
• Simple and computationally efficient.
• Helps avoid vanishing gradient problem.

• Dying ReLU: neurons may output zero for all inputs.
• Not zero-centered.

• Single-shot object detection algorithm.
• Divides image into grid cells; each predicts bounding boxes and class probabilities.

1. Input image is divided into S x S grid.
2. Each grid predicts:
   • Bounding box coordinates (x, y, w, h)
   • Confidence score
   • Class probabilities
3. Non-Maximum Suppression (NMS) removes redundant detections.

• Real-time speed
• End-to-end prediction
• Fewer false positives

• Too complex model with many parameters
• Insufficient training data
• Too many training epochs
• Noisy or irrelevant features
• Lack of regularization

1. Dropout – Randomly disables neurons during training to reduce over-reliance.
2. Data Augmentation – Increases dataset size and variability using transformations. (Also acceptable: L2 regularization, early stopping, cross-validation)

• In deep networks, gradients get smaller as they are backpropagated.
• Early layers receive almost zero updates → learning halts.

• Use ReLU activation instead of sigmoid/tanh
• Apply Batch Normalization
• Use Residual connections (ResNet)
• Use LSTM or GRU for sequential models

• IoU (Intersection over Union)
• Precision / Recall
• mAP (mean Average Precision)
• F1 Score

• Eliminates multiple overlapping boxes for the same object.
• Keeps only the box with the highest confidence.
• Improves detection accuracy and reduces duplicates.

• Introduce non-linearity
• Allow networks to model complex patterns

1. Input Layer

   • Takes image data (e.g., 128x128x3)

2. Convolution Layers

   • Apply filters/kernels to extract features like edges, textures
   • Output: feature maps

3. Activation Function (ReLU)

   • Adds non-linearity

4. Pooling Layers (Max/Average Pooling)

   • Downsamples feature maps to reduce spatial size
   • Helps with translation invariance

5. Flatten Layer

   • Converts 2D data into 1D vector for Dense layers

6. Dense (Fully Connected) Layers

   • Final decision-making layers
   • Often followed by softmax for classification

7. Output Layer

   • Produces final class predictions

• Model performs well on training data but poorly on unseen/test data
• Learns noise or irrelevant details

• Dropout – Randomly deactivate neurons
• Data Augmentation – Increases effective dataset size
• Early Stopping – Stops training when validation loss increases
• Regularization (L2) – Penalizes large weights

• Mathematical functions applied after each layer
• Introduce non-linearity into the model
• Without them, deep models behave like linear functions

• ReLU – Used in hidden layers
• Tanh – Output centered at 0
• Sigmoid – Output between 0 and 1
• Softmax – Used in output layer for multi-class classification

• Single-stage: Fast, less accurate
• Multi-stage: Slower, more accurate

• Comprises two networks:
  1. Generator – Tries to produce fake data
  2. Discriminator – Tries to distinguish real vs fake data
• They are trained adversarially until generator produces realistic data

• Can generate highly realistic data (images, text, audio)
• Used in image synthesis, super-resolution, data augmentation
• Helps in creating synthetic training data for low-resource domains