Course Objectives

• Enhance deep learning tasks with Google Colab and GPU resources.
• Study and implement feedforward neural networks and backpropagation.
• Implement Batch Gradient Descent, Stochastic Gradient Descent, and Mini-batch
Gradient Descent.
• Study and apply batch normalization and dropout for neural networks.
• Train RNNs for sentiment analysis, CNNs for object detection, and GANs for image
generation.

Course Outcomes
At the end of this course, the students will be able to:
• Gain proficiency in using Google Colab for coding, collaborating, and accessing
powerful computing resources like GPUs for faster processing.
• Hands-on experience with libraries like TensorFlow, Keras, or PyTorch, implementing
FFNNs for various data types and tasks.
• Learn the step-by-step process of backpropagation and how gradients are calculated and
used to update weights in a neural network.
• Understand how batch, stochastic, and mini-batch gradient descent affect the efficiency,
accuracy, and computational cost of training models.
• Learn to apply SVD to reduce the dimensionality of datasets, improving computational
efficiency and mitigating issues related to high-dimensional data.
• Understand the techniques like sparsity, dropout, and variational autoencoders to improve
the performance and generalization of autoencoders.
• Gain word2vec embeddings to tasks such as sentiment analysis, document clustering, and
recommendation systems.
• Learn about convolutional neural network (CNN) architectures tailored for object
detection, including feature extraction, detection layers, and bounding box regression.
• Learn the structure and functioning of Long Short-Term Memory (LSTM) networks,
including gates (input, forget, output) and their role in managing long-term dependencies
in sequential data.
• Gain hands-on experience implementing GANs using frameworks like TensorFlow or
PyTorch to generate synthetic handwritten digit images, focusing on data preparation,
model training, and evaluation of generated images
 Deep Learning Lab
List of Experiments/ Exercises
Page Sign of
Exp. No Name of the Experiment
No. Faculty
Getting familiar with the usage of Google Colab and using GPU as
processing unit
Try yourself uploading a dataset and perform basic data pre- 1–6
1
processing on it. Compare the training time with GPU enabled
versus CPU only
Study and implementation of Feed Forward Neural Network.
Conduct an experiment with dataset of your choice to determine the 7 – 13
2 impact of different hyperparameters on the performance of a feed-
forward neural network
Study and Implementation of Back Propagation
3 Write a python code for backward propagation in a neural network 8 – 18
using concepts like error calculation, and learning rate.
Implement Batch Gradient Descent, Stochastic Gradient Descent and Mini
Batch Gradient Descent
4 Implement Gradient Descent for Neural Network (or Logistic 19 – 27
Regression) by Predicting if a person would buy life insurance based
on his age
Study and Implementation of Singular Value Decomposition for
Dimensionality Reduction
Draw the whisker plot for the above solution for creating the
5 28 – 33
distribution of accuracy scores for each configured number of
dimensions and also find out the predicted class by using
combination of SVD transform and logistic regression model
Implementing Autoencoder for Encoding the Real-World Data
6 Write a program to add random noise to the MNIST dataset and train 34 – 38
the autoencoder to reconstruct the clean images.
Implementing Word2Vec for the Real-World Data
7 Implement pre-trained word2vec models from Google or Wikipedia 39 – 41
for improved accuracy and efficiency
Write a program to perform object detection using CNN
8 Write a CNN program using Pascal VOC or COCO dataset and train 42 – 46
the network to detect objects of interest
Study and Implementation of LSTM
9 Write a python code using time series analysis to predict the number 47 – 52
of international airline passengers from January 1949 to December
1960?
Implementation of GAN for Generating Handwritten Digits Images
Implement a Conditional GAN (cGAN) where the generator and
10 discriminator are conditioned on class labels (e.g., digits 0-9 from 53 – 60
the MNIST dataset). Train the cGAN and generate images for each
digit class
Deep Learning Lab Manual R22 III B.Tech I Sem - AI & ML

1. Getting Familiar with the usage of Google Colab and using

GPU as Processing Unit

Introduction
Collaboratory by Google (Google Colab in short) is a Jupyter notebook-based runtime
environment which allows you to run code entirely on the cloud. This is necessary because it
means that you can train large scale ML and DL models even if you don’t have access to a
powerful machine or a high-speed internet access. Google Colab supports both GPU and TPU
instances, which makes it a perfect tool for deep learning and data analytics enthusiasts because
of computational limitations on local machines. Since a Colab notebook can be accessed
remotely from any machine through a browser, it’s well suited for commercial purposes as
well. In this tutorial you will learn:
• Getting around in Google Colab
• Installing python libraries in Colab
• Downloading large datasets in Colab
• Training a Deep learning model in Colab
• Using Tensor Board in Colab
Creating your first .ipynb notebook in colab
Open a browser of your choice and go to colab.research.google.com and sign in using your
Google account. Click on a new notebook to create a new runtime instance. In the top left
corner, you can change the name of the notebook from “Untitled.ipynb“ to the name of your
choice by clicking on it. The cell execution block is where you type your code. To execute the
cell, press shift + enter. The variable declared in one cell can be used in other cells as a global
variable. The environment automatically prints the value of the variable in the last line of the
code block if stated explicitly.
Training a sample tensorflow model
Training a machine learning model in Colab is very easy. The best part about it is not having
to set up a custom runtime environment, it’s all handled for you. For example, let’s look at
training a basic deep learning model to recognize handwritten digits trained on the MNIST
dataset. The data is loaded from the standard Keras dataset archive. The model is very basic, it
categorizes images as numbers and recognizes them.
Setup:
#import necessary libraries
import tensorflow as tf

#load training data and split into train and test sets
mnist = tf.keras.datasets.mnist

(x_train,y_train), (x_test,y_test) = mnist.load_data()

x_train, x_test = x_train / 255.0, x_test / 255.0

Malla Reddy University 1

Deep Learning Lab Manual R22 III B.Tech I Sem - AI & ML

#define model
model = tf.keras.models.Sequential([
tf.keras.layers.Flatten(input_shape=(28,28)),
tf.keras.layers.Dense(128,activation='relu'),
tf.keras.layers.Dropout(0.2),
tf.keras.layers.Dense(10)
])

#define loss function variable

loss_fn = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)

#define optimizer,loss function and evaluation metric

model.compile(optimizer='adam',
loss=loss_fn,
metrics=['accuracy'])

#train the model

model.fit(x_train,y_train,epochs=5)
#test model accuracy on test set
model.evaluate(x_test,y_test,verbose=2)
#extend the base model to predict softmax output
probability_model = tf.keras.Sequential([
model,
tf.keras.layers.Softmax()])

Output:
Downloading data from https://storage.googleapis.com/tensorflow/tf-
keras-datasets/mnist.npz
11490434/11490434 [==============================] - 0s 0us/step
Epoch 1/5
1875/1875 [==============================] - 16s 8ms/step - loss: 0.2975
- accuracy: 0.9144
Epoch 2/5
1875/1875 [==============================] - 7s 3ms/step - loss: 0.1444
- accuracy: 0.9574
Epoch 3/5
1875/1875 [==============================] - 8s 4ms/step - loss: 0.1086
- accuracy: 0.9677
Epoch 4/5
1875/1875 [==============================] - 7s 3ms/step - loss: 0.0871
- accuracy: 0.9726
Epoch 5/5
1875/1875 [==============================] - 8s 4ms/step - loss: 0.0752
- accuracy: 0.9762
313/313 - 1s - loss: 0.0742 - accuracy: 0.9765 - 576ms/epoch - 2ms/step

Downloading a dataset
When you’re training a machine learning model on your local machine, you’re likely to have
trouble with the storage and bandwidth costs that come with downloading and storing the
dataset required for training a model. Deep learning datasets can be massive in size, ranging
between 20 to 50 Gb. Downloading them is most challenging if you’re living in a developing

Malla Reddy University 2

Deep Learning Lab Manual R22 III B.Tech I Sem - AI & ML

country, where getting high-speed internet isn’t possible. The most efficient way to use datasets
is to use a cloud interface to download them, rather than manually uploading the dataset
from a local machine. Thankfully, Colab gives us a variety of ways to download the dataset
from common data hosting platforms.
To download an existing dataset from Kaggle, we can follow the steps outlined below:
1. Go to your Kaggle Account and click on “Create New API Token”. This will
download akaggle.json file to your machine.
2. Go to your Google Colab project file, and run the following commands:
! pip install -q kaggle
from google.colab import files
# choose the kaggle.json file
that you downloaded
files.upload()
! mkdir ~/.kaggle

Downloading the dataset from Google Cloud Platform (GCP) or Google Drive
Google Cloud Platform is a cloud computing and storage platform. You can use it to store large
datasets, and you can import that dataset directly from the cloud into Colab. To upload and
download files on GCP, first you need to authenticate your Google account.
from google.colab import auth

auth.authenticate_user()

After that, install gsutil to upload and download files, and

then init gcloud
!curl https://sdk.cloud.google.com | bash
!gcloud init
Once you have configured these options, you can use the following commands to
download/upload files to and from Google Cloud Storage.

Initiating a runtime with GPU/TPU enabled

Deep learning is a computationally expensive process, a lot of calculations need to be executed
atthe same time to train a model. To mitigate this issue, Google Colab offers us not only the
classicCPU runtime but also an option for a GPU and TPU runtime as well. The CPU runtime
is best for training large models because of the high memory it provides. The GPU runtime
shows better flexibility and programmability for irregular computations, such as small batches
and nonMatMul computations. The TPU runtime is highly-optimized for large batches and
CNNs and has the highest training throughput. If you have a smaller model to train, I suggest
training the model on GPU/TPU runtime to use Colab to its full potential. To create a
GPU/TPU enabled runtime, you can click on runtime in the toolbar menu below the file name.
From there, click on “Change runtime type”, and then select GPU or TPU under the Hardware
Accelerator dropdown menu.

Malla Reddy University 3

Deep Learning Lab Manual R22 III B.Tech I Sem - AI & ML

Malla Reddy University 4

Deep Learning Lab Manual R22 III B.Tech I Sem - AI & ML

Exercise:
1. Try yourself uploading a dataset and perform basic data pre-processing on it. Compare
the training time with GPU enabled versus CPU only.
Solution:

Malla Reddy University 5

Deep Learning Lab Manual R22 III B.Tech I Sem - AI & ML

Malla Reddy University 6

Deep Learning Lab Manual R22 III B.Tech I Sem - AI & ML

2. Study and Implementation of Feed Forward Neural Network

What is a (Neural Network) NN?

• Single neuron == linear regression without applying activation(perceptron)
• Basically, a single neuron will calculate weighted sum of input W(T)×X and then we can
set a threshold to predict output in a perceptron. If weighted sum of input crosses the
threshold, perceptron fires and if not, then perceptron doesn't predict.
• Perceptron can take real values input or boolean values.
• Actually, when w(x)+b=0 the perceptron outputs 0.
• Disadvantage of perceptron is that it only output binary values and if we try to give small
change in weight and bais then perceptron can flip the output. We need some system which
can modify the output slightly according to small change in weight and bias. Here comes
sigmoid function in picture.
• If we change perceptron with a sigmoid function, then we can make slight change in
output. e.g. output in perceptron = 0, you slightly changed weight and bias, output
becomes = 1 but actual output is 0.7. In case of sigmoid, output1 = 0, slight change in
weight and bias, output = 0.7.
If we apply sigmoid activation function then Single neuron will act as Logistic Regression.
we can understand difference between perceptron and sigmoid function by looking at sigmoid
function graph.
Simple NN Model

A visual explanation of how Feed Forward NNs work

First, let’s familiarize ourselves with the basic structure of a Neural Network

Malla Reddy University 7

Deep Learning Lab Manual R22 III B.Tech I Sem - AI & ML

• Input Layer — contains one or more input nodes. For example, suppose you want to
predict whether it will rain tomorrow and base your decision on two variables, humidity
and wind speed. In that case, your first input would be the value for humidity, and the
second input would be the value for wind speed.
• Hidden Layer — this layer houses hidden nodes, each containing an activation function
(more on these later). Note that a Neural Network with multiple hidden layers isknown as
Deep Neural Network.
• Output Layer — contains one or more output nodes. Following the same weather
prediction example above, you could choose to have only one output node generating arain
probability (where >0.5 means rain tomorrow, and ≤0.5 no rain tomorrow). Alternatively,
you could have two output nodes, one for rain and another for no rain. Note, you can use a
different activation function for output nodes vs. hidden nodes.
• Connections — lines joining different nodes are known as connections. These contain
kernels (weights) and biases, the parameters that get optimized during thetraining of a
neural network.

Parameters and activation functions

Let’s take a closer look at kernels (weights) and biases to understand what they do. For
simplicity, we create a basic neural network with one input node, two hidden nodes, and one
output node (1–2–1)

• Kernels (weights) — used to scale input and hidden node values. Each connection
typically holds a different weight.
• Biases — used to adjust scaled values before passing them through an activation
function.
• Activation functions — think of activation functions as standard curves (building
blocks) used by the Neural Network to create a custom curve to fit the training
data.Passing different input values through the network selects different sections
of the standard curve, which are then assembled into a final custom-fit curve.
Loss functions, optimizers, and training
Training Neural Networks involves a complicated process known as backpropagation. I will
not go through a step-by-step explanation of how backpropagation works since it is a big

Malla Reddy University 8

Deep Learning Lab Manual R22 III B.Tech I Sem - AI & ML

enough topic deserving a separate article. Instead, let me briefly introduce you to loss
functions and optimizers and summarize what happens when we “train” a Neural Network.
• Loss — represents the “size” of error between the true values/labels and the predicted
values/labels. The goal of training a Neural Network is to minimize this loss. The smaller
the loss, the closer the match between the true and the predicted data. There are many loss
functions to choose from, with Binary Crossentropy, Categorical Crossentropy, and Mean
Squared Error being the most common.
• Optimizers — are the algorithms used in backpropagation. The goal of an optimizer
isto find the optimum set of kernels (weights) and biases to minimize the loss. Optimizers
typically use a gradient descent approach, which allows them to iteratively find the “best”
possible configuration of weights and biases. The most commonly used ones are SGD,
ADAM, and RMS Prop.
Training a Neural Network is basically fitting a custom curve through the training data until it
can approximate it as well as possible. The graph below illustrates what a custom-fitted curve
could look like in a specific scenario. This example contains a set of data that seem to flip
between 0 and 1 as the value for input increases.
Example: Implementation of Feed Forward Neural Network
import math

import pandas as pd

from keras import models, layers, optimizers, regularizers

import numpy as np

import random

from sklearn import model_selection, preprocessing

import tensorflow as tf

from tqdm import tqdm

import matplotlib.pyplot as plt

file_name = '/content/SAheart.data'

data = pd.read_csv(file_name, sep=',', index_col=0)

data['famhist'] = data['famhist'] == 'Present'

data.head()

n_test = int(math.ceil(len(data) * 0.3))

random.seed(42)

test_ixs = random.sample(list(range(len(data))), n_test)

train_ixs = [ix for ix in range(len(data)) if ix not in test_ixs]

train = data.iloc[train_ixs, :]

test = data.iloc[test_ixs, :]

Malla Reddy University 9

Deep Learning Lab Manual R22 III B.Tech I Sem - AI & ML

print(len(train))

print(len(test))

#features = ['sbp', 'tobacco', 'ldl', 'adiposity', 'famhist', 'typea',

'obesity', 'alcohol', 'age']

features = ['adiposity', 'age']

response = 'chd'

x_train = train[features]

y_train = train[response]

x_test = test[features]

y_test = test[response]

x_train = preprocessing.normalize(x_train)

x_test = preprocessing.normalize(x_test)

hidden_units = 10 # how many neurons in the hidden layer

activation = 'sigmoid' # activation function for hidden layer

l2 = 0.01 # regularization - how much we penalize large

parameter values

learning_rate = 0.01 # how big our steps are in gradient descent

epochs = 5 # how many epochs to train for

batch_size = 16 # how many samples to use for each gradient

descent update

# create a sequential model

model = models.Sequential()

# add the hidden layer

model.add(layers.Dense(input_dim=len(features),

units=hidden_units,

activation=activation))

# add the output layer

model.add(layers.Dense(input_dim=hidden_units,

units=1,

activation='sigmoid'))

# define our loss function and optimizer

model.compile(loss='binary_crossentropy',

# Adam is a kind of gradient descent

Malla Reddy University 10

Deep Learning Lab Manual R22 III B.Tech I Sem - AI & ML

optimizer=optimizers.Adam(lr=learning_rate),

metrics=['accuracy'])

# train the parameters

history = model.fit(x_train, y_train, epochs=10, batch_size=batch_size)

# evaluate accuracy

train_acc = model.evaluate(x_train, y_train, batch_size=32)[1]

test_acc = model.evaluate(x_test, y_test, batch_size=32)[1]

print('Training accuracy: %s' % train_acc)

print('Testing accuracy: %s' % test_acc)

losses = history.history['loss']

plt.plot(range(len(losses)), losses, 'r')

plt.show()

Output

Epoch 1/5
21/21 [==============================] - 1s 2ms/step - loss: 0.6914 -
accuracy: 0.5263
Epoch 2/5
21/21 [==============================] - 0s 2ms/step - loss: 0.6701 -
accuracy: 0.6718
Epoch 3/5
21/21 [==============================] - 0s 2ms/step - loss: 0.6546 -
accuracy: 0.6718
Epoch 4/5
21/21 [==============================] - 0s 2ms/step - loss: 0.6457 -
accuracy: 0.6718
Epoch 5/5
21/21 [==============================] - 0s 2ms/step - loss: 0.6396 -
5/5 [==============================] - 0s 3ms/step - loss: 0.6717 -
accuracy: 0.6115

Malla Reddy University 11

Deep Learning Lab Manual R22 III B.Tech I Sem - AI & ML

accuracy: 0.6718
accuracy: 0.6115
Training accuracy: 0.6718266010284424
Testing accuracy: 0.6115108132362366

Exercise:
1. Conduct an experiment with dataset of your choice to determine the impact of different
hyperparameters on the performance of a feed-forward neural network.
Solution:

Malla Reddy University 12

Deep Learning Lab Manual R22 III B.Tech I Sem - AI & ML

Malla Reddy University 13

Deep Learning Lab Manual R22 III B.Tech I Sem - AI & ML

3. Study and Implementation of Back Propagation

Why use the back propagation algorithm?

Earlier we discussed that the network is trained using 2 passes: forward and backward. At
the end of the forward pass, the network error is calculated, and should be as small as
possible.
If the current error is high, the network didn’t learn properly from the data. What does this
mean? It means that the current set of weights isn’t accurate enough to reduce the network
error and make accurate predictions. As a result, we should update network weights to
reduce the network error.
The back propagation algorithm is one of the algorithms responsible for updating network
weights with the objective of reducing the network error. It’s quite important.
Here are some of the advantages of the back propagation algorithm:
• It’s memory-efficient in calculating the derivatives, as it uses less memory compared to
other optimization algorithms, like the genetic algorithm. This is a very important feature,
especially with large networks.
• The back propagation algorithm is fast, especially for small and medium-sized networks.
As more layers and neurons are added, it starts to get slower as more derivatives are
calculated.
• This algorithm is generic enough to work with different network architectures, like
convolutional neural networks, generative adversarial networks, fully-connected
networks, and more.
• There are no parameters to tune the back propagation algorithm, so there’s lessoverhead.
The only parameters in the process are related to the gradient descent algorithm, like
learning rate

How back propagation algorithm works

How the algorithm works is best explained based on a simple network, like the one given
in the next figure. It only has an input layer with 2 inputs (X1 and X2), and an output layer
with 1 output. There are no hidden layers.
The weights of the inputs are W1 and W2, respectively. The bias is treated as a new input
neuron to the output neuron which has a fixed value +1 and a weight b. Both the weights
and biases could be referred to as parameters.

Malla Reddy University 14

Deep Learning Lab Manual R22 III B.Tech I Sem - AI & ML

Example: Implementation of Backward Propagation

import numpy as np

# Sigmoid activation function

def sigmoid(x):
return 1 / (1 + np.exp(-x))

# Derivative of sigmoid activation function

# Since sigmoid(x) outputs values between 0 and 1, 1 - sigmoid(x)
represents the probability or the activation
# level of a neuron being "off" given the input x
def sigmoid_derivative(x):
return sigmoid(x) * (1 - sigmoid(x))

# Forward pass function

def forward_pass(x, weights):
# Calculate the weighted sum of inputs
weighted_sum = np.dot(x, weights)

# Apply activation function (sigmoid)

output = sigmoid(weighted_sum)

return output

# Backward pass function (gradient descent)

def backward_pass(x, output, actual_output, weights, learning_rate):
# Calculate error
error = actual_output - output

# Compute derivative of the error with respect to the weighted sum

error_derivative = error * sigmoid_derivative(output)

# Update weights using gradient descent

weights += learning_rate * error_derivative * x

return weights, error

# Inputs (features)
x = np.array([0.1, 0.2, 0.3, 0.4])

# Initial weights
weights = np.array([0.5, -0.5, 0.3, -0.3])

# Actual output
actual_output = 1

# Learning rate
learning_rate = 0.01

Malla Reddy University 15

Deep Learning Lab Manual R22 III B.Tech I Sem - AI & ML

# Number of iterations
num_iterations = 10000

# Perform forward pass

output_before = forward_pass(x, weights)

# Perform backward pass (update weights)

for i in range(num_iterations):
weights, error = backward_pass(x, output_before, actual_output,
weights, learning_rate)

# Perform forward pass again after weight update

output_after = forward_pass(x, weights)

# Print results
print("Initial Output:", output_before)
print("Updated Output:", output_after)
print("Error Difference:", error - (actual_output - output_after))
print("Updated Weights after Backward Pass:", weights)

Output
Initial Output: 0.48001065984441826
Updated Output: 0.9734967745349465
Error Difference: 0.4934861146905283
Updated Weights after Backward Pass: [1.72787602 1.95575205 3.98362807
4.61150409]

Exercise:
1. Write a python code for backward propagation in a neural network using concepts like error
calculation, and learning rate.

Solution:

Malla Reddy University 16

Deep Learning Lab Manual R22 III B.Tech I Sem - AI & ML

Malla Reddy University 17

Deep Learning Lab Manual R22 III B.Tech I Sem - AI & ML

Malla Reddy University 18

Deep Learning Lab Manual R22 III B.Tech I Sem - AI & ML

4. Implement Batch Gradient Descent, Stochastic Gradient

Descent and Mini Batch Gradient Descent

Introduction

We will use very simple home prices data set to implement batch and stochastic gradient
descent inpython.

Batch gradient descent uses all training samples in forward pass to calculate cumulative error
and then weadjust weights using derivatives. In stochastic GD, we randomly pick one training
sample, perform forward pass, compute the error and immediately adjust weights.

So, the key difference here is that to adjust weights, batch GD will use all training samples
whereas stochastic GD will use one randomly picked training sample.

Mini batch is intermediate version of batch GD and stochastic GD. In mini gradient descent you
will use abatch of samples in each iteration. For example, if you have total 50 training samples,
you can take a batch of 10 samples, calculate cumulative error for those 10 samples and then
adjust weights.

To summarize it, In SGD we adjust weights after every one sample. In Batch we adjust weights
after going through all samples but in mini batch we do after every m sample (where m is batch
size and it is 0 < m < n, where n is total number of samples).

Gradient descent allows you to find weights (𝑤1, 𝑤2 , 𝑤3 ) and bias in the following linear
equation for housing price prediction.

Example:
Gradient Descent
import numpy as np
import matplotlib.pyplot as plt

# Sample Home Price Dataset (features: area, bedrooms; target: price)

data = {
'area': [2600, 3000, 3200, 3600, 4000],
'bedrooms': [3, 4, 3, 4, 5],
'price': [550000, 565000, 610000, 590000, 680000]
}
# Convert data into numpy arrays
X = np.array([data['area'], data['bedrooms']]).T
y = np.array(data['price'])

def gradient_descent(X, y, learning_rate=0.001, epochs=100):

Malla Reddy University 19

Deep Learning Lab Manual R22 III B.Tech I Sem - AI & ML

num_features = X.shape[1] #Computes the number of features (n)

w = np.ones(num_features) #Initializes the weight vector w
b = 0 #Initializes the bias term (intercept) b to zero
total_samples = X.shape[0] #Computes the total number of samples
(m)
cost_list = []

for epoch in range(epochs):

y_predicted = np.dot(X, w) + b

#w_grad: gradient of the cost function with respect to the weight

#-(2/total_samples): This term is a scalar factor applied to the
gradient.
#It is multiplied by -2 divided by the total number of samples (m).
#This factor is a part of the derivative of the mean squared error
(MSE) loss function, which is commonly used in linear regression
#transpose of the feature matrix X(X.T)

w_grad = -(2/total_samples) * np.dot(X.T, (y - y_predicted))

#This part calculates the sum of the errors (y - y_predicted) across

all samples

# b_grad This variable represents the gradient of the cost function

with respect to the bias term b

b_grad = -(2/total_samples) * np.sum(y - y_predicted)

w = learning_rate * w_grad
b = learning_rate * b_grad

total_cost = np.mean((y - y_predicted) ** 2)

cost_list.append(total_cost)

return w, b, cost_list

# Running GD
w_gd, b_gd, cost_list_gd = gradient_descent(X, y,
learning_rate=0.001, epochs=500)

# Plotting the cost function over epochs

plt.plot(range(len(cost_list_gd))[::-1], cost_list_gd) # Reverse the
x-axis
plt.xlabel('Epochs')
plt.ylabel('Cost')
plt.title('Cost Function Over Epochs (Gradient Descent)')
plt.show()

Malla Reddy University 20

Deep Learning Lab Manual R22 III B.Tech I Sem - AI & ML

Output:

Stochastic Gradient Descent

import numpy as np
import matplotlib.pyplot as plt
from sklearn.preprocessing import MinMaxScaler

# Sample Home Price Dataset (features: area, bedrooms; target:

price)
data = {
'area': [2600, 3000, 3200, 3600, 4000],
'bedrooms': [3, 4, 3, 4, 5],
'price': [550000, 565000, 610000, 590000, 680000]
}

# Convert data into numpy arrays

X = np.array([data['area'], data['bedrooms']]).T
y = np.array(data['price'])

# Scale the features and target using MinMaxScaler

scaler = MinMaxScaler()
scaled_X = scaler.fit_transform(X)
scaled_y = scaler.fit_transform(y.reshape(-1, 1)).flatten()

def stochastic_gradient_descent(X, y_true, epochs,

learning_rate=0.01):
number_of_features = X.shape[1]
w = np.ones(shape=(number_of_features,))
b = 0
total_samples = X.shape[0]
cost_list = []
epoch_list = []

for i in range(epochs):

Malla Reddy University 21

Deep Learning Lab Manual R22 III B.Tech I Sem - AI & ML

random_index = np.random.randint(0, total_samples) # Select

random sample
sample_x = X[random_index]
sample_y = y_true[random_index]

y_predicted = np.dot(w, sample_x) + b

w_grad = -(2/total_samples) * (sample_x * (sample_y -

y_predicted))
b_grad = -(2/total_samples) * (sample_y - y_predicted)

# Update weights and bias using accumulated gradients

w = w - learning_rate * w_grad
b = b - learning_rate * b_grad

cost = np.square(sample_y - y_predicted)

if i % 100 == 0:
cost_list.append(cost)
epoch_list.append(i)

return w, b, cost, cost_list, epoch_list

w_sgd, b_sgd, cost_sgd, cost_list_sgd, epoch_list_sgd =
stochastic_gradient_descent(scaled_X,
scaled_y.reshape(scaled_y.shape[0],), 9000)
# Plotting the cost function over epochs
plt.plot(epoch_list_sgd, cost_list_sgd)
plt.xlabel('Epochs')
plt.ylabel('Cost')
plt.title('Cost Function Over Epochs (Stochastic Gradient Descent)')
plt.show()

Output:

Malla Reddy University 22

Deep Learning Lab Manual R22 III B.Tech I Sem - AI & ML

Mini-Batch Gradient Descent

def mini_batch_gradient_descent(X, y_true, epochs=80, batch_size=2,
learning_rate=0.0001, plot_interval=10):
number_of_features = X.shape[1]
w = np.ones(shape=(number_of_features,))
b =1
total_samples = X.shape[0]
cost_list = []
epoch_list = []

for epoch in range(epochs):

# Shuffle the data
indices = np.arange(total_samples)
np.random.shuffle(indices)
X_shuffled = X[indices]
y_shuffled = y_true[indices]

for j in range(0, total_samples, batch_size):

X_mini_batch = X_shuffled[j:j+batch_size]
y_mini_batch = y_shuffled[j:j+batch_size]

# Compute predictions
y_predicted = np.dot(X_mini_batch, w) + b

# Compute gradients
w_grad = -(2/batch_size) * np.dot(X_mini_batch.T,
(y_mini_batch - y_predicted))
b_grad = -(2/batch_size) * np.sum(y_mini_batch -
y_predicted)

# Update weights and bias using gradients

w -= learning_rate * w_grad
b -= learning_rate * b_grad

# Compute cost and append to list

y_predicted = np.dot(X, w) + b
cost = np.mean(np.square(y - y_predicted))
if epoch % plot_interval == 0:
cost_list.append(cost)
epoch_list.append(epoch)

return w, b, cost_list, epoch_list

# Run mini-batch gradient descent

w_minibatch, b_minibatch, cost_minibatch, epoch_list_minibatch =
mini_batch_gradient_descent(

Malla Reddy University 23

Deep Learning Lab Manual R22 III B.Tech I Sem - AI & ML

scaled_X, scaled_y, epochs=100, batch_size=2, learning_rate=0.1,

plot_interval=5
)

# Calculate total error

#total_error = sum(cost_minibatch)
#print("Total error:", total_error)

# Reverse the order of the lists for plotting

cost_minibatch.reverse()
epoch_list_minibatch.reverse()

# Plot the cost function over epochs

plt.plot(epoch_list_minibatch, cost_minibatch, linestyle='-')
plt.xlabel('Epochs')
plt.ylabel('Cost')
plt.title('Cost Function Over Epochs (Mini-Batch Gradient Descent)')
plt.ylim(reversed(plt.ylim())) # Reverse the y-axis limits
plt.show()

Output:

Malla Reddy University 24

Deep Learning Lab Manual R22 III B.Tech I Sem - AI & ML

Exercise:
1. Implement Gradient Descent for Neural Network (or Logistic Regression) by Predicting
if a person would buy life insurance based on his age.
2. Conduct an experiment to investigate the impact of different mini-batch sizes on the
performance of Mini-Batch Gradient Descent (e.g., 16, 32, 64, 128).

Solution:

Malla Reddy University 25

Deep Learning Lab Manual R22 III B.Tech I Sem - AI & ML

Malla Reddy University 26

Deep Learning Lab Manual R22 III B.Tech I Sem - AI & ML

Malla Reddy University 27

Deep Learning Lab Manual R22 III B.Tech I Sem - AI & ML

5. Study and Implementation of Singular Value Decomposition

for Dimensionality Reduction

Introduction
Reducing the number of input variables for a predictive model is referred to as dimensionality
reduction. Fewer input variables can result in a simpler predictive model that may have better
performance when making predictions on new data. Perhaps the more popular technique for
dimensionality reduction in machine learning is Singular Value Decomposition, or SVD for
short. This is a technique that comes from the field of linear algebra and can be used as a data
preparation technique to create a projectionof a sparse dataset prior to fitting a model.

Dimensionality Reduction and SVD

Dimensionality reduction refers to reducing the number of input variables for a dataset. If your
data is represented using rows and columns, such as in a spreadsheet, then the input variables
are the columns that are fed as input to a model to predict the target variable. Input variables
are also called features.
We can consider the columns of data representing dimensions on an n-dimensional feature
space and the rows of data as points in that space. This is a useful geometric interpretation of
a dataset. Having a large number of dimensions in the feature space can mean that the volume
of that space is very large, and in turn, the points that we have in that space (rows of data) often
represent a small and non-representative sample.
This can dramatically impact the performance of machine learning algorithms fit on data with
many input features, generally referred to as the “curse of dimensionality.” Therefore, it is often
desirable to reduce the number of input features. This reduces the number of dimensions of
the feature space, hence the name “dimensionality reduction.”
A popular approach to dimensionality reduction is to use techniques from the field of linear
algebra. This is often called “feature projection” and the algorithms used are referred to as
“projection methods.” Projection methods seek to reduce the number of dimensions in the
feature space whilst also preserving the most important structure or relationships between the
variables observed in the data.
Singular Value Decomposition, or SVD, might be the most popular technique for
dimensionality reduction when data is sparse. Sparse data refers to rows of data where many
of the values are zero. This is often the case in some problem domains like recommender
systems where a user has a rating for very few movies or songs in the database and zero ratings
for all other cases. Another common example is a bag of words model of a text document,
where the document has a count or frequency for some words and most words have a 0 value.

Examples of sparse data appropriate for applying SVD for dimensionality reduction:
• Recommender Systems
• Customer-Product purchases
• User-Song Listen Counts
• User-Movie Ratings
• Text Classification

Malla Reddy University 28

Deep Learning Lab Manual R22 III B.Tech I Sem - AI & ML

• One Hot Encoding

• Bag of Words Counts
• TF/IDF

SVD can be thought of as a projection method where data with m-columns (features) is
projected into a subspace with m or fewer columns, whilst retaining the essence of the original
data.
The SVD is used widely both in the calculation of other matrix operations, such as matrix
inverse, but also as a data reduction method in machine learning.

Example 1:

import numpy as np
import matplotlib.pyplot as plt
from sklearn.datasets import load_iris

# Step 1: Load the Iris dataset

iris = load_iris()
data = iris.data

# Step 2: Perform SVD

U, S, Vt = np.linalg.svd(data, full_matrices=False) # SVD
decomposition

# Step 3: Reduce dimensions

k = 2 # Number of dimensions to reduce to
reduced_data = np.dot(data, Vt[:k].T) # Project data onto the first
k principal components

# Step 4: Visualize the results

plt.figure(figsize=(10, 5))

# Plot original data

plt.subplot(1, 2, 1)
plt.scatter(data[:, 0], data[:, 1], c=iris.target, cmap='viridis',
label='Original Data')
plt.title('Original Data')
plt.xlabel('Sepal Length (cm)')
plt.ylabel('Sepal Width (cm)')
plt.legend()

# Plot reduced data

plt.subplot(1, 2, 2)
plt.scatter(reduced_data[:, 0], reduced_data[:, 1], c=iris.target,
cmap='viridis', label='Reduced Data')
plt.title('Data After Dimensionality Reduction (k=2)')
plt.xlabel('Principal Component 1')
plt.ylabel('Principal Component 2')
plt.legend()

Malla Reddy University 29

Deep Learning Lab Manual R22 III B.Tech I Sem - AI & ML

plt.tight_layout()
plt.show()

Output:

Example 2:

import numpy as np # for numerical computations in Python

import matplotlib.pyplot as plt

# Step 1: Generate sample data

np.random.seed(0) # Sets the seed for the random number generator to
ensure reproducibility of results
data = np.random.randn(100, 2) # Sample data with 100 points in 2
dimensions

# Step 2: Perform SVD

#U is the left singular vectors matrix.
#S is a 1D array containing the singular values.
#Vt is the right singular vectors matrix
U, S, Vt = np.linalg.svd(data, full_matrices=False) # SVD
decomposition

# Step 3: Reduce dimensions

k = 1 # Number of dimensions to reduce to
reduced_data = np.dot(data, Vt[:k].T) # Project data onto the first
k principal components

# Step 4: Visualize the results

plt.figure(figsize=(10, 5))

# Plot original data

plt.subplot(1, 2, 1)

Malla Reddy University 30

Deep Learning Lab Manual R22 III B.Tech I Sem - AI & ML

plt.scatter(data[:, 0], data[:, 1], color='blue', label='Original

Data')
plt.title('Original Data')
plt.xlabel('Feature 1')
plt.ylabel('Feature 2')
plt.legend()

# Plot reduced data

plt.subplot(1, 2, 2)
plt.scatter(reduced_data[:, 0], np.zeros_like(reduced_data),
color='red', label='Reduced Data')
plt.title('Data After Dimensionality Reduction (k=1)')
plt.xlabel('Principal Component 1')
plt.ylabel('Dimension Reduced to')
plt.legend()
plt.tight_layout()
plt.show()

Output:

Malla Reddy University 31

Deep Learning Lab Manual R22 III B.Tech I Sem - AI & ML

Exercise:
1. Draw the whisker plot for the above solution for creating the distribution of accuracy scores
for each configured number of dimensions and also find out the predicted class by using
combination of SVD transform and logistic regression model.

Solution:

Malla Reddy University 32

Deep Learning Lab Manual R22 III B.Tech I Sem - AI & ML

Malla Reddy University 33

Deep Learning Lab Manual R22 III B.Tech I Sem - AI & ML

6. Implementing Autoencoder for Encoding the Real-World Data

Auto Encoder is a data compression algorithm where the compression and decompression
functions are 1) data-specific, 2) lossy, and 3) learned automatically from examples rather than
engineered by a human. Additionally, in almost all contexts where the term "auto encoder" is
used, the compression and decompression functions are implemented with neural networks.
• Auto encoders are data-specific, which means that they will only be able to compress
data similar to what they have been trained on. This is different from, say, the MPEG-2
Audio Layer III (MP3) compression algorithm, which only holds assumptions about
"sound" in general, but not about specific types of sounds. An auto encoder trained on
pictures of faces would do a rather poor job of compressing pictures of trees, because
the features it would learn would be face-specific.
• Auto encoders are lossy, which means that the decompressed outputs will be degraded
compared to the original inputs (similar to MP3 or JPEG compression). This differs from
losslessarithmetic compression.
• Auto encoders are learned automatically from data examples, which is a useful property:
it means that it is easy to train specialized instances of the algorithm that will perform
well on a specific type of input. It doesn't require any new engineering, just appropriate
training data.
To build an auto encoder, you need three things: an encoding function, a decoding function,
and a distance function between the amount of information loss between the compressed
representation of your data and the decompressed representation (i.e. a "loss" function). The
encoder and decoder will be chosen to be parametric functions (typically neural networks),
and to be differentiable with respect to the distance function, so the parameters of the
encoding/decoding functions can be optimizing to minimize the reconstruction loss, using
Stochastic Gradient Descent. It's simple! And you don't even need to understand any of these
words to start using auto encoders in practice.

Example
import keras
from keras import layers

# This is the size of our encoded representations

encoding_dim = 32 # 32 floats -> compression of factor 24.5,
assuming the input is 784 floats

# This is our input image

input_img = keras.Input(shape=(784,))
# "encoded" is the encoded representation of the input
encoded = layers.Dense(encoding_dim, activation='relu')(input_img)
# "decoded" is the lossy reconstruction of the input
decoded = layers.Dense(784, activation='sigmoid')(encoded)

# This model maps an input to its reconstruction

Malla Reddy University 34

Deep Learning Lab Manual R22 III B.Tech I Sem - AI & ML

autoencoder = keras.Model(input_img, decoded)

# This model maps an input to its encoded representation
encoder = keras.Model(input_img, encoded)
# This is our encoded (32-dimensional) input
encoded_input = keras.Input(shape=(encoding_dim,))
# Retrieve the last layer of the autoencoder model
decoder_layer = autoencoder.layers[-1]
# Create the decoder model
decoder = keras.Model(encoded_input, decoder_layer(encoded_input))
autoencoder.compile(optimizer='adam', loss='binary_crossentropy')
from keras.datasets import mnist
import numpy as np
(x_train, _), (x_test, _) = mnist.load_data()
x_train = x_train.astype('float32') / 255.
x_test = x_test.astype('float32') / 255.
x_train = x_train.reshape((len(x_train),
np.prod(x_train.shape[1:])))
x_test = x_test.reshape((len(x_test), np.prod(x_test.shape[1:])))
print(x_train.shape)
print(x_test.shape)
autoencoder.fit(x_train, x_train,
epochs=50,
batch_size=256,
shuffle=True,
validation_data=(x_test, x_test))
# Encode and decode some digits
# Note that we take them from the *test* set
encoded_imgs = encoder.predict(x_test)
decoded_imgs = decoder.predict(encoded_imgs)
# Use Matplotlib (don't ask)
import matplotlib.pyplot as plt

n = 10 # How many digits we will display

plt.figure(figsize=(20, 4))
for i in range(n):
# Display original
ax = plt.subplot(2, n, i + 1)
plt.imshow(x_test[i].reshape(28, 28))
plt.gray()
ax.get_xaxis().set_visible(False)
ax.get_yaxis().set_visible(False)

# Display reconstruction
ax = plt.subplot(2, n, i + 1 + n)
plt.imshow(decoded_imgs[i].reshape(28, 28))
plt.gray()
ax.get_xaxis().set_visible(False)
ax.get_yaxis().set_visible(False)
plt.show()

Malla Reddy University 35

Deep Learning Lab Manual R22 III B.Tech I Sem - AI & ML

Output:
Downloading data from https://storage.googleapis.com/tensorflow/tf-
keras-datasets/mnist.npz
11490434/11490434 [==============================] - 0s 0us/step
(60000, 784)
(10000, 784)
Epoch 1/50
235/235 [==============================] - 4s 5ms/step - loss: 0.2775
- val_loss: 0.1906
Epoch 2/50
235/235 [==============================] - 1s 4ms/step - loss: 0.1714
- val_loss: 0.1541
Epoch 3/50
235/235 [==============================] - 1s 4ms/step - loss: 0.1447
- val_loss: 0.1341
Epoch 4/50
235/235 [==============================] - 1s 4ms/step - loss: 0.1290
- val_loss: 0.1220
Epoch 5/50
235/235 [==============================] - 1s 4ms/step - loss: 0.1189
- val_loss: 0.1135

Experiment 6 Outcome:

• Apply autoencoders to tasks such as image denoising, anomaly detection in industrial

sensor data, and dimensionality reduction for visualization or feature extraction in
complex datasets.
• Learn how autoencoders create compact, meaningful representations of data in the latent
space.
• Explore techniques like sparsity, dropout, and variational autoencoders to improve the
performance and generalization of autoencoders.

Exercise:
1. Write a program to add random noise to the MNIST dataset and train the autoencoder to
reconstruct the clean images. Compare the reconstructed images with the original images
and evaluate the quality of the reconstruction

Solution:

Malla Reddy University 36

Deep Learning Lab Manual R22 III B.Tech I Sem - AI & ML

Malla Reddy University 37

Deep Learning Lab Manual R22 III B.Tech I Sem - AI & ML

Malla Reddy University 38

Deep Learning Lab Manual R22 III B.Tech I Sem - AI & ML

7. Implementing Word2Vec for the Real-World Data

Introduction

Word2vec is a powerful technique in natural language processing (NLP) that transforms words
into dense vector representations, capturing semantic relationships and contextual meanings.
By embedding words in a continuous vector space, word2vec enables algorithms to interpret
textual data more effectively, facilitating tasks such as sentiment analysis, language translation,
and recommendation systems.

Implementing word2vec involves training neural networks on large corpora of text to learn
these embeddings. This process allows words with similar meanings to have vectors that are
close together in the embedding space, enhancing the model's ability to generalize and
understand language nuances.

In this context, understanding how to implement word2vec for real-world data involves
selecting appropriate training data, choosing the right parameters for the model, and
interpreting the resulting embeddings to derive meaningful insights from textual data.

The basic process for training a word2vec model is as follows:

• Collect a large dataset of text data. This data is typically in the form of a list of
sentences or acollection of documents.
• Tokenize the text data into individual words. This is often done using a tokenizer,
such as the NLTK library in Python.
• Build a vocabulary of all the words in the dataset. This vocabulary will be used to create
the word vectors.
• Train the model. The training process involves iterating over the dataset, and for
each word, predicting the words that are likely to appear in its context. This process
is done using a neural network with a single hidden layer.
• Save the model. After the model is trained, it can be saved for later use.

Using the model: Once the word2vec model is trained, it can be used to perform several
operations such as finding the similarity between words, finding the odd one out, finding the
analogy and more.

Example:

from gensim.models import Word2Vec

from gensim.test.utils import common_texts

# Sample dataset
dataset = [
"natural language processing is a field of computer science",
"word embeddings are used in natural language processing tasks",
"deep learning models have revolutionized natural language processing",
"word2vec is a popular technique for generating word embeddings"
]

Malla Reddy University 39

Deep Learning Lab Manual R22 III B.Tech I Sem - AI & ML

# Tokenize the dataset into words

tokenized_dataset = [text.split() for text in dataset]

# Train word2vec model

model = Word2Vec(tokenized_dataset, vector_size=100, window=5,
min_count=1, workers=4)

# Test the word embeddings

print("Similarity between 'word' and 'embeddings':",
model.wv.similarity('word', 'embeddings'))
print("Most similar words to 'word':",
model.wv.most_similar('word'))

Output:

Similarity between 'word' and 'embeddings': 0.019159097

Most similar words to 'word': [('science', 0.16699635982513428), ('a',

0.13885433971881866), ('is', 0.13150468468666077), ('revolutionized',
0.09771139174699783), ('learning', 0.07175446301698685),
('processing', 0.06410364806652069), ('field', 0.060617249459028244),
('deep', 0.047680601477622986), ('for', 0.04408745467662811),
('used', 0.04276478663086891)]

Exercise:
1. Implement pre-trained word2vec models from Google or Wikipedia for improved
accuracy and efficiency.
Solution

Malla Reddy University 40

Deep Learning Lab Manual R22 III B.Tech I Sem - AI & ML

Malla Reddy University 41

Deep Learning Lab Manual R22 III B.Tech I Sem - AI & ML

8. Perform Object Detection using CNN

Introduction
Object Detection is the process of finding real-world object instances like car, bike, TV,
flowers, and humans in still images or Videos. It allows for the recognition, localization, and
detection of multiple objects within an image which provides us with a much better
understanding of an image as a whole. It is commonly used in applications such as image
retrieval, security, surveillance, and advanced driver assistance systems (ADAS). Image
classification is straight forward, but the differences between object localization and object
detection can be confusing, especially when all three tasks may be just as equally referred to
as object recognition.
Image classification involves assigning a class label to an image, whereas object localization
involves drawing a bounding box around one or more objects in an image. Object detection is
more challenging and combines these two tasks and draws a bounding box around each object
of interest in the image and assigns them a class label. Together, all of these problems are
referred to as object recognition.
As such, we can distinguish between these three computer vision tasks:
• Image Classification: Predict the type or class of an object in an image.
• Input: An image with a single object, such as a photograph.
• Output: A class label (e.g. one or more integers that are mapped to class
labels).
• Object Localization: Locate the presence of objects in an image and indicate
theirlocation with a bounding box.
• Input: An image with one or more objects, such as a photograph.
• Output: One or more bounding boxes (e.g. defined by a point, width, and
height).
• Object Detection: Locate the presence of objects with a bounding box and types
orclasses of the located objects in an image.
• Input: An image with one or more objects, such as a photograph.
• Output: One or more bounding boxes (e.g. defined by a point, width, and
height), and a class label for each bounding box.
Object Detection can be done via multiple ways:
• Feature-Based Object Detection
• Viola Jones Object Detection
• SVM Classifications with HOG Features
• Deep Learning Object Detection

Malla Reddy University 42

Deep Learning Lab Manual R22 III B.Tech I Sem - AI & ML

Example
import cv2
from google.colab.patches import cv2_imshow
import numpy as np

# Load YOLO model and class labels

net = cv2.dnn.readNet("yolov3.weights", "yolov3.cfg")
classes = []
with open("coco.names", "r") as f:
classes = [line.strip() for line in f.readlines()]

# Function to perform object detection

def detect_objects(image_path):
# Read the image
image = cv2.imread(image_path)
height, width, _ = image.shape

# Prepare input blob for the network

blob = cv2.dnn.blobFromImage(image, 1/255.0, (416, 416),
swapRB=True, crop=False)
net.setInput(blob)

# Run forward pass through the network

output_layers_names = net.getUnconnectedOutLayersNames()
outputs = net.forward(output_layers_names)

# Process detections
boxes = []
confidences = []
class_ids = []
for output in outputs:
for detection in output:
scores = detection[5:]
class_id = np.argmax(scores)
confidence = scores[class_id]
if confidence > 0.5:
center_x = int(detection[0] * width)
center_y = int(detection[1] * height)
w = int(detection[2] * width)
h = int(detection[3] * height)
x = int(center_x - w / 2)
y = int(center_y - h / 2)
boxes.append([x, y, w, h])
confidences.append(float(confidence))
class_ids.append(class_id)

# Apply non-max suppression

Malla Reddy University 43

Deep Learning Lab Manual R22 III B.Tech I Sem - AI & ML

indexes = cv2.dnn.NMSBoxes(boxes, confidences,

score_threshold=0.5, nms_threshold=0.4)

# Draw bounding boxes and labels

colors = np.random.uniform(0, 255, size=(len(classes), 3))
if len(indexes) > 0:
for i in indexes.flatten():
x, y, w, h = boxes[i]
label = str(classes[class_ids[i]])
confidence = confidences[i]
color = colors[class_ids[i]]
cv2.rectangle(image, (x, y), (x + w, y + h), color, 2)
cv2.putText(image, f"{label}: {confidence:.2f}", (x, y -
10), cv2.FONT_HERSHEY_SIMPLEX, 0.5, color, 2)

# Display the image with detected objects

cv2_imshow(image)

# Path to the image file

image_path = '/content/2.jpeg' # Replace with your image path

# Perform object detection on the image

detect_objects(image_path)

Output:

Malla Reddy University 44

Deep Learning Lab Manual R22 III B.Tech I Sem - AI & ML

Exercise:
1. Write a CNN program using Pascal VOC or COCO dataset and train the network to detect
objects of interest. Evaluate the model's performance using metrics such as precision,
recall, and mean Average Precision (mAP).
Solution:

Malla Reddy University 45

Deep Learning Lab Manual R22 III B.Tech I Sem - AI & ML

Malla Reddy University 46

Deep Learning Lab Manual R22 III B.Tech I Sem - AI & ML

9. Study and Implementation of LSTM

Introduction
Long short-term memory (LSTM) units (or blocks) are a building unit for layers of a recurrent
neural network (RNN). A RNN composed of LSTM unitsis often called an LSTM network.
A common LSTM unit is composed of a cell, an input gate, an output gate and a forget gate.
The cell is responsible for "remembering" values over arbitrary time intervals; hence the word
"memory" in LSTM. Each of the three gates can be thought of asa "conventional" artificial
neuron, as in a multi-layer (or feed forward) neural network: that is, they compute an activation
(using an activation function) of a weighted sum. Intuitively, they canbe thought as regulators
of the flow of values that goes through the connections of the LSTM; hence the denotation
"gate". There are connections between these gates and the cell.

The expression long short-term refers to the fact that LSTM is a model for the short-term
memory which can last for a long period of time. An LSTM is well-suited to classify, process
and predict time series given time lags of unknown size and duration between important events.
LSTMs were developed to deal with the exploding and vanishing gradient problem when
training traditional RNNs.

Recurrent neural networks have a wide array of applications. These include time series

analysis, document classification, and speech and voice recognition. In contrast to feed
forward artificial neural networks, the predictions made by recurrent neural networks are

dependent on previous predictions.

Draw a straight line. Let us see, if LSTM can learn the relationship of a straight line and

predictit.

Malla Reddy University 47

Deep Learning Lab Manual R22 III B.Tech I Sem - AI & ML

Example
import numpy as np

# Parameters for the straight-line equation y = mx + c

m = 2 # Slope
c = 3 # Intercept

# Generate x values
x_values = np.linspace(0, 10, 100)

# Generate corresponding y values for the straight line

y_values = m * x_values + c

# Display the first few points

print("x values:", x_values[:5])
print("y values:", y_values[:5])

import matplotlib.pyplot as plt

plt.figure(figsize=(8, 6))
plt.plot(x_values, y_values, label="Straight Line", color='blue')
plt.xlabel('x')
plt.ylabel('y')
plt.title('Straight Line Dataset')
plt.legend()
plt.grid(True)
plt.show()

Output:

Malla Reddy University 48

Deep Learning Lab Manual R22 III B.Tech I Sem - AI & ML

from keras.models import Sequential

from keras.layers import LSTM, Dense

# Reshape x_values for LSTM input (samples, timesteps, features)

X = x_values.reshape(len(x_values), 1, 1)

# Define LSTM model

model = Sequential()
model.add(LSTM(50, activation='relu', input_shape=(1, 1)))
model.add(Dense(1))
model.compile(optimizer='adam', loss='mse')

# Train the model

model.fit(X, y_values, epochs=1000, verbose=1)

# Predict the y values

predicted_y_values = model.predict(X, verbose=0)

# Plot the original and predicted straight line

plt.figure(figsize=(8, 6))
plt.plot(x_values, y_values, label="Original Straight Line",
color='blue')
plt.plot(x_values, predicted_y_values, label="Predicted Straight Line",
color='red', linestyle='dashed')
plt.xlabel('x')
plt.ylabel('y')
plt.title('Original and Predicted Straight Lines')
plt.legend()
plt.grid(True)
plt.show()

Output:
Epoch 1/1000
4/4 [==============================] - 6s 28ms/step - loss: 199.4389
Epoch 2/1000
4/4 [==============================] - 0s 6ms/step - loss: 197.8245
Epoch 3/1000
4/4 [==============================] - 0s 6ms/step - loss: 196.1669
.
.
.
Epoch 995/1000
4/4 [==============================] - 0s 9ms/step - loss: 0.0033
Epoch 996/1000
4/4 [==============================] - 0s 11ms/step - loss: 0.0033
Epoch 997/1000
4/4 [==============================] - 0s 10ms/step - loss: 0.0032
Epoch 998/1000
4/4 [==============================] - 0s 11ms/step - loss: 0.0033
Epoch 999/1000

Malla Reddy University 49

Deep Learning Lab Manual R22 III B.Tech I Sem - AI & ML

4/4 [==============================] - 0s 10ms/step - loss: 0.0033

Epoch 1000/1000
4/4 [==============================] - 0s 7ms/step - loss: 0.0033

Exercise:
1. Write a python code using time series analysis to predict the number of international
airline passengers from January 1949 to December 1960?

Solution:

Malla Reddy University 50

Deep Learning Lab Manual R22 III B.Tech I Sem - AI & ML

Malla Reddy University 51

Deep Learning Lab Manual R22 III B.Tech I Sem - AI & ML

Malla Reddy University 52

Deep Learning Lab Manual R22 III B.Tech I Sem - AI & ML

10. Implementation of GAN for Generating Handwritten

Digits Images

Introduction
GANs consist of two neural networks, one trained to generate data and the other trained to
distinguish fake data from real data (hence the “adversarial” nature of the model).
Discriminative vs Generative Models
If you’ve studied neural networks, then most of the applications you’ve come across were likely
implemented using discriminative models. Generative adversarial networks, on the other hand,
are part of a different class of models known as generative models.
Discriminative models are those used for most supervised classification or regression
problems. As an example of a classification problem, suppose you’d like to train a model to
classify images of handwritten digits from 0 to 9. For that, you could use a labeled dataset
containing images of handwritten digits and their associated labels indicating which digit each
image represents.
During the training process, you’d use an algorithm to adjust the model’s parameters. The goal
would be to minimize a loss function so that the model learns the probability.
The Architecture of Generative Adversarial Networks
Generative adversarial networks consist of an overall structure composed of two neural
networks, one called the generator and the other called the discriminator.
The role of the generator is to estimate the probability distribution of the real samples in order
to provide generated samples resembling real data. The discriminator, in turn, is trained to
estimate the probability that a given sample came from the real data rather than being provided
by the generator.
These structures are called generative adversarial networks because the generator and
discriminator are trained to compete with each other: the generator tries to get better at fooling
the discriminator, while the discriminator tries to get better at identifying generated samples.
Example: Students are asked to perform Handwritten Digits Generator with a GAN
import torch
from torch import nn
import matplotlib.pyplot as plt
import torchvision
import torchvision.transforms as transforms

# Set manual seed for reproducibility

torch.manual_seed(111)

# Check for GPU availability and set device

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

Malla Reddy University 53

Deep Learning Lab Manual R22 III B.Tech I Sem - AI & ML

print(f"Using device: {device}")

# Preparing the Training Data

transform = transforms.Compose([
transforms.ToTensor(),
transforms.Normalize((0.5,), (0.5,))
])

train_set = torchvision.datasets.MNIST(
root=".", train=True, download=True,
transform=transform
)

batch_size = 32
train_loader = torch.utils.data.DataLoader(
train_set, batch_size=batch_size, shuffle=True
)

# Plot some samples of the training data

real_samples, mnist_labels = next(iter(train_loader))
plt.figure(figsize=(10, 10))
for i in range(16):
ax = plt.subplot(4, 4, i + 1)
plt.imshow(real_samples[i].reshape(28, 28), cmap="gray_r")
plt.xticks([])
plt.yticks([])
plt.show()

import torch
from torch import nn
import matplotlib.pyplot as plt
import torchvision
import torchvision.transforms as transforms

# Set manual seed for reproducibility

torch.manual_seed(111)

# Check for GPU availability and set device

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
print(f"Using device: {device}")

# Preparing the Training Data

transform = transforms.Compose([
transforms.ToTensor(),
transforms.Normalize((0.5,), (0.5,))
])

train_set = torchvision.datasets.MNIST(

Malla Reddy University 54

Deep Learning Lab Manual R22 III B.Tech I Sem - AI & ML

root=".", train=True, download=True,

transform=transform
)

batch_size = 32
train_loader = torch.utils.data.DataLoader(
train_set, batch_size=batch_size, shuffle=True
)

# Plot some samples of the training data

real_samples, mnist_labels = next(iter(train_loader))
plt.figure(figsize=(10, 10))
for i in range(16):
ax = plt.subplot(4, 4, i + 1)
plt.imshow(real_samples[i].reshape(28, 28), cmap="gray_r")
plt.xticks([])
plt.yticks([])
plt.show()

# Discriminator Model
class Discriminator(nn.Module):
def __init__(self):
super().__init__()
self.model = nn.Sequential(
nn.Linear(784, 1024),
nn.ReLU(),
nn.Dropout(0.3),
nn.Linear(1024, 512),
nn.ReLU(),
nn.Dropout(0.3),
nn.Linear(512, 256),
nn.ReLU(),
nn.Dropout(0.3),
nn.Linear(256, 1),
nn.Sigmoid(),
)

def forward(self, x):

x = x.view(x.size(0), 784)
output = self.model(x)
return output

discriminator = Discriminator().to(device=device)

# Generator Model
class Generator(nn.Module):
def __init__(self):
super().__init__()
self.model = nn.Sequential(

Malla Reddy University 55

Deep Learning Lab Manual R22 III B.Tech I Sem - AI & ML

nn.Linear(100, 256),
nn.ReLU(),
nn.Linear(256, 512),
nn.ReLU(),
nn.Linear(512, 1024),
nn.ReLU(),
nn.Linear(1024, 784),
nn.Tanh(),
)

def forward(self, x):

output = self.model(x)
output = output.view(x.size(0), 1, 28, 28)
return output

generator = Generator().to(device=device)

# Training settings
lr = 0.0001
num_epochs = 6
loss_function = nn.BCELoss()

optimizer_discriminator = torch.optim.Adam(discriminator.parameters(),
lr=lr)
optimizer_generator = torch.optim.Adam(generator.parameters(), lr=lr)

# Training loop
for epoch in range(num_epochs):
for n, (real_samples, mnist_labels) in enumerate(train_loader):
# Data for training the discriminator
real_samples = real_samples.to(device=device)
real_samples_labels = torch.ones((real_samples.size(0),
1)).to(device=device)
latent_space_samples = torch.randn((real_samples.size(0),
100)).to(device=device)
generated_samples = generator(latent_space_samples)
generated_samples_labels = torch.zeros((real_samples.size(0),
1)).to(device=device)
all_samples =
torch.cat((real_samples.view(real_samples.size(0), -1),
generated_samples.view(generated_samples.size(0), -1)))
all_samples_labels = torch.cat((real_samples_labels,
generated_samples_labels))

# Training the discriminator

discriminator.zero_grad()
output_discriminator = discriminator(all_samples)
loss_discriminator = loss_function(output_discriminator,
all_samples_labels)

Malla Reddy University 56

Deep Learning Lab Manual R22 III B.Tech I Sem - AI & ML

loss_discriminator.backward()
optimizer_discriminator.step()

# Data for training the generator

latent_space_samples = torch.randn((real_samples.size(0),
100)).to(device=device)

# Training the generator

generator.zero_grad()
generated_samples = generator(latent_space_samples)
output_discriminator_generated =
discriminator(generated_samples.view(generated_samples.size(0), -1))
loss_generator = loss_function(output_discriminator_generated,
real_samples_labels)
loss_generator.backward()
optimizer_generator.step()

# Show loss
if n == len(train_loader) - 1:
print(f"Epoch: {epoch} Loss D.: {loss_discriminator:.4f}")
print(f"Epoch: {epoch} Loss G.: {loss_generator:.4f}")

# Save models
torch.save(generator.state_dict(), "generator.pth")
torch.save(discriminator.state_dict(), "discriminator.pth")

Output:

Exercise:

Malla Reddy University 57

Deep Learning Lab Manual R22 III B.Tech I Sem - AI & ML

1. Implement a Conditional GAN (cGAN) where the generator and discriminator are
conditioned on class labels (e.g., digits 0-9 from the MNIST dataset). Train the cGAN
and generate images for each digit class.
Solution:

Malla Reddy University 58

Deep Learning Lab Manual R22 III B.Tech I Sem - AI & ML

Malla Reddy University 59

Deep Learning Lab Manual R22 III B.Tech I Sem - AI & ML

Malla Reddy University 60