Types of Deep Learning Framework

Brief history of TensorFlow

• TensorFlow began its life in 2011 as DisBelief, an internal machine

learning neural network system developed at Google

• Due to its rising popularity Google released the project as TensorFlow

under Apache 2.0 open-source license, on November 9, 2015

• The inputs, outputs, and transformations within neural networks are

all represented using tensors, and as a result, neural network
programming utilizes tensors heavily.
 What is TensorFlow?
• TensorFlow is a numerical library written in C++, with Python as one
of its primary APIs

• TensorFlow contains a variety of machine learning and deep

learning algorithms

• TensorFlow can train and run deep neural networks for

handwritten digit classification, image recognition, word
embedding and creation of various sequence models

• In neural networks transformations, input, output etc. are performed

via tensors.
 …
• Tensors represent the connecting edges in any flow diagram called
the Data Flow Graph

• Tensors are defined as multidimensional array or list

• Every tensor is an instance of the Tensor class

• Tensors are identified by Rank, Shape & Type

• Tensors can be created, transformed, and operated upon using functions

of the tf package
TensorFlow 2.0 Features
• Easy model building with Keras and eager execution.

• Robust model deployment in production on any platform.

• Powerful experimentation for research.

• Simplifying the API by cleaning up deprecated APIs and

reducing duplication.
TensorFlow Training & Deployment
Keras
• TensorFlow 2.0 has embraced Keras as the API of choice for the
majority of deep learning development work.

• It is possible to import Keras as a standalone module, but in this

tutorial, we will concentrate on using Keras from within TensorFlow
2.

• Tensorflow is backed by Google, while Pytorch is backed by

Facebook.
Google Colab
• Colab allows anybody to write and execute arbitrary python code
through the browser, and is especially well suited to machine
learning, data analysis and education.

• https://colab.research.google.com/
 Import TensorFlow package
import tensorflow as tf
print("TensorFlow version: {}".format(tf.__version__))
print("Eager execution is: {}".format(tf.executing_eagerly()))
print("Keras version: {}".format(tf.keras.__version__))

For Python applications, Google adheres to the PEP8 standard conventions. In particular,
they use CamelCase for classes (for example, hub.LatestModuleExporter) and
snake_case for functions, methods, and properties (for example,
tf.math.squared_difference).

if tf.config.list_physical_devices('GPU'):
print('Running in GPU')
else:
print('Running in CPU')
First “Hello World” program
Import tensorflow as tf

h = tf.constant("Hello")
w = tf.constant(" World!")
hw = h + w
print(hw)
Basic Math Operations
…
Math Operation Examples
a = tf.constant([3., 3., 3.])

b = tf.constant([2., 2., 2.])

sum = tf.add(a, b) # [ 5. 5. 5. ]

diff = tf.subtract(a, b) # [ 1. 1. 1. ]

prod = tf.multiply(a, b) # [ 6. 6. 6. ]

quot = tf.divide(a, b) # [ 1.5 1.5 1.5 ]

a = tf.constant([3, 3, 3])
…
b = tf.constant([2, 2, 2])

div1 = tf.divide(a, b) # [ 1.5 1.5 1.5 ]

div2 = a / b # [ 1.5 1.5 1.5 ]

div3 = tf.div(a, b) #[111]

div4 = a // b #[111]
Rounding & Comparison
…
Rounding & Comparison Examples
t = tf.constant([-6.5, -3.5, 3.5, 6.5])
r1 = tf.round(t) # [-6. -4. 4. 6.]
r2 = tf.rint(t) # [-6. -4. 4. 6.]
r3 = tf.ceil(t) # [-6. -3. 4. 7.]
r4 = tf.floor(t) # [-7. -4. 3. 6.]
t1 = tf.constant([0, -2, 4, 6])

t2 = tf.constant([[1, 3], [7, 2]])

r1 = tf.argmin(t1) #1

r2 = tf.argmax(t2) #[10]
TensorFlow Datatypes
• TensorFlow will infer the datatype, defaulting to tf.float32 for floats and tf.int32
for integers

t0 = tf.Variable(42)
t1 = tf.Variable(42.0)
t0, t1
• Output

(<tf.Variable 'Variable:0' shape=() dtype=int32, numpy=42>,

<tf.Variable 'Variable:0' shape=() dtype=float32, numpy=42.0>)
…
• Alternatively, the datatype can be explicitly specified, as here:

f64 = tf.Variable(89, dtype = tf.float64)

f64.dtype
tf.float64

Refer below link for all the data types

https://www.tensorflow.org/api_docs/python/tf/dtypes/DType
TensorFlow constants

• t1= tf.constant(4)

• t1

• # <tf.Tensor: shape=(), dtype=int32, numpy=4>

• t1.numpy()

• #4
Rank
• It identifies the number of dimensions of the tensor. The rank of a
tensor tells us how many indexes are required to access (refer to) a
specific data element contained within the tensor data structure.

Shape
• The number of rows and columns together define the shape of
Tensor.

Type
• Type describes the data type assigned to Tensor’s elements.
 Rank 0 Tensor
- Also known as Scalar
- Contains a single value and no axes
- E.g. 34

Sample code
- rank_0 = tf.constant(34)
- print(rank_0)

Output
- tf.Tensor(34, shape=(), dtype=int32)
 Rank 1 Tensor
- Also known as Vector
- Contains a list of values and 1 axis
- E.g. (30, 31, 32, 33, 34)

Sample code
rank_1 = tf.constant([30, 31, 32, 33, 34])
print(rank_1)

Output
tf.Tensor([30 31 32 33 34], shape=(5,),
dtype=int32)
 Rank 2 Tensor
- Also known as Matrix
- Sequence of arrays with 2 axis
- E.g. (30, 31, 32, 33)

Sample code
Output
rank_1 = tf.constant([[30, 31, 32, 33],
[40, 41, 42, 43],
[50, 51, 52, 53]])
print(rank_1)
Visualizing Rank 0, 1 & 2 Tensors
 Rank 3 Tensor
- Also known as Cube
- Contains more than 2 axis
Output
Sample code
tf.Tensor(
rank_3 = tf.constant([ [[[ 0 1 2 3 4]
[[0, 1, 2, 3, 4], [ 5 6 7 8 9]]

[5, 6, 7, 8, 9]], [[10 11 12 13 14]

[15 16 17 18 19]]
[[10, 11, 12, 13, 14],
[[20 21 22 23 24]
[15, 16, 17, 18, 19]], [25 26 27 28 29]]], shape=(3, 2, 5), dtype=int32)
[[20, 21, 22, 23, 24],
[25, 26, 27, 28, 29]],])
Visualizing Rank 3 Tensors
• There are many ways you might visualize a tensor with more than 2-
axis.
• A 3-axis tensor, shape: [3, 2, 5]
Visualizing Rank 4 Tensors
• A rank-4 tensor, shape: [3, 2, 4, 5]
ML Life Cycle

Courtesy : https://www.javatpoint.com/machine-learning-life-cycle
 Analyze Titanic Dataset
On April 15, 1912, during her maiden voyage, the widely considered “unsinkable” RMS Titanic sank after
colliding with an iceberg. While there was some element of luck involved in surviving, it seems some groups
of people were more likely to survive than others. The challenge is to find the likelihood to survive, finding
relevant patterns in the data.
Courtesy : https://www.kaggle.com/danrusei/exploring-tensorflow-keras-dnn

Variable Definition Key

survival Survival 0 = No, 1 = Yes

pclass Ticket class 1 = 1st, 2 = 2nd, 3 = 3rd

sex Sex
Age Age in years
sibsp # of siblings / spouses aboard the Titanic
parch # of parents / children aboard the Titanic
ticket Ticket number
fare Passenger fare
cabin Cabin number
C = Cherbourg, Q = Queenstown, S =
embarked Port of Embarkation
Southampton
High Level Steps
• Upload the data set
• Pre-processing
- Feature engineering
- Fill in missing data
- Encode Categorical & Scale continuous variables
• Build, Train and Evaluate the Model
- Create the Model
- Compile the Model
- Train the Model
- Evaluate the Model
- Predict the Model
Step 1 – Mount the drive & upload the data
set
from google.colab import drive
drive.mount('/content/Titanic')

The data has been split into two groups:

training set (train.csv)

test set (test.csv)

Step 2 – Import all libraries, set plot
properties
import pandas as pd
import numpy as np
# To plot pretty figures
import matplotlib.pyplot as plt
from sklearn.preprocessing import StandardScaler
import tensorflow as tf
from tensorflow import keras
Step 3 – Load the dataset
# Load dataset
train = pd.read_csv("/content/Titanic/MyDrive/Colab Notebooks/train.csv")
test = pd.read_csv("/content/Titanic/MyDrive/Colab Notebooks/test.csv")
# Add a column in each dataset before merging
train['Type'] = 'train'
test['Type'] = 'test'
# Merge train and test
data = train.append(test, sort=False)
# Explore the initial data, first 5 rows of the dataset
data.head()
Step 4 – Pre-process the data
Data preprocessing is a data mining technique that involves
transforming raw data into an understandable format.
Real-world data is often incomplete, inconsistent, lacking in
certain behaviors or trends, and is likely to contain many errors.

• Feature engineering
• Fill in missing data
• Encode Categorical & Scale continuous variables
Feature Engineering
• Feature engineering is the process of using domain knowledge to
extract features from raw data via data mining techniques. These
features can be used to improve the performance of machine learning
algorithms.
• In this example, Titles from the names (like Miss, Mrs etc) could
influence the performance of the model. Therefore those values are
extracted and populated within a new column.
…
# Extract the Title
for name_string in data['Name']:
data['Title'] = data['Name'].str.extract('([A-Za-z]+)\.', expand=True)
# Replace the rare titles
mapping = {'Mlle': 'Miss', 'Ms': 'Miss', 'Mme': 'Mrs', 'Major': 'Other',
'Col': 'Other', 'Dr' : 'Other', 'Rev' : 'Other', 'Capt': 'Other',
'Jonkheer': 'Royal', 'Sir': 'Royal', 'Lady': 'Royal',
'Don': 'Royal', 'Countess': 'Royal', 'Dona': 'Royal'}
data.replace({'Title': mapping}, inplace=True)
#Consolidate Parch and SibSp within a single column, named Family_Size.
data['Family_Size'] = (data['Parch'] + data['SibSp']).astype(int)
data.head()
Fill in the missing data
Although it is possible to train algorithms with missing values, it’s highly
recommended to clean the data.
Strategies to clean up dataset:
• Drop the missing values: Drop the row if a particular feature is missing
• Replace the missing values with mean/median imputation: Calculate
the mean or median of the feature and replace it with the missing
values.
…
# Age is filled with the mean for the corresponding category
data['Age'].fillna(data.groupby(["Sex", "Pclass"])['Age'].transform("mean"), inplace=True)

#Manually add the two missing values for Embarked

data.loc[61,'Embarked'] = 'S'
data.loc[829,'Embarked'] = 'S'

# Fare is filled with the mean.

data['Fare'].fillna(data['Fare'].mean(), inplace = True)
Encode Categorical and Scale Continuous variables

There are two types of data:

• Categorical : Features whose values are taken from a defined set of
values. For instance, days in a week : {Monday, Tuesday, Wednesday,
Thursday, Friday, Saturday, Sunday} is a category because its value is
always taken from this set. Another example could be the Boolean set
: {True, False}
• Numerical : Features whose values are continuous or integer-valued.
They are represented by numbers and possess most of the properties
of numbers. For instance, number of steps you walk in a day, or the
speed at which you are driving your car at.
…
• Encode Categorical Variables
We need to convert all categorical variables into numeric format. The
categorical variables we will be keeping are Embarked, Sex and Title.
• Scale Continuous variables
Standardization is a useful technique to transform attributes with a
Gaussian distribution and differing means and standard deviations to
a standard Gaussian distribution with a mean of 0 and a standard
deviation of 1.
• We can standardize data using scikit-learn with the StandardScaler
class.
…
# Drop the variables that won't be used in the model
data.drop(['Cabin', 'Name', 'Ticket', 'PassengerId', 'SibSp', 'Parch'], axis=1, inplace=True)

# Convert to cateogry dtype

data['Sex'] = data['Sex'].astype('category')
# Convert to category codes
data['Sex'] = data['Sex'].cat.codes
…
# Encode the categorical values
categorical = ['Embarked', 'Title', 'Pclass']
for cat in categorical:
data = pd.concat([data,
pd.get_dummies(data[cat], prefix=cat)], axis=1)
del data[cat]
…
# Scale the numerical values
continuous = ['Age', 'Fare', 'Family_Size']
scaler = StandardScaler()
for val in continuous:
data[val] = data[val].astype('float64')
data[val] = scaler.fit_transform(data[val].values.reshape(-1, 1))

# Checkout the data after all transformations have been applied

data.head()
Step 5 – Split the train and test data set
# Split the train and test dataset
train = data[data['Type'] == 'train'].drop(columns = ['Type', 'Survived'])
train_ = data[data['Type'] == 'train']['Survived']
X_train = train.values
y_train = train_.values

test = data[data['Type'] == 'test'].drop(columns = ['Type', 'Survived'])

X_test = test.values
X_test = X_test.astype(np.float64, copy=False)

#Checkout the shape of the training dataset

train.shape
Step 6 – Create the Model
• There are three ways to build a Model by using the Keras interface:
using Sequential to build the Model in layer order, using functional
API to build any structural Model, and inheriting the Model base class
to build a custom Model. For this example, use the Sequential, layer
by layer Model.

• Sequential model is the simplest kind of Keras model for neural

networks that are just composed of a single stack of layers connected
sequentially.
 …
• A model is a graph of layers.
• Dense layers are the most basic neural network architecture.
• In this layer, all the inputs and outputs are connected to all the neurons in each layer.
…
# Simple model
model1 = keras.models.Sequential()
# 18 units is dimensionality of the output space.
# input_dim specify the size of the input
model1.add(keras.layers.Dense(18, input_dim = X_train.shape[1], activation =
keras.activations.relu))
model1.add(keras.layers.Dense(8, activation = keras.activations.relu))
model1.add(keras.layers.Dense(1, activation = keras.activations.sigmoid))
# Visualize the model
model1.summary()
Step 7 – Compile the Model
• optimizer: This object specifies the training procedure. Pass it
optimizer instances from the tf.keras.optimizers module, such as
tf.keras.optimizers.Adam or tf.keras.optimizers.SGD.
• loss: The function to minimize during optimization. Common choices
include mean square error (mse), categorical_crossentropy, and
binary_crossentropy. Loss functions are specified by name or by
passing a callable object from the tf.keras.losses module.
• metrics: Used to monitor training. These are string names or callables
from the tf.keras.metrics module.
…
# Compiling the model
model1.compile(optimizer = keras.optimizers.SGD(),
loss = keras.losses.binary_crossentropy,
metrics = [tf.keras.metrics.binary_accuracy])
Step 8 - Train the Model
• epochs: Training is structured into epochs. An epoch is one iteration
over the entire input data (this is done in smaller batches).
• batch_size: When passed NumPy data, the model slices the data into
smaller batches and iterates over these batches during training. This
integer specifies the size of each batch.
• validation_data: When prototyping a model, you want to easily
monitor its performance on some validation data.
• validation_split: Float between 0 and 1. Same as validation_data, but
it use a fraction of the training data to be used as validation data.
…

history = model1.fit(X_train, y_train, epochs=100, validation_split=0.2)

val_acc = np.mean(history.history['val_binary_accuracy'])
print("\n%s: %.2f%%" % ('val_acc', val_acc*100))
Step 9 – Evaluate the Model
# Plot the learning curves
pd.DataFrame(history.history).plot(figsize=(8, 5))
plt.grid(True)
plt.gca().set_ylim(0, 1)
plt.show()
…
• Loss is the result of a bad prediction. A loss is a number indicating
how bad the model's prediction was on a single example.

• If the model's prediction is perfect, the loss is zero; otherwise, the

loss is greater. The goal of training a model is to find a set of weights
and biases that have low loss, on average, across all examples. Higher
loss is the worse(bad prediction) for any model.
Step 10 - Predict on test data
submission = pd.read_csv("/content/Titanic/MyDrive/Colab
Notebooks/gender_submission.csv", index_col='PassengerId')
submission['Survived'] = model1.predict(X_test)
submission['Survived'] = submission['Survived'].apply(lambda x:
round(x,0)).astype('int')
submission.to_csv('Titanic_model1.csv')
Thank You