www.anaconda.

co
m Step 2: Click
Download
Step 3: Click 64-Bit Command Line
installer(542 MB) Step 4: Click Next.
Step 5: Read the licensing terms and click “I Agree”.
Step 6: Select an install for “Just Me” unless you’re installing
for all users (which requires Windows Administrator
privileges) and click Next.

Step 7: Choose whether to add Anaconda to your PATH environment variable. We

recommend not adding Anaconda to the PATH environment variable, since this
can interfere with other software. Instead, use Anaconda software by opening
Anaconda Navigator or the Anaconda Prompt from the Start Menu.

Step 8: Choose whether to register Anaconda as your default Python. Unless you
plan on installing and running multiple versions of Anaconda, or multiple versions
of Python, accept the default and leave this box checked.

Step 9: Click the Install button. If you want to watch the packages Anaconda is
installing, click Show Details.

Step 10: Click the Next button.

Step 11: Optional: To install VS Code, click the Install Microsoft VS Code
button. After the install completes click the Next button.

Step 12: After a successful installation you will see the “Thanks for installing
Anaconda” dialog box:

Introduction

Machine learning

Machine learning is a subset of artificial intelligence in the field of computer

science that often uses statistical techniques to give computers the ability to
"learn" (i.e., progressively improve performance on a specific task) with data,
without being explicitly programmed. In the past decade, machine learning has
given us self-driving cars, practical speech recognition, effective web search, and
a vastly improved understanding of the human genome.
Machine learning tasks

Machine learning tasks are typically classified into two broad categories,
depending on whether there is a learning "signal" or "feedback" available to a
learning system:

· Supervised learning: The computer is presented with example inputs and their
desired outputs, given by a "teacher", and the goal is to learn a general rule that
maps inputs to outputs. As special cases, the input signal can be only partially
available, or restricted to special feedback:

· Semi-supervised learning: the computer is given only an incomplete training

signal: a training set with some (often many) of the target outputs missing.

· Active learning: the computer can only obtain training labels for a limited set
of instances (based on a budget), and also has to optimize its choice of objects to
acquire labels for. When used interactively, these can be presented to the user for
labeling.

· Reinforcement learning: training data (in form of rewards and punishments) is

given only as feedback to the program's actions in a dynamic environment, such
as driving a vehicle or playing a game against an opponent.

· Unsupervised learning: No labels are given to the learning algorithm, leaving

it on its own to find structure in its input. Unsupervised learning can be a goal in
itself (discovering hidden patterns in data) or a means towards an end (feature
learning).

Machine Learning Applications

In classification, inputs are divided into two or more classes, and the learner must
produce a model that assigns unseen inputs to one or more (multi-label
classification) of these classes. This is typically tackled in a supervised manner.
Spam filtering is an example of classification, where the inputs are email (or
other) messages and the classes are "spam" and "not spam".

In regression, also a supervised problem, the outputs are continuous rather than discrete.
In clustering, a set of inputs is to be divided into groups. Unlike in classification,
the groups are not known beforehand, making this typically an unsupervised task.

Density estimation finds the distribution of inputs in some space.

Dimensionality reduction simplifies inputs by mapping them into a lower

dimensional space. Topic modeling is a related problem, where a program is given
a list of human language documents and is tasked with finding out which
documents cover similar topics.

Machine learning Approaches

· Decision tree learning

Decision tree learning uses a decision tree as a predictive model, which maps
observations about an item to conclusions about the item's target value.

· Association rule learning

Association rule learning is a method for discovering interesting relations between
variables in large databases.

· Artificial neural networks

An artificial neural network (ANN) learning algorithm, usually called "neural
network" (NN), is a learning algorithm that is vaguely inspired by biological
neural networks. Computations are structured in terms of an interconnected group
of artificial neurons, processing information using a connectionist approach to
computation. Modern neural networks are non-linear statistical data modeling
tools. They are usually used to model complex relationships between inputs and
outputs, to find patterns in data, or to capture the statistical structure in an
unknown joint probability distribution between observed variables.

· Deep learning
Falling hardware prices and the development of GPUs for personal use in the last
few years have contributed to the development of the concept of deep learning
which consists of multiple hidden layers in an artificial neural network. This
approach tries to model the way the human brain processes light and sound into
vision and hearing. Some successful applications of deep learning are computer
vision and speech Recognition.

· Inductive logic programming

Inductive logic programming (ILP) is an approach to rule learning using logic
Programming as a uniform representation for input examples, background
knowledge, and hypotheses. Given an encoding of the known background
knowledge and a set of examples represented as a logical database of facts, an ILP
system will derive a hypothesized logic program that entails all positive and no
negative examples. Inductive programming is a related field that considers any
kind of programming languages for representing hypotheses (and not only logic
programming), such as functional programs.
· Support vector machines
Support vector machines (SVMs) are a set of related supervised learning methods
used for classification and regression. Given a set of training examples, each
marked as belonging to one
of two categories, an SVM training algorithm builds a model that predicts whether
a new example falls into one category or the other.

· Clustering
Cluster analysis is the assignment of a set of observations into subsets (called
clusters) so that observations within the same cluster are similar according to
some pre designated criterion or criteria, while observations drawn from different
clusters are dissimilar. Different clustering techniques make different assumptions
on the structure of the data, often defined by some similarity metric and evaluated
for example by internal compactness (similarity between members of the same
cluster) and separation between different clusters. Other methods are based on
estimated density and graph connectivity. Clustering is a method of unsupervised
learning, and a common technique for statistical data analysis.

· Bayesian networks
A Bayesian network, belief network or directed acyclic graphical model is a
probabilistic graphical model that represents a set of random variables and their
conditional independencies via a directed acyclic graph (DAG). For example, a
Bayesian network could represent the probabilistic relationships between diseases
and symptoms. Given symptoms, the network can be used to compute the
probabilities of the presence of various diseases. Efficient algorithms exist that
perform inference and learning.

· Reinforcement learning
Reinforcement learning is concerned with how an agent ought to take actions in
an environment so as to maximize some notion of long-term reward.
Reinforcement learning algorithms attempt to find a policy that maps states of the
world to the actions the agent ought to take in those states. Reinforcement
learning differs from the supervised learning problem in that correct input/output
pairs are never presented, nor sub-optimal actions explicitly corrected.
· Similarity and metric learning
In this problem, the learning machine is given pairs of examples that are
considered similar and pairs of less similar objects. It then needs to learn a
similarity function (or a distance metric function) that can predict if new objects
are similar. It is sometimes used in Recommendation systems.

· Genetic algorithms
A genetic algorithm (GA) is a search heuristic that mimics the process of natural
selection, and uses methods such as mutation and crossover to generate new
genotype in the hope of finding good solutions to a given problem. In machine
learning, genetic algorithms found some uses in the 1980s and 1990s. Conversely,
machine learning techniques have been used to improve the performance of
genetic and evolutionary algorithms.

· Rule-based machine learning

Rule-based machine learning is a general term for any machine learning method
that identifies, learns, or evolves "rules" to store, manipulate or apply, knowledge.
The defining characteristic of a rule- based machine learner is the identification
and utilization of a set of relational rules that collectively represent the knowledge
captured by the system. This is in contrast to other machine learners that
commonly identify a singular model that can be universally applied to any
instance in order to make a prediction. Rule-based machine learning approaches
include learning classifier systems, association rule learning, and artificial
immune systems.

· Feature selection approach

Feature selection is the process of selecting an optimal subset of relevant features
for use in model construction. It is assumed the data contains some features that
are either redundant or irrelevant, and can thus be removed to reduce calculation
cost without incurring much loss of information. Common optimality criteria
include accuracy, similarity and information measures.
1. Implement and demonstrate the FIND-S algorithm for finding the most
specific hypothesis based on a given set of training data samples. Read the
training data from a
.CSV file.

Find-s Algorithm :
· Load Data set
· Initialize h to the most specific hypothesis in H
· For each positive training instance x
· For each attribute constraint ai in h
If the constraint ai in h is satisfied by x then do nothing
else replace ai in h by the next more general constraint that is satisfied by x
· Output hypothesis h
Input:
finds.csv
Sky Airtemp Humidity Wind Water Forecast Enjoysport
Sunny Warm Normal Strong Warm Same Yes
Sunny Warm High Strong Warm Same Yes
Rain Cold High Strong Warm Change No
Sunny Warm High Strong Cool Change Yes

Step1: Find S read input file

Sky Airtemp Humidity Wind Water Forecast Enjoysport

Sunny Warm Normal Strong Warm Same Yes
Sunny Warm High Strong Warm Same Yes
Rain Cold High Strong Warm Change No
Sunny Warm High Strong Cool Change Yes

Step2: Initialize h to the most specific

hypothesis in H H0=<0,0,0,0,0,0>

Step 3: For each positive training instance x

 4. Output hypothesis h
[Sunny, Warm', '?', 'Strong', '?', '?']

Source Code:

import random

import csv

def read_data(filename):

with open(filename, 'r') as csvfile:

datareader = csv.reader(csvfile, delimiter=',')

traindata = []

for row in datareader:

traindata.append(row)

return (traindata)

h=['phi','phi','phi','phi','phi','phi'

data=read_data('finds.csv')

def isConsistent(h,d):

if len(h)!=len(d)-1:

print('Number of attributes are not same in hypothesis.')

return False

else:

matched=0

for i in range(len(h)):

if ( (h[i]==d[i]) | (h[i]=='any') ):

matched=matched+1

if matched==len(h):

return True

else:
return False

def makeConsistent(h,d):

for i in range(len(h)):

if((h[i] == 'phi')):

h[i]=d[i]

elif(h[i]!=d[i]):

h[i]='any'

return h

print('Begin : Hypothesis :',h)

print('==========================================')

for d in data:

if d[len(d)-1]=='Yes':

if ( isConsistent(h,d)):

pass

else:

h=makeConsistent(h,d)

print ('Training data :',d)

print ('Updated Hypothesis :',h)

print()

print('--------------------------------')

print('==========================================')

print('maximally sepcific data set End: Hypothesis :',h)

Output:

Begin : Hypothesis : ['phi', 'phi', 'phi', 'phi', 'phi', 'phi']

==========================================
Training data : ['Cloudy', 'Cold', 'High', 'Strong', 'Warm', 'Change', 'Yes']

Updated Hypothesis : ['Cloudy', 'Cold', 'High', 'Strong', 'Warm', 'Change']

--------------------------------

Training data : ['Sunny', 'Warm', 'Normal', 'Strong', 'Warm', 'Same', 'Yes']

Updated Hypothesis : ['any', 'any', 'any', 'Strong', 'Warm', 'any']

--------------------------------

Training data : ['Sunny', 'Warm', 'High', 'Strong', 'Warm', 'Same', 'Yes']

Updated Hypothesis : ['any', 'any', 'any', 'Strong', 'Warm', 'any']

--------------------------------

Training data : ['Sunny', 'Warm', 'High', 'Strong', 'Cool', 'Change', 'Yes']

Updated Hypothesis : ['any', 'any', 'any', 'Strong', 'any', 'any']

--------------------------------

Training data : ['Overcast', 'Cool', 'Normal', 'Strong', 'Warm', 'Same', 'Yes']

Updated Hypothesis : ['any', 'any', 'any', 'Strong', 'any', 'any']

--------------------------------

==========================================

maximally sepcific data set End: Hypothesis : ['any', 'any', 'any', 'Strong', 'any', 'any']

OR

import csv

def loadCsv(filename):

lines = csv.reader(open(filename, "r"))

dataset = list(lines)

for i in range(len(dataset)):

dataset[i] = dataset[i]

return dataset
attributes = ['Sky','Temp','Humidity','Wind','Water','Forecast']

print('Attributes =',attributes)

num_attributes = len(attributes)

filename = "finds.csv"

dataset = loadCsv(filename)

print(dataset)

hypothesis=['0'] * num_attributes

print("Intial Hypothesis")

print(hypothesis)

print("The Hypothesis are")

for i in range(len(dataset)):

target = dataset[i][-1]

if(target == 'Yes'):

for j in range(num_attributes):

if(hypothesis[j]=='0'):

hypothesis[j] = dataset[i][j]

if(hypothesis[j]!= dataset[i][j]):

hypothesis[j]='?'

print(i+1,'=',hypothesis)

print("Final Hypothesis")

print(hypothesis)

Output:

Attributes = ['Sky', 'Temp', 'Humidity', 'Wind', 'Water', 'Forecast']

[['sky', 'Airtemp', 'Humidity', 'Wind', 'Water', 'Forecast', 'WaterSport'],

['Cloudy', 'Cold', 'High', 'Strong', 'Warm', 'Change', 'Yes'],

['Sunny', 'Warm', 'Normal', 'Strong', 'Warm', 'Same', 'Yes'],

['Sunny', 'Warm', 'High', 'Strong', 'Warm', 'Same', 'Yes'],

['Cloudy', 'Cold', 'High', 'Strong', 'Warm', 'Change', 'No'],

['Sunny', 'Warm', 'High', 'Strong', 'Cool', 'Change', 'Yes'],

['Rain', 'Mild', 'High', 'Weak', 'Cool', 'Change', 'No'],

['Rain', 'Cool', 'Normal', 'Weak', 'Cool', 'Same', 'No'],

['Overcast', 'Cool', 'Normal', 'Strong', 'Warm', 'Same', 'Yes']]

Intial Hypothesis

['0', '0', '0', '0', '0', '0']

The Hypothesis are

2 = ['Cloudy', 'Cold', 'High', 'Strong', 'Warm', 'Change']

3 = ['?', '?', '?', 'Strong', 'Warm', '?']

4 = ['?', '?', '?', 'Strong', 'Warm', '?']

6 = ['?', '?', '?', 'Strong', '?', '?']

9 = ['?', '?', '?', 'Strong', '?', '?']

Final Hypothesis

['?', '?', '?', 'Strong', '?', '?']

------------------------------------------------------------------------------------------------------------

2. For a given set of training data examples stored in a .CSV file, implement and

demonstrate the Candidate-Elimination algorithm to output a description of the set of all

hypotheses consistent with the training examples.

Candidate-Elimination Algorithm:

1. Load data set

2. G <-maximally general hypotheses in H

3. S <- maximally specific hypotheses in H

4. For each training example d=<x,c(x)>

Case 1 : If d is a positive example

Remove from G any hypothesis that is inconsistent with d

For each hypothesis s in S that is not consistent with d

• Remove s from S.

• Add to S all minimal generalizations h of s such that

• h consistent with d

• Some member of G is more general than h

• Remove from S any hypothesis that is more general than another hypothesis in S

Case 2: If d is a negative example

Remove from S any hypothesis that is inconsistent with d

For each hypothesis g in G that is not consistent with d

*Remove g from G.

*Add to G all minimal specializations h of g such that

o h consistent with d

o Some member of S is more specific than h

• Remove from G any hypothesis that is less general than another hypothesis in G

Input:

finds.csv

Sky Airtemp Humidity Wind Water Forecast Enjoysport

Sunny Warm Normal Strong Warm Same Yes

Sunny Warm High Strong Warm Same Yes

Rain Cold High Strong Warm Change No

Sunny Warm High Strong Cool Change Yes

Step1:

S 0 :{∅,∅,∅,∅,∅,∅}

G 0 : {⟨?,?,?,?,?,?⟩}

1. ⟨ Sunny,Warm,Normal,Strong,Warm,Same⟩,EnjoySport=yes

S 0 :{∅,∅,∅,∅,∅,∅}

S 1 :{⟨Sunny,Warm,Normal,Strong,Warm,Same⟩}

G 0 , G 1 : {⟨?,?,?,?,?,?⟩}

2. ⟨ Sunny,Warm,High,Strong,Warm,Same⟩,EnjoySport=yes

S 0 :{∅,∅,∅,∅,∅,∅}

S 1 :{⟨Sunny,Warm,Normal,Strong,Warm,Same⟩}

S 2 : {⟨Sunny,Warm,?,Strong,Warm,Same⟩}

G 0 , G 1 , G 2 : {⟨?,?,?,?,?,?⟩}

3. ⟨ Rainy,Cold,High,Strong,Warm,Change⟩,EnjoySport=no

S 0 :{∅,∅,∅,∅,∅,∅}

S 1 :{⟨Sunny,Warm,Normal,Strong,Warm,Same⟩}

S 2 , S 3 : {⟨Sunny,Warm,?,Strong,Warm,Same⟩}

G 3 :{⟨Sunny,?,?,?,?,?⟩,⟨?,Warm,?,?,?,?⟩,⟨?,?,?,?,?,Same⟩}

G 0 , G 1 , G 2 : {⟨?,?,?,?,?,?⟩}

4. ⟨ Sunny,Warm,High,Strong,Cool,Change⟩,EnjoySport=yes

S 0 :{∅,∅,∅,∅,∅,∅}

S 1 :{⟨Sunny,Warm,Normal,Strong,Warm,Same⟩}

S 2 , S 3 : {⟨Sunny,Warm,?,Strong,Warm,Same⟩}

S 4 :{⟨Sunny,Warm,?,Strong,?,?⟩}

G 4 :{⟨Sunny,?,?,?,?,?⟩,⟨?,Warm,?,?,?,?⟩}
G 3 :{⟨Sunny,?,?,?,?,?⟩,⟨?,Warm,?,?,?,?⟩,⟨?,?,?,?,?,Same⟩}

G 0 , G 1 , G 2 : {⟨?,?,?,?,?,?⟩}

Source Code:

import numpy as np

import pandas as pd

data = pd.DataFrame(data=pd.read_csv('finds1.csv'))

concepts = np.array(data.iloc[:,0:-1])

target = np.array(data.iloc[:,-1])

def learn(concepts, target):

specific_h = concepts[0].copy()

print("initialization of specific_h and general_h")

print(specific_h)

general_h = [["?" for i in range(len(specific_h))] for i in range(len(specific_h))]

print(general_h)

for i, h in enumerate(concepts):

if target[i] == "Yes":

for x in range(len(specific_h)):

if h[x] != specific_h[x]:

specific_h[x] = '?'

general_h[x][x] = '?'

if target[i] == "No":

for x in range(len(specific_h)):

if h[x] != specific_h[x]:

general_h[x][x] = specific_h[x]

else:
general_h[x][x] = '?'

print(" steps of Candidate Elimination Algorithm",i+1)

print("Specific_h ",i+1,"\n ")

print(specific_h)

print("general_h ", i+1, "\n ")

print(general_h)

indices = [i for i, val in enumerate(general_h) if val == ['?', '?', '?', '?', '?', '?']]

for i in indices:

general_h.remove(['?', '?', '?', '?', '?', '?'])

return specific_h, general_h

s_final, g_final = learn(concepts, target)

print("Final Specific_h:", s_final, sep="\n")

print("Final General_h:", g_final, sep="\n")

INPUT:

ky Airtemp Humidity Wind Water Forecast Enjoysport

Sunny Warm Normal Strong Warm Same Yes

Sunny Warm High Strong Warm Same Yes

Rain Cold High Strong Warm Change No

Sunny Warm High Strong Cool Change Yes

OUTPUT

initialization of specific_h and general_h

['Cloudy' 'Cold' 'High' 'Strong' 'Warm' 'Change']

[['?', '?', '?', '?', '?', '?'], ['?', '?', '?', '?', '?', '?'], ['?', '?', '?', '?', '?', '?'], ['?', '?', '?', '?', '?', '?'], ['?', '?', '?',
'?', '?', '?'], ['?', '?', '?', '?', '?', '?']]

steps of Candidate Elimination Algorithm 8

Specific_h 8

['?' '?' '?' 'Strong' '?' '?']

general_h 8

[['?', '?', '?', '?', '?', '?'], ['?', '?', '?', '?', '?', '?'], ['?', '?', '?', '?', '?', '?'], ['?', '?', '?', 'Strong', '?', '?'], ['?',

'?', '?', '?', '?', '?'], ['?', '?', '?', '?', '?', '?']]

Final Specific_h:

['?' '?' '?' 'Strong' '?' '?']

Final General_h:

[['?', '?', '?', 'Strong', '?', '?']]

-------------------------------------------------------------------------------------------------------------------

3. Write a program to demonstrate the working of the decision tree based ID3 algorithm.

Use an appropriate data set for building the decision tree and apply this knowledge to

classify a new sample.

Input:

finds.csv

Sky Airtemp Humidity Wind Water Forecast Enjoysport

Sunny Warm Normal Strong Warm Same Yes

Sunny Warm High Strong Warm Same Yes

Rain Cold High Strong Warm Change No

Sunny Warm High Strong Cool Change Yes

Source Code:

import numpy as np

import math

from data_loader import read_data

class Node:

def __init__(self, attribute):

self.attribute = attribute

self.children = []

self.answer = ""

def __str__(self):

return self.attribute

def subtables(data, col, delete):

dict = {}

items = np.unique(data[:, col])

count = np.zeros((items.shape[0], 1), dtype=np.int32)

for x in range(items.shape[0]):

for y in range(data.shape[0]):

if data[y, col] == items[x]:

count[x] += 1

for x in range(items.shape[0]):

dict[items[x]] = np.empty((int(count[x]), data.shape[1]), dtype="|S32")

pos = 0

for y in range(data.shape[0]):

if data[y, col] == items[x]:

dict[items[x]][pos] = data[y]

pos += 1

if delete:

dict[items[x]] = np.delete(dict[items[x]], col, 1)

return items, dict

def entropy(S):

items = np.unique(S)

if items.size == 1:

return 0

counts = np.zeros((items.shape[0], 1))

sums = 0

for x in range(items.shape[0]):

counts[x] = sum(S == items[x]) / (S.size * 1.0)

for count in counts:

sums += -1 * count * math.log(count, 2)

return sums

def gain_ratio(data, col):

items, dict = subtables(data, col, delete=False)

total_size = data.shape[0]

entropies = np.zeros((items.shape[0], 1))

intrinsic = np.zeros((items.shape[0], 1))

for x in range(items.shape[0]):

ratio = dict[items[x]].shape[0]/(total_size * 1.0)

entropies[x] = ratio * entropy(dict[items[x]][:, -1])

intrinsic[x] = ratio * math.log(ratio, 2)

total_entropy = entropy(data[:, -1])

iv = -1 * sum(intrinsic)

for x in range(entropies.shape[0]):

total_entropy -= entropies[x]

return total_entropy / iv

def create_node(data, metadata):

#TODO: Co jeśli information gain jest zerowe?

if (np.unique(data[:, -1])).shape[0] == 1:

node = Node("")

node.answer = np.unique(data[:, -1])[0]

return node

gains = np.zeros((data.shape[1] - 1, 1))

for col in range(data.shape[1] - 1):

gains[col] = gain_ratio(data, col)

split = np.argmax(gains)

node = Node(metadata[split])

metadata = np.delete(metadata, split, 0)

items, dict = subtables(data, split, delete=True)

for x in range(items.shape[0]):

child = create_node(dict[items[x]], metadata)

node.children.append((items[x], child))

return node

def empty(size):

s = ""

for x in range(size):

s += " "

return s

def print_tree(node, level):

if node.answer != "":

print(empty(level), node.answer)
return

print(empty(level), node.attribute)

for value, n in node.children:

print(empty(level + 1), value)

print_tree(n, level + 2)

metadata, traindata = read_data("tennis.data")

data = np.array(traindata)

node = create_node(data, metadata)

print_tree(node, 0)

INPUT:

outlook,temperature,humidity,wind,answer

sunny,hot,high,weak,no

sunny,hot,high,strong,no

overcast,hot,high,weak,yes

rain,mild,high,weak,yes

rain,cool,normal,weak,yes

rain,cool,normal,strong,no

overcast,cool,normal,strong,yes

sunny,mild,high,weak,no

sunny,cool,normal,weak,yes

rain,mild,normal,weak,yes
sunny,mild,normal,strong,yes

overcast,mild,high,strong,yes

overcast,hot,normal,weak,yes

rain,mild,high,strong,no

OUTPUT:

outlook

overcast

b'yes'

rain

wind

b'strong'

b'no'

b'weak'

b'yes'

sunny

humidity

b'high'

b'no'

b'normal'

b'yes'

OR

import pandas as pd

import numpy as np

dataset= pd.read_csv('playtennis.csv',names=['outlook','temperature','humidity','wind','class',])

def entropy(target_col):
elements,counts = np.unique(target_col,return_counts = True)

entropy = np.sum([(-counts[i]/np.sum(counts))*np.log2(counts[i]/np.sum(counts)) for i in

range(len(elements))])

return entropy

def InfoGain(data,split_attribute_name,target_name="class"):

total_entropy = entropy(data[target_name])

vals,counts= np.unique(data[split_attribute_name],return_counts=True)

Weighted_Entropy =

np.sum([(counts[i]/np.sum(counts))*entropy(data.where(data[split_attribute_name]==vals[i]).dr

opna()[target_name]) for i in range(len(vals))])

Information_Gain = total_entropy - Weighted_Entropy

return Information_Gain

def ID3(data,originaldata,features,target_attribute_name="class",parent_node_class = None):

if len(np.unique(data[target_attribute_name])) <= 1:

return np.unique(data[target_attribute_name])[0]

elif len(data)==0:

return

np.unique(originaldata[target_attribute_name])[np.argmax(np.unique(originaldata[target_attribut

e_name],return_counts=True)[1])]

elif len(features) ==0:

return parent_node_class

else:

parent_node_class =

np.unique(data[target_attribute_name])[np.argmax(np.unique(data[target_attribute_name],return
_counts=True)[1])]

item_values = [InfoGain(data,feature,target_attribute_name) for feature in features] #Return

the information gain values for the features in the dataset

best_feature_index = np.argmax(item_values)

best_feature = features[best_feature_index]

tree = {best_feature:{}}

features = [i for i in features if i != best_feature]

for value in np.unique(data[best_feature]):

value = value

sub_data = data.where(data[best_feature] == value).dropna()

subtree = ID3(sub_data,dataset,features,target_attribute_name,parent_node_class)

tree[best_feature][value] = subtree

return(tree)

tree = ID3(dataset,dataset,dataset.columns[:-1])

print(' \nDisplay Tree\n',tree)

INPUT:

outlook,temperature,humidity,wind,answer

sunny,hot,high,weak,no

sunny,hot,high,strong,no

overcast,hot,high,weak,yes

rain,mild,high,weak,yes

rain,cool,normal,weak,yes

rain,cool,normal,strong,no

overcast,cool,normal,strong,yes

sunny,mild,high,weak,no
sunny,cool,normal,weak,yes

rain,mild,normal,weak,yes

sunny,mild,normal,strong,yes

overcast,mild,high,strong,yes

overcast,hot,normal,weak,yes

rain,mild,high,strong,no

OUTPUT:

Display Tree

{'outlook': {'Overcast': 'Yes', 'Rain': {'wind': {'Strong': 'No', 'Weak': 'Yes'}}, 'Sunny':

{'humidity': {'High': 'No', 'Normal': 'Yes'}}}}

-------------------------------------------------------------------------------------------------------------------

4. Build an Artificial Neural Network by implementing the Back propagation Algorithm

and test the same using appropriate data sets.

Back propagation Algorithm:

1.Load data set

2. Assign all network inputs and output

3.Initialize all weights with small random numbers, typically between -1 and 1

repeat

for every pattern in the training set

Present the pattern to the network

// Propagated the input forward through the network:

for each layer in the network

for every node in the layer

1. Calculate the weight sum of the inputs to the node

2. Add the threshold to the sum

3. Calculate the activation for the node

end

end

// Propagate the errors backward through the network

for every node in the output layer

calculate the error signal

end

for all hidden layers

for every node in the layer

1. Calculate the node's signal error

15CSL76 ML LAB

Dept. of CSE,MSEC Page 23

2. Update each node's weighin the network

end
end

// Calculate Global Error

Calculate the Error Function

end

while ((maximum number of iterations < than specified) AND

(Error Function is > than specified))

 Input layer with two inputs neurons

 One hidden layer with two neurons

 Output layer with a single neuron

Source Code:

import numpy as np

X = np.array(([2, 9], [1, 5], [3, 6]), dtype=float)

y = np.array(([92], [86], [89]), dtype=float)

X = X/np.amax(X,axis=0) # maximum of X array longitudinally y = y/100

#Sigmoid Function

def sigmoid (x):

return (1/(1 + np.exp(-x)))

#Derivative of Sigmoid Function

def derivatives_sigmoid(x):

return x * (1 - x)

#Variable initialization

epoch=7000 #Setting training iterations

lr=0.1 #Setting learning rate

inputlayer_neurons = 2 #number of features in data set

hiddenlayer_neurons = 3 #number of hidden layers neurons

output_neurons = 1 #number of neurons at output layer

#weight and bias initialization

wh=np.random.uniform(size=(inputlayer_neurons,hiddenlayer_neurons))

bh=np.random.uniform(size=(1,hiddenlayer_neurons))

wout=np.random.uniform(size=(hiddenlayer_neurons,output_neurons))

bout=np.random.uniform(size=(1,output_neurons))

# draws a random range of numbers uniformly of dim x*y

#Forward Propagation

for i in range(epoch):

hinp1=np.dot(X,wh)

hinp=hinp1 + bh

hlayer_act = sigmoid(hinp)

outinp1=np.dot(hlayer_act,wout)

outinp= outinp1+ bout

output = sigmoid(outinp)

#Backpropagation

EO = y-output

outgrad = derivatives_sigmoid(output)
d_output = EO* outgrad

EH = d_output.dot(wout.T)

hiddengrad = derivatives_sigmoid(hlayer_act)

#how much hidden layer wts contributed to error

d_hiddenlayer = EH * hiddengrad

wout += hlayer_act.T.dot(d_output) *lr

# dotproduct of nextlayererror and currentlayerop

bout += np.sum(d_output, axis=0,keepdims=True) *lr

wh += X.T.dot(d_hiddenlayer) *lr

#bh += np.sum(d_hiddenlayer, axis=0,keepdims=True) *lr

print("Input: \n" + str(X))

print("Actual Output: \n" + str(y))

print("Predicted Output: \n" ,output)

INPUT:

No. of hour Slept

(X)

No. of hour Reading

(X)

Percentage of Marks

( Y)

2 9 92

1 5 86

3 6 89

X = (([2, 9], [1, 5], [3, 6])

y = (([92], [86], [89])

Output:

Input:

[[ 0.66666667 1. ]

[ 0.33333333 0.55555556]

[ 1. 0.66666667]]

Actual Output:

[[ 0.92]

[ 0.86]

[ 0.89]]

Predicted Output:

[[ 0.89559591]

[ 0.88142069]

[ 0.8928407 ]]