Classification Algorithm in Machine Learning
As we know, the Supervised Machine Learning algorithm can be broadly
classified into Regression and Classification Algorithms. In Regression
algorithms, we have predicted the output for continuous values, but to
predict the categorical values, we need Classification algorithms.
What is the Classification Algorithm?
The Classification algorithm is a Supervised Learning technique that is used
to identify the category of new observations on the basis of training data. In
Classification, a program learns from the given dataset or observations and
then classifies new observations into a number of classes or groups. Such
as, Yes or No, 0 or 1, Spam or Not Spam, cat or dog, etc. Classes can be
called as targets/labels or categories.
Unlike regression, the output variable of Classification is a category, not a
value, such as "Green or Blue", "fruit or animal", etc. Since the Classification
algorithm is a Supervised learning technique, hence it takes labeled input
data, which means it contains input with the corresponding output.
In the classification algorithm, a discrete output function(y) is mapped to
the input variable(x).
1. y=f(x), where y = categorical output
The best example of an ML classification algorithm is an Email Spam
Detector.
The main goal of the Classification algorithm is to identify the category of a
given dataset, and these algorithms are mainly used to predict the output
for the categorical data.
Classification algorithms can be better understood using the below
diagram. In the below diagram, there are two classes, class A and Class B.
These classes have features that are similar to each other and dissimilar to
other classes.
The algorithm which implements the classification on a dataset is known as
a classifier. There are two types of Classifications:
Binary Classifier: If the classification problem has only two possible
outcomes, then it is called a Binary Classifier.
Examples: YES or NO, MALE or FEMALE, SPAM or NOT SPAM, CAT or
DOG, etc.
Multi-class Classifier: If a classification problem has more than two
outcomes, then it is called as Multi-class Classifier.
Example: Classifications of types of crops, Classification of types of
music.
Naïve Bayes Classifier Algorithm
Naïve Bayes algorithm is a supervised learning algorithm, which is
based on Bayes theorem and used for solving classification
problems.
It is mainly used in text classification that includes a high-dimensional
training dataset.
Naïve Bayes Classifier is one of the simple and most effective
Classification algorithms which helps in building the fast machine
learning models that can make quick predictions.
It is a probabilistic classifier, which means it predicts on the
basis of the probability of an object.
Some popular examples of Naïve Bayes Algorithm are spam
filtration, Sentimental analysis, and classifying articles.
Why is it called Naïve Bayes?
The Naïve Bayes algorithm is comprised of two words Naïve and Bayes,
which can be described as:
Naïve: It is called Naïve because it assumes that the occurrence of a
certain feature is independent of the occurrence of other features.
Such as if the fruit is identified on the bases of colour, shape, and
taste, then red, spherical, and sweet fruit is recognized as an apple.
Hence each feature individually contributes to identify that it is an
apple without depending on each other.
Bayes: It is called Bayes because it depends on the principle of Bayes
theorem.
Bayes' Theorem:
Bayes' theorem is also known as Bayes' Rule or Bayes' law, which is
used to determine the probability of a hypothesis with prior
knowledge. It depends on the conditional probability.
The formula for Bayes' theorem is given as:
where,
P(A|B) is Posterior probability: Probability of hypothesis A on the
observed event B.
P(B|A) is Likelihood probability: Probability of the evidence given that
the probability of a hypothesis is true.
P(A) is Prior Probability: Probability of hypothesis before observing the
evidence.
P(B) is Marginal Probability: Probability of Evidence.
Working of Naïve Bayes' Classifier can be understood with the help of the
below example:
Suppose we have a dataset of weather conditions and corresponding
target variable "Play". So, using this dataset we need to decide that
whether we should play or not on a particular day according to the weather
conditions. So, to solve this problem, we need to follow the below steps:
1. Convert the given dataset into frequency tables.
2. Generate Likelihood table by finding the probabilities of given
features.
3. Now, use Bayes theorem to calculate the posterior probability.
Problem: If the weather is sunny, then the Player should play or not?
Solution: To solve this, first consider the below dataset:
Pl
Outlook
ay
0 Rainy Yes
1 Sunny Yes
2 Overcast Yes
3 Overcast Yes
4 Sunny No
5 Rainy Yes
6 Sunny Yes
7 Overcast Yes
8 Rainy No
9 Sunny No
10 Sunny Yes
11 Rainy No
12 Overcast Yes
13 Overcast Yes
Applying Bayes ‘theorem:
P(Yes|Sunny) = P(Sunny|Yes) *P (Yes)/P(Sunny)
P(Sunny|Yes) = 3/10= 0.3
P(Sunny)= 0.35
P(Yes)=0.71
So, P(Yes|Sunny) = 0.3*0.71/0.35= 0.60
P(No|Sunny) = P(Sunny|No) * P(No)/P(Sunny)
P(Sunny|NO) = 2/4=0.5
P(No)= 0.29
P(Sunny)= 0.35
So, P(No|Sunny) = 0.5*0.29/0.35 = 0.41
So, as we can see from the above calculation that P(Yes|Sunny)>P(No|
Sunny)
Hence on a Sunny day, Player can play the game.
Advantages of Naïve Bayes Classifier:
Naïve Bayes is one of the fast and easy ML algorithms to predict a
class of datasets.
It can be used for Binary as well as Multi-class Classifications.
It performs well in Multi-class predictions as compared to the other
Algorithms.
It is the most popular choice for text classification problems.
Disadvantages of Naïve Bayes Classifier:
Naive Bayes assumes that all features are independent or unrelated,
so it cannot learn the relationship between features.
Applications of Naïve Bayes Classifier:
It is used for Credit Scoring.
It is used in medical data classification.
It can be used in real-time predictions because Naïve Bayes
Classifier is an eager learner.
It is used in Text classification such as Spam filtering and Sentiment
analysis.
