Ai Unit 4
Ai Unit 4
UNIT IV
Learning
1. Forms of Learning:
1.1 Types of Machine Learning
Machine learning is a subset of AI, which enables the machine to automatically learn from data,
improve performance from past experiences, and make predictions. Machine learning contains a
set of algorithms that work on a huge amount of data. Data is fed to these algorithms to train them, and
on the basis of training, they build the model & perform a specific task.
These ML algorithms help to solve different business problems like Regression, Classification,
Forecasting, Clustering, and Associations, etc.
Based on the methods and way of learning, machine learning is divided into mainly four types, which
are:
1. Supervised Machine Learning
2. Unsupervised Machine Learning
3. Semi-Supervised Machine Learning
4. Reinforcement Learning
1. Supervised Machine Learning
As its name suggests, Supervised machine learning is based on supervision. It means in the supervised
learning technique, we train the machines using the "labelled" dataset, and based on the training, the
machine predicts the output. Here, the labelled data specifies that some of the inputs are already mapped
to the output. More preciously, we can say; first, we train the machine with the input and corresponding
output, and then we ask the machine to predict the output using the test dataset.
Let's understand supervised learning with an example. Suppose we have an input dataset of cats and
dog images. So, first, we will provide the training to the machine to understand the images, such as the
shape & size of the tail of cat and dog, Shape of eyes, colour, height (dogs are taller, cats are
smaller), etc. After completion of training, we input the picture of a cat and ask the machine to identify
the object and predict the output. Now, the machine is well trained, so it will check all the features of the
object, such as height, shape, colour, eyes, ears, tail, etc., and find that it's a cat. So, it will put it in the Cat
category. This is the process of how the machine identifies the objects in Supervised Learning.
The main goal of the supervised learning technique is to map the input variable(x) with the
output variable(y). Some real-world applications of supervised learning are Risk Assessment, Fraud
Detection, Spam filtering, etc.
Categories of Supervised Machine Learning
Supervised machine learning can be classified into two types of problems, which are given below:
• Classification
• Regression
a) Classification
Classification algorithms are used to solve the classification problems in which the output variable is
categorical, such as "Yes" or No, Male or Female, Red or Blue, etc. The classification algorithms predict
the categories present in the dataset. Some real-world examples of classification algorithms are Spam
Detection, Email filtering, etc.
Some popular classification algorithms are given below:
• Random Forest Algorithm
• Decision Tree Algorithm
• Logistic Regression Algorithm
• Support Vector Machine Algorithm
b) Regression
Regression algorithms are used to solve regression problems in which there is a linear relationship
between input and output variables. These are used to predict continuous output variables, such as
market trends, weather prediction, etc.
Some popular Regression algorithms are given below:
• Simple Linear Regression Algorithm
• Multivariate Regression Algorithm
• Decision Tree Algorithm
• Lasso Regression
1) Clustering
The clustering technique is used when we want to find the inherent groups from the data. It is a way to
group the objects into a cluster such that the objects with the most similarities remain in one group and
have fewer or no similarities with the objects of other groups. An example of the clustering algorithm is
grouping the customers by their purchasing behaviour.
Some of the popular clustering algorithms are given below:
• K-Means Clustering algorithm
• Mean-shift algorithm
• DBSCAN Algorithm
• Principal Component Analysis
• Independent Component Analysis
2) Association
Association rule learning is an unsupervised learning technique, which finds interesting relations among
variables within a large dataset. The main aim of this learning algorithm is to find the dependency of one
data item on another data item and map those variables accordingly so that it can generate maximum
profit. This algorithm is mainly applied in Market Basket analysis, Web usage mining, continuous
production, etc.
Some popular algorithms of Association rule learning are Apriori Algorithm, Eclat, FP-growth
algorithm.
Advantages and Disadvantages of Unsupervised Learning Algorithm
Advantages:
• These algorithms can be used for complicated tasks compared to the supervised ones because
these algorithms work on the unlabeled dataset.
• Unsupervised algorithms are preferable for various tasks as getting the unlabeled dataset is
easier as compared to the labelled dataset.
Disadvantages:
• The output of an unsupervised algorithm can be less accurate as the dataset is not labelled, and
algorithms are not trained with the exact output in prior.
• Working with Unsupervised learning is more difficult as it works with the unlabelled dataset that
does not map with the output.
3. Semi-Supervised Learning
Semi-Supervised learning is a type of Machine Learning algorithm that lies between Supervised
and Unsupervised machine learning. It represents the intermediate ground between Supervised
(With Labelled training data) and Unsupervised learning (with no labelled training data) algorithms and
uses the combination of labelled and unlabeled datasets during the training period.
Although Semi-supervised learning is the middle ground between supervised and unsupervised
learning and operates on the data that consists of a few labels, it mostly consists of unlabeled data. As
labels are costly, but for corporate purposes, they may have few labels. It is completely different from
supervised and unsupervised learning as they are based on the presence & absence of labels.
To overcome the drawbacks of supervised learning and unsupervised learning algorithms, the
concept of Semi-supervised learning is introduced. The main aim of semi-supervised learning is to
effectively use all the available data, rather than only labelled data like in supervised learning. Initially,
similar data is clustered along with an unsupervised learning algorithm, and further, it helps to label the
unlabeled data into labelled data. It is because labelled data is a comparatively more expensive
acquisition than unlabeled data.
We can imagine these algorithms with an example. Supervised learning is where a student is under the
supervision of an instructor at home and college. Further, if that student is self-analysing the same
concept without any help from the instructor, it comes under unsupervised learning. Under semi-
supervised learning, the student has to revise himself after analyzing the same concept under the
guidance of an instructor at college.
Advantages and disadvantages of Semi-supervised Learning
Advantages:
• It is simple and easy to understand the algorithm.
• It is highly efficient.
• It is used to solve drawbacks of Supervised and Unsupervised Learning algorithms.
Disadvantages:
• Iterations results may not be stable.
• We cannot apply these algorithms to network-level data.
• Accuracy is low.
4. Reinforcement Learning
Disadvantage
• RL algorithms are not preferred for simple problems.
• RL algorithms require huge data and computations.
• Too much reinforcement learning can lead to an overload of states which can weaken the results.
• Decision Tree is a Supervised learning technique that can be used for both classification and
Regression problems, but mostly it is preferred for solving Classification problems. It is a tree-structured
classifier, where internal nodes represent the features of a dataset, branches represent the
decision rules and each leaf node represents the outcome.
• In a Decision tree, there are two nodes, which are the Decision Node and Leaf Node. Decision
nodes are used to make any decision and have multiple branches, whereas Leaf nodes are the output of
those decisions and do not contain any further branches.
• The decisions or the test are performed on the basis of features of the given dataset.
• It is called a decision tree because, similar to a tree, it starts with the root node, which expands
on further branches and constructs a tree-like structure.
• In order to build a tree, we use the CART algorithm, which stands for Classification and
Regression Tree algorithm.
• A decision tree simply asks a question, and based on the answer (Yes/No), it further split the tree
into subtrees.
There are various algorithms in Machine learning, so choosing the best algorithm for the given dataset
and problem is the main point to remember while creating a machine learning model. Below are the two
reasons for using the Decision tree:
• Decision Trees usually mimic human thinking ability while making a decision, so it is easy to
understand.
• The logic behind the decision tree can be easily understood because it shows a tree-like structure.
Root Node: Root node is from where the decision tree starts. It represents the entire dataset, which
further gets divided into two or more homogeneous sets.
Leaf Node: Leaf nodes are the final output node, and the tree cannot be segregated further after getting
a leaf node.
Splitting: Splitting is the process of dividing the decision node/root node into sub-nodes according to
the given conditions.
Branch/Sub Tree: A tree formed by splitting the tree.
Pruning: Pruning is the process of removing the unwanted branches from the tree.
Parent/Child node: The root node of the tree is called the parent node, and other nodes are called the
child nodes.
• Step-1: Begin the tree with the root node, says S, which contains the complete dataset.
• Step-2: Find the best attribute in the dataset using Attribute Selection Measure (ASM).
• Step-3: Divide the S into subsets that contains possible values for the best attributes.
• Step-4: Generate the decision tree node, which contains the best attribute.
• Step-5: Recursively make new decision trees using the subsets of the dataset created in step -3.
Continue this process until a stage is reached where you cannot further classify the nodes and called the
final node as a leaf node.
Example: Suppose there is a candidate who has a job offer and wants to decide whether he should accept
the offer or Not. So, to solve this problem, the decision tree starts with the root node (Salary attribute by
ASM). The root node splits further into the next decision node (distance from the office) and one leaf
node based on the corresponding labels. The next decision node further gets split into one decision node
(Cab facility) and one leaf node. Finally, the decision node splits into two leaf nodes (Accepted offers and
Declined offer). Consider the below diagram:
1. Information Gain:
• Information gain is the measurement of changes in entropy after the segmentation of a dataset
based on an attribute.
• It calculates how much information a feature provides us about a class.
• According to the value of information gain, we split the node and build the decision tree.
• A decision tree algorithm always tries to maximize the value of information gain, and a
node/attribute having the highest information gain is split first. It can be calculated using the below
formula:
• Information Gain= Entropy(S)- [(Weighted Avg) *Entropy(each feature)
Entropy: Entropy is a metric to measure the impurity in a given attribute. It specifies randomness in
data. Entropy can be calculated as:
Entropy(s)= -P(yes)log2 P(yes)- P(no) log2 P(no)
Where,
• S= Total number of samples
• P(yes)= probability of yes
• P(no)= probability of no
2. Gini Index:
• Gini index is a measure of impurity or purity used while creating a decision tree in the
CART(Classification and Regression Tree) algorithm.
• An attribute with the low Gini index should be preferred as compared to the high Gini index.
• It only creates binary splits, and the CART algorithm uses the Gini index to create binary splits.
• Gini index can be calculated using the below formula:
Classification is a process of finding a function which helps in dividing the dataset into classes based on
different parameters. In Classification, a computer program is trained on the training dataset and based
on that training, it categorizes the data into different classes.
The task of the classification algorithm is to find the mapping function to map the input(x) to the discrete
output(y).
Example: The best example to understand the Classification problem is Email Spam Detection. The
model is trained on the basis of millions of emails on different parameters, and whenever it receives a
new email, it identifies whether the email is spam or not. If the email is spam, then it is moved to the
Spam folder.
3.2 Regression:
Regression is a process of finding the correlations between dependent and independent variables. It
helps in predicting the continuous variables such as prediction of Market Trends, prediction of House
prices, etc.
The task of the Regression algorithm is to find the mapping function to map the input variable(x) to the
continuous output variable(y).
Example: Suppose we want to do weather forecasting, so for this, we will use the Regression algorithm.
In weather prediction, the model is trained on the past data, and once the training is completed, it can
easily predict the weather for future days.
3.3 Types of Regression Algorithm:
• Simple Linear Regression
• Multiple Linear Regression
• Polynomial Regression
• Support Vector Regression
• Decision Tree Regression
• Random Forest Regression
The term "Artificial neural network" refers to a biologically inspired sub-field of artificial intelligence
modeled after the brain. An Artificial neural network is usually a computational network based on
biological neural networks that construct the structure of the human brain. Similar to a human brain has
neurons interconnected to each other, artificial neural networks also have neurons that are linked to
each other in various layers of the networks. These neurons are known as nodes.
Artificial neural network tutorial covers all the aspects related to the artificial neural network. In this
tutorial, we will discuss ANNs, Adaptive resonance theory, Kohonen self-organizing map, Building
blocks, unsupervised learning, Genetic algorithm, etc.
What is Artificial Neural Network?
The term "Artificial Neural Network" is derived from Biological neural networks that develop the
structure of a human brain. Similar to the human brain that has neurons interconnected to one another,
artificial neural networks also have neurons that are interconnected to one another in various layers of
the networks. These neurons are known as nodes.
The given figure illustrates the typical diagram of Biological Neural Network.
The typical Artificial Neural Network looks something like the given figure.
Dendrites from Biological Neural Network represent inputs in Artificial Neural Networks, cell nucleus
represents Nodes, synapse represents Weights, and Axon represents Output.
Relationship between Biological neural network and artificial neural network:
BiologicalNeural ArtificialNeural
Network Network
Dendrites Inputs
Cell nucleus Nodes
Synapse Weights
Axon Output
An Artificial Neural Network in the field of Artificial intelligence where it attempts to mimic the
network of neurons makes up a human brain so that computers will have an option to understand things
and make decisions in a human-like manner. The artificial neural network is designed by programming
computers to behave simply like interconnected brain cells.
There are around 1000 billion neurons in the human brain. Each neuron has an association point
somewhere in the range of 1,000 and 100,000. In the human brain, data is stored in such a manner as to
be distributed, and we can extract more than one piece of this data when necessary from our memory
parallelly. We can say that the human brain is made up of incredibly amazing parallel processors.
We can understand the artificial neural network with an example, consider an example of a digital logic
gate that takes an input and gives an output. "OR" gate, which takes two inputs. If one or both the inputs
are "On," then we get "On" in output. If both the inputs are "Off," then we get "Off" in output. Here the
output depends upon input. Our brain does not perform the same task. The outputs to inputs
relationship keep changing because of the neurons in our brain, which are "learning."
The architecture of an artificial neural network:
To understand the concept of the architecture of an artificial neural network, we have to understand
what a neural network consists of. In order to define a neural network that consists of a large number
of artificial neurons, which are termed units arranged in a sequence of layers. Lets us look at various
types of layers available in an artificial neural network.
Artificial Neural Network primarily consists of three layers:
Input Layer:
As the name suggests, it accepts inputs in several different formats provided by the programmer.
Hidden Layer:
The hidden layer presents in-between input and output layers. It performs all the calculations to find
hidden features and patterns.
Output Layer:
The input goes through a series of transformations using the hidden layer, which finally results in output
that is conveyed using this layer.
The artificial neural network takes input and computes the weighted sum of the inputs and includes a
bias. This computation is represented in the form of a transfer function.
It determines weighted total is passed as an input to an activation function to produce the output.
Activation functions choose whether a node should fire or not. Only those who are fired make it to the
output layer. There are distinctive activation functions available that can be applied upon the sort of task
we are performing.
4.1 Advantages of Artificial Neural Network (ANN)
Support Vector Machine or SVM is one of the most popular Supervised Learning algorithms, which is
used for Classification as well as Regression problems. However, primarily, it is used for Classification
problems in Machine Learning.
The goal of the SVM algorithm is to create the best line or decision boundary that can segregate n-
dimensional space into classes so that we can easily put the new data point in the correct category in the
future. This best decision boundary is called a hyperplane.
SVM chooses the extreme points/vectors that help in creating the hyperplane. These extreme cases are
called as support vectors, and hence algorithm is termed as Support Vector Machine. Consider the below
diagram in which there are two different categories that are classified using a decision boundary or
hyperplane:
Example: SVM can be understood with the example that we have used in the KNN classifier. Suppose
we see a strange cat that also has some features of dogs, so if we want a model that can accurately
identify whether it is a cat or dog, so such a model can be created by using the SVM algorithm. We will
first train our model with lots of images of cats and dogs so that it can learn about different features of
cats and dogs, and then we test it with this strange creature. So as support vector creates a decision
boundary between these two data (cat and dog) and choose extreme cases (support vectors), it will see
the extreme case of cat and dog. On the basis of the support vectors, it will classify it as a cat. Consider
the below diagram:
SVM algorithm can be used for Face detection, image classification, text categorization, etc
5.1 Types of SVM
SVM can be of two types:
• Linear SVM: Linear SVM is used for linearly separable data, which means if a dataset can be
classified into two classes by using a single straight line, then such data is termed as linearly separable
data, and classifier is used called as Linear SVM classifier.
• Non-linear SVM: Non-Linear SVM is used for non-linearly separated data, which means if a
dataset cannot be classified by using a straight line, then such data is termed as non-linear data and
classifier used is called as Non-linear SVM classifier.
Linear SVM:
The working of the SVM algorithm can be understood by using an example. Suppose we have a dataset
that has two tags (green and blue), and the dataset has two features x1 and x2. We want a classifier that
can classify the pair(x1, x2) of coordinates in either green or blue. Consider the below image:
So as it is 2-d space so by just using a straight line, we can easily separate these two classes. But there
can be multiple lines that can separate these classes. Consider the below image:
Hence, the SVM algorithm helps to find the best line or decision boundary; this best boundary or region
is called as a hyperplane. SVM algorithm finds the closest point of the lines from both the classes. These
points are called support vectors. The distance between the vectors and the hyperplane is called as
margin. And the goal of SVM is to maximize this margin. The hyperplane with maximum margin is
called the optimal hyperplane.
Non-Linear SVM:
If data is linearly arranged, then we can separate it by using a straight line, but for non-linear data, we
cannot draw a single straight line. Consider the below image:
So to separate these data points, we need to add one more dimension. For linear data, we have used two
dimensions x and y, so for non-linear data, we will add a third dimension z. It can be calculated as:
z=x2 +y2
By adding the third dimension, the sample space will become as below image:
So now, SVM will divide the datasets into classes in the following way. Consider the below image:
Since we are in 3-d Space, hence it is looking like a plane parallel to the x-axis. If we convert it in 2d space
with z=1, then it will become as:
Text classification
SVMs are commonly used in natural language processing (NLP) for tasks such as sentiment analysis,
spam detection, and topic modeling. They lend themselves to these data as they perform well with high-
dimensional data.
Image classification
SVMs are applied in image classification tasks such as object detection and image retrieval. It can also
be useful in security domains, classifying an image as one that has been tampered with.
Bioinformatics
SVMs are also used for protein classification, gene expression analysis, and disease diagnosis. SVMs are
often applied in cancer research (link resides outside ibm.com) because they can detect subtle trends in
complex datasets.
Geographic information system (GIS)
SVMs can analyze layered geophysical structures underground, filtering out the 'noise' from
electromagnetic data. They have also helped to predict the seismic liquefaction potential of soil, which
is relevant to field of civil engineering.
6. Applications of AI:
Natural Language Processing:
large amounts of unstructured text, whether emails, social media conversations, online chats, survey
responses, and many other forms of data.
By bringing NLP into the workplace, companies can analyze data to find what’s relevant amidst the
chaos, and gain valuable insights that help automate tasks and drive business decisions.
So, how can natural language processing make your business smarter? Read on to find out.
Top 11 Natural Language Processing Applications
Natural language processing tools can help businesses analyze data and discover insights, automate
time-consuming processes, and help them gain a competitive advantage.
Let’s take a look at 11 of the most interesting applications of natural language processing in business:
1. Sentiment Analysis
2. Text Classification
3. Chatbots & Virtual Assistants
4. Text Extraction
5. Machine Translation
6. Text Summarization
7. Market Intelligence
8. Auto-Correct
9. Intent Classification
10. Urgency Detection
11. Speech Recognition
1. Sentiment Analysis
Natural language understanding is particularly difficult for machines when it comes to opinions, given
that humans often use sarcasm and irony. Sentiment analysis, however, is able to recognize subtle
nuances in emotions and opinions ‒ and determine how positive or negative they are.
When you analyze sentiment in real-time, you can monitor mentions on social media (and handle
negative comments before they escalate), gauge customer reactions to your latest marketing campaign
or product launch, and get an overall sense of how customers feel about your company.
You can also perform sentiment analysis periodically, and understand what customers like and dislike
about specific aspects of your business ‒ maybe they love your new feature, but are disappointed about
your customer service. Those insights can help you make smarter decisions, as they show you exactly
what things to improve.
2. Text Classification
Text classification, a text analysis task that also includes sentiment analysis, involves automatically
understanding, processing, and categorizing unstructured text.
Let’s say you want to analyze hundreds of open-ended responses to your recent NPS survey. Doing it
manually would take you a lot of time and end up being too expensive. But what if you could train a
natural language processing model to automatically tag your data in just seconds, using predefined
categories and applying your own criteria?
You might use a topic classifier for NPS survey responses, which automatically tags your data by topics
like Customer Support, Features, Ease of Use, and Pricing. Give it a try and see how it performs!
3. Chatbots & Virtual Assistants
Chatbots and virtual assistants are used for automatic question answering, designed to understand
natural language and deliver an appropriate response through natural language generation.
Standard question answering systems follow pre-defined rules, while AI-powered chatbots and virtual
assistants are able to learn from every interaction and understand how they should respond. The best
part: they learn from interactions and improve over time.
These intelligent machines are increasingly present at the frontline of customer support, as they can
help teams solve up to 80% of all routine queries and route more complex issues to human agents.
Available 24/7, chatbots and virtual assistants can speed up response times, and relieve agents from
repetitive and time-consuming queries.
4. Text Extraction
Text extraction, or information extraction, automatically detects specific information in a text, such as
names, companies, places, and more. This is also known as named entity recognition. You can also
extract keywords within a text, as well as pre-defined features such as product serial numbers and
models.
Applications of text extraction include sifting through incoming support tickets and identifying specific
data, like company names, order numbers, and email addresses without needing to open and read every
ticket.
You might also want to use text extraction for data entry. You could pull out the information you need
and set up a trigger to automatically enter this information in your database.
Keyword extraction, on the other hand, gives you an overview of the content of a text, as this free natural
language processing model shows. Combined with sentiment analysis, keyword extraction can add an
extra layer of insight, by telling you which words customers used most often to express negativity
toward your product or service.
5. Machine Translation
Machine translation (MT) is one of the first applications of natural language processing. Even though
Facebooks’s translations have been declared superhuman, machine translation still faces the challenge
of understanding context.
However, if you’ve been an avid user of Google Translate over the years, you’ll know that it has come a
long way since its inception, mainly thanks to huge advances in the field of neural networks and the
increased availability of large amounts of data.
Automated translation is particularly useful in business because it facilitates communication, allows
companies to reach broader audiences, and understand foreign documentation in a fast and cost-
effective way.
6. Text Summarization
Automatic summarization is pretty self-explanatory. It summarizes text, by extracting the most
important information. Its main goal is to simplify the process of going through vast amounts of data,
such as scientific papers, news content, or legal documentation.
There are two ways of using natural language processing to summarize data: extraction-based
summarization ‒ which extracts keyphrases and creates a summary, without adding any extra
information ‒ and abstraction-based summarization, which creates new phrases paraphrasing the
original source. This second approach is more common and performs better.
7. Market Intelligence
Marketers can benefit from natural language processing to learn more about their customers and use
those insights to create more effective strategies.
Analyzing topics, sentiment, keywords, and intent in unstructured data can really boost your market
research, shedding light on trends and business opportunities. You can also analyze data to identify
customer pain points and to keep an eye on your competitors (by seeing what things are working well
for them and which are not).
8. Auto-Correct
Natural Language Processing plays a vital role in grammar checking software and auto-correct functions.
Tools like Grammarly, for example, use NLP to help you improve your writing, by detecting grammar,
spelling, or sentence structure errors.
9. Intent Classification
Intent classification consists of identifying the goal or purpose that underlies a text. Apart from chatbots,
intent detection can drive benefits in sales and customer support areas.
By analyzing customer interactions like emails, chats, or social media posts, you can spot customers that
are ready to purchase. The faster you can detect and classify those leads, the more chances you have of
turning them into customers. Try this email classifier, and sort responses into categories like Interested,
Not Interested, and Unsubscribe.
Finally, looking for customer intent in customer support tickets or social media posts can warn you of
customers at risk of churn, allowing you to take action with a strategy to win them back.
10. Urgency Detection
NLP techniques can also help you detect urgency in text. You can train an urgency detection model using
your own criteria, so it can recognize certain words and expressions that denote gravity or discontent.
This can help you prioritize the most important requests and make sure they don’t get buried under a
pile of unresolved tickets.
Urgency detection helps you improve response times and efficiency, leading to a positive impact on
customer satisfaction.
11. Speech Recognition
Speech recognition technology uses natural language processing to transform spoken language into a
machine-readable format.
Speech recognition systems are an essential part of virtual assistants, like Siri, Alexa, and Google
Assistant, for example. However, there are more and more use cases of speech recognition in business.
For example, adding speech-to-text capabilities to business software, companies are able to
automatically transcribe calls, send emails, and even translate.
Discover Natural Language Processing Tools
Natural language processing has many exciting applications.
Natural language processing tools help businesses process huge amounts of unstructured data, like
customer support tickets, social media posts, survey responses, and more.
Not only are they used to gain insights to support decision-making, but also to automate time-consuming
tasks.
SaaS tools are the most accessible way to get started with natural language processing. With an AI-
platform like MonkeyLearn, you can start using pre-trained models right away, or build a customized
NLP solution in just a few steps (no coding needed).
7.1 Feature Selection techniques selects a subset of the features based on their importance. Document
Frequency is one of the common Feature Selection methods used, where words/features present below
certain frequency in the document are filtered out. Feature Extraction is an optional step in some
business scenarios, where additional features are created from pre-existing features. Clustering
methods is one technique used to add new features.
The final step in this process is the tagging of the data to predefined categories using one of the following
methods:
• Manual tagging
• Rule Based Filtering or string-matching algorithms such as fuzzy matching.
• Learning Algorithms such as Neural Networks that can utilize several hundred features to tag the
Text content. The Learning Algorithms can be classified into two approaches:
• Unsupervised Learning –Applied where there is lack of previously tagged data. Techniques like
clustering and associating rule-based algorithms can be applied to group together similar text. An
example is segmentation of customers into groups based on their details, purchase history and behavior.
These groups can then be further analyzed to identify patterns that allows for customizing customer
approach.
• Supervised Learning –Applied where there is enough volume of accurately categorized data
available. The ML algorithms learns the mapping function between the text and the tags based on already
categorized data. Algorithms such as SVM, Neural Networks, Random Forest are commonly used for text
classification.
Text Classification, both through supervised and unsupervised approach, finds application in various
fields such as social media, marketing, customer experience management, digital media etc. Some of
these will be elucidated below with use cases.
Unsupervised Learning:
The unsupervised approach to text classification looks for similar patterns and structures between text
to group them together. It finds application in real world in scenarios where the data volume is too large
to be completely classified, is in real time or the labels are not predefined.
3.Document Representation: Documents in the collection are typically represented in a way that
makes it easier to compare them to the user's query. Common representations include vector spaces,
term frequency-inverse document frequency (TF-IDF), and word embeddings.
4. Ranking and Scoring: IR systems assign a relevance score to each document in the collection with
respect to a given query. This score reflects the likelihood that the document is relevant to the user's
information needs. Various ranking algorithms, such as BM25, cosine similarity, and learning-to-rank
techniques, are used for this purpose.
5. Retrieval Models: These are mathematical models that formalize how documents are ranked and
scored. Common retrieval models include the Boolean model, vector space model, probabilistic model,
and language modeling.
6. Evaluation Metrics: To assess the performance of an IR system, various metrics like precision, recall,
F1 score, and mean average precision are used. These metrics measure how well the system retrieves
relevant documents and excludes irrelevant ones.
7. Relevance Feedback: In some cases, users can provide feedback on retrieved results, helping to
refine subsequent searches. This feedback can be explicit (e.g., user ratings) or implicit (e.g., click-
through data).
8. Machine Learning Techniques: Machine learning is often used to improve various aspects of
information retrieval. For example, ML algorithms can be used to learn better document
representations, query understanding, and ranking functions. Deep learning models like neural
networks and recurrent neural networks have also been applied to IR tasks.
9. Cross-Modal Retrieval: In addition to text-based retrieval, IR in ML also extends to multimedia
content, where systems aim to retrieve images, videos, or audio based on text queries and vice versa.
Information Retrieval in Machine Learning has practical applications in search engines,
recommendation systems, document classification, content tagging, and more. It plays a crucial role in
making vast amounts of information accessible and useful to users in a variety of domains.
9. Speech recognition
Speech recognition software must adapt to the highly variable and context-specific nature of human
speech. The software algorithms that process and organize audio into text are trained on different
speech patterns, speaking styles, languages, dialects, accents and phrasings. The software also separates
spoken audio from background noise that often accompanies the signal.
To meet these requirements, speech recognition systems use two types of models:
• Acoustic models. These represent the relationship between linguistic units of speech and audio
signals.
• Language models. Here, sounds are matched with word sequences to distinguish between
words that sound similar.
In both cases, the input is an image. For analog image processing, the output is always an image. For
digital image processing, however, the output may be an image or information associated with that
image, such as data on features, characteristics, bounding boxes, or masks.
Today, image processing is widely used in medical visualization, biometrics, self-driving vehicles,
gaming, surveillance, law enforcement, and other spheres.
Here are some of the main purposes of image processing:
• Visualization — Represent processed data in an understandable way. For instance, giving visual
form to objects that aren’t visible
• Image sharpening and restoration — Improve the quality of processed images
• Image retrieval — Help with image search
• Object measurement — Measure objects in an image
• Pattern recognition — Distinguish and classify objects in an image, identify their positions, and
understand the scene
1. Image acquisition — Capture an image with a sensor (such as a camera) and convert it into a
manageable entity (for example, a digital image file). One popular image acquisition method is scraping.
2. Image enhancement — Improve the quality of an image in order to extract hidden information from
it for further processing.
3. Image restoration — Remove possible corruptions from an image in order to get a cleaner version.
This process is mostly based on probabilistic and mathematical models and can be used to get rid of blur
or noise, generate missing pixels, fix camera misfocus, remove watermarks, and eliminate other image
characteristics that may harm the training of a neural network.
4. Color image processing — Improve image quality and analyze image content based on color
information. Depending on the image type, we can talk about pseudocolor processing (when colors are
assigned grayscale values) or RGB processing (for images acquired with a full-color sensor).
5. Image compression and decompression — Reduce or restore the size and resolution of an image.
These techniques are often used for image augmentation when you slightly change an original image to
extend your dataset with quality relevant data. Image augmentation can help improve the way your
neural network model generalizes data and make sure it provides high-quality results.
6. Morphological processing — Describe the shapes and structures of objects in an image to create
datasets for training AI models. In particular, morphological analysis and processing can be applied at
the annotation stage, when you describe what you want your AI model to detect or recognize.
7. Object recognition — Identify specific features of particular objects in an image. AI-based image
recognition often uses such techniques as object detection, object recognition, and segmentation. This
technology is at the core of solutions like driverless automotive systems, medical diagnosis systems,
and AI-powered surveillance.
8. Representation and description — Visualize and describe processed data. Using special
visualization tools, you can turn arrays of numbers and values — the raw output of an AI system — into
readable images suitable for further analysis.
As each of these phases requires processing massive amounts of data, you can’t perform them manually.
The use of AI and machine learning (ML) boosts both the speed of data processing and the quality of
the final result. For instance, with the help of AI platforms, you can successfully accomplish such complex
tasks as object detection, facial recognition, and text recognition. But of course, in order to get high-
quality results, it’s important to pick the right methods and tools for image processing.
Computer vision is a subfield of AI (Artificial Intelligence), which enables machines to derive some
meaningful information from any image, video, or other visual input and perform the required
action on that information. Computer vision is like eyes for an AI system, which means if AI enables
the machine to think, computer vision enables the machines to see and observe the visual inputs.
Computer vision technology is based on the concept of teaching computers to process an image or a
visual input at pixels and derive meaningful information from it. Nowadays, Computer vision is in great
demand and used in different areas, including robotics, manufacturing, healthcare, etc. In this topic, we
will discuss some popular applications of Computer Vision, but before that, let's first understand some
common tasks that are performed by computer vision.
11.1 Below are some common tasks for which computer vision can be used:
• Image Classification: Image classification is a computer vision technique used to classify an
image, such as whether an image contains a dog, a person's face, or a banana. It means that with image
classification, we can accurately predict the class of an object present in an image.
• Object Detection: Object detection uses image classification to identify and locate the objects in
an image or video. With such detection and identification technique, the system can count objects in a
given image or scene and determine their accurate location, along with their labelling. For example, in a
given image, there is one person and one cat, which can be easily detected and classified using the object
detection technique.
• Object Tracking: Object tracking is a computer vision technique used to follow a particular
object or multiple items. Generally, object tracking has applications in videos and real-world
interactions, where objects are firstly detected and then tracked to get observation. Object tracking is
used in applications such as Autonomous vehicles, where apart from object classification and detection
such as pedestrians, other vehicles, etc., tracking of real-time motion is also required to avoid accidents
and follow the traffic rules.
• Semantic Segmentation: Image segmentation is not only about detecting the classes in an image
as image classification. Instead, it classifies each pixel of an image to specify what objects it has. It tries
to determine the role of each pixel in the image.
Although AI and robotics are sometimes used interchangeably, in reality, they are distinct — yet related
— fields. While both AI and robotics can potentially impact various industries and aspects of life in
significant ways, each serves a different purpose and operates in a unique way.
Simply put, AI neural network models are similar to biological neural networks, while robotics is
comparable to the human body. AI refers to the development of systems that can perform tasks that
typically require human intelligence, such as learning, problem-solving and decision-making. These
systems can work autonomously, without the need for constant instructions, since they’re programmed
to learn and adapt on their own.
Robotics, on the other hand, refers to the development of robots that can perform specific physical tasks.
These robots can be programmed to carry out simple, repetitive actions, such as sorting items or
assembling miniscule parts. While AI can be integrated into robotics to enhance the robot’s capabilities
and improve decision-making, it’s not always necessary. Some robotics applications simply require
robots to carry out predictable actions without the need for additional cognitive capabilities. While AI
and robotics are not the same things, they do complement each other and can work together to bring
about a wide range of benefits and advancements in various applications.
AI has made substantial progress in recent years, and its integration with robotics has proven to be a
natural progression. While AI in robotics is not yet widespread, it’s rapidly gaining momentum as AI
systems become more advanced. The combination of AI and robotics holds tremendous potential,
leading to increased productivity and efficiency, improved safety and greater flexibility for workers in a
variety of professions.
One of the key ways in which AI is used in robotics is through machine learning. This technique enables
robots to learn and perform specific tasks through observing and mimicking human actions. AI gives
robots a computer vision that enables them to navigate, detect and determine their reactions
accordingly. This helps them go beyond simply performing repetitive tasks to become true “cognitive
collaborators.”
Another way that AI is used in robotics is through edge computing. AI applications in robotics require
the interpretation of massive amounts of data gathered by robot-based sensors in real time, which is
why this data is analyzed close to the machine, rather than being sent off to the cloud for processing.
This approach provides machines with real-time awareness, enabling robots to act on decisions at a rate
much quicker than human capabilities allow.
AI also helps robots learn to perform specific tasks through the use of various sensors, which may
include:
• Time-of-flight optical sensors
• Temperature and humidity sensors
• Ultrasonic sensors
• Vibration sensors
• Millimeter-wave sensors
These sensors help robots to learn and adapt, making them more intelligent and better equipped to act
and react in different scenarios.
These are just a few of the ways that artificial intelligence is used in conjunction with robotics.
In the world of robotics, AI has proven to be a valuable asset in a variety of applications. From customer
service to manufacturing, AI has made its mark and continues to revolutionize the way we think about
and interact with robots. Let’s take a closer look at some of the key areas where AI is being used
alongside robotics today.
Customer Service: AI-powered chatbots are becoming increasingly common in customer service
applications. These automated service agents can handle simple, repetitive requests without the need
for human involvement. The more these systems interact with humans, the more they learn. And as AI
systems become more sophisticated, we can expect to see more and more robots being used in customer
service in both online and brick-and-mortar environments.
Assembly: AI has proven to be an invaluable tool in robotic assembly applications, especially in complex
manufacturing industries such as aerospace. With the help of advanced vision systems, AI can enable
real-time course correction and can be used to help a robot automatically learn the best paths for certain
processes while in operation.
Packaging: AI is used in the packaging industry to improve efficiency, accuracy and cost-effectiveness.
By continuously refining and saving certain motions made by robotic systems, AI helps make installing
and moving robotic equipment easier for everyone.
Imaging: Across many industries — including assembly and logistics — accurate imaging is crucial.
With the assistance of AI, robots can achieve enhanced visual acuity and image recognition
competencies, enabling greater accuracy in even the smallest of details.
Machine Learning: Machine learning is a powerful tool for robots. By exploring their surroundings,
robots can learn more about their environment, find ways around obstacles and solve problems to
complete tasks more efficiently. From home robots like vacuum cleaners to manufacturing robots in
factories, machine learning is helping robots become more intelligent and adaptable in their work. These
are just a few of the many applications of AI in robotics today. As these technologies continue to expand
and grow in sophistication, it is likely that we will see even more innovative applications in the near
future.
As robotics continue to shape various industries, a robotics engineer plays a critical role in robotic
design, maintenance and functionality. A robotics engineer is a specialist responsible for building,
installing and maintaining the machines that perform tasks in sectors such as manufacturing, security,
aerospace and healthcare.
The day-to-day responsibilities of a robotics engineer include:
• Installing, repairing and testing equipment and components
• Performing predictive maintenance
• Incorporating relevant technical literature into their understanding of system operations
• Identifying new data sources
• Building working relationships
• Ensuring that software solutions meet customer needs
• Developing and deploying AI governance structure to manage ongoing implementation of AI
strategies
• Continuously evaluating and reimagining processes to incorporate conversational AI
• Maintaining knowledge of safety standards and regulations for the safe operation of a system
The future of AI in robotics is vast and exciting. The next stage of AI, known as AGI or Artificial General
Intelligence, holds the potential to reach levels of true human understanding. The key to this is
integrating the computational system of AI with a robot. The robot must possess mobility, senses (such
as touch, vision and hearing) and the ability to interact with physical objects, which will enable the
system to experience immediate sensory feedback from every action it takes. This feedback loop enables
the system to learn and comprehend, bringing it closer to achieving true AGI.
The current focus on AI in robotics is shifting from the question of what tasks robots can perform for
people, to what type of input a robot can provide the AI’s “mind.” By allowing AI to explore and
experiment with real objects, it will be possible for it to approach a deeper understanding, much like a
human child. With this integration of AI and robotics, we can expect to see significant advancements in
a wide range of industries, from manufacturing and healthcare to security and space exploration. The
future of AI in robotics is bright and holds the potential for tremendous progress in how we understand
and interact with the world. By combining the computational power of AI with the physical capabilities
of robots, we are opening up new doors for exploration and innovation, and the potential for true AGI is
within reach.