||Jai Shri Gurudev||

Sri Adichunchanagiri Shikshana Trust

SJB INSTITUTE OF TECHNOLOGY

No.67,BGS Health & Education City, Dr. Vishnuvardhan Road, Kengeri, Bangaluru – 560060

An Autonomous Institute under VTU

Approved by AICTE - New Delhi, Accredited by NAAC A+, Accredited by NBA

Department of
Artificial Intelligence and Machine Learning

ARTIFICIAL INTELLIGENCE
[23AII403]
Prepared by
Mrs. Shashikala AB
Asst. Professor Dept
of AI&ML,SJBIT
WHAT IS INTELLIGENCE?
WHAT IS ARTIFICIAL INTELLIGENCE?
WHAT IS INTELLIGENCE?
Homo Sapiens : The name is Latin for “wise man”
Matter can perceive, understand, predict, and manipulate.

WHAT IS ARTIFICIAL INTELLIGENCE?

Artificial intelligence, or AI: it ARTIFICIAL INTELLIGENCE attempts not just to
understand but also to build intelligent entities.
Introduction to Artificial Intelligence
Homo Sapiens : The name is Latin for “wise man”
Philosophy of AI - Can a machine think and behave like humans do?‖

In Simple Words - Artificial Intelligence is a way of making a computer, a

computer controlled robot, or a software think intelligently, in the similar
manner the intelligent humans think.

Artificial intelligence (AI) is an area of computer science that emphasizes the

creation of intelligent machines that work and react like humans.

AI is accomplished by studying how human brain thinks and how humans learn,
decide, and work while trying to solve a problem, and then using the outcomes of
this study as a basis of developing intelligent software and systems.
 Thinking humanly Thinking rationally
―The exciting new effort to make computers ―The study of mental faculties through
think . .. machines with minds, in the full and the use of computational models.‖
literal sense.‖ (Haugeland, 1985) ― (Charniak and McDermott, 1985)

―[The automation of] activities that we ―The study of the computations that
associate with human thinking, activities such make it possible to perceive, reason, and
as decision-making, problem solving, learning act.‖ (Winston, 1992)
.. .‖ (Bellman, 1978)

Acting humanly Acting rationally

―The art of creating machines that perform ―Computational Intelligence is the
functions that require intelligence when study of the design of intelligent agents.‖
performed by people.‖ (Kurzweil, 1990) (Poole et al., 1998)

―The study of how to make computers do

things at which, at the moment, people are ―AI . . . is concerned with intelligent
better.‖ (Rich and Knight, 1991) be- havior in artifacts.‖ (Nilsson, 1998)
 Acting humanly: The Turing
Test approach
• Alan Turing (1950) developed "Computing machinery and
intelligence":
• "Can machines think?" or "Can machines behave intelligently?"
• Operational test for intelligent behavior: the Imitation Game
• A computer passes the test if a human interrogator, after posing some
written questions, cannot tell whether the written responses come
from a person or from a machine.
• Suggested major components of AI: knowledge, reasoning, language
understanding, learning
The computer would need to posses the following capabilities:
• Natural Language Processing: To enable it to communicate successfully in English.
• Knowledge representation: To store what it knows or hear.
• Automated reasoning: To use the stored information to answer questions and to draw
new conclusions.
• Machine Learning: To adopt to new circumstances and to detect and extrapolate
patterns.

To pass the Total Turing Test

• Computer vision: To perceive objects.
• Robotics: To manipulate objects and move about.
Thinking humanly: The cognitive modeling approach
• If we are going to say that given program thinks like a human, we must have some way
of determining how humans think.
• We need to get inside the actual working of human minds.
• There are 3 ways to do it:

i. Through introspection
Trying to catch our own thoughts as they go

i. Through psychological experiments

Observing a person in action

ii. Through brain imaging

Observing the brain in action
• Comparison of the trace of computer program reasoning steps to traces of human subjects solving
the same problem.

• Cognitive Science brings together computer models from AI and experimental techniques from
psychology to try to construct precise and testable theories of the working of the human mind.

• Now distinct from AI

AI and Cognitive Science fertilize each other in the areas of vision and natural language.

• Once we have a sufficiently precise theory of the mind, it becomes possible to express the theory as
a computer program.

• If the program’s input-output behaviour matches corresponding human behaviour, that is evidence
that the program’s mechanisms could also be working in humans.

• For example, Allen Newell and Herbert Simon, who developed GPS, the “General Problem
Solver”.
Thinking rationally: The “laws of thought” approach

Aristotle was one of the first to attempt to codify ―right thinking,‖ that
is, irrefutable reasoning processes. His syllogisms provided patterns for argument
structures that always yielded correct conclusions when given correct premises.

Eg.
Socrates is a man;
All men are mortal;
Therefore, Socrates is mortal.

These laws of thought were supposed to govern the operation of the mind; their study
initiated the field called logic.
The logicist tradition within artificial intelligence creates intelligent system.

There are two main obstacles to this approach.

• It is not easy to take informal knowledge and state it in the formal terms
required by logical notation, particularly when the knowledge is less than 100%
certain.
• Second, there is a big difference between solving a problem ―in principle and
solving it in practice.
Acting rationally: The rational agent approach
An agent is just something that acts.

All computer programs do something, but computer agents are expected to do more: operate
autonomously, perceive their environment, persist over a prolonged time period, and adapt to
change, and create and pursue goals.

A rational agent is one that acts so as to achieve the best outcome or, when there is uncertainty,
the best expected outcome.

The “thinking rational” approach to AI, the emphasis was on correct inferences. And making
correct inferences is sometimes part of a rational agent.

On the other hand, correct inference is not all of rationality; in some situations, there is no
provably correct thing to do, but something must still be done.

For example, recoiling from a hot stove is a reflex action that is usually more successful than a
slower action taken after careful deliberation.
Foundations of Artificial Intelligence
1. Philosophy

a. Can formal rules be used to draw valid conclusions?

b. How does the mind arise from a physical brain?
c. Where does knowledge come from?
d. How does knowledge lead to action?

Aristotle was the first to formulate a precise set of laws governing the rational part of
the mind. He developed an informal system of syllogisms for proper reasoning, which
in principle allowed one to generate conclusions mechanically, given initial premises.

All dogs are animals;

all animals have four legs;
therefore all dogs have four legs

Ramon Lull proposed reasoning system.

Leonardo da vinci designed 1st Mechanical calculator.
Blaise Pascal built pascaline.
Leibniz built mechanical device that operates on concepts
Dualism: human brain has physical and logical part.

Empiricism : Brain understands through senses.

Induction: Rules are acquired through exposure to its elements.

Logical postivism: All the knowledge can be characterized by the observation of

sentences that corresponds to sensory inputs.

Confirmation theory: Aquisisation of knowledge through the experience by quantifying

the degree of beliefs that should be assigned to the logical sentences based on their
connection to observation that confirm and disconfirm them.

Newell and Simon designed the General problem slover(GPS).

Utilitarianism: Decision makers should consider the interests of the many individuals,
rational decision making based on maximizing utility should apply to all spheres of
human activity, including public decisions made on behalf of many individuals.

Deontological ethics: Doing right things determined by universal laws(don’t lie, don’t
kill)
2. Mathematics
a. What are the formal rules to draw valid conclusions?
b. What can be computed?
c. How do we reason with uncertain information?

Formal representation and proof algorithms: Propositional logic

Computation: Turing tried to characterize exactly which functions are

computable - capable of being computed.

(un)decidability: Incompleteness theory showed that in any formal theory,

there are true statements that are undecidable i.e. they have no proof within
the theory.
― a line can be extended infinitely in both directions‖

(in)tractability: A problem is called intractable if the time required to solve

instances of the problem grows exponentially with the size of the instance.

probability: Predicting the future.

3. Economics

How should we make decisions so as to maximize payoff?

How should we do this when others may not go along?
How should we do this when the payoff may be far in the future?

An Inquiry into the Nature and Causes of the Wealth of Nations

Decision theory, which combines probability theory with utility theory, provides
a formal and complete framework for decisions made under uncertainty.

The most part, economists did not address the third question, how to make
rational decisions when payoffs from actions are not immediate but instead result
from several actions taken in sequence. This topic was pursued in the emerging
field of operations research.

In artificial intelligence, "satisficing" refers to a decision-making strategy where

an AI system chooses a "good enough" solution rather than searching
exhaustively for the absolute best possible outcome
Game Theory, as it applies to Artificial Intelligence (AI), engages in modeling scenarios
among interactive decision-makers. It aims to predict outcomes based on interactions
between AI systems and their environment or between separate AI entities.

The best example of game theory is a classical hypothesis called “Prisoners Dilemma”.
According to this situation, two people are supposed to be arrested for stealing a car. They
have to serve 2-year imprisonment for this. But, the police also suspects that these two
people have also committed a bank robbery. The police placed each prisoner in a separate
cell. Both of them are told that they are suspects of being bank robbers. They are inquired
separately and are not able to communicate with each other.

The prisoners are given two situations:

• If they both confess to being bank robbers, then each will serve 3-year imprisonment
for both car theft and robbery.
• If only one of them confesses to being a bank robber and the other does not, then the
person who confesses will serve 1-year and others will serve 10-year imprisonment.
According to game theory, the prisoners will either confess or deny the bank robbery. So,
there are four possible outcomes :
 4. Neuroscience
Neuroscience is the study of the nervous system, particularly the brain .

Brain consists of nerve cells or neurons. 10^11 neurons.

Neurons are considered as Computational units.
5. Psychology
• How do humans and animals think and act?

Behaviorism movement, led by John Watson(1878-1958).

Behaviorists insisted on studying only objective measures of the percepts(stimulus) given to an

animal and its resulting actions(or response).

Behaviorism discovered a lot about rats and pigeons but had less success at understanding human.

Cognitive psychology, views the brain as an information processing device.

Common view among psychologist that a cognitive theory should be like a computer program.

(Anderson 1980) i.e. It should describe a detailed information processing mechanism whereby some
cognitive function might be implemented.
6. Computer engineering:
How can we build an efficient computer?

For artificial intelligence to succeed, we need two things: intelligence and an artifact. The
computer has been the artifact(object) of choice.

The first operational computer was the electromechanical Heath Robinson, built in 1940
by Alan Turing's team for a single purpose: deciphering German messages.

The first operational programmable computer was the Z-3, the invention of KonradZuse
in Germany in 1941.

The first electronic computer, the ABC, was assembled by John Atanasoff and his student
Clifford Berry between 1940 and 1942 at Iowa State University.

The first programmable machine was a loom, devised in 1805 by Joseph Marie Jacquard
(1752-1834) that used punched cards to store instructions for the pattern to be woven.
7. Control theory and cybernetics
How can artifacts operate under their own control?

Ktesibios of Alexandria (c. 250 B.C.) built the first self-controlling machine: a water
clock with a regulator that maintained a constant flow rate. This invention changed the
definition of what an artifact could do.

Modern control theory, especially the branch known as stochastic optimal control, has as
its goal the design of systems that maximize an objective function over time. This
roughly OBJECTIVE FUNCTION matches our view of Al: designing systems that
behave optimally.

Calculus and matrix algebra- the tools of control theory

The tools of logical inference and computation allowed AI researchers to consider
problems such as language, vision, and planning that fell completely outside the control
theorist’s purview.
8. Linguistics
How does language relate to thought?

In 1957, B. F. Skinner published Verbal Behaviour. This was a comprehensive, detailed

account of the behaviourist approach to language learning, written by the foremost expert
in the field.

Noam Chomsky, who had just published a book on his own theory, Syntactic Structures.
Chomsky pointed out that the behaviourist theory did not address the notion of creativity
in language.

Modern linguistics and AI were ―born at about the same time, and grew up together,
intersecting in a hybrid field called computational linguistics or natural language
processing.

The problem of understanding language soon turned out to be considerably more complex
than it seemed in 1957. Understanding language requires an understanding of the subject
matter and context, not just an understanding of the structure of sentences.
Knowledge representation (the study of how to put knowledge into a form that a computer
can reason with) - tied to language and informed by research in linguistics.
History of Artificial Intelligence
The gestation of artificial intelligence (1943–1955)
The gestation of artificial intelligence (AI) during the period from 1943 to 1955 marked the early
theoretical and conceptual groundwork for the field. This period laid the foundation for the
subsequent development of AI

The birth of artificial intelligence (1956)

The birth of artificial intelligence (AI) in 1956 is commonly associated with the Dartmouth
Conference, a seminal event that took place at Dartmouth College in Hanover, New Hampshire.

Early enthusiasm, great expectations (1952–1969)

The period from 1952 to 1969 in the history of artificial intelligence (AI) was characterized by early
enthusiasm and great expectations. Researchers during this time were optimistic about the potential
of AI and believed that significant progress could be made in creating machines with human-like
intelligence.

A dose of reality (1966–1973)

The period from 1966 to 1973 in the history of artificial intelligence (AI) is often referred to as "A
Dose of Reality." During this time, researchers faced challenges and setbacks that led to a
reevaluation of the initial optimism and expectations surrounding AI.

Knowledge-based systems: The key to power? (1969–1979)

The period from 1969 to 1979 in the history of artificial intelligence (AI) is characterized by a focus
on knowledge-based systems, with researchers exploring the use of symbolic representation of
knowledge to address challenges in AI. This era saw efforts to build expert systems, which were
designed to emulate human expertise in specific domains.
 AI becomes an industry (1980–present)
The period from 1980 to the present marks the evolution of artificial intelligence (AI) into an
industry, witnessing significant advancements, increased commercialization, and widespread
applications across various domains.

The return of neural networks (1986–present)

The period from 1986 to the present is characterized by the resurgence and dominance of neural
networks in the field of artificial intelligence (AI). This era is marked by significant advancements in
the development of neural network architectures, training algorithms, and the widespread adoption of
deep learning techniques.

AI adopts the scientific method (1987–present)

The period from 1987 to the present has seen the adoption of the scientific method in the field of
artificial intelligence (AI), reflecting a more rigorous and empirical approach to research. This shift
has involved the application of experimental methodologies, reproducibility, and a greater emphasis
on evidence-based practices.

The emergence of intelligent agents (1995–present)

The period from 1995 to the present has been marked by the emergence and evolution of intelligent
agents in the field of artificial intelligence (AI). Intelligent agents are autonomous entities that
perceive their environment, make decisions, and take actions to achieve goals.

The availability of very large data sets (2001–present)

The period from 2001 to the present has been characterized by the availability and utilization of very
large datasets in the field of artificial intelligence (AI). This era has witnessed an unprecedented
growth in the volume and diversity of data, providing a foundation for training and enhancing
increasingly sophisticated AI models.
THE STATE OF THE ART

Robotic vehicles: A driverless robotic car named STANLEY sped through the rough terrain of the
Mojave dessert at 22 mph, finishing the 132-mile course first to win the 2005

Speech recognition: A traveler calling United Airlines to book a flight can have the entire
conversation guided by an automated speech recognition and dialog management system.

Autonomous planning and scheduling: A hundred million miles from Earth, NASA’s Remote
Agent program became the first on-board autonomous planning program to control the scheduling
of operations for a spacecraft (Jonsson et al., 2000). REMOTE AGENT generated plans from high-
level goals specified from the ground and monitored the execution of those plans—detecting,
diagnosing, and recovering from problems as they occurred. Successor program MAPGEN (Al-
Chang et al., 2004) plans the daily operations for NASA’s Mars
Exploration Rovers, and MEXAR2 (Cesta et al., 2007) did mission planning—both logistics
and science planning—for the European Space Agency’s Mars Express mission in 2008.Section
1.5. Summary 29

Game playing: IBM’s DEEP BLUE became the first computer program to defeat the
world champion in a chess match when it bested Garry Kasparov by a score of 3.5 to 2.5 in
an exhibition match (Goodman and Keene, 1997). Kasparov said that he felt a “new kind of
intelligence” across the board from him. Newsweek magazine described the match as “The
brain’s last stand.” The value of IBM’s stock increased by $18 billion. Human champions
studied Kasparov’s loss and were able to draw a few matches in subsequent years, but the
most recent human-computer matches have been won convincingly by the computer.
Spam fighting: Each day, learning algorithms classify over a billion messages as spam,
saving the recipient from having to waste time deleting what, for many users, could comprise 80%
or 90% of all messages, if not classified away by algorithms. Because the spammers are continually
updating their tactics, it is difficult for a static programmed approach to keep up, and learning
algorithms work best (Sahami et al., 1998; Goodman and Heckerman, 2004).

Logistics planning: During the Persian Gulf crisis of 1991, U.S. forces deployed a
Dynamic Analysis and Replanning Tool, DART (Cross and Walker, 1994), to do automated logistics
planning and scheduling for transportation. This involved up to 50,000 vehicles, cargo, and people
at a time, and had to account for starting points, destinations, routes, and conflict resolution among
all parameters.

Robotics: The iRobot Corporation has sold over two million Roomba robotic vacuum
cleaners for home use. The company also deploys the more rugged PackBot to Iraq and
Afghanistan, where it is used to handle hazardous materials, clear explosives, and identify
the location of snipers.

Machine Translation: A computer program automatically translates from Arabic to English,

allowing an English speaker to see the headline “Ardogan Confirms That Turkey Would Not Accept
Any Pressure, Urging Them to Recognize Cyprus.” The program uses a statistical model built from
examples of Arabic-to-English translations and from examples of English text totaling two trillion
words (Brants et al., 2007). None of the computer scientists on the team speak Arabic, but they do
understand statistics and machine learning algorithms.
Intelligent Agents

Agents and environment

An agent is anything that can be viewed as perceiving its environment through sensors
and acting upon that environment through actuators.

A human agent has eyes, ears, and other organs for sensors and hands, legs, vocal tract,
and so on for actuators.

A robotic agent might have cameras and infrared range finders for sensors and various
motors for actuators.

A software agent receives keystrokes, file contents, and network packets as sensory
inputs and acts on the environment by displaying on the screen, writing files, and
sending network packets.
The term percept to refer to the agent’s perceptual inputs at any given instant.

An PERCEPT SEQUENCE agent’s percept sequence is the complete history of everything

the agent has ever perceived.

In general, an agent’s choice of action at any given instant can depend on the entire
percept sequence observed to date, but not on anything it hasn’t perceived.

Tabulating the agent function that describes any given agent. The table is, of course, an
external characterization of the agent. Internally, the agent function for an artificial
agent will be implemented by an AGENT PROGRAM.
AGENT FUNCTION describes agent’s behavior by mapping any given percept
sequence to an action.
-To describe any given agent, we have to tabulate the agent function – and this will
typically be a very large table(potentially infinitely large table)
-The table can be constructed by trying out all possible percept sequences and
recording which actions the agent does in response.
-This table is external characterization of the agent.
-Agent function is abstract mathematical description.

AGENT PROGRAM is an internal implementation of the agent function for an artificial

agent.
-it is a concrete implementation, running within some physical system.
GOOD BEHAVIOR: THE CONCEPT OF RATIONALITY

We learnt that we should design agents that “act rational”

1. How do we define ‘acting rationally’ so we can write programs?
2. Should we consider the environment where the agent will be deployed?

A rational agent is one that does the right thing.(Generally)

-In other words, every entry in the table for the agent function is filled out correctly

What does it mean to do the right thing?

- We consider the consequences of the agent’s behavior.
- When an agent is plunked down in an environment, it generates a sequence of actions
according to percepts it receives
- The sequence of actions causes the environment to go through a sequence of states.
- If this sequence of environment is desirable, the agent has performed well.

- We are interested in environment states not agent states

- we should not define success in terms of the agent’s opinion
- The agent could achieve a perfect rationality simply by deluding itself that its
performance was perfect
- Human agents, for example, are notorious for “sour grapes”
Design Performance Measure(with Environment Focus)

It is better to design ‘performance measures’ according to what one actually wants in the
environment, rather than according to how one thinks the agent should behave.

Eg: for the vaccum agent, we propose to measure the performance by the amount of dirt
cleaned up in a single 8-hours shift.
Rationality

Rational at any given time depends on four things:

• The performance measure that defines the criterion of success.
• The agent's prior knowledge of the environment.
• The actions that the agent can perform.
• The agent's percept sequence to date.
A definition of a rational agent: For each possible percept sequence, a rational agent
should select an action that is expected to maximize its performance measure, given the
evidence provided by the percept sequence and whatever built-in knowledge the agent
has.
Let us assume the following:
The performance measure awards one point for each clean square at each
time step, over a ―lifetime‖ of 1000 time steps.

Prior knowledge The geography of the environment known but the dirt
distribution and the initial location of the agent are not. Clean squares stay
clean and sucking cleans the current square. The Left and Right actions move
the agent left and right except when this would take the agent outside the
environment, in which case the agent remains where it is.

The only available actions are Left , Right , and Suck .

Precept sequence: The agent correctly perceives its location and whether that
location contains dirt.
Omniscience, learning, and autonomy

An omniscient agent knows the actual outcome of its actions and can act accordingly; but
omniscience is mpossible in reality.

LEARNING Our definition requires a rational agent not only to gather information but
also to learn as much as possible from what it perceives. The agent’s initial configuration
could reflect some prior knowledge of the environment, but as the agent gains experience
this may be modified and augmented.

To the extent that an agent relies on the prior knowledge of its designer rather than
AUTONOMY on its own percepts, we say that the agent lacks autonomy
The nature of environment
Task environments, which are essentially the ―problems‖ to which rational agents are
the solutions.‖
To specify the performance measure, the environment, and the agent’s actuators and
sensors called the PEAS (Performance, Environment, Actuators, Sensors) description.
In designing an agent, the first step must always be to specify the task environment as
fully as possible.

PEAS description of an automated taxi driver.

Agent Type Performance Environment Actuators Sensors

Measure
Taxi driver Safe, fast, legal, Roads, other Steering, Cameras,
comfortable traffic, accelerator, sonar,
trip, pedestrians, brake, signal, speedometer,
maximize customers horn, display GPS, odometer,
profits accelerometer,
engine sensors,
keyboard
The performance measure to which we would like our automated driver to aspire?
Desirable qualities include getting to the
• correct destination;
• minimizing fuel consumption and wear and tear;
• minimizing the trip time or cost;
• minimizing violations of traffic laws and disturbances to other drivers;
• maximizing safety and passenger comfort;
• maximizing profits.

What is the driving environment that the taxi will face?

• Any taxi driver must deal with a variety of roads, ranging from rural lanes and urban
alleys to 12-lane freeways.
• The roads contain other traffic, pedestrians, stray animals, road works, police cars,
puddles, and potholes.
• The taxi must also interact with potential and actual passengers
Properties of Task Environments:

1. Fully observable vs. partially observable

• If an agent’s sensors give it access to the complete state of the environment at each point
in time, then we say that the task environment is fully observable.

• A task environment is effectively fully observable if the sensors detect all aspects that
are relevant to the choice of action; relevance, in turn, depends on the performance
measure.

• Fully observable environments are convenient because the agent need not maintain any
internal state to keep track of the world.

• An environment might be partially observable because of noisy and inaccurate sensors

or because parts of the state are simply missing from the sensor data

• —for example, a vacuum agent with only a local dirt sensor cannot tell whether there is
dirt in other squares, and an automated taxi cannot see what other drivers are thinking. If
the agent has no sensors at all then the environment is unobservable.
Single agent vs. multiagent:

The distinction between single-agent and multi agent environments may seem simple
enough. For example, an agent solving a crossword puzzle by itself is clearly in a single-
agent environment, whereas an agent playing chess is in a two agent
environment.

Deterministic vs. stochastic.

If the next state of the environment is completely determined by the current state and the
action executed by the agent, then we say the environment is deterministic; otherwise, it is
stochastic.

Episodic vs. sequential:

• In an episodic task environment, the agent’s experience is divided into atomic
episodes. In each episode the agent receives a percept and then performs a single
action.

• In sequential environments, on the other hand, the current decision could affect all
future decisions.
Static vs. dynamic:
• If the environment can change while an agent is deliberating, then we say the
environment is dynamic for that agent; otherwise, it is static.

• Static environments are easy to deal with because the agent need not keep looking at
the world while it is deciding on an action, nor need it worry about the passage of
time.

• Dynamic environments, on the other hand, are continuously asking the agent what it
wants to do; if it hasn’t decided yet, that counts as deciding to do nothing.
Discrete vs. continuous:
• The discrete/continuous distinction applies to the state of the environment, to the way
time is handled, and to the percepts and actions of the agent.

• For example, the chess environment has a finite number of distinct states (excluding
the clock). Chess also has a discrete set of percepts and actions.

• Taxi driving is a continuous-state and continuous-time problem: the speed and location
of the taxi and of the other vehicles sweep through a range of continuous values and do
so smoothly over time.
Properties of the Agent’s State of Knowledge

Known vs. unknown

• Describes the agent’s (or designer’s) state of knowledge about the ―laws of
physics of the environment

o if the environment is known, then the outcomes (or outcome probabilities if

stochastic) for all actions are given.
o if the environment is unknown, then the agent will have to learn how it
works in order to make good decisions

Known not equal to Fully observable a known environment can be partially

observable (Ex: a solitaire card games)

an unknown environment can be fully observable (Ex: a game I don’t know the
rules of)
Task Observable Agents Deterministic Episodic Static Discrete
Environment

Cross word
puzzle
Image
analysis
Interactive
English tutor
The structure of agents

Agent = Architecture + Program

AI Job: design an agent program implementing the agent function

The agent program runs on some computing device with physical sensors and
actuators: the agent architecture

All agents have the same skeleton:

Input: current percepts
Output: action
Program: manipulates input to produce output.

• The agent function takes the entire percept history as input

• The agent program takes only the current percept as input.
• If the actions need to depend on the entire percept sequence, the agent will
have to remember the percepts
The Table-Driven Agent
The table represents explicitly the agent function Ex: the simple
vacuum cleaner
Agents can be grouped into five classes based on their degree of
perceived intelligence and capability. All these agents can improve
their performance and generate better action over the time. These
are given below:
•Simple Reflex Agent
•Model-based reflex agent
•Goal-based agents
•Utility-based agent
•Learning agent
Simple reflex agents
• The Simple reflex agents are the simplest agents. These agents take decisions
on the basis of the current percepts and ignore the rest of the percept history.
• These agents only succeed in the fully observable environment.
• The Simple reflex agent does not consider any part of percepts history during
their decision and action process.
• The Simple reflex agent works on Condition-action rule, which means it maps
the current state to action. Such as a Room Cleaner agent, it works only if there
is dirt in the room.
• Problems for the simple reflex agent design approach:
• They have very limited intelligence
• They do not have knowledge of non-perceptual parts of the current state
• Mostly too big to generate and to store.
• Not adaptive to changes in the environment.
function SIMPLE-REFLEX-AGENT(percept) returns an
action
persistent: rules, a set of condition–action rules
state ← INTERPRET-INPUT(percept)
rule ← RULE-MATCH(state, rules)
action ← rule.ACTION
return action
Model-based reflex agent
The Model-based agent
can work in a partially observable environment, and track the situ
ation. A model-based agent has two important factors:
• Model: It is knowledge about "how things happen in the
world," so it is called a Model-based agent.

• Internal State: It is a representation of the current

state based on percept history.
These agents have the model, "which is knowledge of the world"
and based on the model they perform actions.
Updating the agent state requires information about:
• How the world evolves
•How the agent's action affects the world.
function MODEL-BASED-REFLEX-AGENT(percept) returns an action
persistent: state, the agent’s current conception of the world state
model, a description of how the next state depends on current state
and action
rules, a set of condition–action rules
action, the most recent action, initially none
state ← UPDATE-STATE(state, action, percept, model)
rule ← RULE-MATCH(state, rules)
action ← rule.ACTION
return action
Goal-based agents

• The knowledge of the current state environment is not always sufficient to decide for an agent
to what to do.
• The agent needs to know its goal which describes desirable situations.
• Goal-based agents expand the capabilities of the model-based agent by having the "goal"
information.
• They choose an action, so that they can achieve the goal.

• These agents may have to consider a long sequence of possible actions before deciding whether
the goal is achieved or not. Such considerations of different scenario are called searching and
planning, which makes an agent proactive.
• Sometimes goal-based action selection is straightforward: for example when
goal satisfaction results immediately from a single action.
• Sometimes it will be trickier: for example, when the agent has to consider long sequences of
twists and turns to find a way to achieve the goal.
• Search and planning are the subfields of AI devoted to finding action sequences that achieve
the agent’s goals.
Utility-based agents

• These agents are similar to the goal-based agent but provide an extra component of utility
measurement which makes them different by providing a measure of success at a given state.
• Utility-based agent act based not only goals but also the best way to achieve the goal.
• The Utility-based agent is useful when there are multiple possible alternatives, and an agent has
to choose in order to perform the best action.
• The utility function maps each state to a real number to check how efficiently each action
achieves the goals.
Utility-based Agents advantages wrt. goal-based:

• with conflicting goals, utility specifies and appropriate tradeoff

• with several goals none of which can be achieved with certainty, utility selects proper tradeoff
between importance of goals and likelihood of success
• still complicate to implement
• require sophisticated perception, reasoning, and learning
• may require expensive computation.
 Learning Agents

Problem Previous agent programs describe methods for selecting actions

• How are these agent programs programmed?
• Programming by hand inefficient and ineffective!
• Solution: build learning machines and then teach them (rather than instruct th
em)
Advantage: robustness of the agent program toward initially-
unknown environments
• Performance element: selects actions based on percepts Corresponds to the previo
us agent programs
• Learning element: introduces improvements uses feedback from the critic on how
the agent is doing determines improvements for the performance element
• Critic tells how the agent is doing wrt. performance standard
• Problem generator: suggests actions that will lead to new and
informative experiences forces exploration of new stimulating scenarios
Example: Taxi Driving
• After the taxi makes a quick left turn across three lanes, the critic observes the sh
ocking language used by other drivers.

• From this experience, the learning element formulates a rule saying this was a ba
d action.
• The performance element is modified by adding the new rule.
• The problem generator might identify certain areas of behavior in need of
improvement, and suggest trying out the brakes on different road surfaces under
different conditions.