Chapter 2

Intelligent Agents

In which we discuss the nature of agents, perfect or otherwise, the

diversity of environments, and the resulting menagerie of
agent types.
 Outline

2.1 Agents and Environments

2.2 Good Behavior: The Concept of Rationality
2.3 The Nature of Environments
2.4 The Structure of Agents
2.5 Summary
 Agents
An agent is anything that can be viewed as perceiving its environment through sensors
and acting upon that environment through actuators.

A human agent has eyes, ears, and other organs for sensors and hands, legs, vocal tract,
and so on for actuators.

A robotic agent might have cameras and infrared range finders for sensors and various
motors for actuators.

A software agent receives keystrokes, file contents, and network packets as sensory
inputs and acts on the environment by displaying on the screen, writing files, and sending
network packets.
Agents interaction with environment
 Percept Sequence, Agent Function and Agent
Program
Percept : refers to the agent’s perceptual inputs at any given instant.

Percept Sequence: complete history of everything the agent has ever perceived.

Agent’s choice of action : depends on the entire percept sequence observed to date, but not
on anything it hasn’t perceived.

Agents and Environment

Vacuum-Cleaner World
 Partial Tabulation-Vacuum Cleaner World

Notion of an agent is meant to be a tool for analyzing systems, not an absolute

characterization that divides the world into agents and non-agents.
 Outline

2.1 Agents and Environments

2.2 Good Behavior: The Concept of Rationality
2.3 The Nature of Environments
2.4 The Structure of Agents
2.5 Summary
 THE CONCEPT OF RATIONALITY
What is rational at any given time depends on four things:

• The performance measure that defines the criterion of success.

• The agent’s prior knowledge of the environment.
• The actions that the agent can perform.
• The agent’s percept sequence to date.

Definition of a Rational Agent:

For each possible percept sequence, a rational agent should select an action that is expected to
maximize its performance measure, given the evidence provided by the percept sequence and
whatever built-in knowledge the agent has.
 Rational Agents

 An agent should strive to "do the right thing", based on what it can perceive
and the actions it can perform. The right action is the one that will cause
the agent to be most successful

 Performance measure: An objective criterion for success of an agent's

behavior. E.g., performance measure of a vacuum-cleaner agent could be
amount of dirt cleaned up, amount of time taken, amount of electricity
consumed, amount of noise generated, etc.
 Vacuum Cleaner: Is this a rational agent?
• The performance measure awards one point for each clean square at each
time step, over a “lifetime” of 1000 time steps.
• The “geography” of the environment is known a priori but the dirt distribution
and the initial location of the agent are not. Clean squares stay clean and
sucking cleans the current square. The Left and Right actions move the agent
left and right except when this would take the agent outside the environment, in
which case the agent remains where it is.
• The only available actions are Left , Right, and Suck.
• The agent correctly perceives its location and whether that location contains
dirt.
 The Concept of Rationality
Omniscience. An omniscient agent knows the actual outcome of its actions and can act
accordingly; but omniscience is impossible in reality.

A rational agent should choose the “looking” action before stepping into the street,
because looking helps maximize the expected performance.

Doing actions in order to modify future percepts—sometimes called information

gathering.

A second example of information gathering is provided by the exploration that must be

undertaken by a vacuum-cleaning agent in an initially unknown environment.

Rational agent not only to gather information but also to learn as much as possible from
what it perceives.

To the extent that an agent relies on the prior knowledge of its designer rather than
on its own percepts, we say that the agent lacks autonomy.
 The Concept of Rationality
Rationality is distinct from omniscience (all- knowing with infinite knowledge)
Rational != omniscient

 Agents can perform actions in order to modify future percepts so as to obtain

useful information (information gathering, exploration)
 An agent is to learn as much as possible from what it perceives.
 An agent is autonomous if its behavior is determined by its own experience
(with ability to learn and adapt)

 Rational exploration, learning, autonomy

 Outline

2.1 Agents and Environments

2.2 Good Behavior: The Concept of Rationality
2.3 The Nature of Environments
2.4 The Structure of Agents
2.5 Summary
 The nature of Environments

Task environments, which are essentially the “problems” to which rational

agents are the “solutions.” We begin by showing how to specify a task
environment, illustrating the process with a number of examples. We then
show that task environments come in a variety of flavors. The flavor of the task
environment directly affects the appropriate design for the agent program.
 Specifying the task environment

PEAS (Performance, Environment, Actuators, Sensors)

- Performance measure: Safe, fast, legal, comfortable trip, maximize profits

- Environment: Roads, other traffic, pedestrians, customers, weather
- Actuators: Steering wheel, accelerator, brake, signal, horn
- Sensors: Cameras, sonar, speedometer, GPS, odometer, engine sensors,
keyboard
PEAS Description
Examples
 Fully Observable or Partially Observable
Environment

Fully observable environments are convenient because the agent need not
maintain any internal state to keep track of the world. An environment might
be partially observable because of noisy and inaccurate sensors or because
parts of the state are simply missing from the sensor data.
—for example, a vacuum agent with only a local dirt sensor cannot tell
whether there is dirt in other squares, and an automated taxi cannot see what
other drivers are thinking.
 Unobservable Environment

If the agent has no sensors at all then the environment is

unobservable. One might think that in such cases
the agent’s plight is hopeless, but, as we discuss in
Chapter 4, the agent’s goals may still be achievable,
sometimes with certainty.
 Single agent vs. multiagent

The distinction between single-agent and multiagent environments may

seem simple enough.

Chess is a competitive multiagent environment. In the taxi-driving environment, on the other

hand, avoiding collisions maximizes the performance measure of all agents, so it is a partially
cooperative multiagent environment. It is also partially competitive because, for example, only
one car can occupy a parking space. The agent-design problems in multiagent environments are
often quite different from those in single-agent environments; for example, communication often
emerges as a rational behavior in multiagent environments; in some competitive environments,
randomized behavior is rational because it avoids the pitfalls of predictability.
 Environment Types

 Deterministic (vs. stochastic): The next state of the environment is completely determined
by the current state and the action executed by the agent. (We ignore uncertainty that arises
purely from the actions of other agents in a multiagent environment; thus, a game can be
deterministic even though each agent may be unable to predict the actions of the others.)
Stochastic: The next state can't determined by current state. It generally implies that
uncertainty about outcomes is quantified in terms of probabilities; - E.g. taxi driving
 An environment is uncertain if it is not fully observable or not deterministic.
 A nondeterministic environment is one in which actions are characterized by their
possible outcomes, but no probabilities are attached to them.
 Episodic vs. sequential Environment

⚫ Episodic (vs. sequential): The agent's experience is divided into atomic

"episodes" (each episode consists of the agent perceiving and then
performing a single action), and the choice of action in each episode
depends only on the episode itself. Many classification tasks are episodic.

⚫ In sequential environments, the current decision could affect all future

decisions. E.g. Chess and taxi driving are sequential: in both cases, short-
term actions can have long- term consequences.
 Static, Semi-dynamic, Dynamic environment

• Static (vs. dynamic): The environment is unchanged while an agent is

deliberating. Crossword puzzles are static.
• Semi-dynamic: if the environment itself does not change with the
passage of time but the agent's performance score does. E.g. Chess,
when played with a clock.
• Dynamic: If the environment can change while an agent is deliberating;
Taxi driving is dynamic: the other cars and the taxi itself keep moving
while the driving algorithm dithers about what to do next.
 Discrete (vs. continuous) environment

• Discrete (vs. continuous): A limited number of distinct, clearly

defined percepts and actions. For example, the chess environment
has a finite number of distinct states.

• Taxi driving is a continuous-state and continuous-time problem: the

speed and location of the taxi and of the other vehicles sweep
through a range of continuous values and do so smoothly over time.
 Known vs Unknown Properties of Agents
Known(vs. unknown): this distinction refers not to the environment itself but to
the agent's (or designer's) state of knowledge about the "laws of physics" of the
environment. In a known environment, the outcomes for all actions are given.
• In unknown environment, the agent will have to learn how it works in order to
make good decisions.
• Known/unknown is different from observable/ unobservable
• Known environment can be partially observable - Solitaire card games. Know
all rules but unable to see other cards
• Unknown environment can be fully observable - In video game, entire game
state on screen but don't know about buttons
Examples
Table-lookup agent
 Table-lookup agent

10^250,000,000,000 entries for an hour's driving.

At least 10^150 entries for chess.

Drawbacks:
- Huge table
- Take a long time to build the table
- No autonomy
- Even with learning, need a long time to learn
 Outline

2.1 Agents and Environments

2.2 Good Behavior: The Concept of Rationality
2.3 The Nature of Environments
2.4 The Structure of Agents
2.5 Summary