Representation
The form of knowledge
COMP 20090: Introduction to Cognitive Science
Salt Lake City
COMP 20090: Introduction to Cognitive Science
COMP 20090: Introduction to Cognitive Science
COMP 20090: Introduction to Cognitive Science
Symbols stand for things
What is explicit, and what can you figure out but is
not explicit?
COMP 20090: Introduction to Cognitive Science
A symbol stands in
correspondence to
something else
Do you think a symbol has to
look like the thing is corresponds
with?
Which symbols you are familiar
with do not look like the thing
they refer to?
COMP 20090: Introduction to Cognitive Science
These are imprints. They also
stand in correspondence to
something.
What aspects of the lips/hand
are captured by the imprints?
What aspects are not captured?
COMP 20090: Introduction to Cognitive Science
REPRESENTATION
We say that something represents something else if:
[1] It stands in correspondence to that thing, and
[2] It plays a role within a system because it has
that relationship of correspondence.
COMP 20090: Introduction to Cognitive Science
The relationship between a representation and its
referent can be arbitrary
... or non-arbitrary
Which systems do these symbols function in?
Which elements on a map stand in arbitrary relationship
to their referents, and which are non-arbitrary?
COMP 20090: Introduction to Cognitive Science
The spatial layout of the retina is retained through
several transformations.
Places that are near each other stay near each other
in the initial stages of processing.
Q: Is this more like a symbol or an imprint?
What is the difference between representation
and re-presentation?
COMP 20090: Introduction to Cognitive Science
A retinotopic map in
primary visual cortex of
a monkey
What is preserved
between retina and
cortex?
What is altered or
distorted?
COMP 20090: Introduction to Cognitive Science
In primary somato-sensory
cortex, different amounts of
cortex are devoted to the
various parts of the body.
This is one way the word
representation is used.
We say that this shows
cortical representation of
sensation in the skin
COMP 20090: Introduction to Cognitive Science
COMP 20090: Introduction to Cognitive Science
A tonotopic map in cat auditory cortex. The numbers
refer to the frequencies each area is sensitive to. (All
naturally occurring sounds are composed of a mixture of
frequencies from low to high)
COMP 20090: Introduction to Cognitive Science
These examples are all of primary sensory cortex. We
have somatotopic, retinotopic and tonotopic maps.
The structure of the stimulus is somewhat preserved
in the pattern of neural sensitivities.
Distortions occur, because some aspects of a signal are
more important than others. Compare sensitivity of
lips to heels. Compare foveal vision (high focus, where
you are looking directly) with peripheral vision (what
you see out of the corner of your eye).
COMP 20090: Introduction to Cognitive Science
Are these maps representations?
Do they correspond to something?
Do they function within a system by virtue of
that correspondence?
COMP 20090: Introduction to Cognitive Science
A big mystery: About-ness, or Intentionality
If thinking cat is thinking about a
cheeseburger, and the
cheeseburger is not there, there
must be something in his neural
activity that corresponds to the
cheeseburger.
COMP 20090: Introduction to Cognitive Science
Some pattern of activity is “about” the cheeseburger.
This relationship of “about-ness” is also called
“intentionality”.
The cat, so it is argued, must have a representation of the
cheeseburger that can stand for the cheeseburger
within his thinking system.
Hard Question: Would a representation of a
cheeseburger have to look like a cheeseburger? Would
it have to taste like a cheeseburger? Does it need any
of the properties of a cheeseburger?
COMP 20090: Introduction to Cognitive Science
Propositional Representations
If thoughts are like sentences, then a representation of a
cheeseburger is a kind of symbol
I want a
s are tasty
Yes! We have no s
COMP 20090: Introduction to Cognitive Science
In propositional representation, there is no need for
the symbol to physically resemble its referent.
I want a
s are tasty
Yes! We have no s
COMP 20090: Introduction to Cognitive Science
Moving beyond sentence-like representations...
COMP 20090: Introduction to Cognitive Science
Picture a hippopotamus..........
COMP 20090: Introduction to Cognitive Science
Picture a hippopotamus..........
COMP 20090: Introduction to Cognitive Science
Are mental images like pictures?
Are the elements spatially arranged?
Can one thing block another?
Are the elements fully specified?
Can you see things you didn’t think of?
COMP 20090: Introduction to Cognitive Science
Not all thought and imagination is like language!
The great mental imagery debate:
Are the representations that are used in thinking like
symbols and propositions?
Do they bear any necessary relation to their referents?
Are they all symbols?
Are brains big symbol manipulating machines?
COMP 20090: Introduction to Cognitive Science
Shepard and others
• Shepard and Metzler (1974):
– Are two stimuli identical except for
rotation?
COMP 20090: Introduction to Cognitive Science
Another.....
COMP 20090: Introduction to Cognitive Science
And another.....
COMP 20090: Introduction to Cognitive Science
Results
COMP 20090: Introduction to Cognitive Science
An analogy
We can represent numerical quantities in several ways
Each form represents the same thing, but the operations
that each representation supports may be different: easy
with some representation and hard with others
Do the following addition using either arabic numerals or
roman numerals:
3,456 + 284 or MMMCDLVI + CCLXXXIV
COMP 20090: Introduction to Cognitive Science
Grand Challenge 1:
Given a phonebook, look up the number of Mr.
Thelonious Winterbotham
COMP 20090: Introduction to Cognitive Science
Grand Challenge 2:
Given a phonebook, find who owns the number
2824758
COMP 20090: Introduction to Cognitive Science
Grand Challenge 3:
Name three bands who are a bit like Nirvana
COMP 20090: Introduction to Cognitive Science
Grand Challenge 4:
Name three bands where the third letter in the
band’s name is ‘e’
COMP 20090: Introduction to Cognitive Science
Grand Challenge 5:
Name three words that rhyme with flap
COMP 20090: Introduction to Cognitive Science
Some challenges are harder than others.
For Challenges 1 & 2, you can see why this is: the
information in the phonebook is organized in a fashion
that makes it easy to find a number given a name, but
nearly impossible to find a name given a number.
The knowledge represented in the phonebook is
represented in a specific fashion. This hugely affects
how that knowledge can be accessed and manipulated.
COMP 20090: Introduction to Cognitive Science
Now, what do Grand Challenges 3, 4 and 5 tell you
about how some knowledge is represented in your
own brain?
COMP 20090: Introduction to Cognitive Science
Knowledge Representation
COMP 20090: Introduction to Cognitive Science
Some things I know:
The capital of Ireland is Dublin.
I know how to tie my shoelaces.
I can find my way from my bedroom to the kitchen in
the dark
Bonus question: where is the knowledge in each of
these? Is is plausibly in your head?
COMP 20090: Introduction to Cognitive Science
The capital of Ireland is Dublin.
This is explicit knowledge.
It is disconnected from any specific context
I can state this nugget of wisdom, irrespective of where I
currently am
There must be some sense in which my brain(+body) represent,
or contain, this fact, even if that is just as the potential to utter it.
We can refer to this potential as a representation
We have vast amounts of this kind of explicit knowledge
We can infer some things about how it is represented by looking
at the kind of questions that are easy or hard to answer.
COMP 20090: Introduction to Cognitive Science
Maybe everything is unconnected to everything else
CAT DOG
...has fur ...has hair
...is an animal ...is an animal
...chases mice ...chases cats
...meows ...barks
...dislikes water ...likes water
...has a tail ...has a tail
...likes to be ...likes to be
petted....... petted........
This seems to be inefficient
COMP 20090: Introduction to Cognitive Science
tail
has-a has-a
CAT DOG
wears is-a is-a wears
fur mammal hair
is-a
animal
Semantic Network
COMP 20090: Introduction to Cognitive Science
Classical categories
• Some features are necessary for X to be an
instance of Y
• Some set of features are sufficient for X to
be an instance of Y
• What do you think the necessary and
sufficient conditions for being a member of
the following categories:
A bird
A pope
A planet
COMP 20090: Introduction to Cognitive Science
Lakoff: Women, Fire and Dangerous Things
COMP 20090: Introduction to Cognitive Science
Dyirbal langauge (from Lakoff...)
COMP 20090: Introduction to Cognitive Science
What makes a chair a chair?
COMP 20090: Introduction to Cognitive Science
Resemblance Theories: Prototype vs. Exemplar
The Standard = The acting representative of the category
The “standard” prototype The “standard” exemplar
A prototype is an abstract An exemplar is a concrete
representation that is derived from representation. A specific instance
the “center of mass” of the that happens to be the most active in
features of all the objects in the memory. More typical members tend
category. to be the most active, on average.
COMP 20090: Introduction to Cognitive Science
Prototypes Exemplars
A prototype is an An exemplar is one specific
abstraction, based on example of a concept
previous experience
Storing prototypes is Storing exemplars is
cheap (there are few), expensive, but every
but you need a lot of instance of a concept can
experience to develop a function as an exemplar
prototype
COMP 20090: Introduction to Cognitive Science
• Eleanore Rosch
– Categories are created by generalization and
specialization of the basic level
– At the basic level, categories are maximally
distinct, i.e. They maximize perceived
similarity among category members and
minimize perceived similarities across
contrasting categories
COMP 20090: Introduction to Cognitive Science
• Eleanore Rosch
– Categories are created by generalization and
specialization of the basic level
– At the basic level, categories are maximally
distinct, i.e. They maximize perceived
similarity among category members and
minimize perceived similarities across
contrasting categories
COMP 20090: Introduction to Cognitive Science
Which of these are basic level categories?
1. Activity, Sport, Football, Under-15s Football
2. Object, Furniture, Table, Dressing Table
3. Animal, Bird, Gull, Herring Gull
COMP 20090: Introduction to Cognitive Science
Is everything you know
represented in your brain?
COMP 20090: Introduction to Cognitive Science
External memory....... is it important?
COMP 20090: Introduction to Cognitive Science
The homuncular fallacy.
The Cartesian theatre.
COMP 20090: Introduction to Cognitive Science
The experience of building walking robots
Rodney Brooks: MIT
Brooks: When I started building walking robots, I built six legged robots
and rather than have them compute ahead of time a stable way of walking
I made the legs very sensitive to things that they touched in the
environment and made it so that it was safe for the robot to fall down and
had it learn to scramble over rough terrain, feeling its way as it went
instead of sitting back looking at the whole terrain in front and computing
the optimal path through, my robots got in down dirty with the
environment and interacted with the environment at every step.
COMP 20090: Introduction to Cognitive Science
The [external] world is its
own best representation
--Rodney Brooks
COMP 20090: Introduction to Cognitive Science
Problems with a ‘Brain-in-a-Vat”
COMP 20090: Introduction to Cognitive Science
Much of the discipline of Artificial Intelligence has been
the attempt to find the most useful form of knowledge
representation
COMP 20090: Introduction to Cognitive Science
COMP 20090: Introduction to Cognitive Science
The Uncanny Valley
COMP 20090: Introduction to Cognitive Science
COMP 20090: Introduction to Cognitive Science
How do we build a system that incorporates
human-like knowledge of the world?
COMP 20090: Introduction to Cognitive Science
Approach 1: Add More Facts
COMP 20090: Introduction to Cognitive Science
Terry Winograd’s SHRDLU program
Person: PICK UP A BIG RED BLOCK.
Computer: OK. (does it)
Person: GRASP THE PYRAMID.
Computer: I DON'T UNDERSTAND WHICH
PYRAMID YOU MEAN.
Person: FIND A BLOCK WHICH IS TALLER
THAN THE ONE YOU ARE HOLDING AND
PUT IT INTO THE BOX.
Computer: BY "IT", I ASSUME YOU MEAN
THE BLOCK WHICH IS TALLER THAN THE
ONE I AM HOLDING.
Computer: OK. (does it)
Person: WHAT DOES THE BOX CONTAIN?
Computer: THE BLUE PYRAMID AND THE
BLUE BLOCK.
Person: WHAT IS THE PYRAMID
SUPPORTED BY?
Computer: THE BOX.
... etc
Sample dialogue available under ʻReading and
Resourcesʼ
COMP 20090: Introduction to Cognitive Science
Some great things about SHRDLU:
It could learn (some things)
It knew what it knew and what it didn’t know
It interacted using natural language
Problems: No common sense
Successes in various ‘blocks worlds’ did not scale
to systems dealing with the real world
Sample problem:
Nora is in the house
Where is Nora’s nose?
COMP 20090: Introduction to Cognitive Science
Cyc Reasoning System
Knowledge
Users
Knowledge User Interface Other
Authors (with Natural Language Dialog) Applications
Knowledge Entry
Cyc Reasoning
Modules
Cyc API
Tools
Cyc
Ontology &
Knowledge
Base
Interface to
External Data Sources
Add more External
Data Data Web Text Other
Bases Pages Sources KBs
facts...... Sources
COMP 20090: Introduction to Cognitive Science
Cyc Foundation:
“Imagine a world in which every single person is given
free access to programs that reason with the sum of all
human knowledge. That's what we're doing.”
Cyc Reasoning System
Knowledge
Users
Knowledge User Interface Other
Authors (with Natural Language Dialog) Applications
Knowledge Entry
Cyc Reasoning
Modules
Cyc API
Tools
Cyc
Ontology &
Knowledge
Base
Interface to
External Data Sources
Add more External
Data Data Web Text Other
Bases Pages Sources KBs
facts...... Sources
COMP 20090: Introduction to Cognitive Science
Common sense reasoning is hard (for computers)
The set of facts which might be relevant to solving any
problem is essentially infinite. Small toy world
solutions don’t scale up to real world situations.
This is the frame problem.
Nora goes into the house. Where is Nora’s nose?
COMP 20090: Introduction to Cognitive Science
Moravec's paradox
contrary to traditional assumptions, the uniquely human faculty of
reason (conscious, intelligent, rational thought) requires very little
computation, but that the unconscious sensorimotor skills and
instincts that we share with the animals require enormous
computational resources. (wikipedia)
COMP 20090: Introduction to Cognitive Science
Steven Pinker
The main lesson of thirty-five years of AI research is that the hard problems
are easy and the easy problems are hard. The mental abilities of a four-year-old
that we take for granted – recognizing a face, lifting a pencil, walking across a
room, answering a question – in fact solve some of the hardest engineering
problems ever conceived.... As the new generation of intelligent devices
appears, it will be the stock analysts and petrochemical engineers and parole
board members who are in danger of being replaced by machines. The
gardeners, receptionists, and cooks are secure in their jobs for decades to
come.
(In Defense of Dangerous Ideas, 2007)
COMP 20090: Introduction to Cognitive Science
Perceptually guided exploratory action may not require a
complex cognitive architecture supporting abstract
representation:
COMP 20090: Introduction to Cognitive Science
