Guy A.

Orban
Physiol Rev 88:59-89, 2008. doi:10.1152/physrev.00008.2007

You might find this additional information useful...

This article cites 331 articles, 190 of which you can access free at:
http://physrev.physiology.org/cgi/content/full/88/1/59#BIBL

Medline items on this article's topics can be found at http://highwire.stanford.edu/lists/artbytopic.dtl

on the following topics:
Psychology .. Visual Processing
Veterinary Science .. Cerebral Cortex
Veterinary Science .. Medial Superior Temporal Area

Updated information and services including high-resolution figures, can be found at:
http://physrev.physiology.org/cgi/content/full/88/1/59

Additional material and information about Physiological Reviews can be found at:
http://www.the-aps.org/publications/prv

Downloaded from physrev.physiology.org on October 9, 2008

This information is current as of October 9, 2008 .

Physiological Reviews provides state of the art coverage of timely issues in the physiological and biomedical sciences. It is
published quarterly in January, April, July, and October by the American Physiological Society, 9650 Rockville Pike, Bethesda
MD 20814-3991. Copyright © 2005 by the American Physiological Society. ISSN: 0031-9333, ESSN: 1522-1210. Visit our
website at http://www.the-aps.org/.
 Physiol Rev 88: 59 – 89, 2008;
doi:10.1152/physrev.00008.2007.

Higher Order Visual Processing in Macaque Extrastriate Cortex

GUY A. ORBAN

Laboratorium voor Neuro- en Psychofysiologie, K. U. Leuven Medical School, Leuven, Belgium

I. Introduction: Scope of the Review 60

II. The Starting Point: Simple Attribute Selectivity in V1 60
A. Orientation and spatial frequency selectivity 60
B. Direction and speed selectivity 61
C. Other selectivities 61
D. Receptive field issues 61
III. The Framework: Layout and Connections of the Extrastriate Cortex 62

Downloaded from physrev.physiology.org on October 9, 2008

IV. A Property Derived From the Antagonistic Surround: Speed Gradient Selectivity in MT/V5 63
A. Antagonistic surrounds in MT/V5 63
B. Speed gradient selectivity of MT/V5 neurons 64
C. Extraction of speed gradients beyond MT/V5 65
V. A Property Arising From the Combination of Inputs: Pattern Direction Selectivity of MT/V5 Neurons 66
A. Use of multiple contour motion: pattern direction selectivity 66
B. Use of other information: moving terminators 66
C. Limits of motion integration 67
VI. Optic Flow Component Selectivity in the MT/V5 Complex and Beyond 68
A. Selectivity of the MSTd for optic flow 68
B. Selectivity of MSTd neurons for heading directions 69
C. Mixing of visual with vestibular and pursuit signals in MSTd: out of scope 70
D. Optic flow selectivity of other extrastriate regions 70
VII. Segmentation Mechanisms: Figure-Ground Segregation and Depth Ordering in Areas V2, V4, and
MT/V5 71
A. Distinction between internal and external contours of objects: selectivity for nonluminance
defined boundaries 71
B. Segmentation or depth ordering in static images: border ownership and surface representations 73
C. Segmentation of moving planes in MT/V5 74
VIII. Stereoscopic Processing: Three-Dimensional Shape Selectivity in Far Extrastriate Cortex 75
A. Higher order disparity selectivity in TEs, part of the infero-temporal complex 75
B. Exquisite coding of three-dimensional shape from disparity by TEs neurons 76
C. The invariance of three-dimensional shape selectivity in TEs 77
D. Selectivity of CIP neurons for first-order disparity 77
E. Three-dimensional shape from disparity selectivity in other cortical regions 78
IX. Two-Dimensional Shape Processing in Infero-Temporal Cortex 78
A. The starting point of shape selectivity in V4 79
B. Shape processing in posterior IT: building simple shape parts 80
C. Shape processing in anterior IT: manipulating shape dimensions 80
X. Concluding Remarks 81

Orban, GA. Higher Order Visual Processing in Macaque Extrastriate Cortex. Physiol Rev 88: 59 – 89, 2008;
doi:10.1152/physrev.00008.2007.—The extrastriate cortex of primates encompasses a substantial portion of the
cerebral cortex and is devoted to the higher order processing of visual signals and their dispatch to other parts of
the brain. A first step towards the understanding of the function of this cortical tissue is a description of the
selectivities of the various neuronal populations for higher order aspects of the image. These selectivities present in
the various extrastriate areas support many diverse representations of the scene before the subject. The list of the
known selectivities includes that for pattern direction and speed gradients in middle temporal/V5 area; for heading
in medial superior temporal visual area, dorsal part; for orientation of nonluminance contours in V2 and V4; for
curved boundary fragments in V4 and shape parts in infero-temporal area (IT); and for curvature and orientation in
depth from disparity in IT and CIP. The most common putative mechanism for generating such emergent selectivity
is the pattern of excitatory and inhibitory linear inputs from the afferent area combined with nonlinear mechanisms
in the afferent and receiving area.

www.prv.org 0031-9333/08 $18.00 Copyright © 2008 the American Physiological Society 59

60 GUY A. ORBAN

I. INTRODUCTION: SCOPE OF THE REVIEW (5) was one of the first extrastriate areas to be discovered.
Interestingly, recent functional imaging studies in which
The visual system of primates consists of three main monkey and human extrastriate cortical regions are di-
parts: a projection from the retina to the primary visual rectly compared (201, 202, 325) have revealed a more
cortex or striate cortex, retinal projections to subcortical extensive processing of motion signals in human com-
visual centers, and higher order or associative visual cor- pared with monkey cortex. Specialization for motion pro-
tical areas beyond the primary visual cortex. This latter cessing has been reported even in human striate cortex
cortical expanse, the so-called extrastriate cortex, pro- (232). The importance of motion processing may be re-
vides the outputs to the other cerebral systems and is the lated to the greater mobility of humans and even more so
portion of the cortex where most of the analysis of the to the extensive use of their hands to manipulate objects,
visual signals is performed. Thus any understanding of particularly tools, a proposal that has recently received
the behavioral role of vision is contingent on unraveling direct support (296). We will review two novel selectivi-
the function of extrastriate cortex. This cortical region is ties that emerge in MT/V5: pattern direction selectivity
greatly expanded in primates compared with other mam- and selectivity for speed gradients. The third aspect of
mals and has a characteristic architecture in primates. All motion processing is the extraction of optic flow compo-
primates share not only V1 and neighboring V2 and V3 nents in MSTd, one of the areas receiving input from
areas, but also middle temporal (MT)/V5 area and likely MT/V5. Much less progress has been made regarding the

Downloaded from physrev.physiology.org on October 9, 2008

V3A (121). The neuronal operations preformed by this processing of static object attributes. Although selectivity
associative cortex are the topic of this review. In partic- for object shape has long since been attributed to infero-
ular, we shall focus on the novel selectivities, for complex temporal (IT) cortex (86), progress has been slowed for
aspects of the retinal image, that arise in these cortical several reasons. Many different types of stimuli can be
regions, beyond the selectivity for simple features that is used for its exploration, and it is not clear that the many
characteristic of V1 neurons. Thus we do not include in man-made or abstract stimuli that are used in most IT
this review any further elaboration of these simple selec- studies are particularly useful for revealing the underlying
tivities such as their invariance for position or size. For function of this part of cortex. Furthermore, it has only
example, the relative sensitivity to achromatic and isolu- recently become clear that shape is also processed in
minant gratings is size invariant in V2 but not V1 (290). We parietal regions (48, 274). Recently, two other advances
also restrict the review to studies using subjects that were have been made with respect to extrastriate function. On
passive with respect to the visual stimulus, either alert or one hand, a number of studies suggest that early extra-
anesthetized. Such studies reveal the basic visual process- striate cortex, i.e., cortical regions in the immediate vicin-
ing capabilities of extrastriate neurons that can be mod- ity of striate cortex, notably V2 and V3, plays a role in
ulated in many ways such as by the task, by attention, etc. segmentation and the definition of figure-ground relation-
These modulatory influences are often summarized as ships. On the other hand, some advances have been made
top-down signals. With regard to that terminology, we are in unraveling the processing of stereoscopic information
restricting the review to bottom-up processing. Attention beyond the simple specification of depth.
and task-related signals are only two examples of what
are generally referred to as extraretinal signals. These
II. THE STARTING POINT: SIMPLE ATTRIBUTE
extraretinal signals also include signals originating from
SELECTIVITY IN V1
other senses, inputs related to the motor system, such as
proprioceptive or vestibular signals, or even signals orig-
inating in the motor structures (corollary discharges). In A. Orientation and Spatial Frequency Selectivity
this review we restrict ourselves to the processing of the
retinal signals in extrastriate cortex in either the forward Hubel and Wiesel discovered the prototypical selec-
(from lower order to higher order areas) or backward tivity of striate neurons: orientation selectivity, first in
(from higher order to lower order areas) direction. cats (104, 105) and later in monkeys (106). They also
The extrastriate cortex includes a large number of described the receptive field organization of orientation
cortical regions, some of them well known, many of them selective neurons: simple, complex, and hypercomplex
far less explored. Thus the review is also limited by the neurons. Today these hypercomplex neurons are more
data that are available. The aspect of visual processing generally referred to as end-stopped neurons (205), since
that has received the most attention is motion processing, they could be of either the simple or complex variety
probably because the parameters of motion are relatively (269) and to avoid the wiring implications of the term
simple, motion can be readily manipulated in display hypercomplex. Hubel and Wiesel (108, 109) also de-
systems, and the initial stage of motion processing be- scribed a functional organization for orientation selectiv-
yond V1 was discovered early on. Indeed, the visual area ity: the columnar organization of preferred orientations.
V5 (53) or middle temporal (MT) area in the caudal STS This organization has been confirmed by several tech-

Physiol Rev • VOL 88 • JANUARY 2008 • www.prv.org

 HIGHER ORDER VISUAL PROCESSING 61

niques including optical imaging (19, 194), deoxyglucose speed tunings for different spatial frequencies and to
labeling (110, 326), and calcium imaging (195). Since the disentangle temporal frequency selectivity from speed
time of the original Hubel and Wiesel studies, orientation tuning. Following this strategy, Priebe et al. (235) were
selectivity has been investigated quantitatively with a va- able to show that there is a major difference between the
riety of stimuli, including bars and gratings, in both anes- direction-selective simple and complex cells of V1. In
thetized and alert animals (36, 50, 90, 155, 246, 247, 270, simple cells, spatial and temporal frequencies are separa-
331). Hubel and Wiesel (104) proposed that the orienta- ble, and only in complex cells is there evidence for speed
tion selectivity of simple cells arose from the excitatory tuning. This speed tuning is relatively similar to that of
convergence of a set of geniculate afferents, the receptive MT/V5 neurons (233), although the relationship between
fields (RFs) of which were aligned. This is also the arche- speed tunings for patterns other than gratings and the
type of one sort of mechanism with which to construct predictions derived from the tuning for gratings can differ
cortical selectivity: a specific pattern of excitatory inputs between V1 complex cells and MT/V5 neurons (235).
to the neuron. This is an interesting issue, as it tells us the
limitations of our present technology, insofar as the exact
C. Other Selectivities
circuit generating this selectivity is still under discussion.
Evidence has been provided for the contribution of genic-
V1 neurons are tuned along axes in color space when
ulate inputs (67, 123, 245), but evidence that intracortical

Downloaded from physrev.physiology.org on October 9, 2008

tested with isoluminant stimuli (51, 91, 152, 312, 337; for
inhibition plays a role has also been obtained (281, 284).
review, see Refs. 79, 153). Compared with geniculate neu-
In particular, recent studies of the dynamism of orienta-
rons, the peaks of the chromatic tunings are much more
tion selectivity have revealed several inhibitory mecha-
widely distributed in V1 than in the LGN (152, 337). Chro-
nisms, both tuned and untuned (247, 275, 285, 344, 345).
matic tuning is invariant for contrast (289) and size (290)
Together with orientation, spatial frequency defines the
in only a minority of V1 neurons. Color induction effects,
power spectrum of images. Spatial frequency selectivity
i.e., the shift of the color of a stimulus away from the
in V1 neurons has been described quantitatively (29, 50,
color of the background, have been documented for V1
64, 69, 271). This selectivity is determined by factors similar
neurons in alert animals (337).
to those implicated in orientation selectivity, mainly the
Initial studies by Hubel and Wiesel (107) failed to
pattern of lateral geniculate nucleus (LGN) afferents and
document disparity selectivity in V1 neurons. Later
intracortical inhibitory inputs, whether tuned or not (344).
studies in awake animals (229, 230, 236) reported tun-
ing for horizontal disparity as well as asymmetric dis-
B. Direction and Speed Selectivity parity response curves (far and near cells). Since then,
it has been shown that V1 neurons are selective for
absolute, not relative, disparity (42), indicating that
Hubel and Wiesel (104, 106) first discovered direction
they only signal position in depth relative to the fixation
selectivity in V1 neurons. Since then, direction selectivity
point, not another stimulus. Furthermore, in central
has been studied quantitatively with moving edges, bars,
vision, V1 is specialized for horizontal disparity (39),
and gratings in anesthetized and alert preparations (2, 50,
but this vanishes in the more peripheral visual field
69, 206, 270, 287). There is now agreement that direction
representation of V1 (61, 62, 80). Although the disparity
selectivity in V1 is concentrated in the laminae that
selectivity present at the level of V1 neurons imposes
project to MT/V5: layers 4B and 6 (90, 92, 161, 206).
bounds on stereoscopic perception (192, 193), V1 neurons
Recordings from large numbers of V1 neurons reveal a
are far removed from perception. They respond even to
bimodal distribution of direction selectivity indices, sug-
anticorrelated stereograms (41) which elicit little or no
gesting that direction-selective neurons are a separate
depth perception (for review, see Ref. 40).
population of V1 neurons (90, 270).
Thus V1 neurons are selective for simple attributes
The speed selectivity of V1 neurons was initially in-
covering all aspects of vision: shape, motion, color, tex-
vestigated with moving bars (206). This revealed an ec-
ture, and depth.
centricity and laminar dependence of the speed sensitiv-
ity, with neurons having preferences for faster speeds
occurring in laminae 4B and 6 and at larger eccentricities. D. Receptive Field Issues
Thus, even when eccentricities are matched, one has to
exercise caution when comparing the speed sensitivities Hubel and Wiesel (104, 105) introduced two dis-
of V1 and MT/V5 neurons: overall V1 neurons respond to tinctions in the receptive field organizations of V1 cells
slower motion than MT/V5 neurons (177, 178, 198); how- with oriented RFs: that between simple and complex
ever, speed ranges are more similar between the popula- cells and that between end-stopped and end-free cells.
tions of direction-selective neurons (235). By using grat- While the distinction between simple and complex is
ings rather than bars, it becomes possible to compare the routinely used in many V1 studies, see, e.g., speed

Physiol Rev • VOL 88 • JANUARY 2008 • www.prv.org

62 GUY A. ORBAN

tuning above, the role of end-stopped cells has received extrastriate regions had been described (Fig. 1A), and
far less attention. A substantial fraction (⬃25%) of V1 each of these areas was, on average, connected recip-
neurons are end-stopped (98, 265, 269), but these neu- rocally with a dozen other areas. The complexity of this
rons are difficult to observe in awake animals because wiring diagram has suggested to Van Essen et al. (328)
of their exquisite stimulus requirements, and their num- that the visual system must be a dynamic system that
bers may even be underestimated by the almost sys- adapts itself to the needs of the subject, depending on
tematic use of extended stimuli such as gratings. None- the task he needs to perform with his vision.
theless, these neurons are most abundant in the super- Such task-dependent processing is beyond the scope of
ficial layers, those projecting to extrastriate cortex. the present review, but see Reference 210 for an
Initially, these neurons were considered to contribute early review and Reference 133 for a recent demonstra-
primarily to the analysis of shape, since they had been tion.
shown to respond to corners, end points of lines, and
There is still disagreement, however, about the
curved stimuli (52, 98, 105, 330, 347). It has been sug-
exact parcellation at higher levels in the temporal and
gested by Bishop and co-workers (172) that end-stop-
parietal cortex (327). These various possibilities can be
ping might be useful for restricting disparity tuning in
visualized in Caret (329), including the new parcellation
directions both parallel and orthogonal to the preferred
orientation. This mechanism might partially explain the of the IPS by Lewis and Van Essen (157, 158). In fact,

Downloaded from physrev.physiology.org on October 9, 2008

specialization for horizontal disparities in central vi- cortical areas are identified by a set of four independent
sion mentioned above, but the role of end-stopped cells criteria: cyto- and myeloarchitectonic organization,
in disparity processing in V1 of the monkey still has to connection pattern with other cortical and subcortical
be evaluated. End-stopped neurons have been shown to regions, retinotopic organization, and functional prop-
play a similar role in motion processing, insofar as erties. In the past, these properties were often docu-
end-stopped neurons can encode direction of motion in mented within disparate studies, and progress towards
two dimensions, while end-free cells can only encode an exact parcellation of the extrastriate cortex has
the direction orthogonal to their preferred orientation consequently been slow. This less than satisfactory
and thus suffer from the aperture problem (i.e., they state of affairs may now change with the advent of fMRI
signal motion in one direction, whatever the actual motion in the awake monkey (324). This technique allows the
direction, Ref. 216). End-free cells will respond to an infinity sampling of a wide range of functional properties in the
of velocity vectors, ranging from the shortest vector orthog- same set of subjects (186), as well as revealing the
onal to the preferred orientation to the longest one nearly retinotopic organization (68) and even more recently,
parallel to the preferred orientation. Indeed, it was docu- includes the ability to trace anatomical connections by
mented long ago (340) that V1 neurons can respond to very means of electrical stimulation (63). A number of addi-
fast speeds for bars oriented parallel to the preferred orien- tional regions have been documented since the Van
tation. End-stopped cells should not respond under these Essen et al. (328) compilation. In the parietal cortex,
conditions, although this has not been explicitly tested. this includes V6 and V6A (76, 166), AIP (183, 253), and
Finally, it is worth mentioning that even at the level some of the subdivisions of the inferior parietal lobule
of macaque V1, the classical receptive field from which (IPL) (253). In the superior temporal sulcus (STS), this
excitatory responses are evoked is already surrounded includes the stereo part of TE (TEs) (118) in the ante-
by an antagonistic region that suppresses the responses
rior tip of the lower bank, and lower superior temporal
evoked from the center (35, 120, 129, 231, 266, 282), as
area (LST) and the middle part of superior temporal
has also been reported for area 17 of the cat (45, 89,
polysensory region (STPm) in the lower and upper
167, 187) and owl monkey (6). In many studies the
bank of midlevel STS, respectively (186).
surround is considered to include the end-stopping re-
gions (45, 266), but the sensitivity of end-stopping to Finally, it is now well established that the extra-
contrast (347) casts doubt on this view. In addition to striate cortex is organized into two parallel streams,
inhibitory influences from beyond the classical recep- one dorsal or occipito-parietal stream and one ventral
tive field, facilitatory influences have also been shown or occipito-temporal stream (Fig. 1B). Originally (321),
that arise mainly from the end regions of the RFs (122). it was suggested that these two streams process differ-
ent visual attributes, but more recently the difference in
III. THE FRAMEWORK: LAYOUT AND the behavioral goal of their processing has been em-
CONNECTIONS OF THE phasized (81). Indeed, there is increasing evidence that
EXTRASTRIATE CORTEX a number of attributes such as two- and three-dimen-
sional shape are processed in both pathways (48, 201),
The last tabulation of extrastriate regions was pub- even when there is no arbitrary mapping between shape
lished some time ago (65, 328). At that time, roughly 30 and response (82).

Physiol Rev • VOL 88 • JANUARY 2008 • www.prv.org

 HIGHER ORDER VISUAL PROCESSING 63

FIG. 1. The extrastriate cortex in macaque

Downloaded from physrev.physiology.org on October 9, 2008

monkey. A: lateral, medial, and flattened view
of one hemisphere with the eyeball attached.
Colors indicate visual regions; green and light
blue indicate motor and auditory cortex, respec-
tively. [Modified from Van Essen et al. (328).] B:
the two pathways of extrastriate cortex for the
ventral pathway. SC (A) and CS (B), colliculus
superior.

IV. A PROPERTY DERIVED FROM THE surrounds and attempted to demonstrate their isotropy
ANTAGONISTIC SURROUND: SPEED by comparing the suppressive effects of the two halves
GRADIENT SELECTIVITY IN MT/V5 of the surround. Raiguel et al. (242) documented the
strength and size of these surrounds by fitting the inte-
A. Antagonistic Surrounds in MT/V5 gral of a differences of Gaussian function to the diam-
eter-response curves of a large number of MT/V5 neu-
Tanaka et al. (307) described the antagonistic sur- rons. The surround radius averaged 3.3 times that of
rounds of MT/V5 neurons and demonstrated their direc- CRF, a value larger than that for V1 neurons (266).
tion selectivity. The surround was maximally suppres- Surrounds were weakest and smallest in the input lay-
sive when the motion was in the same direction as the ers 4 and 6, where they measured only 2.5 times the
preferred direction of classical receptive field (CRF). radius of the CRF, a value close to that of V1 neurons.
The suppression was also maximal when the relative Surrounds were stronger in the superficial layers and in
speed was unity. They documented the large size of the layer 5 where they reached five times the size of the

Physiol Rev • VOL 88 • JANUARY 2008 • www.prv.org

64 GUY A. ORBAN

CRF. These data provide evidence that the surrounds in

MT/V5 are not just a reflection of those contained in the
V1 input, but arise from further processing in MT itself
(21). Just as in V1, however, the surrounds are contrast
dependent and vanish at low contrast (215). As a con-
sequence, MT neurons will respond better to a large
stimulus with a low contrast than to one with a high
contrast. How this fits with changes in speed percep-
tion with varying contrast is still unclear (142, 159,
215, 234).

B. Speed Gradient Selectivity of MT/V5 Neurons

Initially, two alternative views have been proposed as

mechanisms for the detection of speed gradients: hot-
spots in the CRF itself (314) or a surround-based mecha-

Downloaded from physrev.physiology.org on October 9, 2008

nism (342). These mechanisms differ both in spatial origin
and in sign, since the CRF is excitatory and the surround
suppressive. Subsequent work has indicated that the main
mechanism is surround based. In anesthetized monkeys,
about half (27/57) of MT/V5 neurons were selective for the
direction of the speed gradient, and different neurons
were tuned to different directions of speed gradients cor-
responding to different tilts in depth (direction in which
the 3-dimensional surface is angled away from the fronto-
parallel plane, Ref. 341). These authors also showed that
the speed gradient selectivity critically depended on the
surround, since all of the selective neurons had antago-
nistic surrounds and the selectivity was strongly reduced
when the surround was masked. Further studies (343)
showed that the majority of MT/V5 neurons have nonho-
mogeneous antagonistic surrounds, as predicted by com-
putational models (16, 77). Figure 2 illustrates the stimuli
used to map the excitatory CRF (A), to unmask the pres-
ence of a surround (spatial summation test, B), and to
map the spatial distribution of the surround effects either
coarsely (in 1 of 8 directions around the CRF, C) or in FIG. 2. Asymmetric surround in middle temporal (MT)/V5 neuron.

detail (same spacing as excitatory mapping, D). The neu- A–D: stimuli used to map the classical receptive field (CRF) and the
antagonistic surround. E–H: the result for a single MT/V5 neuron (red,
ron in Figure 2 had a strong surround effect (F), which excitation; blue, suppression; yellow, no effect). Single, square patch of
arose from an antagonistic region located above the CRF moving random dots presented at different positions (A) yields the CRF
(G and H). Such unimodal surrounds allow the neuron to (E). A circular patch of moving random dots (B) of increasing diameter
yields the summation curve in F. The double stimulation with one
compute a spatial derivative of speed (thus a speed gra- central patch and one peripheral patch in 8 different positions (C) yields
dient), provided that the inhibitory surround influence is one suppression map (G). The double stimulation with one square
itself speed dependent. In many MT/V5 neurons, this is central patch and one small peripheral stimulus in 24 positions yields a
detailed suppression map (H). In F–H, poststimulus time histograms are
indeed the case (341): their surrounds are strongly asym- shown for selected conditions. In G, the arrow points to the position
metric but only when the stimuli in the surround moved at with strongest suppression, and bottom histograms indicate mean re-
the same or faster speed than the dots moving over the sponse for complete stimulation of surround and central patch only.
[From Orban (200), with permission from Elsevier.]
CRF. About half the MT/V5 neurons had asymmetric sur-
rounds with a single major suppressive region on one side
of the CRF. Another quarter of MT/V5 neurons had two
suppressive regions on either side of the CRF, and a final gain control or normalization as in V1 (254). They could
quarter possessed a suppressive surround that indeed also remove common motion (22) or provide modulatory
encircled the CRF (343). These latter uniform surrounds signals for segmentation in depth (see below). The double
could perform more general functions such as contrast symmetric regions, on the other hand, might be involved

Physiol Rev • VOL 88 • JANUARY 2008 • www.prv.org

 HIGHER ORDER VISUAL PROCESSING 65

in the extraction of curved surfaces from motion (342). C. Extraction of Speed Gradients Beyond MT/V5
Notice that a seemingly obvious function of MT sur-
rounds, the extraction of discontinuities in the velocity Selectivity for speed gradients has also been docu-
field, was not supported by the data: MT/V5 neurons are mented in MSTd, a region receiving input from MT/V5 and
not selective for either the orientation or the position of a which is known to process optic flow (55, 83, 144, 255,
kinetic boundary (169). 321). Duffy and Wurtz (58) noted that the speed gradients
These studies in the anesthetized monkey have been superimposed on flow components significantly affect the
recently replicated in the alert monkey by Nguyenkim and firing of MSTd neurons in the awake monkey. Sugihara et
DeAngelis (191), who used large stimuli involving the al. (298) superimposed speed gradients onto rotatory
surround, similar to those used by Xiao et al. (341). These flow, producing rotating planes with various three-dimen-
authors confirmed the selectivity of MT/V5 neurons for sional orientations. A substantial fraction (43/97) of MSTd
speed gradients and provided an important control test: neurons were found to be selective for the tilt, i.e., the
the selectivity for the speed gradient is invariant with direction of the speed gradient. Selectivity for slant (the
respect to the average speed in the display. Furthermore, amount by which the 3-dimensional surface is angled
they showed that MT/V5 neurons are frequently selective away from the fronto-parallel plane) was also observed in
for disparity gradients and occasionally even texture gra- nearly half the MSTd neurons.

Downloaded from physrev.physiology.org on October 9, 2008

dients, but to a lesser degree than for speed gradients. In anterior superior temporal polysensory area
Furthermore, the selectivity for the three cues combined (STPa), neurons are also selective for optic flow compo-
reflected largely that for the speed gradients (200; J. D. nents (10, 186), and a fraction of the population is selec-
Nguyenkim, unpublished data). Although the optimal tilt tive for three-dimensional structure from motion (9).
for the different gradients was not always congruent, These authors used a transparent sphere similar to the
Nguyenkim and DeAngelis (191) obtained evidence for hollow cylinder used in the perceptual experiments of
increased selectivity when cues were combined. Siegel and Andersen (278). While the sphere stimulus is
The selectivity of MT/V5 neurons for disparity gradi- characterized by second-order speed variations, it does
ents might be thought of being similarly surround-based not allow the parameters describing the surface to be
since MT/V5 neurons also exhibit surround effects in the easily manipulated. A number of STPa neurons responded
disparity domain (24). Nguyenkim and DeAngelis (190), at the onset of three-dimensional structure from motion,
however, provided evidence that they arose instead from and some of these were selective for the axis or rotation
heterogeneities in the receptive field. This finding further and were size invariant.
suggests that the selectivity for disparity gradients does
reflect spatial variations in position disparity rather than
orientation disparity. That the gradient selectivity for dis-
parity arises from the CRF, while that for speed arises
from interaction between the surround and the CRF might
reflect the well-documented difference in the coding of
speed and disparity at this level. Speed tuning in MT/V5
neurons is rather coarse (145, 174, 233), whereas that of
disparity is finer (46, 175). Hence, capturing the value of
the gradient may require a larger distance for speed than
for disparity, explaining the need to resort to interactions
between the surround and the CRF to extract speed gra-
dients. Indeed, as mentioned above, surrounds are on
average 3.3 times larger than the CRF in MT/V5 (242). It is
worth pointing out that in MT/V5 a columnar organization
has been described not only for direction of motion (2)
but also disparity (44). This organization favors the sort of
precise readout of disparity values required for building
gradient-sensitive hot spots in the CRF.
Finally, one study reported an impairment of struc-
ture-from-motion perception after a lesion of MT/V5
(8), indicating that this area is a critical component
FIG. 3. Extraction of moving contours (A) and intersections (C) in
of the three-dimensional shape-from-motion extraction V1 and their integration in MT/V5 (B and D). Hatching indicates RFs. The
pathway. image of the moving object is a diamond moving to the right.

Physiol Rev • VOL 88 • JANUARY 2008 • www.prv.org

66 GUY A. ORBAN

V. A PROPERTY ARISING FROM THE earity is attributed to the direction-selective V1 neurons,

COMBINATION OF INPUTS: PATTERN and supposedly reflects both nontuned and tuned compo-
DIRECTION SELECTIVITY OF MT/V5 NEURONS nents of normalization, the latter introduced by, e.g., a
surround. The linear operation is the combination (sum-
In principle, a small oriented receptive field such as ming) of excitatory and inhibitory inputs from these di-
that of a V1 neuron suffers from the aperture problem in rection-selective V1 neurons, whereas the second nonlin-
the sense that it can measure only the velocity component earity is due to the spike threshold of the MT neurons
orthogonal to its preferred orientation, which is compat- themselves. The most important factors determining pat-
ible with an infinite number of velocity vectors, as the tern direction selectivity are the pattern of excitatory and
parallel component of the velocity is unknown (Fig. 3A). inhibitory inputs, underscoring the importance of this
Images of real objects are generally bound by several mechanism, first proposed by Hubel and Wiesel (104) to
intersecting contours, and these intersections/corners account for orientation selectivity of V1 neurons. Notice,
provide additional information with which to determine however, that the modeling study (254) revealed that the
the velocity of the object motion. MT/V5 neurons have patterns of excitatory and inhibitory weights were equally
been shown to use both of the sources of information important, while initially only the pattern of excitatory
provided by images of moving objects: multiple contours LGN inputs had been emphasized. It is worth mentioning
(Fig. 3B) and the intersections between the contours that the success of the particular model developed by

Downloaded from physrev.physiology.org on October 9, 2008

(Fig. 3D). Rust et al. (254) does not need to imply that the particular
implementation of the components used in that model is
unique. For example, the first nonlinearity may arise from
A. Use of Multiple Contour Motion: Pattern synchronization between direction-selective inputs, as
Direction Selectivity has been documented for LGN input to simple cells (283),
while the second nonlinearity may partially reflect intra-
Movshon et al. (180) introduced plaid stimuli, made cortical inhibitory interactions from the surrounds of
by the superposition of two sinusoidal gratings angled 90 MT/V5 neurons (242), or other intracortical inhibitory
or 135° apart, to test the integration of multiple moving inputs.
contours. Testing these stimuli on 108 MT/V5 neurons,
they showed that while a large fraction of MT/V5 neurons
(⬃40%) signaled the direction of motion of each of the B. Use of Other Information: Moving Terminators
component gratings (component direction selective
cells), another group of ⬃25% of MT neurons was selec- Moving plaids contain only contour information. In-
tive for the direction of the pattern (pattern direction deed, Movshon et al. (181) have emphasized the impor-
selective cells), and a third remained unclassified. These tance of perfect superposition of the two gratings in the
proportions have remained remarkably stable in subse- plaids to create additive plaids. Failure to do so by using
quent studies (286, 295). In contrast, all V1 neurons were nonadditive plaids, in which the intersections have the
component direction selective or remained unclassified. same contrast as the contours, actually generates an ad-
This was later confirmed for V1 neurons identified as ditional low-contrast component at the intersections.
projecting to MT/V5 (182). Rodman and Albright (250) Hence, the plaid studies reveal only one aspect of the
showed that pattern and component direction selectivity solution to the aperture problem (Fig. 3, A and B). They
correlated with the angle between the preferred orienta- completely eliminate the contribution from two-dimen-
tion of a given MT neuron and its preferred direction of sional features of an image such as end points, corners,
motion. Recently, Smith et al. (286) studied the time and intersections which allow accurate two-dimensional
course of component and pattern direction-selective neu- velocity measurements (Fig. 3C). These features, called
rons’ responses in MT. Component cells had a 6 ms terminators (278), could also contribute to solving the
shorter average latency than pattern cells, and the selec- aperture problem, especially considering that a class of
tivity of the latter neurons took ⬃50 –75 ms to develop, V1 neurons, the end-stopped cells, are able to signal their
suggesting that the integration of two contours takes motion (216). To study the effects of terminators, Pack
time. and Born (213) introduced a field of short lines that
Rust et al. (254) tested MT neurons with a whole moved either at right angle to their orientation or at
family of plaid stimuli parameterized by the angle be- angles of 45°. In response to such a tilted stimulus, all MT
tween the component gratings. They showed that the neurons gradually altered their preferred direction over a
range of responses of MT neurons ranging from strong period of 75 ms to eventually coincide with that for or-
component selectivity to complete pattern selectivity thogonal stimuli. While the time course is similar to that
could be captured by a cascade model including two observed by Smith et al. (286) for pattern direction selec-
nonlinearities sandwiching a linearity. The initial nonlin- tivity, this pattern of behavior was observed in all MT

Physiol Rev • VOL 88 • JANUARY 2008 • www.prv.org

 HIGHER ORDER VISUAL PROCESSING 67

neurons, and not just in the 25% that were pattern direc-
tion selective. Pack et al. (211) confirmed this result with
nonadditive plaids, which also contain terminators. With
this stimulus, they obtained 60% pattern-selective neurons
and 6% component cells, probably reflecting the low con-
trast of the terminators. Since all studies of the Movshon
group were performed on anesthetized animals, they re-
corded from the same animals under anesthesia and ob-
served only 7% pattern-selective cells and 45% compo-
nent-selective cells. The latter result has been disputed,
however (181, 212). One possibility is that by using a
different anesthetic, isoflurane rather than the sufentanil
used by the Movshon group, Pack et al. (211) strength- FIG. 4. Types of visual signals sent by different laminae in V1 to
ened the GABA inhibition, which may have upset one or MT/V5.
several of the inhibitory mechanisms involved in the in-
tegration of the contour or terminator signals. Studies in
the LGN have indeed observed differences using the two of inputs from V1 to MT/V5. The terminator signals might

Downloaded from physrev.physiology.org on October 9, 2008

anesthetics (291). reach MT through the projection of layer 4B, in which
To study the integration of terminator signals with many of the neurons are end-stopped. The contour infor-
contour signals, Pack et al. (214) used barber pole-like mation might reach MT via layer 6, where many direction-
stimuli of which the elongation was manipulated. These selective neurons are end free (Fig. 4). It is worth noting
stimuli are created by showing a square-wave grating that anatomically, the projection from layer 4B dominates
inside a rectangular window at two different orientations. the inputs into MT. One would thus expect terminator
The results confirm that integration of terminator signals signals to dominate over contour signals. This has been
takes time. The steady-state angular deviations of the observed by monitoring the MT activity through smooth
preferred directions of MT neurons were very close to the pursuit eye movements, the control signals of which tran-
deviations predicted by the integration of the sole termi- sit through MT (84, 136, 137, 188). When monkeys track a
nator motion vectors for three different aperture geome- single bar whose orientation can be tilted compared with
tries. The deviations largely vanished when the straight- the direction of motion, smooth pursuit initially follows
edge aperture was replaced by an aperture with 0.4° the component predictions, but after 50 –100 ms gradually
indentations, a situation in which the barber pole illusion turns to the pattern prediction (23).
vanishes. The deviations in preferred direction induced by
the change in orientation of the rectangular window did
not depend on the size of the aperture nor on its position C. Limits of Motion Integration
in the receptive field. In conclusion, the use of terminator
vectors to compute object motion is useful only when the Integration of motion signals is relevant only when
terminators are intrinsic to the object (Fig. 3, C and D). In these signals belong to the image of the same object. In
the classical barber pole stimulus, terminators are intrin- both of the situations in which MT neurons integrate
sic. Under these conditions, the MT neurons seem mainly contour vectors or terminator vectors, this integration is
to integrate the terminator vectors at the expense of the reduced when there is sufficient visual evidence indicat-
contour vectors. It has been shown that the end-stopped ing that the contours or terminators belong to different
direction-selective neurons of V1 can provide the termi- objects. For the plaid stimuli, which test integration of
nator motion vectors to MT neurons. It has also been contours, the limit is the transparency of the motions of
shown that end-stopping takes time to develop (216), the two gratings. Any condition that promotes the impres-
which might at least partially explain the time that MT sion of transparency reduces the pattern component be-
neurons need to accurately signal the direction of motion havior in all MT neurons (295). For the barber pole stim-
of the pattern-containing terminators. The remainder uli, the introduction of an occlusion cue indicates that the
probably reflects the integration of the V1 signals, since grating is located behind the aperture, making the termi-
the indentation experiment (214) shows that the scale of nators extrinsic (or accidental). To make the terminators
these V1 signals is much smaller than the RF size in MT/V5 extrinsic, Pack et al. (214) surrounded the aperture by a
neurons. Indeed, it has recently been shown that in IT, bright frame which introduces an occlusion cue and re-
nonlinear integration of afferent signals develops over a duced the barber pole illusion. This manipulation greatly
similar time scale (34). reduced the deviations in the preferred directions but did
The two mechanisms of motion integration are not not abolish them. To contrast the effect of extrinsic and
incompatible and might, in fact, depend on the two types intrinsic terminators, they placed parallel bars next to the

Physiol Rev • VOL 88 • JANUARY 2008 • www.prv.org

68 GUY A. ORBAN

aperture and changed the orientation of these flanking particularly for small stimuli, confirming this subregion’s
bars. This change shifted the preferred directions of the role in the analysis of visual trajectories and the genera-
MT neurons as predicted, although not completely. The tion of pursuit. Recent data have extended the scope
effect was stronger than that obtained by Duncan et al. of this MSTv functionality to include the control of arm
(60), who introduced occlusion by a stereo cue on oppo- trajectories (112). Recent imaging data (186) have shown
site sides of a square aperture. This weaker effect may be that FST reacts very differently from the MST regions and
due to the larger distance between the occlusion cue and that FST could be at the origin of an action-processing
the aperture in the Duncan et al. experiment, in which the pathway veering off ventrally into the STS and involving
occlusion cue could act only through the surround of the newly defined/recognized regions LST and STPm. Thus
MT/V5 neurons. It is interesting to note that MT/V5 re- MT dispatches motion signals to its three satellites (Fig. 5)
ceives afferents not just from V1, which is the major input, for further processing of self motion (MSTd), trajectories
but also from V2 and V3 (21). We will see that neurons in of moving objects (MSTv), and actions/motion of animate
V2 are sensitive to occlusion cues and might contribute to entities (FST). In this review we concentrate on MSTd,
the ordering of stimuli in depth (see below). Input from which has been far better investigated that the other two
these early extrastriate areas might provide the MT neu- regions.
rons with the signals required to restrict integration of
motion signals to those belonging to the same object

Downloaded from physrev.physiology.org on October 9, 2008

(Fig. 4). A. Selectivity of the MSTd for Optic Flow

The seminal observation by Tanaka’s group (255)

VI. OPTIC FLOW COMPONENT SELECTIVITY IN
that a number of MSTd neurons are selective for expan-
THE MT/V5 COMPLEX AND BEYOND
sion/contraction or for rotation but do not respond to
translation was the starting point of the research into the
The signals from MT/V5 are sent in parallel to a
role of MSTd in the processing of optic flow (for review of
number of neighboring regions. Initially, Ungerleider and
this early work, see Ref. 199). Next it became clear that
Desimone (321) distinguished two such regions, the me-
MSTd cells selective for only expansion/contraction or
dial superior temporal visual area (MST) and the floor of
rotation were in the minority and that most MSTd cells
the superior temporal visual area (FST). Subsequently,
were selective for multiple flow components (55, 83, 144).
Komatsu and Wurtz (136) proposed a further distinction
Since expansion/contraction and rotation are basically
between dorsal and ventral MST, the latter being involved
spatial configurations of local translations, it became es-
in smooth pursuit. This distinction was further supported
sential, for any meaningful analysis of those MSTd neu-
by the work of Tanaka et al. (310), reporting that optic
rons selective for translation, to define a criterion allow-
flow selectivity was observed mainly in the dorsal part,
ing one to determine whether, in addition to their trans-
implying a role in the analysis of self-motion. Neurons of
lation selectivity, they are selective for expansion/
ventral MST (MSTv) were more selective for translation,
contraction or rotation. Position invariance, tested over a
wide region of the visual field encompassing the RF for
translation, is such a criterion (144). If the neuron is not
selective for the flow component but only for translation,
the spatial response map for the flow component will be
located on either side of the translation RF for the two
directions of the flow component. In contrast, a neuron
selective for a flow component will have a spatial re-
sponse map for only one direction of the flow component,
and this map will overlap the translation RF. It should be
noted that position invariance requires explicit testing; a
large RF does not in itself guarantee position invariance.
Furthermore, the position invariance criterion draws a
sharp distinction between MT/V5 neurons that have no
selectivity for radial motion or rotation and MSTd neu-
rons which can be selective for these higher order mo-
tions. This criterion established that selectivity for radial
motion or rotation is a novel type of selectivity emerging
at the level of MSTd (144). Given that selectivity was
FIG. 5. Different types of motion signals processed by MT/V5 and its observed for two of the first-order components of optic
satellites. flow, expansion/contraction and rotation, the question

Physiol Rev • VOL 88 • JANUARY 2008 • www.prv.org

 HIGHER ORDER VISUAL PROCESSING 69

then arose as to whether MSTd decomposes optic flow. inhibitory inputs are involved is not yet clear, how-
The answer to this question was negative, since very few ever (56).
neurons are selective for the third component of flow,
deformations (144), and because the response of an MSTd
neuron decreases when the component of flow for which B. Selectivity of MSTd Neurons for
the neuron is selective is mixed with increasing amounts Heading Directions
of a different component (209). The MSTd neurons do not
calculate derivatives of flow but signal how well the flow The visual processing of MSTd neurons can be sum-
present on the retina matches their preferred flow com- marized as a template matching with radial motion, rota-
ponent or mixture of components. tion, and translation, or their combinations. This paved
Generally, the RFs of MSTd neurons are described as the way for a radical change in the investigation of MSTd
very large and difficult to map even in awake animals. processing, initiated by Duffy and Wurtz (57): testing the
Quantitative measurements of the sizes of these RFs have selectivity for heading direction specified by the position
shown that they are indeed large, but do not span the of the focus of expansion (FOE). Initially the position of
entire visual field (163, 241). This is important, otherwise the FOE was varied in the fronto-parallel plane (57, 147,
translation-selective neurons could not contribute to 217, 218). Only recently was a variation of the FOE posi-
the determination of heading direction (see below). The tion in the horizontal plane added (163), which is essential

Downloaded from physrev.physiology.org on October 9, 2008

Raiguel et al. (241) study also showed that the ipsilateral for capturing the role of cells selective for radial motion.
representation of the visual field was much more exten- Indeed, if the FOE varies only in the fronto-parallel or
sive in MSTd than in MT/V5 and that RF size did not vertical direction, the problem can be solved with only
change with eccentricity in MSTd. MSTd neurons are translation-selective neurons (147). So far, only one study
broadly sensitive to the speed of radial motion or rotation has completely investigated heading in three-dimensional
(208, 308) and exhibit strong spatial summation, with space using 26 directions of heading (88). Remarkably,
this study reveals that nearly all MSTd neurons (251/255)
antagonistic surrounds being less frequent than in MT/V5
are selective for heading in three dimensions (Fig. 6A)! As
(56, 144, 308). Tanaka and Saito (308) and Geesaman and
far as we are aware, no effort has been made to investi-
Andersen (78) showed that the selectivity for radial mo-
gate the effects of speed along these 26 directions in
tion, rotation, or translation of MSTd neurons did not
space. The second important finding of this study is that
depend on the carrier of the motion, whether it consisted
all directions of heading in space are equally represented
of random dot patterns, windmills, rings, squares, or even
(Fig. 6B): there is no preference for straight ahead trajec-
non-Fourier stimuli. In the same vein it has recently been
tories or trajectories along the ground plane. This may
reported that MSTd neurons even respond to Glass pat-
seem surprising, but only because we tend to think of
terns evoking the perception of rotation (141).
monkeys as moving around as humans do. In fact, they
In a very careful study, Tanaka et al. (306) investi- behave very differently insofar as much of their locomo-
gated the different cues present in flow stimuli and con- tion consists of jumping and moving about in the treetops
cluded that the selectivity of MSTd neurons for radial and rather than walking or running on the ground. Once one
rotatory motion reflects the spatial pattern of translations recognizes that, for the life-style of monkeys, an assess-
present in these motion stimuli. Indeed, a set of eight ment of self motion in all directions of space is useful, the
translations in appropriate positions are undistinguish- role of MSTd neurons selective for rotation or mixtures
able from the real rotation or radial motion for MSTd containing rotation becomes clearer. These neurons
neurons. The selectivity of MSTd neurons for optic flow might analyze rotation around the axis of heading, which
might therefore arise from a combination of excitatory itself can take any direction in space. Such an analysis is,
MT inputs. These inputs not only have to be direction of course, very different from that of rotation with respect
selective but must arise from neurons with RFs at the to gravity or space, which is analyzed by the vestibular
appropriate locations. Thus, to build on the work of Rust system.
et al. (254) and Brincat and Connor (33), it may well be Heading direction is an instantaneous measurement;
that a NL/L/NL cascade model also applies to flow selec- its integration yields information about the path followed.
tivity in MSTd neurons. Such a model generates some An initial study found little effect of temporal integration
degree of position invariance (33), and it may not be in the selectivity of MTSd neurons (219). Yet using a more
necessary to repeat the configuration of excitatory inputs, natural stimulation, consistent with following the same
as initially suggested by Tanaka et al. (306). In the in- rotatory path in opposite directions, Froehler and Duffy
stance of the MSTd neuron model, the initial nonlinearity (72) showed that a number of MSTd neurons are selective
could easily be provided by the suppressive surrounds of for the path followed by the subject. Finally, heading
the MT/V5 neurons, which are particularly abundant in discrimination is impeded by the presence of a moving
the superficial layers of MT/V5 (242). To what extent object. Logan and Duffy (163) investigated the heading

Physiol Rev • VOL 88 • JANUARY 2008 • www.prv.org

70 GUY A. ORBAN

FIG. 6. Selectivity of the dorsal part of the me-

dial superior temporal visual area (MSTd) neurons
for heading direction in 3 dimensions. Elevation and
azimuth describe all directions in space. A: response
of 3 MSTd neurons as a function of elevation and
azimuth. B: preferred directions of headings of all
MSTd neurons studied. [Modified from Gu et al. (88),
copyright 2006 by the Society for Neuroscience.]

Downloaded from physrev.physiology.org on October 9, 2008

coding of MSTd neurons for optic flow stimuli alone, a neurons are influenced by pursuit (26, 217, 218, 276) and
moving object alone, and the combination of the two that, in general, the compensation for the shift of the FOE
when the object moved with the flow or in the opposite by pursuit is not complete, even if it is improved by the
direction. While MSTd neurons could derive the heading presence of multiple depth planes (323).
from the moving object responses, the combination of the Similarly, the integration of visual and vestibular sig-
two stimulations, when in agreement, was largely domi- nals in MSTd is also beyond the scope of this review.
nated by the optic flow signals. Yet when the object MSTd receives vestibular input (32, 54), which may be
moved in directions opposite to the optic flow, the head- stronger in the postero-medial part of MSTd (88). The role
ing signals arising from the MSTd neurons were strongly of this combination of signals might be more about the
reduced, indicating strong interactions between the two conversion of heading signals into a head-centered refer-
types of signals. This result also shows that the presence ence (88) than the generation of a robust heading repre-
of moving object responses in MSTd does not necessarily sentation (218). The combination of these different sig-
contradict the concept of an MSTd specialized for pro- nals in MSTd has triggered considerable interest in mod-
cessing self-motion (Fig. 5). eling. One view is that MSTd neurons are actually basis
functions useful for representing multiple variables: the
FOE in different coordinates, eye motion, and position
C. Mixing of Visual With Vestibular and Pursuit (18). To what extent these signals are used in the brain is
Signals in MSTd: Out of Scope unclear. Far greater attention to the output of this region
will be necessary to understand the computations in
If the heading direction problem is to be solved by which MSTd is involved, one of these likely being that of
using the location of the FOE, this necessarily implies that self motion.
neurons involved in heading have to be influenced by
pursuit eye movements. Indeed, it is well known that the
FOE is a function of both the heading and the pursuit eye D. Optic Flow Selectivity of Other
movements, and extraretinal signals have also been im- Extrastriate Regions
plicated in heading during pursuit (15, 338). A complete
review of these studies is beyond the scope of this review. MSTd projects to a number of other extrastriate ar-
Suffice it to say that there is ample evidence that MSTd eas, including STPa, in the upper bank of the anterior STS,

Physiol Rev • VOL 88 • JANUARY 2008 • www.prv.org

 HIGHER ORDER VISUAL PROCESSING 71

VIP in the intraparietal sulcus, and 7a, the posterior part more particularly the outline of the object in the image,
of the IPL. Optic flow selectivity has been observed in all i.e., the external contour of the object. Many IT neurons
three areas. Recording in STPa, Anderson and Siegel (10) encode simple shapes whether defined by motion, texture
observed a substantial fraction of neurons selective for difference, or luminance difference (264). For a number
optic flow, especially expansion. Only a small proportion of IT neurons, this invariance extends to orientation se-
of these neurons were responsive to translation. It re- lectivity as measured with gratings defined by this same
mains unclear whether the region from which they re- set of cues (263). It was subsequently shown that IT
corded corresponds to STPm defined by Nelissen et al. neurons are also selective for shapes defined by disparity
(186) by, amongst other fMRI criteria, its responsiveness differences (304). It is important to note that the differ-
to optic flow. ences in disparity or texture, used in these studies (264,
MSTd and MT/V5 project to VIP in the fundus of the 304), are steep steplike changes or discontinuities and are
intraparietal sulcus. The properties of the neurons in VIP very different from the smooth changes in those features
are surprisingly similar to those of MSTd (30, 267). In fact, that constitute cues to the three-dimensional shape of
the selectivity of VIP neurons for heading is also similar to objects, i.e., the curvature or orientation of their surfaces
that of MSTd neurons (349). VIP also receives vestibular in three dimensions. These discontinuities generally
inputs (31). Two differences are noteworthy. First, com- evoke a sharp impression of a contour, although physi-
pensation for pursuit seems more complete than in MSTd, cally no contour is present in the image. Thus these

Downloaded from physrev.physiology.org on October 9, 2008

suggesting that heading might be coded in head-centered contours are created by the brain very much as color is.
coordinates (349). VIP is also more multimodal than The computational importance of this cue convergence is
MSTd, as it also receives somatosensory input (11, 59) the distinction between internal and external contours of
and auditory input (272). objects. Indeed, at the edge of an object, i.e., its external
Neurons in area 7a are also selective for optic flow contours, it is likely that many aspects of the image
(179, 244, 256, 257, 280, 294). Phinney and Siegel (228) change: luminance and color, texture, depth, motion, etc.
have reported interactions between speed and optic flow In that sense the extraction of such nonluminance-defined
selectivity in 7a. Steinmetz et al. (294) and Merchant et al. contours is essential for delineating the objects and thus
(176) reported that 7a neurons are also especially sensi- represents a first step in figure-ground segregation. On the
tive to expansion stimuli. other hand, internal contours corresponding to surface
PEc neurons on the medial side of the superior pari- markings or shadows will exhibit mainly luminance changes
etal lobule are also responsive to optic flow stimuli (240), without changes in depth, motion, or texture. Thus shape-
but this region, part of area 5, is only indirectly linked selective IT neurons onto which different cues converge
with MSTd and VIP. PEc is an area involved in the control are likely to encode the shape of objects or object parts.
of arm movements (66). An association between radial This in turn poses the question as to which of the levels
flow and the control of arm trajectories was suggested preceding IT first gives rise to this selectivity for nonlu-
earlier by Steinmetz et al. (294). Likewise, VIP projects to minance-defined contours. In principle, this could be V1,
area F4 (249), an area involved in the planning of arm but this possibility is generally considered unlikely, since
movements. STPa is responsive to the observation of selectivity for many of the features upon which these
actions, including hand actions (225). Thus it may well be nonluminance-defined contours are based, first emerges
that selectivity to optic flow does not necessarily reflect only at the level of V1. A notable exception might be
involvement in heading and self-motion processing. contours defined by temporal texture (37), as temporal
frequency influences geniculate neurons.
VII. SEGMENTATION MECHANISMS:
FIGURE-GROUND SEGREGATION AND 1. Illusory contours
DEPTH ORDERING IN AREAS V2, V4, (anomalous or subjective contours)
AND MT/V5
The reports of von der Heydt and co-workers (226,
334, 335) drew attention to early extrastriate cortex, par-
A. Distinction Between Internal and External ticularly V2, for its role in the processing of nonlumi-
Contours of Objects: Selectivity for nance-defined contours. These authors showed that V2
Nonluminance Defined Boundaries neurons were selective for the orientation of illusory con-
tours, that this selectivity was similar to that for lumi-
Far in the ventral pathway, in area TE, the anterior nance-defined stimuli, and that the response to illusory
part of infero-temporal cortex, neurons are selective for contours could not be explained by any spurious stimu-
two-dimensional shape. It has been shown that at this lation of the RF. They studied two types of contours:
high level, neurons are invariant for the cue defining the those arising from abutting gratings and those bridging
shape. Shape here refers to two-dimensional shape and gaps as in the Kanisza triangle. The first type can be seen

Physiol Rev • VOL 88 • JANUARY 2008 • www.prv.org

72 GUY A. ORBAN

as a discontinuity in texture reflecting an object bound- gratings, while severely impairing the discrimination of
ary. The second type corresponds to a discontinuity in small differences in motion direction.
luminance induced by the figural elements (the inducers) Very few V1 neurons and only a small proportion
at the ends of the contour. Recently, it has been shown (13/113) of V2 neurons were selective for kinetic bound-
that cortical neurons (V2 neurons more than V1 neurons) ary orientation when tested using kinetic gratings in
can even signal illusory contours defined by a step in which motion was either orthogonal or parallel to the
disparity between two surfaces of equal luminance (99) or grating orientation (168). In V4, selectivity for kinetic
illusory contours defined by placing an occluder in the boundary orientation was more evident, as there were
near plane (14). Peterhans and von der Heydt (226) pro- more neurons selective for this attribute in proportion to
posed that the selectivity for illusory contours in V2 arose the number of neurons selective for luminance grating
by pooling appropriate signals from V1 and V2 end- orientation, even though the overall proportion was still
stopped neurons (98, 100) and combining them with those small (52/482) (184). As was the case for illusory con-
from end-free V1 neurons, an operation reminiscent of tours, the higher order (V4) neurons had shorter latencies
that for the integration of motion signals in MT (Fig. 4). In than lower order (V2), suggesting that the kinetic re-
V2, neurons selective for illusory contours are located sponses in V2 might represent feedback signals from V4.
mostly in the pale but also in the thick stripes (98, 227). It is difficult to compare latencies across studies, since
In these initial studies, virtually no V1 cell was found small changes in stimulus parameters may change the

Downloaded from physrev.physiology.org on October 9, 2008

to be selective for illusory contours. Subsequent studies latency values. However, the latency differences between
have observed small fractions of V1 neurons selective for V2 and V4 are differences in relative latencies, taking
illusory contours, however. In some of these studies, it latency for luminance-defined stimuli as the reference. In
could be argued that luminance cues were present in V2, the latency of response to kinetic boundaries is 50 – 60
some of the stimuli used (17, 85), but in others that was ms longer than that to luminance-defined boundaries in
kinetic orientation-selective neurons. In nonselective neu-
not the case (150, 243). Alternative explanations might
rons, this differences was reduced to 20 –30 ms, and the
relate to laminar positions in V1 (150), since end-stopped
latency of kinetic and uniform motion responses was
neurons are more frequent in superficial layers (98). Fur-
similar (168). In V4, the response latency for kinetic grat-
thermore, it has been argued that the latency of responses
ings was only 25–30 ms longer than that for luminance
to illusory contours is longer in V1 than in V2 and that the
defined gratings, was not different for selective or nonse-
V1 responses represent feedback from V2 (150). Whether
lective neurons, and was similar to that for uniform mo-
this feedback is required for the emergence of the percept
tion (184). These latency data are compatible with the
of a sharp contour is unclear. It might simply be intended
following hypotheses: 1) responses to kinetic patterns in
to keep V1 and V2 in register. It has also been suggested nonselective neurons reflect motion responses in V2 and
that illusory contours deactivate a number of V1 neurons, V4; 2) responses in kinetic selective and nonselective V4
a proposal intended to disentangle responses to real and neurons are assembled in parallel from the V2 motion
illusory contours (243). input, according to different combination rules (see be-
low); and 3) the kinetic selective V2 neurons receive their
2. Kinetic and other contours inputs from selective V4 neurons. Furthermore, it is im-
portant to notice that the kinetic orientation-selective V4
Kinetic contours are created by differences in the neurons exhibit three key properties required for the
direction or speed of motion between two abutting ran- representation of cue invariant boundaries: 1) orientation
dom dot fields. Such contours, although generated by selectivity for an impoverished stimulus that 2) was in-
motion, do not themselves move. Originally it was thought variant for changes in the carrier, in this case direction of
that orientation selectivity for such stimuli would arise in motion, and that 3) matched the selectivity for orientation
MT/V5, but MT/V5 neurons are not selective for the ori- of luminance-defined stimuli. It is noteworthy that a small
entations or positions of kinetic boundaries (169). This number (9/452) of V4 neurons were selective for the ori-
result was obtained by testing MT/V5 neurons with edges entations of kinetic grating but not luminance gratings.
and gratings in which motion was either orthogonal or Leventhal et al. (156) observed V2 neurons that were
parallel to the boundary orientation. Results were very orientation selective for texture-defined boundaries. For
similar for gratings and edges. MT/V5 neurons signal only this type of boundary, as for illusory contours, the pre-
the local motion present in this spatial pattern of transla- ferred orientation and tuning widths were similar to those
tions, as is the case for flow components (144). The lack obtained with luminance-defined stimuli. In addition, the
of MT/V5 involvement in kinetic boundary processing was authors showed that the gain of the tuning function in-
confirmed by the lesion study of Lauwers et al. (149). creased when the saliency of the border increased and
Lesions of MT/V5, even large ones, little affect the dis- when a subliminal luminance-defined bar was added to a
crimination of small differences in orientation of kinetic weak texture-defined bar. Von der Heydt et al. (336) re-

Physiol Rev • VOL 88 • JANUARY 2008 • www.prv.org

 HIGHER ORDER VISUAL PROCESSING 73

ported that a fraction of the V2 neurons is tuned to the internal markings, it has only solved half the problem of
orientation of disparity-defined edges. These authors figure-ground segregation, i.e., to determine which parts
compared responses of V2 neurons to optimally oriented in the image correspond to the figure (object image) and
luminance-defined and disparity-defined squares at differ- which correspond to the background. Indeed, at the level
ent positions. They observed edge-selective responses in of V4, the visual information is still carried by local RFs
most V2 neurons for both types of figures and observed a that process only part of the object boundary. It has been
correlation between the preferred orientations of the two shown that even V1 neurons respond to the presence of a
types of edge responses: the mean difference between the figure over their RF (146, 150, 351; but see Ref. 252). V2
preferred orientation was only 2.7° (9.2° SD) for the seven neurons respond similarly, and in either area this re-
neurons tuned for orientation with both types of stimuli. sponse is little affected by attention (170). These re-
To further test the proposition that these cyclopean edge- sponses, which need time to develop (between 60 and 150
selective responses represent a cue invariant boundary ms in Marcus and Van Essen, Ref. 170), have been taken
signal, Bredfeldt and Cumming (28) tested V2 neurons as evidence that even V1 neurons contribute to the seg-
with single disparity edges at various positions and ori- mentation of figure from background, although these
entations for both signs of the edge, along with uniform- long-latency signals most likely reflect feedback from
disparity random dot stereograms. In these tests, V2 neu- higher order areas. Whether these responses provide in-
rons’ orientation selectivity for disparity steps, although dications that V1 and V2 neurons process object surfaces

Downloaded from physrev.physiology.org on October 9, 2008

broadly correlated with that for luminance-defined stim- (encompassed by the object boundaries) is less clear.
uli, is not as selective as orientation selectivity for lumi- This question can be addressed by testing the effect of
nance stimuli. More importantly, the disparity edge re- presenting a figure at different positions relative to the
sponses frequently (⬎50%) originated from different loca- RF, as done by von der Heydt et al. (336) for disparity-
tions in the RF and often (again ⬎50%) the orientation defined figures and Friedman et al. (71) for color figures.
selectivity depended on the choice of the disparities de- These tests have revealed that the vast majority of V1 and
fining the edge. Bredfeldt and Cumming (28) conclude V2 neurons signal the presence of the edge of the figure
that cue invariance is not achieved at the level of V2 and (Fig. 7A) and only ⬃20% the presence of the figure itself.
that this first step towards invariance can be accounted The latter neurons may contribute to the analysis of ob-
for by feed-forward projections using the appropriate ject surfaces by signaling, e.g., their color and/or lumi-
combination of excitatory inputs. A similar scheme has nance. This is in marked contrast to findings at a much
been proposed for V2 neurons by Ito and Komatsu (113) higher level. In studying three-dimensional shape ex-
to explain corner responses and by Orban and Gulyas tracted from disparity in IT neurons, Janssen et al. (115)
(203) to explain selectivity in cortical neurons for kinetic observed equal numbers of neurons signaling the depth
boundary orientation. Because similar proposals have structure of boundaries and surfaces.
been made for illusory contour orientation selectivity (98, Further experiments, however, have indicated that in
100), it may be that once again a cascade model, in which V2, even edge cells can provide information about which
two nonlinearities sandwich a linear combination of in- side of the boundary the actual figure is on. By manipu-
puts, is applicable to these selectivities in V2 and V4 lating the contrast polarity of edges and their inclusion in
neurons. a square figure, Zhou et al. (350) demonstrated that a
In conclusion, there is mounting evidence that as the number of early extrastriate neurons can signal the side
visual message travels from V1 to V4 over V2, cue invari- on which the figure is located with respect to the edge
ant boundary representations gradually emerge, perhaps (Fig. 7B). Such neurons were observed in V2 and V4, but
earlier (that is, in V2) for illusory and texture-defined far fewer were found in V1 (Fig. 7C). A number of these
boundaries and rather late (in V4) for kinetic and perhaps, neurons can additionally signal the luminance/color of a
disparity-defined edges. For illusory contours as well as figure for which they signal the location with respect to
for kinetic contours, it has been proposed that the selec- the RF (top right corner in squares of Fig. 7C). It is
tivity observed at lower levels (V1 for illusory contours noteworthy that these border ownership signals emerge
and V2 for kinetic contours) reflects feedback signals. rapidly, ⬃10 –25 ms after response onset, much faster
than the figure-ground signals recorded by Marcus and
Van Essen (170). Thus, even signals that require integra-
B. Segmentation or Depth Ordering in Static tion over long distances, far beyond the classical RF (see
Images: Border Ownership and review in Albright and Stoner, Ref. 4) may emerge rela-
Surface Representations tively quickly. In fact, surround effects have a relatively
similar time course (20 –30 ms after response onset, Ref.
Even if, at the level of V4, the brain manages to 12). In a subsequent study, Qiu and von der Heydt (239)
distinguish between the external contours or boundaries confirmed the presence of boundary ownership signals in
of objects and internal contours reflecting shadows or V2. Indeed, if these V2 neurons signal that a figure is

Physiol Rev • VOL 88 • JANUARY 2008 • www.prv.org

74 GUY A. ORBAN

FIG. 7. Edge neurons and border ownership selec-

tivity. A: response of a surface and an edge neuron (V2)
as a function of the position of a square figure. [From
Friedman et al. (71), with permission from Blackwell
Publishing.] B: schematic indication of four types of
neurons (stripes indicate RF) signaling the direction of
the figure with respect to the edge (b) or not (a) and
signaling the polarity of the figure (c) or not (d). C:
distribution of contrast polarity discrimination (c-d)
and side of ownership discrimination (b-a) in V1, V2,
and V4. [Modified from Zhou et al. (350).]

Downloaded from physrev.physiology.org on October 9, 2008

present on a given side of the edge, then these neurons (140), where the addition of disparity is not necessary for
should also respond to the presence, in stereograms, of neurons to complete the shape.
“near” figures on the same side. By systematically com-
paring responses to figural occlusion and disparity cues, C. Segmentation of Moving Planes in MT/V5
these authors showed that this convergence indeed oc-
curred in a significant number of V2 neurons, but only Objects are generally opaque; thus occlusion is the
rarely in V1. rule between objects at different depths. A moving object
So far, most studies addressing the ordering of sur- near the observer will therefore dynamically occlude
faces in depth have studied the simple situation of a single other objects at greater distance from the observer.
figure on a background, with three notable exceptions. Hegde et al. (93) have suggested in a psychophysical
Zhou et al. (350) showed that about half the edge cells in study that second-order or non-Fourier motion stimuli
V2 and V4 (plus a small proportion in V1) that signal such as contrast modulated moving stimuli may signal
border ownership for a single figure can also signal border dynamic occlusion. Thus the response of MT/V5 neurons
ownership for overlapping figures. Bakin et al. (14) to these non-Fourier motion stimuli (3, 196) might signal
showed that the neural basis for contour completion, that the dynamic occlusion of one object by another, even
is, the facilitation of neural responses to stimuli located when the objects are not distinguished from one another
within the RF by contextual lines lying outside the RF, is by luminance differences.
blocked by an orthogonal line intersecting the contour, Full transparency is relatively rare in natural scenes.
but is recovered when the orthogonal line is placed in a Exceptions include shadows and to some extent foliage,
“near” depth plane. This recovery was observed more especially fine foliage. To disentangle moving shadows
frequently in V2 than in V1. Sugita (299) showed that V1 from moving objects, the visual system should be able to
neurons do not respond to an optimal moving bar when it process transparent motion signals. Snowden et al. (288)
is partially occluded by a small patch. Response was compared responses of V1 and MT/V5 neuron responses
restored by adding crossed disparity to the patch so that to moving random dots or to transparent motion in which
it appeared to be in front of the bar, while adding un- two sets of random dots moved in opposite directions. V1
crossed disparity had no effect. Notice that this type of neurons responded equally well to random dot and trans-
completion is very different from that observed in IT parent motion, while MT/V5 neurons responded less

Physiol Rev • VOL 88 • JANUARY 2008 • www.prv.org

 HIGHER ORDER VISUAL PROCESSING 75

strongly to transparent motion than to the random dot near disparities (103), which might relate to the process-
motion. The inverse relationship found between direction ing of objects segregated from the background. This lower-
selectivity and response to transparent motion suggested order disparity selectivity in V1, V2, and V4 neurons sig-
that it was the inhibition responsible for suppressing re- nals only position in depth (but see below for V4 neu-
sponses in the nonpreferred direction that decreased the rons). Higher-order disparity refers to gradients of
response to the transparent motion. Indeed, this inhibi- disparity which signal orientation or curvature in depth.
tion was stronger in MT/V5 than in V1. These authors Neuronal selectivity for higher order disparity has been
noted that for the same reason, MT/V5 neurons responded documented mainly in two cortical regions: the caudal
less strongly to kinetic gratings containing opposed mo- part of the lateral bank of IPS, CIP, explored by Sakata,
tion in segregated bands than to uniformly moving ran- Taira, and colleagues, and a small region in lower bank of
dom dot patterns. This agrees with the findings of Marcar the STS, TEs, explored by our group (for review, see Refs.
et al. (169) that MT/V5 neurons do not signal kinetic 204, 260).
boundary orientation (see above). The distinction be-
tween V1 and MT/V5 was further explored by using paired
and unpaired dot patterns (238). Both types of stimuli A. Higher Order Disparity Selectivity in TEs, Part
include dots moving in opposite directions, but the former of the Infero-Temporal Complex
is locally balanced and appears to flicker, while the latter

Downloaded from physrev.physiology.org on October 9, 2008

is unbalanced and gives the impression of two transparent The Janssen et al. (117) study, reporting that a frac-
surfaces. While V1 neurons respond equally to these two tion of IT neurons were selective for three-dimensional
types of stimuli, MT/V5 neurons respond far less well to shape defined by disparity, was not only the first study to
the paired than to the unpaired dot patterns. This sug- report selectivity for second-order disparity stimuli, but it
gested that in MT/V5, the second step in motion process- was also the first to report disparity selectivity as such in
ing following V1, suppression occurs when locally differ- the ventral stream. Indeed, stereo has been classically
ent directions of motion are present in the image. The aim associated with the dorsal stream (322, 328), although
of this suppression is to reduce the response to flicker, a lesion studies had indicated some involvement of the
process that is incomplete in MT/V5 but continues in ventral stream in stereoscopic processing (38, 237, 268).
MSTd (144). Subsequently, Bradley et al. (27) reported Many subsequent studies have confirmed that stereo is
that the suppression in a transparent display could be processed in the ventral stream (95, 96, 101–103, 302–304,
decreased by introducing disparity between the two sets 319, 320,339). In these studies of the Leuven group, ver-
of moving dots. Thus MT/V5 neurons, while rejecting tical disparity gradients were imposed on textured sur-
motion noise (flicker), can still represent transparent sur- faces included in relatively complex outlines, about 5° in
faces at different depths. While this does not apply to diameter. Higher order selectivity was demonstrated in IT
moving shadows, it might be useful for discerning a mov- neurons by showing that the selectivity for curved sur-
ing object through moving foliage. In general, this prop- faces of opposite sign (convex and concave) did not
erty will be useful in cluttered dynamic scenes since it depend on the position in depth of the surfaces. The use
might resolve the three-dimensional structure of such of position invariance as a criterion is reminiscent of the
scenes. This property of MT/V5 neurons has been used in test used by Lagae et al. (144) to demonstrate higher order
a depth-ordering task (25, 87), sometimes referred to a motion selectivity in MSTd neurons.
structure from motion task. In fact, depth ordering is Subsequent studies (118) indicated that neurons se-
indeed useful in case of partial occlusions, typical of lective for three-dimensional shape defined by disparity
cluttered scenes. were not scattered throughout IT, but were concentrated
in a small region in the rostral part of the lower bank of
the STS. This region, labeled TEs (119), houses many
VIII. STEREOSCOPIC PROCESSING: neurons selective for three-dimensional shape from dis-
THREE-DIMENSIONAL SHAPE SELECTIVITY parity, in contrast to the convexity of IT. The two parts of
IN FAR EXTRASTRIATE CORTEX IT also differ in their degree of binocular summation,
which is stronger in TEs than in lateral TE (118). Since the
Neurons in V1 are selective for horizontal disparity, anatomical connectivity of this lower STS region is also
but this is a selectivity for absolute disparities (42), sig- different from the remainder of the convexity (165, 261),
naling only position in depth relative to the fixation point. Janssen et al. (118) proposed that TEs is a separate cor-
In V2, a fraction of the neurons are selective for relative tical region linked to the IPS.
disparity (311), signaling position in depth with respect to Finally, TEs neurons were shown to be endowed
another plane. Such signals probably underlie the preci- with another higher order property that had been fre-
sion of stereoacuity. Neurons in V2 are also subject to quently postulated but never observed: the rejection of
disparity capture (14). V4 neurons are often selective for false matches such as those in anticorrelated stereograms

Physiol Rev • VOL 88 • JANUARY 2008 • www.prv.org

76 GUY A. ORBAN

(116). In contrast to V1 neurons (41), TEs neurons, which not reverse at any position in depth. This criterion sup-
are selective for three-dimensional shape depicted by cor- poses that vergence eye movements by the monkey are
related random dot stereograms (RDS), do not respond negligible. Generally, the position of only one eye was
selectively to anticorrelated RDS. In this respect anticor- recorded, but Janssen et al. (119) have shown that this
related RDS are similar to decorrelated RDS, which also suffices to detect vergence eye movements, provided
evoke no differential responses from TEs neurons. Thus, enough trials are averaged. Furthermore, a number of
at the level of TEs, the so-called “stereo correspondence higher order neurons were recorded while the positions
problem” (171) is solved. This need not imply that it has of both eyes were monitored (115) and the absence of
not already been solved at some earlier level. Recent vergence eye movements directly demonstrated. Thus
results suggest that the false matches are greatly reduced these studies confirmed the validity of our definition of
in V4 (302), but not in V2 (7). higher order neurons. In the initial study (117), this crite-
rion was implemented by the requirement that the re-
B. Exquisite Coding of Three-Dimensional Shape sponse to the preferred shape at its optimal position
From Disparity by TEs Neurons should exceed the response to the nonpreferred shape at
any position. Subsequently (119), this requirement was
In their initial studies, Janssen et al. (117, 118) em- quantified by an index comparing the best position for the
nonpreferred shape to the worst position of the preferred

Downloaded from physrev.physiology.org on October 9, 2008

phasized the selectivity of TEs neurons for second-order
disparity stimuli. In fact, TEs houses neurons selective for shape. The ratio of these responses did not exceed a
all three orders of depth signaled by disparity. Figure 8 factor of 2 in higher order neurons and was generally
shows an example of zero-order, first-order, and second- smaller than 1.5. For the cell in Figure 8A, the ratio
order disparity-selective TEs neurons. The defining crite- exceeded 5. A simple disparity test with fronto-parallel
rion for a higher order neuron was a selectivity that did surfaces sufficed to confirm that this cell was of zero

FIG. 8. Types of TEs neurons. A–C: PSTHs indicating average responses of zero-order (A), first-order (B), and second-order (C) selective
neurons. D: responses of TEs neurons selective for the 3-dimensional shape of the edges of surfaces (i) and of texture inside the edges (ii). The
horizontal lines below the PSTH indicate stimulus duration. In A, all stimuli are indicated above the corresponding PSTHs. In B–D, icons show only
the preferred stimulus polarity. Vertical bars indicate 60 spikes/s (A), 30 spikes/s (B and C), and 65 spikes/s (D). [From Orban et al. (204), copyright
2006 with permission from Elsevier.]

Physiol Rev • VOL 88 • JANUARY 2008 • www.prv.org

 HIGHER ORDER VISUAL PROCESSING 77

order: the cell was a “near” neuron (229). First-order also showed that TEs neurons can encode the orientation
neurons were position-in-depth invariant and responded of the three-dimensional curvature and can, in all likeli-
as well to the three-dimensional shapes as to a planar hood, combine selectivity for orthogonally oriented cur-
three-dimensional surface tilted in depth (Fig. 8B). In the vatures as captured by the shape index of Koenderink
Liu et al. study (160), these neurons were shown to be (132). Two orthogonal curvatures define convex or con-
tuned for the tilt (3-dimensional orientation) in depth. cave ridges (one curvature of zero), convex or concave
Finally, second-order neurons were invariant for position half spheres (both curvatures of the same sign), or sad-
in depth and responded selectively to shapes curved in dles (curvatures of opposite sign).
depth but not to first-order stimuli (Fig. 8C). In about half
of these, the first-order approximation, a wedge, evoked a
C. The Invariance of Three-Dimensional Shape
significantly weaker response than the original curved
Selectivity in TEs
stimulus (Fig. 8C). In the other half, this approximation
was as effective as that stimulus, reminiscent of the V4
The selectivity for depth structure was found to be
neurons tuned to the orientation of smooth curves or
invariant in TEs for changes in fronto-parallel position
sharp angles (221). Note that zero-order approximations
and in size (119), as has been observed for two-dimen-
were generally not effective in higher-order neurons, es-
sional shape selectivity (114, 164, 273, 305, 331, 333). The
pecially second-order neurons, the few exceptions being

Downloaded from physrev.physiology.org on October 9, 2008

invariance for fronto-parallel position complements the
first-order neurons, as illustrated in Figure 8B.
invariance for position in depth already reported in
It is worthwhile to emphasize the exquisite sensitivity
the first study (117), defining a region in three-dimen-
of TEs neurons for small changes in three-dimensional
sional space in which TEs neurons maintain their three-
structure. The difference between curved stimuli and
dimensional shape selectivity.
their linear approximations is only one example. Most
As mentioned earlier, the two-dimensional shape se-
neurons remained selective for the sign of curvature up to
lectivity of IT neurons has been shown to be cue invariant
the smallest amplitude of depth variation (0.03°) tested. In
(264, 304). In the same vein, the three-dimensional shape
addition, most neurons were sensitive to differences in
selectivity of TEs neurons has also been shown to be
the amplitude of depth variation within convex or concave
depth-cue invariant. We opted for a comparison of selec-
stimuli. Their response usually decreased monotonically
tivities for the disparity and texture cues. TEs neurons are
with decreasing amplitude, but in some cases was tuned
selective for tilt specified by disparity but also those
to a preferred amplitude. Thus TEs neurons can signal
specified by texture gradients (160), and the preferred tilt
very precisely the shape of the object in the third dimen-
is similar for the two cues. In addition, the selectivity for
sion (depth structure), and since they are also two-dimen-
tilt specified by texture was shown to be invariant for
sional shape selective, they provide a complete three-
texture type, for slant, and for binocular versus monocu-
dimensional shape representation of objects.
lar presentations.
In the original studies, the depth variation was ap-
So far, these properties of TEs neurons have not been
plied to the outline as well as the texture inside the
modeled. It is clear that some properties such as the fine
outline of the surface stimuli. Hence, the selectivity for
sensitivity to curvature magnitude pose severe challenges
the curvature of three-dimensional surfaces of TEs neu-
for the cascade model that is proposed to operate in many
rons could reflect selectivity for the depth structure of
cortical areas. Perhaps in this case a surround-based
either the edges or the texture pattern inside the edges. In
mechanism is more suited for extracting these disparity
fact, TEs neurons can be selective for depth structure of
gradients. For example, a tuned near neuron having a
either component of the surface stimuli (115). This was
surround with two opponent regions, as described in
demonstrated by testing TEs neurons with additional
MT/V5 for motion, and perhaps with lateral inhibitory
stimuli some of which lacked edges in depth (double
connections from a tuned far neuron with a smaller RF,
curved “surface” stimuli in Fig. 8D), others of which
would generate a selectivity for a convex surface, be it
lacked texture in depth (decorrelated and solid RDS in
with relatively restricted position invariance.
Fig. 8D). The neuron in the top part of Figure 8D retains
its selectivity with decorrelated RDS and solid stereo-
grams in which only the boundary carries depth informa- D. Selectivity of CIP Neurons for
tion, while losing it when the edges are removed in the First-Order Disparity
doubly curved stimuli. This neuron was thus selective for
the three-dimensional shape of the edges (3-dimensional Shikata et al. (277) reported that neurons in the
edge neuron). The neuron in the bottom part of Figure 8D caudal part of the lateral bank of IPS were selective for
reacted in exactly the opposite way and was selective the tilt of stereoscopic surfaces. This caudal region has
for the depth structure of the texture inside the edges been referred to as cIPS (258), CIP (300), or posterior LIP
(3-dimensional surface neuron). In the same study, we (185) and probably corresponds to pIPS as defined by

Physiol Rev • VOL 88 • JANUARY 2008 • www.prv.org

78 GUY A. ORBAN

Denys et al. (48) and to LOP as defined by Lewis and Van ger selectivity in CIP and TEs compared with MT/V5,
Essen (157). Although that initial study (277) established however, suggests that MT/V5 represents one of the early
the disparity selectivity of the CIP neurons, it is only in stages in the extraction of three-dimensional shape and
Taira et al. (300) that the higher order nature of the three-dimensional surface orientation extraction.
selectivity was established by showing invariance for
changes in the fixation distance. It has been reported in
abstract form that CIP neurons have also solved the cor- IX. TWO-DIMENSIONAL SHAPE PROCESSING
respondence problem (124). Importantly, Tsutsui et al. IN INFERO-TEMPORAL CORTEX
(317) have demonstrated that inactivation of CIP inter-
feres with judgments about surface tilt. It has been known for a considerable time that IT
So far, only first-order selectivity has been demon- neurons respond to images of complex objects (86), in-
strated in CIP, although it has also been suggested that cluding biological entities such as faces and hands (49,
second-order selectivity is present (125). Cue conver- 224). Although face stimuli have received a great deal of
gence has been documented for CIP neurons, for the attention (128, 173, 297, 315, 348), it is not clear how
combination of texture and disparity (318), as well as for important natural stimuli, such as faces, body parts, and
perspective and disparity (317). Unlike the stimuli in the animals, are to the actual function of infero-temporal
TEs studies, the outlines were always very simple cortex.

Downloaded from physrev.physiology.org on October 9, 2008

(squares or circles), and stimuli were small solid figures In a very influential set of experiments, Tanaka et al.
(317) or large textured surfaces (318). Finally, CIP neu- (309) showed that images of complex objects can be
rons, rather than being selective for the orientation in reduced, without loss of response from infero-temporal
depth of surfaces (surface orientation selective), can be neurons, to “critical features” which generally consist of
selective for the three-dimensional orientation of elon- more or less complex geometrical parts of the object
gated stimuli (axis orientation selective, Ref. 259). image. These experiments supported the view that ob-
jects, including many man-made objects, are represented
by their parts and that these parts are predominantly
E. Three-Dimensional Shape From Disparity defined by their geometrical description in flat images
Selectivity in Other Cortical Regions (164, 305). This has gradually shifted studies of IT neurons
towards their two-dimensional shape selectivity, although
V1 neurons display no higher-order disparity selec- in the initial study (309) some of the elaborate neurons
tivity (192) and are selective for anticorrelated RDS as clearly required combinations of shape with texture
well as for correlated RDS (41). Thus most of the prop- and/or color to be responsive. Subsequently, the same
erties of TEs and CIP neurons reflect processing beyond group (130), using a similar procedure along the entire
V1. V4 provides input to IT, and V4 neurons are selective ventral pathway, showed that the critical features of neu-
for the orientation in depth of elongated stimuli (102) but rons become increasingly complex as one advances along
not for surfaces curved in depth (95). Thus either TEs the ventral pathway, culminating in the features of the
neurons acquire their higher order selectivity through so-called elaborate neurons of TE (anterior part of IT
local connections in TEs or TEO, or TEs receives its cortex). Finally, critical features were found to be clus-
selective input from IPS, presumably AIP (165). Indeed, it tered in IT (73), and further studies have suggested that
has been suggested that selectivity for three-dimensional the various critical features of object images are repre-
orientation is a property of neurons along the lateral bank sented by the pattern of active and inactive clusters in IT
of IPS (185). The selectivity of AIP neurons for real (316), with some clusters specifically representing the
objects supposedly supports their role in the control of links between the object parts (346). In the preceding
grasping (183). Whether or not the selectivity for real sections, many of the higher order selectivities were for
three-dimensional objects is based on selectivity for higher order parameters in the image such as three-di-
three-dimensional shape is presently being investi- mensional orientation or position in space of the FOE.
gated (293). The notion of critical feature suggests that in IT, the
Some three-dimensional orientation selectivity, higher order selectivity is rather a selectivity for a com-
based on disparity, has been reported for MT/V5 neurons plex “configuration” (e.g., association of shape parts or of
(see above), which also have intermediate properties with shape elements) than one for complex parameters. This
respect to responses to anticorrelated RDS (143, 190). might be taken as an indication that the processing in IT
Nguyenkim and DeAngelis (190) reported that the pre- is different and perhaps more qualitative, although a crit-
ferred tilt of MT/V5 neurons, specified by disparity, did ical feature can be seen as a point in a high-dimensional
not depend on slant. Thus the origin of higher order space. Some support for such a qualitative type of pro-
disparity selectivity and the distribution of this selectivity cessing is provided by the results of Kayaert et al. (127),
throughout the visual system remain unclear. The stron- who found that IT neurons are more sensitive to changes

Physiol Rev • VOL 88 • JANUARY 2008 • www.prv.org

 HIGHER ORDER VISUAL PROCESSING 79

in nonaccidental properties (invariant for changes of ori- they showed 1) that a substantial fraction of V4 neurons
entation in depth of the object) than for changes in metric are tuned for the orientation of a given angle or curve; 2)
properties (dependent on orientation in depth). that this selectivity could not be accounted for by lower
The finding that the removal of internal contours, selectivity for, e.g., orientation of one of the edges of the
texture, or color does not affect the responses of many IT angle; and 3) that this selectivity was invariant over the
neurons (138) has further contributed to make two-di- position in the RF (221). These results, in V4, are different
mensional shape selectivity, understood as selectivity for from those obtained by Ito and Komatsu (113) in V2
the outlines of object images, the canonical property of IT where cells can be tuned for the orientation of angles but
neurons. While this is certainly a key property of IT respond just as strongly to the angles and the component
neurons, one should keep in mind (Fig. 9) that 1) we are edges, suggesting that these V2 neurons represent an
still uncertain about exactly what it is that the IT neurons intermediate step leading to V4 angle selectivity. It is
represent, since living organisms are far more important worth noting that the selectivity for curvature in V4 is
to the monkey than most man-made objects; 2) the rep- probably based on a mechanism different from the cur-
resentation of objects, including animate entities, may not vature selectivity of end-stopped V1 neurons (52, 330).
be the only goal of IT processing, and representation of Instead of resulting from a balance between the extent
scenes is perhaps just as important; and 3) the processing and strength of the inhibitory end regions (205) and the
of two-dimensional shape is important for building these excitatory RF, the selectivity in V4 more likely arises from

Downloaded from physrev.physiology.org on October 9, 2008

“object” and scene representations but processing of the combined input from end-stopped or end-free neurons
other aspects such as three-dimensional shape and mate- (220) having the appropriate preferred orientation and RF
rial properties, including volumetric texture and color, location (Fig. 10). This sort of representation is more
are also important (1, 134, 135, 139, 251, 301). Thus IT invariant for changes in contrast and possibly for cues
neurons might be selective for complex image attributes defining the contour.
other than two-dimensional shape, and two-dimensional In a second step, Pasupathy and Connor (222) tested
shape itself may be integrated into even higher attributes. V4 neurons selective for curvature with a large parame-
terized set of 366 stimuli constructed by combining con-
A. The Starting Point of Shape Selectivity in V4 vex and concave boundary elements into closed shapes.
They observed that individual neurons were selective for
As the visual message reaches V4, the figures have one to several neighboring curvatures, generally convex-
been segregated from the background (see above), and ities, placed in particular angular position with respect to
the analysis of these figures, particularly the shape of the shape center. Thus V4 neurons encode complex
their boundary, can now begin. In an influential set of shapes in terms of moderately complex contour configu-
experiments, Pasupathy and Connor (221) have provided rations and positions. It is noteworthy that the position
strong evidence that curvature, a contour feature present is a “relative” position with respect to the shape and its
in angles and curves, is represented in V4 as an interme- center, which is only possible once the shape has been
diate step between the orientation and spatial frequency segregated from the background by the preliminary
selectivity in V1 and the complex shape selectivity in IT. processing along the V1, V2, V4 path (see above). Fi-
Using a large parametric set of contours in which the nally, Pasupathy and Connor (223) reported that the
acuteness and orientation of convex and concave angles population of curvature-selective V4 neurons repre-
and curves, both sharp and smooth, were manipulated, sents complete shapes as aggregates of curved bound-

FIG. 9. Different aspects of the image

processed by V1, V4 and infero-temporal
(IT). Both geometry and material proper-
ties contribute to the description of ani-
mate entities (animals, conspecifics, and
body parts) and/or objects and of scenes.

Physiol Rev • VOL 88 • JANUARY 2008 • www.prv.org

80 GUY A. ORBAN

and posterior part of TE) integrate multiple contour ele-

ments such as those coded in V4 using linear and nonlin-
ear mechanisms. Indeed, the responses to the stimulus set
were widely distributed but could be modeled by nonlin-
ear integration of one to six subunits, each selective for a
contour element in a given relative position. The average
(over 109 neurons) correlation between observed re-
sponses and those predicted by the model was 0.7, indi-
cating that the model explained half the variability in the
posterior IT responses. Both excitatory and inhibitory
inputs were integrated by the IT neurons, but only ex-
citatory inputs were integrated nonlinearly. Nonlinearity
was related to the sparseness of the response. Shape
FIG. 10. Two definitions of curvature. Balance of excitation from tuning in the sense of relative responses to different
classical RF and inhibition from the end regions in V1, V4 from two RFs
located and oriented appropriately (with or without endstopping) yields shapes was position and size invariant. Again, this higher
selectivity for the angle/curve or for the angle only. order selectivity is that for a “configuration,” not a com-
plex parameter, underscoring the difference in processing

Downloaded from physrev.physiology.org on October 9, 2008

between IT and other parts of extrastriate cortex.
ary fragments. To estimate the population representa- Subsequently, Brincat and Connor (34) studied the
tion of a shape, they scaled each cell’s tuning peak with time course of the integration. They observed that the
the response of that cell to that shape, summed across linear integration was fast and that nonlinear integration
cells, and smoothed. The resulting population surface required ⬃60 additional milliseconds, a finding reminis-
(coordinates: curvature and angular position) con- cent of observations in MT/V5 for pattern direction selec-
tained several peaks that could be used to reconstruct tive responses (213, 286). This temporal evolution could
the original shape. be modeled by recurrent connections within the area,
Others have found little difference between areas V1, which have been shown to produce nonlinear selectivity
V2, and V4 using a large set of grating and contour stimuli for a conjunction of inputs that are initially combined
(97). The stimulus set included radial and hyperbolic linearly (262). These studies underscore the generality of
gratings that were originally used to demonstrate the cascade models, including a linear combination of input
responsiveness to complex patterns in V4 neurons (74, sandwiched between two nonlinear stages. On the other
75) and V2 neurons (94). This result is reminiscent of the hand, they also indicate the diversity of the possible im-
apparent similarity in responses to optic flow components plementations of the nonlinearities.
in MT/V5 and MSTd (144). It is only when further tests,
such as the position invariance test, were used that the
underlying difference between MT/V5 neurons selective C. Shape Processing in Anterior IT: Manipulating
only for translation direction and MSTd neurons selec- Shape Dimensions
tive for flow components became evident. More elaborate
tests, such as those reviewed above, are probably neces- The shape selectivity of anterior TE neurons has
sary to differentiate between areas V1, V2, and V4. Inter- been studied with regard to several aspects, such as in-
estingly, Hegdé and Van Essen (97) observed in their MDS variance (114, 264, 313, 333; for reviews, see Refs. 164,
analysis a clear segregation between grating and contour 305), similarity between shapes (197), and the influence of
stimuli in V4, which can be seen as an indication that, at training (13, 20, 131) and input statistics (47). An initial
this level, shape or contour processing and texture pro- study using radial frequency as a global parameter of
cessing begin to diverge. shape (273) failed to reveal any systematic effects for this
single parameter, and it is only recently that various shape
parameters have been manipulated systematically and
B. Shape Processing in Posterior IT: Building tested in anterior TE neurons. Kayaert et al. (126) manip-
Simple Shape Parts ulated various curvatures in rectangular and triangular
shapes and observed monotonic response curves with the
Using a strategy similar to that of Pasupathy and strongest responses to the most sharply curved shapes.
Connor (222), Brincat and Connor (33) constructed a This is rather similar to results obtained in V4 by Pasu-
parametric stimulus set of two-dimensional silhouette pathy and Connor (222) who observed strong responses
shapes by systematically integrating convex, straight, and to maximally curved convex contour fragments. Thus this
concave contour elements at specific orientations and tuning could simply reflect a selectivity present in the
positions. They showed that posterior IT neurons (in TEO inputs to anterior TE. This simple account is more prob-

Physiol Rev • VOL 88 • JANUARY 2008 • www.prv.org

 HIGHER ORDER VISUAL PROCESSING 81

however, exactly what that wider set should be. Even if

we accept that these monotonic curves genuinely repre-
sent neuronal behavior in IT, it is unclear what they
encode. One possibility is that, exactly as at lower levels
in the system, they encode stimulus dimensions just as
bell-shaped tunings do. While it is true that V1 neurons are
tuned for orientation, direction, and spatial frequency,
one should remember that monotonic curves have also
been reported for disparity (229, 230) and for speed (199,
206, 207). Furthermore, tuning for spatial frequency might
reflect the frequency envelope, imposed by the peripheral
visual apparatus, leaving only circular variables such as
orientation and direction for which tunings have been
observed at the level of V1. Thus, at early levels, mono-
tonic curves might be as useful for encoding stimulus
dimensions as bell-shaped tuning curves are. On the other
hand, at the level of IT, monotonic curves may have a

Downloaded from physrev.physiology.org on October 9, 2008

different interpretation. Because extreme values of shape
parameters are more likely to be features that define
shape parts, they may evoke stronger responses in IT
neurons. Thus the smooth variation of shape dimensions
would have effects similar to the more stepwise reduction
of object images as preformed by Tanaka et al. (309).
Similar monotonic response curves have been ob-
served by Leopold et al. (154), who tested face-selective
TE neurons with stimuli in a face space defined by four
different faces and their average. These neurons re-
sponded more strongly to the individual faces than to the
average and even more so to a caricature that extrapo-
lated beyond the actual face. An explanation in terms of
typical shape parts seems difficult at first glance, since a
face already contains all the defining parts. Yet it may be
that the response variations along the axis average face-
identity face in fact reflected some covariation in a face
part dimension, such as interocular distance or distance
nose-to-mouth distance (70). An extreme value for such a
dimension might be indicative of the identity of the face
and therefore encoded by IT neurons.

X. CONCLUDING REMARKS
FIG. 11. Preferences of anterior TE neurons for extreme shapes of
a parameterized set. Smoothed average response for all responsive IT
neurons to the shapes in four different sets. [Modified from Debaene et Neurons in the extrastriate cortex are endowed with
al. (47).] selectivity for higher order aspects of the image which is
absent or rare in V1. Many examples of higher order
lematic for the more recent results obtained by Debaene selectivity come from near-extrastriate areas, such as V2,
et al. (47). These authors used a rapid serial presentation V4, or MT/V5. At these levels, higher order selectivity is
to test a wide range of stimuli divided into five stimulus generally that for a complex parameter. Most of the se-
sets, each of which was parameterized by two variables lectivity described here can be accounted for by a simple
(Fig. 11). Again, anterior TE neurons responded maxi- pattern of excitatory and inhibitory linear inputs, supple-
mally to the extreme stimuli in each set. These monotonic mented with nonlinear mechanisms in the afferent and
response curves are difficult to interpret in these IT stud- receiving area. Thus the feed-forward projections seem to
ies. Indeed, the choice of the stimuli is arbitrary, and a largely determine the processing in the visual system,
larger stimulus set could, in principle, reveal a tuning for although feedback may play a role in figure-ground seg-
an optimal value. Given the stimuli used, it is unclear, mentation (111). One of the functions to which feedback

Physiol Rev • VOL 88 • JANUARY 2008 • www.prv.org

82 GUY A. ORBAN

connections might contribute is the generation of antag- oculomotor signals in MSTd and the convergence of
onistic surrounds present at various levels in the visual visual, vestibular, oculomotor, auditory, and somato-sen-
cortex. These antagonistic surrounds perform a variety of sory signals in VIP.
functions, such as gain control, rejection of uniformity, The results described in this review span 20 years of
generation of selectivity, or integration of secondary cues. single-cell recording and are a testimony to the strength of
We know far less about properties in far-extrastriate the technique, the only one to date able to give insight into
regions, such as the intraparietal sulcus, or infero-tempo- the operations performed in the brain at the neuronal
ral cortex. A possible reason is that we do not know the level. Yet, they also show that progress has been slow. As
goals of the visual processing well enough, especially for indicated above, there might be conceptual reasons for
the species in which we most often investigate the visual this lack of headway, but there are also methodological
system, the monkey. It may well be that the higher we reasons. Single-cell studies are very labor intensive, in-
climb into the hierarchy of the system, the more the clining researchers to go for assured results and to be
processing of visual signals is tailored to the specific rather conservative in their choice of recording sites.
behavioral needs of the species under study. This is ex- Indeed, only a small number of areas have been explored
emplified by the tuning for heading directions in MSTd. It well enough to have at least hints of their perceptual role.
came as a surprise that all directions in space were This may change rapidly, as we have now a scouting
equally represented rather than emphasizing the forward technique available: functional imaging in the awake mon-

Downloaded from physrev.physiology.org on October 9, 2008

direction or directions parallel to the ground plane. This is key. This technique (324) allows one to test a wide range
surprising only from the standpoint of our human needs; of novel stimuli over the whole visual system rather
it is far less surprising when we consider that a monkey quickly and to find out which areas are involved in their
with his arboreal life-style can jump or fall in any direc- processing. Furthermore, this technique also allows the
tion. It may well be that the types of object images we invasive studies in the monkey to be linked in a scientif-
have tested on IT neurons are too human-oriented and ically sound way to human functional imaging data, al-
that a monkey normally needs to recognize and categorize lowing a full exploitation of the monkey model.
other types of stimuli, most generally living animals or
ACKNOWLEDGMENTS
conspecifics. Thus, while it has proven fruitful to link
neurophysiological studies of the visual system with This review was written while the author held the Euro-
pean chair at the Collège de France, Paris.
psychophysics, it might equally be useful to consider an
I am indebted to Y. Celis, A. Verhulst, and G. Meulemans for
ethological perspective to better understand what the
help with the references and figures and to Dr. S. Raiguel for
monkey uses his vision for. comments on an earlier version of this manuscript.
Even if it turns out that human and nonhuman pri- Address for reprint requests and other correspondence:
mates use behaviorally similar visual information, it will G. A. Orban, K.U. Leuven, Medical School, Laboratorium voor
still be important to consider the visual system from its Neuro- en Psychofysiologie, Campus Gasthuisberg, Herestraat
output side, the connections with other brain regions. 49, Bus 1021, BE-3000 Leuven, Belgium (e-mail: guy.orban@med.
Indeed, vision is useful only when it delivers a message to kuleuven.be).
other parts of the brain. Thus reversing the current trend
of considering vision from the perspective of the eye and REFERENCES
the input and instead emphasizing the other brain parts
1. Aggelopoulos NC, Rolls ET. Scene perception: inferior temporal
receiving visual information and the output will help de- cortex neurons encode the positions of different objects in the
fine the goals of the visual processing. Without such scene. Eur J Neurosci 22: 2903–2916, 2005.
knowledge, attempts to investigate higher order visual 2. Albright TD. Direction and orientation selectivity of neurons in
visual area MT of the macaque. J Neurophysiol 52: 1106 –1130,
processing can have little hope of success. Knowing the 1984.
behavioral goals will specify the “end products” that 3. Albright TD. Form-cue invariant motion processing in primate
vision has to deliver, and one can then trace back how the visual cortex. Science 255: 1141–1143, 1992.
4. Albright TD, Stoner GR. Contextual influences on visual process-
representations of these end products gradually emerge ing. Annu Rev Neurosci 25: 339 –379, 2002.
at the different levels in the extrastriate cortex. Undoubt- 5. Allman JM, Kaas JH. A representation of the visual field in the
edly these end products must be as robust as possible and caudal third of the middle tempral gyrus of the owl monkey (Aotus
trivirgatus). Brain Res 31: 85–105, 1971.
will require the convergence of the different types of 6. Allman J, Miezin F, McGuinness E. Stimulus specific responses
information pertinent to their computation. Thus empha- from beyond the classical receptive field: neurophysiological mech-
sis will shift to the convergence of different types of anisms for local-global comparisons in visual neurons. Annu Rev
Neurosci 8: 407– 430, 1985.
information, either within the visual system as we have 7. Allouni AK, Thomas OM, Solomon SG, Krug K, Parker AJ.
seen for shape, texture, and color in the description of Local and global binocular matching in V2 of the awake macaque.
object parts by IT neurons, or between sensory systems in Soc Neurosci Abstr 510.8, 2005.
8. Andersen RA, Bradley DC, Shenoy KV. Neural mechanisms for
multimodal areas, such as in several parietal regions. We heading and structure from motion perception. Cold Spring Har-
have noted the combination of visual, vestibular, and bor Symp Quant Biol 61: 15–25, 1996.

Physiol Rev • VOL 88 • JANUARY 2008 • www.prv.org

 HIGHER ORDER VISUAL PROCESSING 83

9. Anderson KC, Siegel RM. Three-dimensional structure-from-mo- 35. Cavanaugh JR, Bair W, Movshon JA. Nature and interaction of
tion selectivity in the anterior superior temporal polysensory area, signals from the receptive field center and surround in macaque V1
STPa, of the behaving monkey. Cereb Cortex 15: 1299 –1307, 2005. neurons. J Neurophysiol 88: 2530 –2546, 2002.
10. Anderson KC, Siegel RM. Optic flow selectivity in the anterior 36. Celebrini S, Thorpe S, Trotter Y, Imbert M. Dynamics of ori-
superior temporal polysensory area, STPa, of the behaving mon- entation coding in area V1 of the awake primate. Vis Neurosci 10:
key. J Neurosci 9: 2681–2692, 1999. 811– 825, 1993.
11. Avillac M, Deneve S, Olivier E, Pouget A, Duhamel JR. Refer- 37. Chaudhuri A, Albright TD. Neuronal responses to edges defined
ence frames for representing visual and tactile locations in parietal by luminance vs. temporal texture in macaque area V1. Vis Neu-
cortex. Nat Neurosci 8: 941–949, 2005. rosci 14: 949 –962, 1997.
12. Bair W, Cavanaugh JR, Movshon JA. Time course time-distance 38. Cowey A, Gross CG. Effects of foveal prestriate and inferotem-
relationships for surround suppression in macaque V1 neurons. poral lesions on visual discriminations. Exp Brain Res 11: 128 –144,
J Neurosci 23: 3189 –3205, 2003. 1970.
13. Baker CI, Behrmann M, Olson CR. Impact of learning on repre- 39. Cumming BG. An unexpected specialization for horizontal dispar-
sentation of parts and wholes in monkey inferotemporal cortex. ity in primate primary visual cortex. Nature 418: 633– 636, 2002.
Nat Neurosci 5: 1210 –1216, 2002. 40. Cumming BG, DeAngelis GC. The physiology of stereopsis. Annu
14. Bakin JS, Nakayama K, Gilbert CD. Visual responses in monkey Rev Neurosci 24: 203–238, 2001.
areas V1 and V2 to three-dimensional surface configurations. 41. Cumming BG, Parker AJ. Responses of primary visual cortical
J Neurosci 20: 8188 – 8198, 2000. neurons to binocular disparity without depth perception. Nature
15. Banks MS, Ehrlich SM, Backus BT, Crowell JA. Estimating 389: 280 –283, 1997.
heading during real and simulated eye movements. Vision Res 36: 42. Cumming BG, Parker AJ. Binocular neurons in V1 of awake
431– 443, 1996. monkeys are selective for absolute, not relative, disparity. J Neu-
16. Baracas GT, Albright TD. Contribution of area MT to perception rosci 19: 5602–5618, 1999.

Downloaded from physrev.physiology.org on October 9, 2008

of three-dimensional shape: a computational study. Vision Res 36: 43. DeAngelis GC, Freeman RD, Ohzawa I. Length and width tuning
869 – 887, 1996. of neurons in the cat’s primary visual cortex. J Neurophysiol 71:
17. Baumann R, van der Zwan R, Peterhans E. Figure-ground seg- 347–374, 1994.
regation at contours: a neural mechanism in the visual cortex of the 44. DeAngelis GC, Newsome WT. Organization of disparity-selective
alert monkey. Eur J Neurosci 9: 1290 –1303, 1997. neurons in macaque area MT. J Neurosci 19: 1398 –1415, 1999.
18. Ben Hamed S, Page W, Duffy C, Pouget A. MSTd neuronal basis 45. DeAngelis GC, Robson JG, Ohzawa I, Freeman RD. Organiza-
functions for the population encoding of heading direction. J Neu- tion of suppression in receptive fields of neurons in cat visual
rophysiol 90: 549 –558, 2003. cortex. J Neurophysiol 68: 144 –163, 1992.
19. Blasdel GG. Differential imaging of ocular dominance and orien- 46. DeAngelis GC, Uka T. Coding of horizontal disparity and velocity
tation selectivity in monkey striate cortex. J Neurosci 12: 3115– by MT neurons in the alert macaque. J Neurophysiol 89: 1094 –1111,
3138, 1992. 2003.
20. Booth MC, Rolls ET. View-invariant representations of familiar 47. Debaene W, Premereur E, Vogels R. Properties of shape tuning
objects by neurons in the inferior temporal visual cortex. Cereb of macaque inferior temporal neurons examined using Rapid Serial
Cortex 8: 510 –523, 1998. Visual Presentation. J Neurophysiol. In press.
21. Born RT, Bradley DC. Structure and function of visual area MT. 48. Denys K, Vanduffel W, Fize D, Nelissen K, Peuskens H, Van
Annu Rev Neurosci 28: 157–189, 2005. Essen D, Orban GA. The processing of visual shape in the cere-
22. Born RT, Groh JM, Zhao R, Lukasewycz SJ. Segregation of bral cortex of human and nonhuman primates: a functional mag-
object and background motion in visual area MT: effects of micro- netic resonance imaging study. J Neurosci 24: 2551–2565, 2004.
stimulation on eye movements. Neuron 26: 725–734, 2000. 49. Desimone R, Albright TD, Gross CG, Bruce C. Stimulus-selec-
23. Born RT, Pack CC, Ponce CR, Yi S. Temporal evolution of tive properties of inferior temporal neurons in the macaque. J Neu-
2-dimensional direction signals used to guide eye movements. rosci 4: 2051–2062, 1984.
J Neurophysiol 95: 284 –300, 2006. 50. De Valois RL, Albrecht DG, Thorell LG. Spatial frequency
24. Bradley DC, Andersen RA. Center-surround antagonism based selectivity of cells in macaque visual cortex. Vision Res 22: 545–
on disparity in primate area MT. J Neurosci 18: 7552–7565, 1998. 549, 1982.
25. Bradley DC, Chang GC, Andersen RA. Encoding of three-dimen- 51. De Valois RL, Cottaris NP, Elfar SD, Mahon LE, Wilson JA.
sional structure-from-motion by primate area MT neurons. Nature Some transformations of color information from lateral geniculate
392: 714 –717, 1998. nucleus to striate cortex. Proc Natl Acad Sci USA 97: 4997–5002,
26. Bradley DC, Maxwell M, Andersen RA, Banks MS, Shenoy KV. 2000.
Mechanisms of heading perception in primate visual cortex. Sci- 52. Dobbins A, Zucker SW, Cynader MS. Endstopped neurons in the
ence 273: 1544 –1547, 1996. visual cortex as a substrate for calculating curvature. Nature 329:
27. Bradley DC, Qian N, Andersen RA. Integration of motion and 438 – 441, 1987.
stereopsis in middle temporal cortical area of macaques. Nature 53. Dubner R, Zeki SM. Response properties and receptive fields of
373: 609 – 611, 1995. cells in an anatomically defined region of the superior temporal
28. Bredfeldt CE, Cumming BG. A simple account of cyclopean edge sulcus in the monkey. Brain Res 35: 528 –532, 1971.
responses in macaque v2. J Neurosci 26: 7581–7596, 2006. 54. Duffy CJ. MST neurons respond to optic flow and translational
29. Bredfeldt CE, Ringach DL. Dynamics of spatial frequency tuning movement. J Neurophysiol 80: 1816 –1827, 1998.
in macaque V1. J Neurosci 22: 1976 –1984, 2002. 55. Duffy CJ, Wurtz RH. Sensitivity of MST neurons to optic flow
30. Bremmer F, Duhamel JR, Ben Hamed S, Graf W. Heading stimuli. I. A continuum of response selectivity to large-field stimuli.
encoding in the macaque ventral intraparietal area (VIP). Eur J Neurophysiol 65: 1329 –1345, 1991.
J Neurosci 16: 1554 –1568, 2002. 56. Duffy CJ, Wurtz RH. Sensitivity of MST neurons to optic flow
31. Bremmer F, Klam F, Duhamel JR, Ben Hamed S, Graf W. stimuli. II. Mechanisms of response selectivity revealed by small-
Visual-vestibular interactive responses in the macaque ventral field stimuli. J Neurophysiol 65: 1346 –1359, 1991.
intraparietal area (VIP). Eur J Neurosci 16: 1569 –1586, 2002. 57. Duffy CJ, Wurtz RH. Response of monkey MST neurons to optic
32. Bremmer F, Kubischik M, Pekel M, Lappe M, Hoffmann KP. flow stimuli with shifted centers of motion. J Neurosci 15: 5192–
Linear vestibular self-motion signals in monkey medial superior 5208, 1995.
temporal area. Ann NY Acad Sci 871: 272–281, 1999. 58. Duffy CJ, Wurtz RH. Medial superior temporal area neurons
33. Brincat SL, Connor CE. Underlying principles of visual shape respond to speed patterns in optic flow. J Neurosci 17: 2839 –2851,
selectivity in posterior inferotemporal cortex. Nat Neurosci 7: 880 – 1997.
886, 2004. 59. Duhamel JR, Colby CL, Goldberg ME. Ventral intraparietal area
34. Brincat SL, Connor CE. Dynamic shape synthesis in posterior of the macaque: congruent visual and somatic response properties.
inferotemporal cortex. Neuron 49: 17–24, 2006. J Neurophysiol 79: 126 –136, 1998.

Physiol Rev • VOL 88 • JANUARY 2008 • www.prv.org

84 GUY A. ORBAN

60. Duncan RO, Albright TD, Stoner GR. Occlusion and the inter- 85. Grosof DH, Shapley RM, Hawken MJ. Macaque V1 neurons can
pretation of visual motion: perceptual and neuronal effects of signal “illusory” contours. Nature 365: 550 –552, 1993.
context. J Neurosci 20: 5885–5897, 2000. 86. Gross CG, Rocha-Miranda CE, Bender DB. Visual properties of
61. Durand JB, Celebrini S, Trotter Y. Neural bases of stereopsis neurons in inferotemporal cortex of the macaque. J Neurophysiol
across visual field of the alert macaque monkey. Cereb Cortex 35: 96 –111, 1972.
doi:10.1093/cercor/bhl050, 2006. 87. Grunewald A, Bradley DC, Andersen RA. Neural correlates of
62. Durand JB, Zhu S, Celebrini S, Trotter Y. Neurons in parafoveal structure-from-motion perception in macaque V1 and MT. J Neu-
areas V1 and V2 encode vertical and horizontal disparities. J Neu- rosci 22: 6195– 6207, 2002.
rophysiol 88: 2874 –2879, 2002. 88. Gu Y, Watkins PV, Angelaki DE, DeAngelis GC. Visual and
63. Ekstrom LB, Bonmassar G, Tootell RBH, Roelfsema PR, nonvisual contributions to three-dimensional heading selectivity in
Vanduffel W. Investigating FEF microstimulation and distractor the medial superior temporal area. J Neurosci 26: 73– 85, 2006.
effects in visual cortex with awake monkey fMRI. Soc Neurosci 89. Gulyás B, Orban GA, Duysens J, Maes H. The suppressive
Abstr 703.8, 2006. influence of moving textured backgrounds on responses of cat
64. Everson RM, Prashanth AK, Gabbay M, Knight BW, Sirovich striate neurons to moving bars. J Neurophysiol 57: 1767–1791,
L, Kaplan E. Representation of spatial frequency and orientation 1987.
in the visual cortex. Proc Natl Acad Sci USA 95: 8334 – 8338, 1998. 90. Gur M, Kagan I, Snodderly DM. Orientation and direction selec-
65. Felleman DJ, Van Essen DC. Distributed hierarchical processing tivity of neurons in V1 of alert monkeys: functional relationships
in the primate cerebral cortex. Cereb Cortex 1: 1– 47, 1991. and laminar distributions. Cereb Cortex 15: 1207–1221, 2005.
66. Ferraina S, Garasto MR, Battaglia Mayer A, Ferraresi P, 91. Hanazawa A, Komatsu H, Murakami I. Neural selectivity for hue
Johnson PB, Lacquaniti F, Caminiti R. Visual control of hand- and saturation of colour in the primary visual cortex of the mon-
reaching movement: activity in parietal area 7m. Eur J Neurosci 9: key. Eur J Neurosci 12: 1753–1763, 2000.
1090 –1095, 1997. 92. Hawken MJ, Parker AJ, Lund JS. Laminar organization and

Downloaded from physrev.physiology.org on October 9, 2008

67. Ferster D, Chung S, Wheat H. Orientation selectivity of thalamic contrast sensitivity of direction-selective cells in the striate cortex
input to simple cells of cat visual cortex. Nature 380: 249 –252, of the Old World monkey. J Neurosci 8: 3541–3548, 1988.
1996. 93. Hegdé J, Albright TD, Stoner GR. Second-order motion conveys
68. Fize D, Vanduffel W, Nelissen K, Denys K, Chef d’Hotel C, depth-order information. J Vision 4: 838 – 842, 2004.
Faugeras O, Orban GA. The retinotopic organization of primate 94. Hegdé J, Van Essen DC. Selectivity for complex shapes in pri-
dorsal V4 and surrounding areas: a functional magnetic resonance mate visual area V2. J Neurosci 20: 61, 2000.
imaging study in awake monkeys. J Neurosci 23: 7395– 406, 2003. 95. Hegdé J, Van Essen DC. Stimulus dependence of disparity coding
69. Foster KH, Gaska JP, Nagler M, Pollen DA. Spatial and tempo- in primate visual area V4. J Neurophysiol 93: 620 – 626, 2005.
ral frequency selectivity of neurones in visual cortical areas V1 and 96. Hegdé J, Van Essen DC. Role of primate visual area V4 in the
V2 of the macaque monkey. J Physiol 365: 331–363, 1985. processing of 3-D shape characteristics defined by disparity. J Neu-
70. Freiwald WA, Tsao DY, Tootell RBH, Livingstone MS. Single rophysiol 94: 2856 –2866, 2005.
unit recording in an fMRI-identified macaque face patch. II. Coding
97. Hegdé J, Van Essen DC. A comparative study of shape represen-
along multiple feature axes. Soc Neurosci Abstr 362.6, 2005.
tation in macaque visual areas V2 and V4. Cereb Cortex doi:10.1093/
71. Friedman HS, Zhou H, von der Heydt R. The coding of uniform
cercor/bhl020, 2006.
colour figures in monkey visual cortex. J Physiol 548: 593– 613,
98. Heider B, Meskenaite V, Peterhans E. Anatomy and physiology
2003.
of a neural mechanism defining depth order and contrast polarity at
72. Froehler MT, Duffy CJ. Cortical neurons encoding path and
illusory contours. Eur J Neurosci 12: 4117– 4130, 2000.
place: where you go is where you are. Science 295: 2462–2465, 2002.
99. Heider B, Spillmann L, Peterhans E. Stereoscopic illusory con-
73. Fujita I, Tanaka K, Ito M, Cheng K. Columns for visual features
tours— cortical neuron responses and human perception. J Cogn
of objects in monkey inferotemporal cortex. Nature 360: 343–346,
1992. Neurosci 14: 1018 –1029, 2002.
74. Gallant JL, Braun J, Van Essen DC. Selectivity for polar, hyper- 100. Heitger F, Rosenthaler L, von der Heydt R, Peterhans E,
bolic, Cartesian gratings in macaque visual cortex. Science 259: Kubler O. Simulation of neural contour mechanisms: from simple
100 –103, 1993. to end-stopped cells. Vision Res 32: 963–981, 1992.
75. Gallant JL, Connor CE, Rakshit S, Lewis JW, Van Essen DC. 101. Hinkle DA, Connor CE. Disparity tuning in macaque area V4.
Neural responses to polar, hyperbolic, Cartesian gratings in area V4 Neuroreport 12: 365–369, 2001.
of the macaque monkey. J Neurophysiol 76: 2718 –2739, 1996. 102. Hinkle DA, Connor CE. Three-dimensional orientation tuning in
76. Galletti C, Gamberini M, Kutz DF, Baldinotti I, Fattori P. The macaque area V4. Nat Neurosci 5: 665– 670, 2002.
relationship between V6 and PO in macaque extrastriate cortex. 103. Hinkle DA, Connor CE. Quantitative characterization of disparity
Eur J Neurosci 21: 959 –970, 2005. tuning in ventral pathway area V4. J Neurophysiol 94: 2726 –2737,
77. Gautama T, Van Hulle MM. Function of center-surround antag- 2005.
onism for motion in visual area MT/V5: a modeling study. Vision 104. Hubel DH, Wiesel TN. Receptive fields, binocular interaction and
Res 41: 3917–3930, 2001. functional architecture in the cat’s visual cortex. J Physiol 160:
78. Geesaman BJ, Andersen RA. The analysis of complex motion 106 –154, 1962.
patterns by form/cue invariant MSTd neurons. J Neurosci 16: 4716 – 105. Hubel DH, Wiesel TN. Receptive fields and functional architec-
4732, 1996. ture in two nonstriate visual areas (18 and 19) of the cat. J Neu-
79. Gegenfurtner KR, Kiper DC. Color vision. Annu Rev Neurosci rophysiol 28: 229 –289, 1965.
26: 181–206, 2003. 106. Hubel DH, Wiesel TN. Receptive fields and functional architec-
80. Gonzalez F, Justo MS, Bermudez MA, Perez R. Sensitivity to ture of monkey striate cortex. J Physiol 195: 215–243, 1968.
horizontal and vertical disparity and orientation preference in 107. Hubel DH, Wiesel TN. Stereoscopic vision in macaque monkey.
areas V1 and V2 of the monkey. Neuroreport 14: 829 –32, 2003. Cells sensitive to binocular depth in area 18 of the macaque mon-
81. Goodale MA, Milner AD. Separate visual pathways for perception key cortex. Nature 225: 41– 42, 1970.
and action. Trends Neurosci 15: 20 –25, 1992. 108. Hubel DH, Wiesel TN. Uniformity of monkey striate cortex: a
82. Goodale MA, Westwoord DA. An evolving view of duplex vision: parallel relationship between field size, scatter, magnification fac-
separate but interacting cortical pathways for perception and ac- tor. J Comp Neurol 158: 295–305, 1974.
tion. Curr Opin Neurobiol 14: 203–211, 2004. 109. Hubel DH, Wiesel TN. Sequence regularity and geometry of ori-
83. Graziano MS, Andersen RA, Snowden RJ. Tuning of MST neu- entation columns in the monkey striate cortex. J Comp Neurol 158:
rons to spiral motions. J Neurosci 14: 54 – 67, 1994. 267–293, 1974.
84. Groh JM, Born RT, Newsome WT. How is a sensory map read 110. Hubel DH, Wiesel TN, Stryker MP. Anatomical demonstration of
out? Effects of microstimulation in visual area MT on saccades and orientation selectivity in macaque monkey. J Comp Neurol 177:
smooth pursuit eye movements. J Neurosci 17: 4312– 4330, 1997. 361–380, 1978.

Physiol Rev • VOL 88 • JANUARY 2008 • www.prv.org

 HIGHER ORDER VISUAL PROCESSING 85

111. Hupé JM, James AC, Payne BR, Lomber SG, Girard P, Bullier 136. Komatsu H, Wurtz RH. Relation of cortical areas MT and MST to
J. Cortical feedback improves discrimination between figure and pursuit eye movements. I. Localization and visual properties of
background by V1, V2 and V3 neurons. Nature 394: 784 –787, 1998. neurons. J Neurophysiol 60: 580 – 603, 1988.
112. Ilg UJ, Schumann S. Primate area MST-I is involved in the gen- 137. Komatsu H, Wurtz RH. Modulation of pursuit eye movements by
eration of goal-directed eye and hand movements. J Neurophysiol stimulation of cortical areas MT and MST. J Neurophysiol 62:
97: 761–771, 2007. 31– 47, 1989.
113. Ito M, Komatsu H. Representation of angles embedded within 138. Köteles K, Vogels R, Orban GA. Coding of material properties in
contour stimuli in area V2 of macaque monkeys. J Neurosci 24: macaque inferior-temporal cortex. J Vision 4: 124, 2004.
3313–3324, 2004. 139. Kovács G, Sáry G, Köteles K, Chadaide Z, Tompa T, Vogels R,
114. Ito M, Tamura H, Fujita I, Tanaka K. Size and position invari- Benedek G. Effects of surface cues on the macaque inferior tem-
ance of neuronal responses in monkey inferotemporal cortex. poral cortical responses. Cereb Cortex 13: 178 –188, 2003.
J Neurophysiol 73: 218 –226, 1995. 140. Kovacs G, Vogels R, Orban GA. Selectivity of macaque inferior
115. Janssen P, Vogels R, Liu Y, Orban GA. Macaque inferior tempo- temporal neurons for partially occluded shapes. J Neurosci 15:
ral neurons are selective for three-dimensional boundaries and 1984 –1997, 1995.
surfaces. J Neurosci 21: 9419 –9429, 2001. 141. Krekelberg B, Dannenberg S, Hoffmann KP, Bremmer F, Ross
116. Janssen P, Vogels R, Liu Y, Orban GA. At least at the level of J. Neural correlates of implied motion. Nature 424: 674 – 677, 2003.
inferior temporal cortex, the stereo correspondence problem is 142. Krekelberg B, van Wezel RJA, Albright TD. Interactions be-
solved. Neuron 37: 693–701, 2003. tween speed and contrast tuning in the middle temporal area:
117. Janssen P, Vogels R, Orban GA. Macaque inferior temporal implications for the neural code for speed. J Neurosci 26: 8988 –
neurons are selective for disparity-defined three-dimensional 8998, 2006.
shapes. Proc Natl Acad Sci USA 96: 8217– 8222, 1999. 143. Krug K, Cumming BG, Parker AJ. Comparing perceptual signals
118. Janssen P, Vogels R, Orban GA. Selectivity for 3D shape that of single V5/MT neurons in two binocular depth tasks. J Neuro-

Downloaded from physrev.physiology.org on October 9, 2008

reveals distinct areas within macaque inferior temporal cortex. physiol 92: 1586 –1596, 2004.
Science 288: 2054 –2056, 2000. 144. Lagae L, Maes H, Raiguel S, Xiao D, Orban GA. Responses of
119. Janssen P, Vogels R, Orban GA. Three-dimensional shape coding macaque STS neurons to optic flow components: a comparison of
in inferior temporal cortex. Neuron 27: 385–397, 2000. areas MT and MST. J Neurophysiol 71: 1597–1626, 1994.
120. Jones HE, Grieve KL, Wang W, Sillito AM. Surround suppres- 145. Lagae L, Raiguel S, Orban GA. Speed and direction selectivity of
sion in primate V1. J Neurophysiol 86: 2011–2028, 2001. macaque middle temporal neurons. J Neurophysiol 69: 19 –39,
121. Kaas JH, Lyon DC. Visual cortex organization in primates: thoer- 1993.
ies of V3 and adjoining visual areas. Prog Brain Res 134: 285–295, 146. Lamme VA. The neurophysiology of figure-ground segregation in
2001. primary visual cortex. J Neurosci 15: 1605–1615, 1995.
122. Kapadia MK, Westheimer G, Gilbert CD. Spatial distribution of 147. Lappe M, Bremmer F, Pekel M, Thiele A, Hoffmann KP. Optic
contextual interactions in primary visual cortex and in visual per- flow processing in monkey STS: a theoretical and experimental
ception. J Neurophysiol 84: 2048 –2062, 2000.
approach. J Neurosci 16: 6265– 6285, 1996.
123. Kara P, Pezaris JS, Yurgenson S, Reid RC. The spatial receptive
148. Lappe M, Duffy CJ. Optic flow illusion and single neuron behav-
field of thalamic inputs to single cortical simple cells revealed by
iour reconciled by a population model. Eur J Neurosci 11: 2323–
the interaction of visual and electrical stimulation. Proc Natl Acad
2331, 1999.
Sci USA 99: 16261–16266, 2002.
149. Lauwers K, Saunders R, Vogels R, Vandenbussche E, Orban G.
124. Katsuyama N, Naganuma T, Sakata H, Taira M. Representation
Impairment in motion discrimination tasks is unrelated to amount
of 3D curvature in the caudal intraparietal (CIP) area of macaque.
of damage to superior temporal sulcus motion areas. J Comp
Soc Neurosci Abstr 751.11, 2004.
Neurol 420: 539 –557, 2000.
125. Katsuyama N, Yamashsita A, Sawada K, Tsutsui K, Taira M.
Architectonic structures and 3D-selective neurons in the caudal 150. Lee TS, Mumford D, Romero R, Lamme VA. The role of the
intraparietal area of Japanese macaque (Macaca fuscata). Soc Neu- primary visual cortex in higher level vision. Vision Res 38: 2429 –
rosci Abstr 510.6, 2005. 2454, 1998.
126. Kayaert G, Biederman I, Op de Beeck HP, Vogels R. Tuning for 151. Lee TS, Nguyen M. Dynamics of subjective contour formation in
shape dimensions in macaque inferior temporal cortex. Eur J Neu- the early visual cortex. Proc Natl Acad Sci USA 98: 1907–1911,
rosci 22: 212–224, 2005. 2001.
127. Kayaert G, Biederman I, Vogels R. Shape tuning in macaque 152. Lennie P, Krauskopf J, Sclar G. Chromatic mechanisms in stri-
inferior temporal cortex. J Neurosci 23: 3016 –3027, 2003. ate cortex of macaque. J Neurosci 10: 649 – 669, 1990.
128. Kiani R, Esteky H, Tanaka K. Differences in onset latency of 153. Lennie P, Movshon JA. Coding of color and form in the genicu-
macaque inferotemporal neural responses in primate and non- lostriate visual pathway (invited review). J Opt Soc Am A Opt
primate faces. J Neurophysiol 94: 1587–1596, 2005. Image Sci Vis 22: 2013–2033, 2005.
129. Knierim JJ, Van Essen DC. Neuronal responses to static texture 154. Leopold DA, Bondar IV, Giese MA. Norm-based face encoding
patterns in area V1 of the alert macaque monkey. J Neurophysiol by single neurons in the monkey inferotemporal cortex. Nature
67: 961–980, 1992. 442: 572–575, 2006.
130. Kobatake E, Tanaka K. Neuronal selectivities to complex object 155. Leventhal AG, Thompson KG, Liu D, Zhou Y, Ault SJ. Con-
features in the ventral visual pathway of the macaque cerebral comitant sensitivity to orientation direction, color of cells in layers
cortex. J Neurophysiol 71: 856 – 867, 1994. 2, 3, 4 of monkey striate cortex. J Neurosci 15: 1808 –1818, 1995.
131. Kobatake E, Wang G, Tanaka K. Effects of shape-discrimination 156. Leventhal AG, Wang Y, Schmolesky MT, Zhou Y. Neural corre-
training on the selectivity of inferotemporal cells in adult monkeys. lates of boundary perception. Vis Neurosci 15: 1107–1118, 1998.
J Neurophysiol 80: 324 –330, 1998. 157. Lewis JW, Van Essen DC. Mapping of architectonic subdivisions
132. Koenderink JJ. Solid Shape. Cambridge, MA: MIT Press, 1990, p. in the macaque monkey, with emphasis on parieto-occipital cortex.
1– 699. J Comp Neurol 428: 79 –111, 2000.
133. Koida K, Komatsu H. Effects of task demands on the responses of 158. Lewis JW, Van Essen DC. Corticocortical connections of visual,
color-selective neurons in the inferior temporal cortex. Nat Neu- sensorimotor, multimodal processing areas in the parietal lobe of
rosci 10: 108 –116, 2007. the macaque monkey. J Comp Neurol 428: 112–137, 2000.
134. Komatsu H, Ideura Y. Relationships between color, shape, pat- 159. Liu J, Newsome WT. Correlation between speed perception and
tern selectivities of neurons in the inferior temporal cortex of the neural activity in the middle temporal visual area. J Neurosci 25:
monkey. J Neurophysiol 70: 677– 694, 1993. 711–722, 2005.
135. Komatsu H, Ideura Y, Kaji S, Yamane S. Color selectivity of 160. Liu Y, Vogels R, Orban GA. Convergence of depth from texture
neurons in the inferior temporal cortex of the awake macaque and depth from disparity in macaque inferior temporal cortex.
monkey. J Neurosci 12: 408 – 424, 1992. J Neurosci 24: 795–3800, 2004.

Physiol Rev • VOL 88 • JANUARY 2008 • www.prv.org

86 GUY A. ORBAN

161. Livingstone MS, Hubel DH. Anatomy and physiology of a color 185. Nakamura H, Kuroda T, Wakita M, Kusunoki M, Kato A,
system in the primate visual cortex. J Neurosci 4: 309 –356, 1984. Mikami A, Sakata H, Itoh K. From three-dimensional space
162. Livingstone MS, Pack CC, Born RT. Two-dimensional substruc- vision to prehensile hand movements: the lateral intraparietal area
ture of MT receptive fields. Neuron 30: 781–793, 2001. links the area V3A and the anterior intraparietal area in macaques.
163. Logan DJ, Duffy CJ. Cortical area MSTd combines visual cues to J Neurosci 21: 8174 – 8187, 2001.
represent 3-D self-movement. Cereb Cortex 16: 1494 –1507, 2006. 186. Nelissen K, Vanduffel W, Orban GA. Charting the lower superior
164. Logothetis NK, Sheinberg DL. Visual object recognition. Annu temporal region, a new motion-sensitive region in monkey superior
Rev Neurosci 19: 577– 621, 1996. temporal sulcus. J Neurosci 26: 5929 –5947, 2006.
165. Luppino G. Organization of the posterior parietal lobe and of 187. Nelson JI, Frost BJ. Orientation-selective inhibition from beyond
parietofrontal connections. In: From Monkey Brain to Human the classic visual receptive field. Brain Res 139: 359 –365, 1978.
Brain: A Fyssen Foundation Symposium Bradford Books, edited 188. Newsome WT, Wurtz RH, Dursteler MR, Mikami A. Deficits in
by Dehaene S, Duhamel JR, Hauser MD, and Rizzolatti G. Cam- visual motion processing following ibotenic acid lesions of the
bridge, MA: MIT Press, 2005, p. 235–252. middle temporal visual area of the macaque monkey. J Neurosci 5:
166. Luppino G, Hamed SB, Gamberini M, Matelli M, Galletti C. 825– 840, 1985.
Occipital (V6) and parietal (V6A) areas in the anterior wall of the 190. Nguyenkim JD, DeAngelis GC. Disparity-based coding of three-
parieto-occipital sulcus of the macaque: a cytoarchitectonic study. dimensional surface orientation by macaque middle temporal neu-
Eur J Neurosci 21: 3056 –3076, 2005. rons. J Neurosci 23: 7117–7128, 2003.
167. Maffei L, Fiorentini A. The unresponsive regions of visual corti- 191. Nguyenkim JD, DeAngelis GC. Macaque MT neurons are selec-
cal receptive fields. Vision Res 16: 1131–1139, 1976. tive for 3D surface orientation defined by multiple cues. Soc Neu-
168. Marcar VL, Raiguel SE, Xiao D, Orban GA. Processing of kinet- rosci Abstr 368.12, 2004.
ically defined boundaries in areas V1 and V2 of the macaque 192. Nienborg H, Bridge H, Parker AJ, Cumming BG. Receptive field
monkey. J Neurophysiol 84: 2786 –2798, 2000. size in V1 neurons limits acuity for perceiving disparity modulation.

Downloaded from physrev.physiology.org on October 9, 2008

169. Marcar VL, Xiao DK, Raiguel SE, Maes H, Orban GA. Process- J Neurosci 24: 2065–2076, 2004.
ing of kinetically defined boundaries in the cortical motion area MT 193. Nienborg H, Bridge H, Parker AJ, Cumming BG. Neuronal
of the macaque monkey. J Neurophysiol 74: 1258 –1270, 1995. computation of disparity in V1 limits temporal resolution for de-
170. Marcus DS, Van Essen DC. Scene segmentation and attention in tecting disparity modulation. J Neurosci 25: 10207–10219, 2005.
primate cortical areas V1 and V2. J Neurophysiol 88: 2648 –2658, 194. Obermayer K, Blasdel GG. Geometry of orientation and ocular
2002. dominance columns in monkey striate cortex. J Neurosci 13: 4114 –
171. Marr D, Poggio T. A computational theory of human stereo vision. 4129, 1993.
Proc R Soc Lond B Biol Sci 204: 301–328, 1979. 195. Ohki K, Chung S, Ch’ng YH, Kara P, Reid RC. Functional
172. Maske R, Yamane S, Bishop PO. End-stopped cells and binocular imaging with cellular resolution reveals precise micro-architecture
depth discrimination in the striate cortex of cats. Proc R Soc Lond in visual cortex. Nature 433: 597– 603, 2005.
B Biol Sci 229: 257–762, 1986. 196. O’Keefe LP, Movshon JA. Processing of first- and second-order
173. Matsumoto N, Okada M, Sugase-Miyamoto Y, Yamane S,
motion signals by neurons in area MT of the macaque monkey. Vis
Kawano K. Population dynamics of face-responsive neurons in the
Neurosci 15: 305–317, 1998.
inferior temporal cortex. Cereb Cortex 15: 1103–1112, 2005.
197. Op de Beeck H, Wagemans J, Vogels R. Inferotemporal neurons
174. Maunsell JHR, Van Essen DC. Functional properties of neurons
represent low-dimensional configurations of parameterized shapes.
in middle temporal visual area of the macaque monkey. I. Selectiv-
Nat Neurosci 4: 1244 –1252, 2001.
ity of stimulus direction, speed, orientation. J Neurophysiol 49:
198. Orban GA. Motion processing in monkey striate cortex. In: Cere-
1127–1147, 1983.
bral Cortex. Primary Visual Cortex in Primates, edited by Peters
175. Maunsell JHR, Van Essen DC. Functional properties of neurons
A, Rockland KS. New York: Plenum, 1994, vol. 10, p. 413– 441.
in middle temporal visual area of the macaque monkey. II. Binoc-
ular interactions and sensitivity to binocular disparity. J Neuro- 199. Orban GA. Visual processing in macaque area MT/V5 and its
physiol 49: 1148 –1167, 1983. satellites (MSTd and MSTv). In: Cerebral Cortex, Extrastriate Cor-
176. Merchant H, Battaglia-Mayer A, Georgopoulos AP. Functional tex in Primates,edited by. Rockland KS, Kaas JH, Peters A. New
organization of parietal neuronal responses to optic-flow stimuli. York: Plenum, 1997, vol. 12, p. 359 – 434.
J Neurophysiol 90: 675– 682, 2003. 200. Orban GA. Three-dimensional shape: cortical mechanisms of
177. Mikami A, Newsome WT, Wurtz RH. Motion selectivity in ma- shape extraction. In: The Senses: A Comprehensive Reference,
caque visual cortex. I. Mechanisms of direction and speed selec- edited by Hoy RR, Sheperd GM, Basbaum AI, Kaneko A, Westheimer
tivity in extrastriate area MT. J Neurophysiol 55: 1308 –1327, 1986. G. Oxford, UK: Elsevier. In press.
178. Mikami A, Newsome WT, Wurtz RH. Motion selectivity in ma- 201. Orban GA, Claeys K, Nelissen K, Smans R, Sunaert S, Todd
caque visual cortex. II. Spatiotemporal range of directional inter- JT, Wardak C, Durand JB, Vanduffel W. Mapping the parietal
actions in MT and V1. J Neurophysiol 55: 1328 –1339, 1986. cortex of human and non-human primates. Neuropsychologia 44:
179. Motter BC, Mountcastle VB. The functional properties of the 2647–2667, 2006.
light-sensitive neurons of the posterior parietal cortex studied in 202. Orban GA, Fize D, Peuskens H, Denys K, Nelissen K, Sunaert
waking monkeys: foveal sparing and opponent vector organization. S, Todd J, Vanduffel W. Similarities and differences in motion
J Neurosci 1: 3–26, 1981. processing between the human and macaque brain: evidence from
180. Movshon JA, Adelson EH, Gizzi MS, Newsome WT. The anal- fMRI. Neuropsychologia 41: 1757–1768, 2003.
ysis of moving visual patterns. In: Pattern Recognition Mechanims, 203. Orban GA, Gulyás B. Image segregation by motion: cortical mech-
edited by Chagas C, Gattass R, Gross C. Vatican City: Pontificiae anisms and implementation in neural networks. In: Neural Com-
Academiae Scientiarium Scripta Varia, 1985, vol. 54, p. 117–151. puters, edited by Eckmiller R, von der Malsburg C. Berlin: Springer-
181. Movshon JA, Albright TD, Stoner GR, Majaj NJ, Smith MA. Verlag, 1988, p. 149 –158.
Cortical responses to visual motion in alert and anesthetized mon- 204. Orban GA, Janssen P, Vogels R. Extracting 3D structure from
keys. Nat Neurosci 6: 3, 2003. disparity. Trends Neurosci 29: 466 – 473, 2006.
182. Movshon JA, Newsome WT. Visual response properties of striate 205. Orban GA, Kato H, Bishop PO. End-zone region in receptive
cortical neurons projecting to area MT in macaque monkeys. fields of hypercomplex and other striate neurons in the cat. J Neu-
J Neurosci 16: 7733–7741, 1996. rophysiol 42: 818 – 832, 1979.
183. Murata A, Gallese V, Luppino G, Kaseda M, Hideo S. Selectiv- 206. Orban GA, Kennedy H, Bullier J. Velocity sensitivity and direc-
ity for the shape, size, orientation of objects for grasping in neurons tion selectivity of neurons in areas V1 and V2 of the monkey:
of monkey parietal area AIP. J Neurophysiol 83: 2580 –2601, 2000. influence of eccentricity. J Neurophysiol 56: 462– 480, 1986.
184. Mysore SG, Vogels R, Raiguel SE, Orban GA. Processing of 207. Orban GA, Kennedy H, Maes H. Response to movement of
kinetic boundaries in macaque V4. J Neurophysiol 95: 1864 –1880, neurons in areas 17 and 18 of the cat: velocity sensitivity. J Neu-
2006. rophysiol 45: 1043–1058, 1981.

Physiol Rev • VOL 88 • JANUARY 2008 • www.prv.org

 HIGHER ORDER VISUAL PROCESSING 87

208. Orban GA, Lagae L, Raiguel S, Xiao D, Maes H. The speed 234. Priebe NJ, Lisberger SG. Estimating target speed from the pop-
tuning of medial superior temporal (MST) cell responses to optic- ulation response in visual area MT. J Neurosci 24: 1907–1916, 2004.
flow components. Perception 24: 269 –285, 1995. 235. Priebe NJ, Lisberger SG, Movshon JA. Tuning for spatiotempo-
209. Orban GA, Lagae L, Verri A, Raiguel S, Xiao D, Maes H, Torre ral frequency and speed in directionally selective neurons of ma-
V. First-order analysis of optical flow in monkey brain. Proc Natl caque striate cortex. J Neurosci 26: 2941–2950, 2006.
Acad Sci USA 89: 2595–2599, 1992. 236. Prince SJ, Pointon AD, Cumming BG, Parker AJ. Quantitative
210. Orban GA, Vogels R. The neuronal machinery involved in succes- analysis of the responses of V1 neurons to horizontal disparity in
sive orientation discrimination. Prog Neurobiol 55: 117–147, 1998. dynamic random-dot stereograms. J Neurophysiol 87: 191–208,
211. Pack CC, Berezovskii VK, Born RT. Dynamic properties of 2002.
neurons in cortical area MT in alert and anaesthetized macaque 237. Ptito A, Zatorre RJ, Larson WL, Tosoni C. Stereopsis after
monkeys. Nature 414: 905–908, 2001. unilateral anterior temporal lobectomy. Dissociation between local
212. Pack CC, Berezovskii VK, Born RT. Cortical responses to visual and global measures. Brain 114: 1323–1333, 1991.
motion in alert and anesthetized monkeys. Nat Neurosci 6: 3– 4, 238. Qian N, Andersen RA. Transparent motion perception as detec-
2003. tion of unbalanced motion signals. II. Physiology. J Neurosci 14:
213. Pack CC, Born RT. Temporal dynamics of a neural solution to the 7367–7380, 1994.
aperture problem in visual area MT of macaque brain. Nature 409: 239. Qiu FT, von der Heydt R. Figure and ground in the visual cortex:
1040 –1042, 2001.
V2 combines stereoscopic cues with Gestalt rules. Neuron 47:
214. Pack CC, Gartland AJ, Born RT. Integration of contour and
155–166, 2005.
terminator signals in visual area MT of alert macaque. J Neurosci
240. Raffi M, Squatrito S, Maioli MG. Neuronal responses to optic
24: 3268 –3280, 2004.
flow in the monkey parietal area PEc. Cereb Cortex 12: 639 – 646,
215. Pack CC, Hunter JN, Born RT. Contrast dependence of suppres-
sive influences in cortical area MT of alert macaque. J Neuro- 2002.
241. Raiguel S, Van Hulle MM, Xiao DK, Marcar VL, Lagae L, Orban

Downloaded from physrev.physiology.org on October 9, 2008

physiol 93: 1809 –1815, 2005.
216. Pack CC, Livingstone MS, Duffy KR, Born RT. End-stopping GA. Size and shape of receptive fields in the medial superior
and the aperture problem: two-dimensional motion signals in temporal area (MST) of the macaque. Neuroreport 8: 2803–2808,
macaue V1. Neuron 39: 671– 680, 2003. 1997.
217. Page WK, Duffy CJ. MST neuronal responses to heading direction 242. Raiguel S, Van Hulle MM, Xiao DK, Marcar VL, Orban GA.
during pursuit eye movements. J Neurophysiol 81: 596 – 610, 1999. Shape and spatial distribution of receptive fields and antagonistic
218. Page WK, Duffy CJ. Heading representation in MST: sensory motion surrounds in the middle temporal area (V5) of the macaque.
interactions and population encoding. J Neurophysiol 89: 1994 – Eur J Neurosci 7: 2064 –2082, 1995.
2013, 2003. 243. Ramsden BM, Hung CP, Roe AW. Real and illusory contour
219. Paolini M, Distler C, Bremmer F, Lappe M, Hoffmann KP. processing in area V1 of the primate: a cortical balancing act. Cereb
Responses to continuously changing optic flow in area MST. J Neu- Cortex 11: 648 – 665, 2001.
rophysiol 84: 730 –743, 2000. 244. Read HL, Siegel RM. Modulation of responses to optic flow in
220. Pasupathy A. Neural basis of shape representation in the primate area 7a by retinotopic and oculomotor cues in monkey. Cereb
brain. Prog Brain Res 154: 293–313, 2006. Cortex 7: 647– 661, 1997.
221. Pasupathy A, Connor CE. Responses to contour features in 245. Reid RC, Alonso JM. Specificity of monosynaptic connections
macaque area V4. J Neurophysiol 82: 2490 –2502, 1999. from thalamus to visual cortex. Nature 378: 281–284, 1995.
222. Pasupathy A, Connor CE. Shape representation in area V4: 246. Ringach DL, Hawken MJ, Shapley R. Dynamics of orientation
position-specific tuning for boundary conformation. J Neuro- tuning in macaque primary visual cortex. Nature 387: 281–284,
physiol 86: 2505–2519, 2001. 1997.
223. Pasupathy A, Connor CE. Population coding of shape in area V4. 247. Ringach DL, Hawken MJ, Shapley R. Dynamics of orientation
Nat Neurosci 5: 1332–1338, 2002. tuning in macaque V1: the role of global and tuned suppression.
224. Perrett DI, Rolls ET, Caan W. Visual neurones responsive to J Neurophysiol 90: 342–352, 2003.
faces in the monkey temporal cortex. Exp Brain Res 47: 329 –342, 248. Ringach DL, Shapley RM, Hawken MJ. Orientation selectivity in
1982. macaque V1: diversity and laminar dependence. J Neurosci 22:
225. Perrett DI, Smith PA, Mistlin AJ, Chitty AJ, Head AS, Potter 5639 –5651, 2002.
DD, Broennimann R, Milner AD, Jeeves MA. Visual analysis of 249. Rizzolatti G, Luppino G. The cortical motor system. Neuron 31:
body movements by neurones in the temporal cortex of the 889 –901, 2001.
macaque monkey: a preliminary report. Behav Brain Res 16: 153– 250. Rodman HR, Albright TD. Single-unit analysis of pattern-motion
170, 1985. selective properties in the middle temporal visual area (MT). Exp
226. Peterhans E, von der Heydt R. Mechanisms of contour percep- Brain Res 75: 53– 64, 1989.
tion in monkey visual cortex II Contours bridging gaps. J Neurosci
251. Rolls ET, Aggelopoulos NC, Zheng F. The receptive fields of
9: 1749 –1763, 1989.
inferior temporal cortex neurons in natural scenes. J Neurosci 23:
227. Peterhans E, von der Heydt R. Functional organization of area
339 –348, 2003.
V2 in the alert macaque. Eur J Neurosci 5: 509 –524, 1993.
252. Rossi AF, Desimone R, Ungerleider LG. Contextual modulation
228. Phinney RE, Siegel RM. Speed selectivity for optic flow in area 7a
in primary visual cortex of macaques. J Neurosci 21: 1698 –1709,
of the behaving macaque. Cereb Cortex 10: 413–21, 2000.
229. Poggio GF, Fischer B. Binocular interaction and depth sensitivity 2001.
in striate and prestriate cortex of behaving rhesus monkey. J Neu- 253. Rozzi S, Calzavara R, Belmalih A, Borra E, Gregoriou GG,
rophysiol 40: 1392–1405, 1977. Matelli M, Luppino G. Cortical connections of the inferior pari-
230. Poggio GF, Gonzales F, Krause F. Stereoscopic mechanisms in etal cortical convexity of the macaque monkey. Cereb Cortex 16:
monkey visual cortex: binocular correlation and disparity selectiv- 1389 –1417, 2006.
ity. J Neurosci 8: 4531– 4550, 1988. 254. Rust NC, Mante V, Simoncelli EP, Movshon JA. How MT cells
231. Polat U, Mizobe K, Pettet MW, Kasamatsu T, Norcia AM. analyze the motion of visual patterns. Nat Neurosci 9: 1421–1431,
Collinear stimuli regulate visual responses depending on cell’s 2006.
contrast threshold. Nature 391: 580 –584, 1998. 255. Saito H, Yukie M, Tanaka K, Hikosaka K, Fukada Y, Iwai E.
232. Preuss TM, Qi H, Kaas JH. Distinctive compartmental organiza- Integration of direction signals of image motion in the superior
tion of human primary visual cortex. Proc Nat Acad Sci USA 96: temporal sulcus of the macaque monkey. J Neurosci 6: 145–157,
11601–11606, 1999. 1986.
233. Priebe NJ, Cassanello CR, Lisberger SG. The neural represen- 256. Sakata H, Shubutani H, Ito Y, Tsurugai K. Parietal visual neu-
tation of speed in macaque area MT/V5. J Neurosci 23: 5650 –5661, rons respond to rotary movements of the visual stimulus. J Physiol
2003. Soc Jpn 46: 455, 1984.

Physiol Rev • VOL 88 • JANUARY 2008 • www.prv.org

88 GUY A. ORBAN

257. Sakata H, Shibutani H, Ito Y, Tsurugai K. Parietal cortical 281. Sillito AM. The contribution of inhibitory mechanisms to the
neurons responding to rotary movement of visual stimulus in receptive field properties of neurons in the striate cortex of the cat.
space. Exp Brain Res 61: 658 – 663, 1986. J Physiol 250: 305–329, 1975.
258. Sakata H, Taira M, Kusunoki M, Murata A, Tanaka Y. The 282. Sillito AM, Griever KL, Jones HE, Cudeiro J, Davis J. Visual
TINS Lecture: the parietal association cortex in depth perception cortical mechanisms detecting focal orientation discontinuities.
and visual control of hand action. Trends Neurosci 20: 350 –357, Nature 378: 492– 496, 1995.
1997. 283. Sillito AM, Jones HE, Gerstein GL, West DC. Feature-linked
259. Sakata H, Taira M, Kusunoki M, Murata A, Tanaka Y, Tsutsui synchronization of thalamic relay cell firing induced by feedback
K. Neural coding of 3D features of objects for hand action in the from the visual cortex. Nature 369: 479 – 482, 1994.
parietal cortex of the monkey. Philos Trans R Soc Lond B Biol Sci 284. Sillito AM, Kemp JA, Milson JA, Berardi N. A re-evaluation of
353: 1363–1373, 1998. the mechanisms underlying simple cell orientation selectivity.
260. Sakata H, Tsutsui K, Taira M. Toward an understanding of the Brain Res 194: 517–520, 1980.
neural processing for 3D shape perception. Neuropsychologia 43: 285. Smith MA, Bair W, Movshon JA. Dynamics of suppression in
151–161, 2005. macaque primary visual cortex. J Neurosci 26: 4826 – 4834, 2006.
261. Saleem KS, Suzuki W, Tanaka K, Hashikawa T. Connections 286. Smith MA, Majaj NJ, Movshon JA. Dynamics of motion signaling
by neurons in macaque area MT. Nat Neurosci 8: 220 –228, 2005.
between anterior inferotemporal cortex and superior temporal sul-
287. Snowden RJ, Treue S, Andersen RA. The response of neurons in
cus regions in the macaque monkey. J Neurosci 20: 5083–5101,
areas V1 and MT of the alert rhesus monkey to moving random dot
2000.
patterns. Exp Brain Res 88: 389 – 400, 1992.
262. Salinas E, Abbott LF. A model of multiplicative neural responses
288. Snowden RJ, Treue S, Erickson RG, Andersen RA. The re-
in parietal cortex. Proc Natl Acad Sci USA 93: 11956 –11961, 1996. sponse of area MT and V1 neurons to transparent motion. J Neu-
263. Sáry Gy Vogels R, Kovács GY, Orban GA. Responses of monkey rosci 11: 2768 –2785, 1991.
inferior temporal neurons to luminance-, motion-, texture-defined

Downloaded from physrev.physiology.org on October 9, 2008

289. Solomon SG, Lennie P. Chromatic gain controls in visual cortical
gratings. J Neurophysiol 73: 1341–1354, 1995. neurons. J Neurosci 25: 4779 – 4792, 2005.
264. Sáry GY, Vogels R, Orban GA. Cue-invariant shape selectivity of 290. Solomon SG, Peirce JW, Lennie P. The impact of suppressive
macaque inferior temporal neurons. Science 260: 995–997, 1993. surrounds on chromatic properties of cortical neurons. J Neurosci
265. Sceniak MP, Hawken MJ, Shapley R. Visual spatial character- 24: 148 –160, 2004.
ization of macaque V1 neurons. J Neurophysiol 85: 1873–1887, 291. Solomon SG, White AJR, Martin PR. Temporal contrast sensi-
2001. tivity in the lateral geniculate nucleus of a New World monkey, the
266. Sceniak MP, Ringach DL, Hawken MJ, Shapley R. Contrast’s marmoset Callithrix jacchus. J Physiol 517: 907–917, 1999.
effect on spatial summation by macaque V1 neurons. Nat Neurosci 292. Spillmann HB, Peterhans E. Stereoscopic illusory contours-cor-
2: 733–739, 1999. tical neuron responses and human perception. J Cogn Neurosci
267. Schaafsma SJ, Duysens J. Neurons in the ventral intraparietal 14:1018 –1029, 2002.
area of awake macaque monkey closely resemble neurons in the 293. Srivastava S, Orban GA, Janssen P. Selectivity for three-dimen-
dorsal part of the medial superior temporal area in their responses sional shape in macaque posterior parietal cortex. Soc Neurosci
to optic flow patterns. J Neurophysiol 76: 4056 – 4068, 1996. Abstr 407.9, 2006.
268. Schiller PH. The effects of V4 and middle temporal (MT) area 294. Steinmetz MA, Motter BC, Duffy CJ, Mountcastle VB. Func-
lesions on visual performance in the rhesus monkey. Vis Neurosci tional properties of parietal visual neurons: radial organization of
10: 717–746, 1993. directionalities within the visual field. J Neurosci 7: 177–191, 1987.
269. Schiller PH, Finlay BL, Volman SF. Quantitative studies of sin- 295. Stoner GR, Albright TD. Neural correlates of perceptual motion
gle-cell properties in monkey striate cortex. I. Spatiotemporal coherence. Nature 358: 412– 414, 1992.
organization of receptive fields. J Neurophysiol 39: 1288 –1319, 296. Stout D, Chaminade T. The evolutionary neuroscience of tool
1976. making. Neuropsychologia 45: 1091–1100, 2007.
270. Schiller PH, Finlay BL, Volman SF. Quantitative studies of sin- 297. Sugase Y, Yamane S, Ueno S, Kawano K. Global and fine infor-
gle-cell properties in monkey striate cortex. II. Orientation speci- mation coded by single neurons in the temporal visual cortex.
ficity and ocular dominance. J Neurophysiol 39: 1320 –1333, 1976. Nature 400: 869 – 873, 1999.
271. Schiller PH, Finlay BL, Volman SF. Quantitative studies of sin- 298. Sugihara H, Murakami I, Shenoy KV, Andersen RA, Komatsu
gle-cell properties in monkey striate cortex. III. Spatial frequency. H. Response of MSTd neurons to simulated 3D orientation of
J Neurophysiol 39: 1334 –1351, 1976. rotating planes. J Neurophysiol 87: 273–285, 2002.
272. Schlack A, Sterbing-D’Angelo SJ, Hartung K, Hoffmann KP, 299. Sugita Y. Grouping of image fragments in primary visual cortex.
Bremmer F. Multisensory space representations in the macaque Nature 401: 269 –272, 1999.
ventral intraparietal area. J Neurosci 25: 4616 – 4625, 2005. 300. Taira M, Tsutsui KI, Jiang M, Yara K, Sakata H. Parietal
neurons represent surface orientation from the gradient of binoc-
273. Schwartz EL, Desimone R, Albright TD, Gross CG. Shape
ular disparity. J Neurophysiol 83: 3140 –3146, 2000.
recognition and inferior temporal neurons. Proc Natl Acad Sci USA
301. Tamura H, Tanaka K. Visual response properties of cells in the
80: 5776 –5778, 1983.
ventral and dorsal parts of the macaque inferotemporal cortex.
274. Sereno AB, Maunsell JHR. Shape selectivity in primate lateral
Cereb Cortex 11: 384 –399, 2001.
intraparietal cortex. Nature 395: 500 –503, 1998. 302. Tanabe S, Takahiro D, Umeda K, Fujita I. Disparity-tuning
275. Shapley R, Hawken M, Ringach DL. Dynamics of orientation characteristics of neuronal responses to dynamic random-dot ste-
selectivity in the primary visual cortex and the importance of reograms in macaque visual area V4. J Neurophysiol 94: 2683–2699,
cortical inhibition. Neuron 38: 689 – 699, 2003. 2005.
276. Shenoy KV, Crowell JA, Andersen RA. Pursuit speed compen- 303. Tanabe S, Umeda K, Fujita I. Rejection of false matches for
sation in cortical area MSTd. J Neurophysiol 88: 2630 –2647, 2002. binocular correspondence in macaque visual cortical area V4.
277. Shikata E, Tanaka Y, Nakamura H, Taira M, Sakata H. Selec- J Neurosci 24: 8170 – 8180, 2004.
tivity of the parietal visual neurones in 3D orientation of surface of 304. Tanaka H, Takanori U, Yoshiyama K, Kato M, Fujita I. Pro-
stereoscopic stimuli. Neuroreport 7: 2389 –2394, 1996. cessing of shape defined by disparity in monkey inferior temporal
278. Shimojo S, Silverman GH, Nakayama K. Occlusion and the cortex. J Neurophysiol 85: 735–744, 2001.
solution to the aperture problem for motion. Vision Res 29: 619 – 305. Tanaka K. Inferotemporal cortex, object vision. Annu Rev Neu-
626, 1989. rosci 19: 109 –139, 1996.
279. Siegel RM, Andersen RA. The perception of structure from visual 306. Tanaka K, Fukada Y, Saito HA. Underlying mechanisms of the
motion in monkey and man. J Cogn Neurosci 2: 306 –319, 1990. response specificity of expansion/contraction and rotation cells in
280. Siegel RM, Read HL. Analysis of optic flow in the monkey parietal the dorsal part of the medial superior temporal area of the macaque
area 7a. Cereb Cortex 7: 327–346, 1997. monkey. J Neurophysiol 62: 642– 656, 1989.

Physiol Rev • VOL 88 • JANUARY 2008 • www.prv.org

 HIGHER ORDER VISUAL PROCESSING 89

307. Tanaka K, Hikosaka K, Saito H, Yukie M, Fukada Y, Iwai E. 329. Van Essen DC, Dickson J, Harwell J, Hanlon D, Anderson CH,
Analysis of local and wide-field movements in the superior tempo- Drury HA. An integrated software system for surface-based anal-
ral visual areas of the macaque monkey. J Neurosci 6: 134 –144, yses of cerebral cortex. J Am Med Information Assoc 8: 443– 459,
1986. 2001.
308. Tanaka K, Saito H. Analysis of motion of the visual field by 330. Versavel M, Orban GA, Lagae L. Responses of visual cortical
direction, expansion/contraction, rotation cells clustered in the neurons to curved stimuli and chevrons. Vision Res 30: 235–248,
dorsal part of the medial superior temporal area of the macaque 1990.
monkey. J Neurophysiol 62: 626 – 641, 1989. 331. Vogels R. Categorization of complex visual images by rhesus
309. Tanaka K, Saito H, Fukada Y, Moriya M. Coding visual images monkeys. Part 2: single-cell study. Eur J Neurosci 11: 1239 –1255,
of objects in the inferotemporal cortex of the macaque monkey. 1999.
J Neurophysiol 66: 170 –189, 1991. 332. Vogels R, Orban GA. Quantitative study of striate single unit
310. Tanaka K, Sugita Y, Moriya M, Saito H. Analysis of object responses in monkeys performing an orientation discrimination
motion in the ventral part of the medial superior temporal area of task. Exp Brain Res 84: 1–11, 1991.
the macaque visual cortex. J Neurophysiol 69: 128 –142, 1993. 333. Vogels R, Orban GA. Coding of stimulus invariances by inferior
311. Thomas OT, Cumming BG, Parker AJ. A specialization for rel- temporal neurons. In: Progress in Brain Research: Extragenicu-
ative disparity in V2. Nat Neurosci 5: 472– 478, 2002. lostriate Mechanisms Underlying Visually-Guided Orientation
312. Thorell LG, De Valois RL, Albrecht DG. Spatial mapping of Behavior, edited by Norita M, Bando T, Stein B. Amsterdam:
monkey V1 cells with pure color and luminance stimuli. Vision Res Elsevier, 1996, vol. 112, p. 195–211.
24: 751–769, 1984. 334. Von der Heydt R, Peterhans E. Mechanisms of contour percep-
313. Tovee MJ, Rolls ET, Azzopardi P. Translation invariance in the tion in monkey visual cortex. I. Lines of pattern discontinuity.
responses to faces of single neurons in the temporal visual cortical J Neurosci 9: 1731–1748, 1989.
areas of the alert macaque. J Neurophysiol 72: 1049 –1060, 1994. 335. Von der Heydt R, Peterhans E, Baumgartner G. Illusory con-

Downloaded from physrev.physiology.org on October 9, 2008

314. Treue S, Andersen RA. Neural responses to velocity gradients in tours and cortical neuron responses. Science 224: 1260 –1262, 1984.
macaque cortical area MT. Vis Neurosci 13: 797– 804, 1996. 336. Von der Heydt R, Zhou H, Friedman HS. Representation of
315. Tsao DY, Freiwald WA, Tootell RBH, Livingstone MS. A cor- stereoscopic edges in monkey visual cortex. Vision Res 40: 1955–
tical region consisting entirely of ACE-selective cells. Science 311: 1967, 2000.
670 – 674, 2006. 337. Wachter T, Sejnowski TJ, Albright TD. Representation of color
316. Tsunoda K, Yamane Y, Nishizaki M, Tanifuji M. Complex ob- stimuli in awake macaque primary visual cortex. Neuron 37: 681–
jects are represented in macaque inferotemporal cortex by the 691, 2003.
combination of feature columns. Nat Neurosci 4: 832– 838, 2001. 338. Warren WH Jr, Hannon DJ. Eye movements and optical flow. J
Opt Soc Am 7: 160 –169, 1990.
317. Tsutsui K, Jiang M, Yara K, Sakata H, Taira M. Integration of
339. Watanabe M, Tanaka H, Uka T, Fujita I. Disparity-selective
perspective and disparity cues in surface-orientation-selective neu-
neurons in area V4 of macaque monkeys. J Neurophysiol 87: 1960 –
rons of area CIP. J Neurophysiol 86: 2856 –2867, 2001.
1973, 2002.
318. Tsutsui K, Sakata H, Naganuma T, Taira M. Neural correlates
340. Wurtz RH. Comparison of effects of eye movements and stimulus
for perception of 3D surface orientation from texture gradient.
movements on striate cortex neurons of the monkey. J Neuro-
Science 298: 409 – 412, 2002.
physiol 32: 987–994, 1969.
319. Uka T, Tanabe S, Watanabe M, Fujita I. Neural correlates of fine
341. Xiao DK, Marcar VL, Raiguel SE, Orban GA. Selectivity of
depth discrimination in monkey inferior temporal cortex. J Neuro- macaque MT/V5 neurons for surface orientation in depth specified
sci 25: 10796 –10802, 2005. by motion. Eur J Neurosci 9: 956 –964, 1997.
320. Uka T, Tanaka H, Yoshiyama K, Kato M, Fujita I. Disparity 342. Xiao DK, Raiguel S, Marcar V, Koenderink J, Orban GA.
selectivity of neurons in monkey inferior temporal cortex. J Neu- Spatial heterogeneity of inhibitory surrounds in the middle tempo-
rophysiol 84: 120 –132, 2000. ral visual area. Proc Natl Acad Sci USA 92: 11303–11306, 1995.
321. Ungerleider LG, Desimone R. Cortical connections of visual area 343. Xiao DK, Raiguel S, Marcar V, Orban GA. The spatial distribu-
MT in the macaque. J Comp Neurol 248: 190 –222, 1986. tion of the antagonistic surround of MT/V5 neurons. Cereb Cortex
322. Ungerleider LG, Mishkin M. Two cortical visual systems. In: 7: 662– 677, 1997.
Analysis of Visual Behavior,edited by Ingle DJ, Goodale MA, 344. Xing D, Ringach DL, Shapley R, Hawken MJ. Correlation of
Mansfield RJW. Cambridge, MA: MIT Press, 1982, p. 549 –586. local and global orientation and spatial frequency tuning in macaque
323. Upadhyay UD, Page WK, Duffy CJ. MST responses to pursuit V1. J Physiol 557: 923–933, 2004.
across optic flow with motion parallax. J Neurophysiol 84: 818 – 345. Xing D, Shapley RM, Hawken MJ, Ringach DL. Effect of stim-
826, 2000. ulus size on the dynamics of orientation selectivity in macaque V1.
324. Vanduffel W, Fize D, Mandeville JB, Nelissen K, Van Hecke P, J Neurophysiol 94: 799 – 812, 2005.
Rosen BR, Tootell RB, Orban GA. Visual motion processing 346. Yamane Y, Tsunoda K, Matsumoto M, Phillips AN, Tanifuji M.
investigated using contrast agent-enhanced fMRI in awake behav- Representation of the spatial relationship among object parts by
ing monkeys. Neuron 32: 565–577, 2001. neurons in macaque inferotemporal cortex. J Neurophysiol 96:
325. Vanduffel W, Fize D, Peuskens H, Denys K, Sunaert S, Todd 3147–3156, 2006.
JT, Orban GA. Extracting 3D from motion: differences in human 347. Yazdanbakhsh A, Livingstone MS. End stopping in V1 is sensi-
and monkey intraparietal cortex. Science 298: 413– 415, 2002. tive to contrast. Nat Neurosci 9: 697–702, 2006.
326. Vanduffel W, Tootell RBH, Schoups A, Orban GA. The organi- 348. Young MP, Yamane S. Sparse population coding of faces in the
zation of orientation selectivity throughout macaque visual cortex. inferotemporal cortex. Science 256: 1327–1331, 1992.
Cerebral Cortex 12: 647– 662, 2002. 349. Zhang T, Heuer HW, Britten KH. Parietal area VIP neuronal
327. Van Essen DC. Organization of visual areas in macaque and hu- responses to heading stimuli are encoded in head-centered coor-
man cerebral cortex. In: The Visual Neurosciences, edited by dinates. Neuron 42: 993–1001, 2004.
Chalupa LM, Werner JS. Cambridge, MA: MIT Press, 2004, vol. 1, p. 350. Zhou H, Friedman HS, von der Heydt R. Coding of border
507–521. ownership in monkey visual cortex. J Neurosci 20: 6594 – 6611,
328. Van Essen DC, Anderson CH, Felleman DJ. Information pro- 2000.
cessing in the primate visual system: an integrated systems per- 351. Zipser K, Lamme VA, Schiller PH. Contextual modulation in
spective. Science 255: 419 – 423, 1992. primary visual cortex. J Neurosci 16: 7376 –7389, 1996.

Physiol Rev • VOL 88 • JANUARY 2008 • www.prv.org