9.
13 The Human Brain Class 7
Category Selectivity, Controversies,
and MVPA
Outline:
I. Leftovers from Experimental Design
event-related versus blocked designs
why use functional regions of interest?
two-factor designs, main effects, interactions
II. Category Selective Regions of Visual Cortex
and Haxby’s Important Critique
III. Neural Decoding
IV. Short Quiz at 12:17
1
Decisions toward an Actual Experiment
1. What exact conditions will you run in each experiment?
strive for minimal pairs that manipulate just one mental process
this is the crux of the matter in experimental design
2. What task will subject do in the scanner?
for visual experiments usually passive viewing or 1-back
don’t have diff tasks for diff stimuli, that could introduce a _______?
3. Will you have “baseline” conditions? Of what? Why?
for vision, staring at a cross (no eye movements)
useful to have a baseline of minimal visual processing
to look at not just the difference in response, but the RATIO
MEH
2
Decisions toward an Actual Experiment
1. What exact conditions will you run in each experiment?
strive for minimal pairs that manipulate just one mental process
this is the crux of the matter in experimental design
2. What task will subject do in the scanner?
for visual experiments usually passive viewing or 1-back
don’t have diff tasks for diff stimuli, that could introduce a _______?
3. Will you have “baseline” conditions? Of what? Why?
for vision, staring at a cross (no eye movements)
useful to have a baseline of minimal visual processing
to look at not just the difference in response, but the RATIO
need some kind of baseline to measure selectivity.
WOW
one the other hand there is no perfect baseline
3
Decisions toward an Actual Experiment
Suppose you get to scan ten subjects for one hour each.
4. Will you assign different conditions to different subjects, or have
each subject do all conditions?
Whenever possible, run all conditions within subjects.
Suppose 1/3 of class was always graded by Heather, 1/3 always by Dana, and 1/3
always by Anya? Would that be fair?
Neither is it “fair” to use different people’s brains for different conditions.
When is a completely within-subjects design not possible?
5. How many “runs” will you include, and which conditions will
happen in each run (e.g., 1 cond/ run, or all conds in each run, etc.)?
Have all conditions within each run if possible, so differences across runs
(e.g. in how sleepy the subject is) affect all conditions equally.
7.If .multiple conditions per run, will they be clumped or interleaved?
8. What rate of presentation?
9. What order of stimuli/conditions within or across runs?
Many tradeoffs here...
10. How exactly will you analyze your data? 4
Blocked (clumped) vs. Event-related (mixed)
Source: Buckner 1998
What is the challenge with rapid mixed?
Figures © 1998 Wiley-Liss, Inc. All rights reserved. This content is excluded from our Creative Commons
license. See https://ocw.mit.edu/fairuse for more information. Source: Hum. Brain Mapping 6:373–377, 1998. 5
Observed: the sum of all of these:
Uh-oh.
The crazy thing:
All these events add up almost linearly, so with enough repetitions
of each condition, you can pull out the response to each condition.
It just takes a lot of trials.
Tradeoff:
Blocked: Big effects w/out
lots of data.
Event-related: conditions
unpredictable, but need
more data to detect
differences ($$$).
Photos & images © sources unknown. All rights reserved. This content is excluded from
our Creative Commons license. See https://ocw.mit.edu/fairuse for more information.
6
9.13 The Human Brain Class 6
Experimental Design
Outline:
I. Leftovers from Experimental Design
event-related versus blocked designs
why use functional regions of interest?
two-factor designs, main effects, interactions
II. Category Selective Regions of Visual Cortex
and Haxby’s Important Critique
III. Neural Decoding
IV. Short Quiz at 12:17
7
Why Use Functional Regions of Interest (fROIs)
Defined in Each Subject Individually
with a Localizer Scan
Brains differ from one another and so cannot be
perfectly aligned.
It is like trying to align faces:
No matter how hard you try, someone’s mouth will
land on someone else’s chin
If you were a dermatologist studying skin cancers that arise on the upper
lip, it would be sloppy to just align photos and chose one location.
That may or may not be the upper lip in any individual face.
Instead, find each person’s upper lip, then study that.
Similarly, the exact location of the FFA varies from subject to subject.
So, if you want to study it, you have to first find it with a localizer scan
in each subject, then you can measure its response to new conditions.
Superimposed faces © BioMotionLab/York University. All rights reserved. This content is excluded
from our Creative Commons license. See https://ocw.mit.edu/fairuse for more information. 8
Standard Designs
So far, we’ve been talking about the simplest possible
experimental design:
• Manipulate one factor with two levels (“conditions”), e.g.:
e.g., faces & objects, snakes & nonsnakes, moving & stationary...
of course, we could have more than two conditions:
or we could get fancy and...
• Manipulate 2 factors orthogonally (a “2x2 design”), e.g……..
for example...
9
“Factorial Designs”
faces objects
• What are the F/O Monitor for
two factors? Atten- a a Face/obj
• What are the ded repetitions
levels
(conditions) F/O
Un- Monitor for
within each? a a
Atten- letter
ded Repetitions
This design enables us to ask…..
1. Does the response (in a given region) depend on stimulus category?
Photos & images © sources unknown. All rights reserved. This content is excluded from
our Creative Commons license. See https://ocw.mit.edu/fairuse for more information. 10
“Factorial Designs”
faces objects
Atten- a
ded a
vs.
Un-
Atten- a a
ded
A “main effect” of stimulus type
1. Does the response (in a given region) depend on stimulus category?
Photos & images © sources unknown. All rights reserved. This content is excluded from
our Creative Commons license. See https://ocw.mit.edu/fairuse for more information. 11
“Factorial Designs”
faces objects
Atten- a
ded a
vs.
Un-
Atten- a a
ded
A “main effect” of attention
2. Does the response (in a given region) depend on attention?
Photos & images © sources unknown. All rights reserved. This content is excluded from
our Creative Commons license. See https://ocw.mit.edu/fairuse for more information. 12
“Factorial Designs”
faces objects F-O
What might the
data look like and
what would the Atten- a
different ded a
outcomes
mean?….. vs.
Un-
Atten- a a
ded
A difference of differences
An interaction of stim categ x attention
3. Does the effect of stim category depend on attention?
Photos & images © sources unknown. All rights reserved. This content is excluded from
our Creative Commons license. See https://ocw.mit.edu/fairuse for more information. 13
Main Effects vs Interactions
Main effect of factor X: an overall effect of X (i.e., difference betwn X1 and X2).
An interaction of factor X and factor Y: The effect of X depends on Y (& vv)
attended
unattended
FFA Response FFA Response FFA Response FFA Response
Objs Faces Objs Faces Objs Faces Objs Faces
No main effect of stimtype Main effect of stimtype Main effect of stim Main effect of st
No main effect of atten No main effect of atten Main effect of atten Main effect of atten
No intxn of st x att. No intxn of st x att. No intxn of st x att. Intxn of st x att.
What does this mean? What does this mean? What does this mean?
What is the key sign of an interaction?
Do the lines need to cross? 14
9.13 The Human Brain Class 6
Experimental Design
Outline:
I. Leftovers from Experimental Design
event-related versus blocked designs
why use functional regions of interest?
two-factor designs, main effects, interactions
II. Category Selective Regions of Visual Cortex
and Haxby’s Important Critique
III. Neural Decoding
IV. Short Quiz at 12:17
15
Category Selective Regions in Visual Cortex
Faces Bodies Places
PPA
places
EBA bodies
What else?
FFA
faces
16
Do other
regions exist
that are
X X X X
selective for
other
categories? X X X X
Some
disagreement X X X X X
about:
Tool regions?
Hands?
X X X X
Downing et al. (2006)
No one finds selective responses for cars, chairs, food, or lots of others.
So, some categories are “special”.
Photos & images © sources unknown. All rights reserved. This content is excluded from
Or are they? 17
our Creative Commons license. See https://ocw.mit.edu/fairuse for more information.
Visual Cortical Regions Selectively Responsive to
visual categories:
Faces Bodies Places
PPA
places
EBA bodies
Ongoing Controversies:
• Discrete regions vs gradients.
FFA
• Categories, or their correlated
faces features?
• How specific?
The most serious challenge... 18
Visual Cortical Regions Selectively Responsive to
visual categories:
Faces Bodies Places
PPA
places
Jim Haxby
EBA bodies
Even if FFA responds weakly to chairs & cars,
That does not mean it does not represent
FFA information about chairs and cars.
faces Even low responses could hold information
if the pattern of response across voxels is
Photo of Haxby © source unknown. All rights reserved. This
content is excluded from our Creative Commons license. See
different for each category.
https://ocw.mit.edu/fairuse for more information.
How would we tell? 19
Does the FFA hold Information about Nonfaces (e.g., cars versus chairs)?
1. Collect fMRI response to chairs and cars, for each voxel in FFA.
2. Repeat in same subject.
3. Now ask: is the pattern more similar within a category… than between
Within
category
between
categories
What does this If r(Within) > r(Between)
method reveal
about FFA?
the region contains info. distinguishing cars & chairs!
Images above © sources unknown. All rights reserved. This content is excluded from
our Creative Commons license. See https://ocw.mit.edu/fairuse for more information. 20
Does the Pattern of Response Across Voxels in the FFA
Contain Information about Nonfaces? YES!
Haxby et al (2001): yes
“Regions such as the …. ‘FFA’ are not dedicated to
representing only ... human faces.. but, rather, are part of a
more extended representation for all objects”.
Spiridon & Kanwisher (2002): no
O’Toole, Haxby et al. (2005): not very much
“preferred regions for faces & houses are not well suited to object
classifications that do not involve faces and houses, respectively.”
Reddy & Kanwisher (2007): uh, a little
OK does that mean I am toast?
Think about why? Why not?
What other evidence suggests this is not the whole story? 21
Visual Cortical Regions Selectively Responsive to
visual categories:
Faces Bodies Places
PPA
places
Jim Haxby
EBA bodies
These regions contain small amounts of
information about “nonpreferred” categories.
FFA An important critique of the selectivity story.
faces But Haxby’s pattern analysis is also important
for another reason….
Photo of Haxby © source unknown. All rights reserved. This
We can ask: what information is present?
content is excluded from our Creative Commons license. See
https://ocw.mit.edu/fairuse for more information. That is, we can “decode” neural responses 22
Neural Decoding with fMRI
Can you read the mind with fMRI?
Or at least tell what the person saw?
1. Train your decoder.
Can we tell what stimulus Given a pattern of fMRI response
the person saw? across voxels in a particular brain
region (e.g., V1 or FFA or EBA):
How can try this?
Figures & images on this and the next two pages © sources unknown. All rights reserved. This content is
excluded from our Creative Commons license. For more information, see https://ocw.mit.edu/fairuse.
23
Neural Decoding with fMRI
Can you read the mind with fMRI?
Or at least tell what the person saw?
Machine LearningPattern Classifier
BOLD Response
fMRI
Stimulus across Voxels
2. Test your decoder.
??
Does this work? A little bit. Won’t work for forcing testimony.
But don’t panic. But good enough for science. Sometimes.
Yet. Many versions… 24
Varieties of Neural Decoding
1. Can use decoding methods on many types of neural data.
Time course of
Magnetoencephalography (MEG) Response across information
sensors (at tx) extraction
let’s compare
Firing Rate “Neural population
Monkey Neurophysiology across Neurons decoding”
“Multiple Voxel
BOLD Response Pattern Analysis
fMRI across Voxels (MVPA)”
of an ROI
of whole
brain
2. Many decoding methods: Haxby-style correlations, Machine Learning (SVMs, deep nets)
25
Neural Decoding:
A Direct Comparison of fMRI versus Neurons
Monkey Neurophysiology
167 neurons in AM
For each neuron, measure
response to each of 5 different
faces
Question:
What information Monkey fMRI
about faces is ~100 voxels
represented in AM? For each voxel, measure
Ask 2 ways…. BOLD response to each of 5
Finding: different faces
Can decode face identity from populations of neurons (neurophys),
Not from populations of voxels in the same region (fMRI).
How can this be? Photos or MRI machine & brain drawing © sources unknown. Brain figure © Society for
Neuroscience. All rights reserved. This content is excluded from our Creative Commons
license. See https://ocw.mit.edu/fairuse for more information. Figure source: J. Dubois,
What are the implications? et al. (2015) J Neurosci, 35 (6). doi: https://doi.org/10.1523/JNEUROSCI.4037-14.2015
26
A Powerful Use of MVPA: Testing Invariance
1. Train your decoder.
2. Test your decoder.
SHOE
What information is represented here?
Very specific templates? Or more abstract (“invariant”) representations?
To find out: Train on one set of stimuli, and test on another.
e.g. Are there representations of shoes that are invariant to
color and viewpoint? Figures & images © sources unknown. All rights reserved. This content is excluded from
our Creative Commons license. For more information, see https://ocw.mit.edu/fairuse.
the concept of shoe? 27
MIT OpenCourseWare
https://ocw.mit.edu/
9.13 The Human Brain
Spring 2019
For information about citing these materials or our Terms of Use, visit: https://ocw.mit.edu/terms.
28
