Binding Action and Emotion in Social Understanding

Francesca Ferri1,2*, Sjoerd J. H. Ebisch2,3, Marcello Costantini2,3, Anatolia Salone3,4, Giampiero Arciero5,
Viridiana Mazzola5, Filippo Maria Ferro4, Gian Luca Romani2,3, Vittorio Gallese1,6*
1 Department of Neuroscience, University of Parma, Parma, Italy, 2 ITAB - Institute for Advanced Biomedical Technologies, Foundation University ‘‘G. d’Annunzio’’, Chieti,
Italy, 3 Department of Neuroscience and Imaging, University ‘‘G. d’Annunzio’’, Chieti-Pescara, Italy, 4 Institute of Psychiatry, University ‘‘G. d’Annunzio’’, Chieti-Pescara,
Italy, 5 IPRA - Istituto di Psicologia e Psicoterapia Post Razionalista, Roma, Italy, 6 Brain Center for Social and Motor Cognition, Italian Institute of Technology, Parma, Italy

Abstract
In social life actions are tightly linked with emotions. The integration of affective- and action-related information has to be
considered as a fundamental component of appropriate social understanding. The present functional magnetic resonance
imaging study aimed at investigating whether an emotion (Happiness, Anger or Neutral) dynamically expressed by an
observed agent modulates brain activity underlying the perception of his grasping action. As control stimuli, participants
observed the same agent either only expressing an emotion or only performing a grasping action. Our results showed that
the observation of an action embedded in an emotional context (agent’s facial expression), compared with the observation
of the same action embedded in a neutral context, elicits higher neural response at the level of motor frontal cortices,
temporal and occipital cortices, bilaterally. Particularly, the dynamic facial expression of anger modulates the re-enactment
of a motor representation of the observed action. This is supported by the evidence that observing actions embedded in
the context of anger, but not happiness, compared with a neutral context, elicits stronger activity in the bilateral pre-central
gyrus and inferior frontal gyrus, besides the pre-supplementary motor area, a region playing a central role in motor control.
Angry faces not only seem to modulate the simulation of actions, but may also trigger motor reaction. These findings
suggest that emotions exert a modulatory role on action observation in different cortical areas involved in action
processing.

Citation: Ferri F, Ebisch SJH, Costantini M, Salone A, Arciero G, et al. (2013) Binding Action and Emotion in Social Understanding. PLoS ONE 8(1): e54091.
doi:10.1371/journal.pone.0054091
Editor: Giuseppe di Pellegrino, University of Bologna, Italy
Received February 24, 2012; Accepted December 10, 2012; Published January 17, 2013
Copyright: ß 2013 Ferri et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted
use, distribution, and reproduction in any medium, provided the original author and source are credited.
Funding: This work was supported by the EU grant TESIS to Vittorio Gallese. The funders had no role in study design, data collection and analysis, decision to
publish, or preparation of the manuscript.
Competing Interests: The authors have declared that no competing interests exist.
* E-mail: francesca.ferri@nemo.unipr.it (FF); vittorio.gallese@unipr.it (VG)

Introduction activity underlying the perception of a goal-related action. What is

the relevance of such investigation?
Emotions play an important role in shaping social interchange. Imagine to enter a room and see the guy sitting at the table in
During everyday social situations actions are tightly linked with front of you grasping a bottle. How can you guess what he is going
emotions. Indeed, motor behaviours are frequently characterized to do? Whether he is going to drink, because thirsty, or going to
by emotional colouring rather than being mechanically per- throw the bottle towards the door you have just opened, because
formed, and emotions, manifest in body movements [1], are often he has got notice of dismissal from work? His facial expression
motivators of actions [2,3]. Empirical evidence for the link revealing his emotional state may cue the intention behind his
between action and emotion has been provided by numerous action. In other words, the integration of emotion- and action-
behavioural and neurophysiological investigations. For example, it related information may facilitate the recognition of the emotional
has been shown that Transcranial Magnetic Stimulation (TMS) intention of an observed action and, consequently, trigger an
induced motor evoked potentials are larger while observing appropriate reaction.
pleasant and unpleasant compared to neutral images [4]. In the At present, the neural mechanisms underlying action and
same vein, corticospinal excitability facilitation during action emotional state understanding have been extensively, but sepa-
observation is higher in amplitude following the presentation of rately studied [8,9,10,11,12,13,14,15,16,17,18,19,20,21,22]. How-
negative than positive stimuli [5]. Differently, imitative tendency ever, given the tight link between action and emotion, not only the
seems to be increased specifically by negative emotional stimuli recognition of the former or the latter needs to be investigated, but
[6]. Finally, there is evidence that emotional states influence also also or rather their combination.
the execution of future movements [7]. All these studies provide There is by now large consensus on attributing changes in
evidence supporting the link between emotions and motor neural activity produced by the observation of others’ behaviour to
behaviour. More precisely, they demonstrate that emotional an action observation-execution matching mechanism. This has
stimuli affect motor responses as well as action processing. The typically been interpreted in the context of a ‘‘Mirror Mechanism’’
present study aims at investigating whether and how the emotional (MM). In humans, the MM has been shown to characterize the
context in which an action is embedded, or, more specifically, the activation of the lower part of the pre-central gyrus, the posterior
emotion expressed by an observed agent, modulates the brain part of the inferior frontal gyrus, the rostral part of the inferior

PLOS ONE | www.plosone.org 1 January 2013 | Volume 8 | Issue 1 | e54091

 Action and Emotion

parietal lobe, and regions within the intraparietal sulcus. It has matrix, 464 mm in-plane resolution; FOV = 256 mm, no gap). A
been proposed that the MM underpins the shared representation high-resolution structural image was acquired at the end of the
of one’s own executed actions and others’ observed actions; for session via a 3D MPRAGE sequence (170 sagittal slices, voxel size:
reviews see [23,24,25]. In addition to this ‘‘core’’ motor MM, an 1.2561.2561.20 mm, TR = 8.6 ms, TE = 4.0 ms, 1926192 im-
emotional cortical circuit endowed with similar mirror properties age matrix, FOV = 240 mm).
has been proposed, comprising additional brain regions, such as
the insula and adjacent frontal operculum, subserving the Stimuli and Conditions
representation of emotional bodily states [26,27,28]; for a review The experimental stimuli consisted of three sets of coloured
see [29,30]. movies: 1) ‘‘Emotion-Action’’ (EA), showing an actor (torso, face
It has been recently argued that specific Embodied Simulation and arms of either a male or a female) grasping one of four
(ES) processes, instantiated by means of MMs (MM-driven ES, different objects (bottle, pencil case, receiver or CD case placed on
[31]) might play a constitutive role in mind-reading, meaning that a table) with the right hand and facially expressing anger,
‘‘people reuse their own mental states or processes in functionally happiness or no emotion; 2) ‘‘Emotion’’ (E) showing only the face
attributing them to others, where the extent and reliability of such of the actor (either a male or a female) expressing anger, happiness
reuse and functional attribution depend on the simulator’s bodily or no emotion; and 3) ‘‘Action’’ (A), showing only the hand action
resources and their being shared with the target’s bodily (the field of view was such that the face did not appear). Hence, the
resources’’ [31]. Although MM-driven ES has been so far studied experiment comprised the following 7 conditions: 1a) Angry
in the domains of action, emotion and sensation separately, it is Action (AA, the actor grasped an object expressing anger); 1b)
likely that a given mental state or process (e.g., anger) can be Happy Action (HA, the actor grasped an object expressing
simulated in parallel across several of these domains (e.g., at the happiness); 1c) Neutral Action (NA, the actor grasped an object
sensory-motor and visceromotor level). How can these domains be with a neutral facial expression); 2a) Angry Face (AF, expression of
integrated? anger); 2b) Happy Face (HF, expression of happiness); 2c) Neutral
The present study addresses this issue focusing on the Face (NF, a face expressing no emotion); 3) Action (A, one of the
modulatory role of emotions on action observation within cortical objects being grasped). All the Emotion conditions were dynamic.
regions involved in the processing of action- and emotion- related The actors in the video clips were seen from a frontal point of
information. Previous studies seem to suggest that some brain view. Actors and different type of objects were presented in equal
regions, such as the pre-central gyrus, the superior temporal sulcus proportions. Two professional actors, a female and a male, were
and the insula are actually recruited while separately observing enrolled as models for the videos (Created by VM and GA). The
both hand and face actions cueing the agent’s affective state and kinematics of all the presented hand actions was identical in order
shaping an emotional context (e.g., [8,10]). However, as far as we to avoid that the action emotional context could be inferred by
are aware, it is still unknown how such emotional context hand kinematics. To obtain such identity, we applied the Blue
modulates neural activity related to action processing. Screen technique, that is, a technique for compositing two images
By means of functional Magnetic Resonance Imaging (fMRI), or frames together in which a color (or a small color range) from
we investigated whether the observation of the same grasping one image is removed (made transparent), revealing another image
action, either embedded in a context that cues the emotional state behind it. It was applied to our stimuli in order to superimpose on
of the agent (positive or negative) or in absence of a contextual the same trunk different dynamic facial expressions.
emotional cue elicits the same or differential neural activity. The
effect of emotional context on action observation was tested by Design and Procedure
means of both a region of interest analysis and voxel-wise The rapid event-related fMRI paradigm consisted of four scans.
contrasts. In each scan 12 movies were presented for each of the seven
experimental conditions (AA, HA, NA, A, AF, HF, NF). Each
Materials and Methods movie lasted 1800 ms and was preceded by a randomized, non-
predictable intertrial interval ranging from 2000 to 5000 ms
Participants during which a black fixation cross was presented in the centre of
Twenty-two healthy young adults (8 female, mean age: 27.6 a white screen (see figure 1). Participants were instructed to
years; range: 21–35), all right-handed [32]; (handedness index carefully watch the whole scene. To make sure participants paid
.0.8), participated in the present study. All participants had attention to the experimental stimuli, 8 control trials were
normal or corrected-to-normal vision (correction ,0.75) and were randomly inserted in the video sequence of each scan. These
naı̈ve as to the purposes of the experiment. Participants gave their unpredictable trials were followed by a question mark lasting
written informed consent to participate in the study and were paid 2000 ms followed by a written request (6000 ms) to imitate either
for their participation. The study was approved by the Ethics the action (4 trials) or the emotion (4 trials) (see figure 1). In total,
Committee of the ‘‘G. d’Annunzio’’ University, Chieti, and was our experiment consisted of 336 passive observation trials (48 for
conducted in accordance with the ethical standards of the 1964 each experimental condition) and 32 imitation trials (16 for actions
Declaration of Helsinki. and 16 for emotions), presented in pseudo-randomized order.
Subjects lay supine in the scanner with the arms outstretched
fMRI Data Acquisition beside the abdomen. Visual stimuli were projected onto a back-
All images were collected with a 1.5 T Philips Achieva scanner projection screen situated behind the subject’s head and were
operating at the Institute of Advanced Biomedical Technologies visible in a mirror (10615 cm). Sound-attenuating headphones
(I.T.A.B. Fondazione ‘‘G. d’Annunzio’’, Chieti, Italy). Functional were used to muffle scanner noise. Participants were instructed to
images were acquired with a gradient echo EPI sequence. Each carefully watch the whole scene.
subject underwent four scans, each including 216 consecutive
volumes comprising 26 consecutive ascending 4-mm-thick slices Data Analysis
oriented parallel to the anterior-posterior commissure and Functional MRI data were preprocessed and analysed using
covering the whole brain (TR = 2.4 s, TE = 50 ms, 64664 image SPM8 (Wellcome Department of Cognitive Neurology, Institute of

PLOS ONE | www.plosone.org 2 January 2013 | Volume 8 | Issue 1 | e54091

 Action and Emotion

Figure 1. Stimuli and experimental paradigm. Examples of the three sets of coloured movies used in the visual stimulation. Lower panel:
description of the experimental paradigm.
doi:10.1371/journal.pone.0054091.g001

Neurology, London). For each subject, functional images were first experimental conditions (AA, HA, NA, A, AF, HF, NF), relative to
spatially corrected for head movements using a least-squares the intertrial baseline. At the second stage, contrast images from all
approach and six-parameters rigid body spatial transformations subjects were entered into a full-factorial model, as implemented in
[33]. The realigned functional images were then corrected for SPM8. We first selected regions responding more during at least
difference in timing between slices, using the middle slice acquired one of the seven conditions relative to the intertrial baseline. The
in time as a reference. The high-resolution anatomical image and resulting statistical parametric map of the F statistic was
the functional images were coregistered and then stereotactically thresholded at p,0.001, corrected for multiple comparisons over
normalized to the Montreal Neurological Institute (MNI) brain the total amount of analysed brain volume using ‘‘Family Wise
template used in SPM8. Functional images were re-sampled with Error’’ (FWE). Based on this map, we created regions of interest
a voxel size of 36363 mm and spatially smoothed with a three- (ROIs) for further analyses, by grouping together, for each
dimensional Gaussian filter of 8 mm full width at half maximum to regional peak, all neighbouring voxels at a maximum distance of
accommodate anatomical variations between subjects [33]. Images 32 mm from the peak.
were subsequently analysed using a random-effects approach. At To localize and visualize the activated clusters we used the
the first stage, the time series of functional MR images obtained BrainShow software [36,37] implemented in Matlab (MathWorks
from each participant were analysed separately. The effects of the Inc., MA). The BrainShow software was also used to project group
experimental paradigm were estimated on a voxel-by-voxel basis, activations onto the standard MNI template and to assign
according to the general linear model extended to allow the anatomical labels [38].
analysis of fMRI data as a time series [34]. The onset of each trial As a second step, for each identified region, we computed the
constituted a neural event, that was modeled through a canonical estimated beta values in each condition (relative to the intertrial
hemodynamic response function, chosen to represent the relation- baseline), by spatially averaging the pre-processed time series
ship between neuronal activation and blood flow changes [35]. across all voxels in the region and re-estimating the individual
Imitation and question mark periods were modelled as separate general linear models on these averaged time series. Such regional
conditions and then excluded from further analyses. hemodynamic response estimates were then used to perform the
These single-subject models were used to compute seven following two-tailed simple effect analysis between the Emotion-
contrast images per subject, each representing the estimated Action conditions (AA ? HA; AA ? NA; AA ? A; HA ? NA;
amplitude of the hemodynamic response in one of the seven HA ? A; NA ? A) as well as between the Emotion conditions (AF

PLOS ONE | www.plosone.org 3 January 2013 | Volume 8 | Issue 1 | e54091

 Action and Emotion

? HF; AF ? NF and HF ? NF). According to the Bonferroni was significantly higher during AA than all the other action
correction method the alpha level was set at 0.008 for the contrasts conditions (HA, NA, A) [T(21) .3.740; all ps .0.001].
between the Emotion-Action conditions and 0.0167 for the Furthermore, activation during observation of both AF and
contrasts between the Emotion conditions. Additionally, every HF was higher than that during observation of NF [T(21)
single condition was compared with the intertrial baseline by .4.413; all ps,0.001] (figures 2 and 3). The BOLD signal
means of one sample t-test. levels in this cluster were significantly higher than the intertrial
Finally, the effect of emotional context on action observation baseline for all the experimental conditions [all ps ,.05].
was tested by means of whole-brain voxel-wise contrasts. We
investigated cortical regions differentiating between observation of Right Inferior Frontal Gyrus
actions embedded in emotional contexts as compared to actions The cluster centred on the rIFG pars opercularis (37%),
embedded in neutral context. To this aim, AA ? NA and HA ? encompassed the pars triangularis (25%) and the PCG (28%).
NA comparisons were performed (p,0.05 FDR). It should be BOLD signal was significantly higher during observation of
noted that such contrasts also control for low-level visual a grasping action embedded in the emotional context of anger
activations. (AA) than in a non emotional context (NA) [T(21) = 3.253;
p,0.004]. Moreover, BOLD signal was higher during observation
Results of AA compared to A [T(21) = 3.592; p,0.002] (figures 2 and 3).
The BOLD signal levels in this cluster were significantly higher
ROI-based Analysis than the intertrial baseline for all the experimental conditions [all
From the group-level whole-brain analysis of functional MR ps ,.05].
images, with a statistical threshold of p,0.001 (FWE corrected),
we identified nine different cortical regions where BOLD signal Right Temporal Cortex
was significantly higher during at least one of the experimental Two activation clusters mapped in the right temporal cortices.
conditions (AA, HA, NA, A, AF, HF, NF), relative to the The first, centred in Middle Temporal Gyrus (MTG), extended to
intertrial baseline. The nine active regions were located in the Middle Occipital Gyrus (MOG), Inferior Occipital Gyrus (IOG)
left pre-central gyrus (PCG), the right inferior frontal gyrus and inferior temporal gyrus (ITG). Average BOLD signal in this
(IFG), the right temporal cortices, the parietal lobe (PL) cluster was significantly higher during the observation of a grasping
bilaterally and the occipital cortex (OC) bilaterally (table 1, action performed by a person expressing anger compared to the
figures 2 and 3). same action performed by a person expressing no emotion
[T(21) = 3.293; p,0.003]. A further modulation of the neural
Left Precentral Gyrus activity in this region resulted for the observation of both AF and
The cluster centred on the left inferior PCG (48%) extended HF compared to NF [T(21) .4.502; all ps ,0.001], and AF versus
to the superior PCG (14%) and the IFG (13%). BOLD signal HF [T(21) = 2.783; p,0.011]. The second cluster, centred in

Figure 2. Regions showing different activation during the observation of any experimental condition compared with the intertrial
baseline. Group activation data are rendered on the cortical surface of a ‘‘canonical’’ brain (Mazziotta et al., 1995). Plots represent estimated beta
values for the AE and A conditions. Vertical bars indicate standard errors. Horizontal bars represent statistical differences.
doi:10.1371/journal.pone.0054091.g002

PLOS ONE | www.plosone.org 4 January 2013 | Volume 8 | Issue 1 | e54091

 Action and Emotion

Figure 3. Estimated beta values for the E conditions.

doi:10.1371/journal.pone.0054091.g003

MTG, extended to the superior temporal sulcus (MTG/STS). expressing no emotion (NA) [T(21) = 5.245; p,0.001]. A further
Simple effect analysis of the average regional BOLD response modulation of the neural activity in this region resulted for the
showed that activation in this cluster was significantly higher for observation of both AF and HF compared to NF [T(21) .3.630;
the observation of AA, HA and NA compared to A [T(21) .5.086; all ps ,0.002] (figures 2 and 3). The BOLD signal levels in this
p,0.001 always] and for the observation of a grasping action cluster were significantly higher than the intertrial baseline for all
performed by a person expressing either anger or happiness (AA, the experimental conditions [all ps ,.05].
HA) compared to the same action performed by a person

Table 1. Montreal Neurological Institute (MNI) coordinates of peaks of relative activation in the cortical regions where BOLD signal
was significantly different during observation of any experimental condition compared with the intertrial baseline.

Anatomical region Side z-score Cluster size (voxel) Main local maxima (MNI)

x y z

CC L/R Inf 1755 9 281 6

23 2102 6
224 284 24
MOG L Inf 699 248 275 3
SOG R Inf 393 24 284 36
MTG R Inf 605 48 272 23
R Inf 184 54 239 3
PL R Inf 356 227 260 51
L Inf 344 30 254 54
IFG R 7.06 120 48 12 27
PCG L Inf 151 245 3 39

The alfa level was set at 0.001 (FWE Corrected).

Notes: CC, calcarine cortex; MOG, middle occipital gyrus; SOG, superior occipital gyrus; MTG, middle temporal gyrus; PL, parietal lobe; IFG, inferior frontal gyrus; PCG,
precentral gyrus.
doi:10.1371/journal.pone.0054091.t001

PLOS ONE | www.plosone.org 5 January 2013 | Volume 8 | Issue 1 | e54091

 Action and Emotion

Left Parietal Lobe agent, modulates the brain activity underlying the perception of
The cluster was centred on left SPL with 73% of the voxels a goal-related action. The experimental conditions included short
falling within this area. Other voxels fell within the IPL (22%). movies depicting an actor performing a grasping action while
Activation due to the observation of action embedded in non expressing an emotion (Happiness, Anger or Neutral), an actor
emotional/emotional context and action alone did not differ from only expressing one of these emotions, or an actor only performing
each other (figures 2 and 3). The BOLD signal levels in this cluster a grasping action (figure 1). These conditions allowed the
were significantly higher than the intertrial baseline for the Action identification of brain regions sensitive to the modulatory role of
and all Emotion-Action conditions [all ps ,.05], but not for the emotions on action observation.
Emotion conditions [all ps ..05 for AF, HF and NF].
Frontal Cortices
Right Parietal Lobe Among cortical frontal areas ventral PCG and IFG, bilaterally,
The cluster’s peak value was centred on the right superior were more activated by observation of actions embedded in the
parietal lobe (SPL) with 49% of the voxels falling within this context of anger compared to the observation of actions embedded
region, 23% of the voxels falling within the inferior parietal lobe in a neutral context. Ventral PCG, as well as IFG, are part of
(IPL) and 13% within the angular gyrus (table 1). BOLD signal in a fronto-parietal network critical for action representation [39,40].
this cluster was significantly higher during the observation of AF The ventral PCG has been recently proposed to be, at the
[T(21) = 3.116; p,0.005] and HF [T(21) = 3.065; p,0.006] faces functional level, the human counterpart of monkey area F5 [41],
compared to NF. Activation due to the observation of action where mirror neurons were firstly described [42]. The coordinates
embedded in non emotional/emotional context and action alone of local maxima of the PCG clusters in the present study (245, 3,
did not differ from each other (figures 2 and 3). The BOLD signal 39; 51, 6, 39) are similar to the average coordinates (y = 0, z = 41)
levels in this cluster were significantly higher than the intertrial of activations reported in fourteen out of twenty-three contrasts,
baseline for all the experimental conditions [all ps ,.05]. reviewed by Morin and Grezes [41], comparing neuronal response
elicited by the observation of an action made by a living being with
Occipital Cortex another visual control stimulus. Based on this evidence, these
The activation clusters in the occipital cortex were centred in authors proposed that the ventral PCG shares the visual properties
the calcarine cortex (CC), the left middle occipital gyrus (MOG) of ‘‘mirror’’ neurons found in area F5 of the macaque brain.
and the right superior occipital gyrus (SOG). The cluster centred Imaging data in humans suggest a role of PCG and IFG in
in the CC extended to the left MOG, the left SOG and the right coding the goal of the action [43] and, from another perspective,
lingual gyrus (LG). Simple effect analysis of the average regional in the understanding of the agent’s motor intention, driven by the
BOLD response showed that activation in this cluster was context in which the action is embedded [13]. The present results
significantly higher for the observation of AA, HA and NA expand current knowledge by showing that PCG and IFG likely
compared to A [T(21) .3.317; all ps ,0.003]. The cluster centred bind the motor information about grasping with the emotional
in the left MOG extended to the inferior occipital gyrus (IOG) and information extracted from the agent’s face, in order to code the
the middle temporal gyrus (MTG). Simple effect analysis of the motor goal of the action. This holds for anger, but not for
average regional BOLD response showed that activation in this happiness (figures 2 and 4). This result suggests that the angry
cluster was significantly higher for the observation of Actions context is combined with the motor representation of the observed
embedded in emotional context (both AA and HA) compared to action, likely contributing to the immediate ascription of the
NA [T(21) .3.765; all ps ,0.001]. The cluster centred in the right emotional intention associated with it [44]. As a consequence, it
SOG extended to MOG. Simple effect analysis of the average might be hypothesized that this triggers an immediate interaction/
regional BOLD response showed that activation in this cluster was re(-en)action from the observer. This hypothesis is supported by
significantly higher for the observation of AA than A [T(21) the modulated activity in pre-SMA while observing angry actions
.4.024; p,0.001] and for the observation of HF than NF [T(21) compared to the observation of neutral actions (figure 3).
.2.993; p,0.007]. The BOLD signal levels in this cluster were Pre-SMA plays a central role in the control of motor
significantly higher than the intertrial baseline for all conditions behaviour. The higher activation for Angry than for Neutral
[all ps ,.05] (figures 2 and 3). The BOLD signal levels in this Action can be interpreted in the light of the role of pre-SMA in
cluster were significantly higher than the intertrial baseline for all the shaping of self-initiated reactions [45]. More specifically,
the experimental conditions [all ps ,.05]. a possible interpretation is that the negative emotional context
connoted the perceived action as potentially threatening and,
Neural Mapping of Observing Angry and Happy Actions hence, triggered a reaction in the observer. Oliveri and
Compared to Neutral Actions colleagues [45] tested a similar hypothesis by means of a TMS
When contrasting the effect of observing either Angry Action or study. They delivered single-pulse TMS over the left primary
Happy Action with Neutral Action (AA ? NA or HA ? NA, motor cortex (M1), after a conditioning stimulation of the left
respectively), the following activation differences were found. SMA, while participants carried out movements in response to
Regarding AA relative to NA, higher activation was found in the pictures with negative or neutral emotional content. Results
left IFG (48% pars triangularis and 33% pars opercularis), left pre- showed that conditioning of SMA by means of TMS selectively
SMA, right inferior PCG, MTG/STS bilaterally and left middle enhanced M1 excitability during the execution of movements
OC. Regarding HA, relative to NA, higher activation was found in triggered by visual cues with negative emotional content, but
the MTG/STS, left Fusiform Gyrus (FG), right Lingual Gyrus not by visual-neutral cues. Our results are in line with this
(LG), middle and inferior OC bilaterally (table 2, figure 4). finding and with evidence provided by a meta-analysis [46]
showing that the SMA/pre-SMA complex is involved in
different cognitive functions including attention to one’s own
Discussion
action and valuation of other people’s behaviour.
The present study aimed at investigating whether the emotional Alternatively, one may argue that activations in pre-SMA
context, that is, an emotion dynamically expressed by an observed recruitment is related to overt or covered shifts of attention,

PLOS ONE | www.plosone.org 6 January 2013 | Volume 8 | Issue 1 | e54091

 Action and Emotion

Figure 4. Action-Emotion.Action Neutral. Regions activated either in Angry Action Vs Neutral Action contrast (red) or in Happy Action Vs
Neutral Action (green) contrast.
doi:10.1371/journal.pone.0054091.g004

especially in the angry action condition, as it is more salient Parietal Cortices

than happy and neutral action. Although we agree that angry The right and the left parietal cortices, despite being both
action attracts more attention than the other conditions, we engaged in hand action’s perception, were not affected by the
believe that attentional shifting might not entirely account for emotional context. This could be related to the fact the both
our activations. Indeed, if it was the case it remains to be clusters were centred on and mostly extended within SPL. Data
explained why the very same areas are activated also by the from action imitation and observation studies demonstrated that
observation of action alone and emotion alone, regardless of while the abstract aspects of the action (such as its goal or
emotional valence. intention) are represented in IPL, in addition to IFG, the specific
(kinematic) aspects of the action [11,47] are represented in SPL.

Table 2. Action-Emotion.Action Neutral.

Angry Action vs Neutral Action Happy Action vs Neutral Action

Cluster size Main local maxima Cluster size Main local maxima
Anatomical region Side (voxel) (MNI) (voxel) (MNI)

x y z z-score x y z z-score

OC L/R 20 224 299 23 3.97 18 33 293 26 3.52

226 23 284 6 4.13
47 233 278 29 4.63
LG R 55 21 266 26 4.27
FG L 24 236 263 212 4.50
MTG/STS R 168 57 245 3 4.60 233 54 245 9 5.28
L 85 251 245 9 5.05 198 254 242 9 5.27
PCG R 13 51 6 39 3.84
pre-SMA L 121 23 6 66 4.35
IFG L 21 236 12 24 3.78

Local maxima of cortical clusters responding to action embedded in emotional context as compared to actions embedded in neutral context The alfa level was set at
0.05 (FDR Corrected).
Notes: OC, occipital cortex; LG, lingual gyrus; FG, fusiform gyrus; MTG, middle temporal gyrus; STS, superior temporal sulcus; PCG, precentral gyrus; pre-SMA, pre-
supplementary motor area; IFG, inferior frontal gyrus.
doi:10.1371/journal.pone.0054091.t002

PLOS ONE | www.plosone.org 7 January 2013 | Volume 8 | Issue 1 | e54091

 Action and Emotion

The absence of a pattern of modulation depending on the Differently, at the level of PCG/IFG it mainly occurs when the
emotional context is consistent with this previous knowledge. In action is performed by angry, but not happy, agents (figure 2 and
fact, during the observation of hand actions performed in different 4). Selective response for anger in premotor cortex, but not in
emotional contexts, but with identical kinematics, contextual temporal cortices, has been previously observed comparing brain
differences should not affect the representation of the kinematic activations elicited during the perception of anger, threat and
aspects of the action mapping in SPL. Viewing dynamic face neutral behaviours [56]. Even if we did not explicitly assess coding
movements led to significant increases in the BOLD signal only in of intentional states, our results might suggest that when viewing
the right parietal cortex, where both angry and happy faces actions performed with anger, rather than happiness, an important
elicited a stronger activity than neutral faces. These results priority for the brain is to represent the emotion state associated
complement also previous data about the body parts specificity with it.
of the parietal cortices in the processing of human actions [10,48] With our current data, we are not able to disentangle whether
by adding information related to the effect of the emotional the activations in PCG/IFG reveal emotion-related modulations
context in which the action is performed. of motor simulation or, alternatively, the preparation of motor
response required by the situation. Indeed, coping with angry
Other Brain Regions agents as compared to happy and neutral agents, may require
Finally, regions in bilateral posterior temporal cortex, largely additional behavioral adjustments. In this case, the asymmetry of
overlapping with MTG/STS, and in left occipital cortex showed angry and happy faces would be mainly due to the fact that angry
increased activation during observation of action embedded in an faces lead to stronger arousal boosting activation in movement-
emotional context, produced by both angry and happy faces. This related areas, thus, triggering motor reaction. This hypothesis
is in keeping with the data obtained by Wyk and colleagues [49], would be supported by the specific effect of anger found in pre-
who proposed that STS is sensitive to the congruency between SMA, which plays a central role in the control of motor behaviour
action and the agent’s intention established by a previous (figure 3). Finally, as it is known that negative emotional context
emotional expression. In general, STS is considered to play an facilitates imitative action tendencies [6] and that automatic
important role in the perception of social acts [50]. In particular, imitation is mediated by the MM [57,58], it is also possible that in
right STS seems to be specifically involved in facial emotion our study angry faces trigger action imitation, rather than a motor
recognition rather than in general face processing [51]. As shown reaction. That said, on the basis of our results, we propose that the
in figures 2 and 3, its greater engagement during the emotional modulatory role of emotions on action perception, mainly mapped
conditions might be critical for the emotional modulation of the within PCG/IFG and MTG/STS in the present study, could be
entire action representation system [8,10]. In fact, the superior viewed as the necessary step towards a more comprehensive social
temporal cortex, which is connected to the limbic system via the understanding and shaping appropriate social interaction.
insula [52], is hypothesized to code an early visual description of
the action and to send this information to the action representation
Acknowledgments
network [53,54].
The authors thank Mauro Gianni Perrucci for technical assistance.
Conclusions
The results of the present study suggest that MTG/STS and Author Contributions
PCG/IFG, which are part of the action observation system [55], Conceived and designed the experiments: FF SJHE AS GA VM VG.
possibly combine action-related information with the specific Performed the experiments: FF SJHE AS. Analyzed the data: FF MC.
agent’s affective state. At the level of MTG/STS this occurs Contributed reagents/materials/analysis tools: GA VM GLR FMF VG.
regardless of the specific emotional context (i.e. Anger, Happiness). Wrote the paper: FF MC SJHE VG.

References
1. Damasio AR (2003) Looking for Spinoza: Joy, Sorrow and the Feeling Brain. 13. Iacoboni M, Molnar-Szakacs I, Gallese V, Buccino G, Mazziotta JC, et al.
New York: Harcourt. (2005) Grasping the intentions of others with one’s own mirror neuron system.
2. Frijda N (1986) The emotions. Cambridge: Cambridge University Press. PLoS Biol 3: e79.
3. Frijda N (2006) The laws of emotions. Mawwah NJ: Erlbaum. 14. Järveläinen J, Schürmann M, Hari R (2004) Activation of the human primary
4. Hajcak G, Molnar C, George MS, Bolger K, Koola J, et al. (2007) Emotion motor cortex during observation of tool use. NeuroImage 23: 187–192.
facilitates action: A transcranial magnetic stimulation study of motor cortex 15. Knutson K, McClellan E, Grafman J (2008) Observing social gestures: an fMRI
excitability during picture viewing. Psychophysiology 44: 91–97. study. Experimental Brain Research 188: 187–198.
5. Enticott PG, Harrison BA, Arnold SL, Nibaldi K, Segrave RA, et al. (2012) 16. Lee T-W, Josephs O, Dolan RJ, Critchley HD (2006) Imitating expressions:
Emotional valence modulates putative mirror neuron activity. Neurosci Lett emotion-specific neural substrates in facial mimicry. Social Cognitive and
508: 56–59. Affective Neuroscience 1: 122–135.
6. Grecucci A, Koch I, Rumiati RI (2011) The role of emotional context in 17. Lestou V, Pollick FE, Kourtzi Z (2007) Neural Substrates for Action
facilitating imitative actions. Acta Psychologica 138: 311–315. Understanding at Different Description Levels in the Human Brain. Journal
7. Coombes S, Tandonnet C, Fujiyama H, Janelle C, Cauraugh J, et al. (2009) of Cognitive Neuroscience 20: 324–341.
Emotion and motor preparation: A transcranial magnetic stimulation study of 18. Pichon S, de Gelder B, Grezes J (2008) Emotional modulation of visual and
corticospinal motor tract excitability. Cognitive, Affective, & Behavioral motor areas by dynamic body expressions of anger. Soc Neurosci 3: 199–212.
Neuroscience 9: 380–388. 19. Piefke M, Kramer K, Korte M, Schulte-Rüther M, Korte JM, et al. (2009)
8. Carr L, Iacoboni M, Dubeau M-C, Mazziotta JC, Lenzi GL (2003) Neural Neurofunctional modulation of brain regions by distinct forms of motor
mechanisms of empathy in humans: A relay from neural systems for imitation to cognition and movement features. Human Brain Mapping 30: 432–451.
limbic areas. Proceedings of the National Academy of Sciences 100: 5497–5502. 20. Sato W, Kochiyama T, Yoshikawa S, Naito E, Matsumura M (2004) Enhanced
9. de Lange FP, Spronk M, Willems RM, Toni I, Bekkering H (2008) neural activity in response to dynamic facial expressions of emotion: an fMRI
Complementary Systems for Understanding Action Intentions. Current Biology study. Cognitive Brain Research 20: 81–91.
18: 454–457. 21. Thioux M, Gazzola V, Keysers C (2008) Action understanding: how, what and
10. Grosbras M-H, Paus T (2006) Brain Networks Involved in Viewing Angry why. Curr Biol 18: R431–434.
Hands or Faces. Cerebral Cortex 16: 1087–1096. 22. van de Riet WA, Grezes J, de Gelder B (2009) Specific and common brain
11. Hamilton AF, Grafton ST (2006) Goal representation in human anterior regions involved in the perception of faces and bodies and the representation of
intraparietal sulcus. J Neurosci 26: 1133–1137. their emotional expressions. Soc Neurosci 4: 101–120.
12. Hamilton AF, Grafton ST (2008) Action outcomes are represented in human 23. Gallese V (2009) Motor abstraction: a neuroscientific account of how action
inferior frontoparietal cortex. Cereb Cortex 18: 1160–1168. goals and intentions are mapped and understood. Psychol Res 73: 486–498.

PLOS ONE | www.plosone.org 8 January 2013 | Volume 8 | Issue 1 | e54091

 Action and Emotion

24. Rizzolatti G, Sinigaglia C (2008) Further reflections on how we interpret the 41. Morin O, Grezes J (2008) What is ‘‘mirror’’ in the premotor cortex? A review.
actions of others. Nature 455: 589. Neurophysiol Clin 38: 189–195.
25. Rizzolatti G, Sinigaglia C (2010) The functional role of the parieto-frontal 42. Gallese V, Fadiga L, Fogassi L, Rizzolatti G (1996) Action recognition in the
mirror circuit: interpretations and misinterpretations. Nat Rev Neurosci 11: premotor cortex. Brain 119 (Pt 2): 593–609.
264–274. 43. Johnson-Frey SH, Maloof FR, Newman-Norlund R, Farrer C, Inati S, et al.
26. Jabbi M, Bastiaansen J, Keysers C (2008) A common anterior insula (2003) Actions or Hand-Object Interactions? Human Inferior Frontal Cortex
representation of disgust observation, experience and imagination shows and Action Observation. Neuron 39: 1053–1058.
divergent functional connectivity pathways. PLoS ONE 3: e2939. 44. Gallese V, Sinigaglia C (2011) How the body in action shapes the self. Journal of
27. Jabbi M, Swart M, Keysers C (2007) Empathy for positive and negative Consciousness Studies 18: 117–143.
emotions in the gustatory cortex. NeuroImage 34: 1744–1753. 45. Oliveri M, Babiloni C, Filippi MM, Caltagirone C, Babiloni F, et al. (2003)
28. Wicker B, Keysers C, Plailly J, Royet JP, Gallese V, et al. (2003) Both of us Influence of the supplementary motor area on primary motor cortex excitability
disgusted in My insula: the common neural basis of seeing and feeling disgust. during movements triggered by neutral or emotionally unpleasant visual cues.
Neuron 40: 655–664. Experimental Brain Research 149: 214–221.
29. Keysers C, Gazzola V (2007) Integrating simulation and theory of mind: from 46. Seitz RJ, Nickel J, Azari NP (2006) Functional modularity of the medial
self to social cognition. Trends Cogn Sci 11: 194–196. prefrontal cortex: Involvement in human empathy. Neuropsychology 20: 743–
30. Molenberghs P, Hayward L, Mattingley JB, Cunnington R (2011) Activation 751.
patterns during action observation are modulated by context in mirror system 47. Iacoboni M, Woods RP, Brass M, Bekkering H, Mazziotta JC, et al. (1999)
areas. Neuroimage. Cortical mechanisms of human imitation. Science 286: 2526–2528.
31. Gallese V, Sinigaglia C (2011) What is so special about embodied simulation? 48. Thompson JC, Hardee JE, Panayiotou A, Crewther D, Puce A (2007) Common
Trends in Cognitive Sciences 15: 512–519. and distinct brain activation to viewing dynamic sequences of face and hand
32. Oldfield RC (1971) The assessment and analysis of handedness: the Edinburgh
movements. NeuroImage 37: 966–973.
inventory. Neuropsychologia 9: 97.
49. Wyk BC, Hudac CM, Carter EJ, Sobel DM, Pelphrey KA (2009) Action
33. Friston KJ, Ashburner J, Frith CD, Poline JB, Heather JD, et al. (1995) Spatial
understanding in the superior temporal sulcus region. Psychol Sci 20: 771–777.
Registration and Normalization of Images. Human Brain Mapping 2: 165–189.
50. Allison T, Puce A, McCarthy G (2000) Social perception from visual cues: role of
34. Worsley KJ, Friston KJ (1995) Analysis of fMRI time-series revisited–again.
the STS region. Trends Cogn Sci 4: 267–278.
Neuroimage 2: 173–181.
35. Friston KJ, Fletcher P, Josephs O, Holmes A, Rugg MD, et al. (1998) Event- 51. Narumoto J, Okada T, Sadato N, Fukui K, Yonekura Y (2001) Attention to
related fMRI: characterizing differential responses. Neuroimage 7: 30–40. emotion modulates fMRI activity in human right superior temporal sulcus.
36. Galati G, Committeri G, Pitzalis S, Pelle G, Patria F, et al. (2011) Intentional Cognitive Brain Research 12: 225–231.
signals during saccadic and reaching delays in the human posterior parietal 52. Augustine J R (1996) Circuitry and functional aspects of the insular lobe in
cortex. European Journal of Neuroscience 34: 1871–1885. primates including humans. Brain Research Reviews 22: 229–244.
37. Galati G, Committeri G, Spitoni G, Aprile T, Di Russo F, et al. (2008) A 53. Iacoboni M, Koski LM, Brass M, Bekkering H, Woods RP, et al. (2001)
selective representation of the meaning of actions in the auditory mirror system. Reafferent copies of imitated actions in the right superior temporal cortex.
Neuroimage 40: 1274–1286. Proceedings of the National Academy of Sciences 98: 13995–13999.
38. Tzourio-Mazoyer N, Landeau B, Papathanassiou D, Crivello F, Etard O, et al. 54. Rizzolatti G, Luppino G (2001) The cortical motor system. Neuron 31: 889–901.
(2002) Automated Anatomical Labeling of Activations in SPM Using 55. Kilner JM (2011) More than one pathway to action understanding. Trends in
a Macroscopic Anatomical Parcellation of the MNI MRI Single-Subject Brain. Cognitive Sciences 15: 352–357.
NeuroImage 15: 273. 56. Pichon S, de Gelder B, Grezes J (2009) Two different faces of threat. Comparing
39. Buccino G, Binkofski F, Fink GR, Fadiga L, Fogassi L, et al. (2001) Action the neural systems for recognizing fear and anger in dynamic body expressions.
observation activates premotor and parietal areas in a somatotopic manner: an Neuroimage 47: 1873–1883.
fMRI study. Eur J Neurosci 13: 400–404. 57. Cook R, Bird G, Lünser G, Huck S, Heyes C (2012) Automatic imitation in
40. Buccino G, Vogt S, Ritzl A, Fink GR, Zilles K, et al. (2004) Neural circuits a strategic context: players of rock–paper–scissors imitate opponents’ gestures.
underlying imitation learning of hand actions: an event-related fMRI study. Proceedings of the Royal Society B: Biological Sciences 279: 780–786.
Neuron 42: 323–334. 58. Heyes C (2011) Automatic imitation. Psychol Bull 137: 463–483.

PLOS ONE | www.plosone.org 9 January 2013 | Volume 8 | Issue 1 | e54091