ORIGINAL RESEARCH ARTICLE

published: 01 July 2014

doi: 10.3389/fpsyg.2014.00607

Moral judgment reloaded: a moral dilemma validation

study
Julia F. Christensen 1*, Albert Flexas 1 , Margareta Calabrese 1 , Nadine K. Gut 1,2,3 and Antoni Gomila 1
1
Psychology, Evolution and Cognition (IFISC-CSIC), University of the Balearic Islands, Palma, Spain
2
School of Psychology and Neuroscience, University of St Andrews, St Andrews, UK
3
Strathclyde Institute of Pharmacy and Biomedical Sciences, University of Strathclyde, Glasgow, UK

Edited by: We propose a revised set of moral dilemmas for studies on moral judgment. We
Jean Decety, University of Chicago, selected a total of 46 moral dilemmas available in the literature and fine-tuned them
USA
in terms of four conceptual factors (Personal Force, Benefit Recipient, Evitability, and
Reviewed by:
Intention) and methodological aspects of the dilemma formulation (word count, expression
Jan Van Den Stock, KU Leuven,
Belgium style, question formats) that have been shown to influence moral judgment. Second,
Florian Cova, University of Geneva, we obtained normative codings of arousal and valence for each dilemma showing that
Switzerland emotional arousal in response to moral dilemmas depends crucially on the factors
*Correspondence: Personal Force, Benefit Recipient, and Intentionality. Third, we validated the dilemma set
Julia F. Christensen, Department of
confirming that people’s moral judgment is sensitive to all four conceptual factors, and to
Psychology, University of the
Balearic Islands, University Campus, their interactions. Results are discussed in the context of this field of research, outlining
Carretera de Valldemossa, km. 7.5, also the relevance of our RT effects for the Dual Process account of moral judgment.
Building: Guillem Cifre de Colonya, Finally, we suggest tentative theoretical avenues for future testing, particularly stressing
07122 Palma, Spain
e-mail: julia.christensen@uib.es
the importance of the factor Intentionality in moral judgment. Additionally, due to the
importance of cross-cultural studies in the quest for universals in human moral cognition,
we provide the new set dilemmas in six languages (English, French, German, Spanish,
Catalan, and Danish). The norming values provided here refer to the Spanish dilemma set.

Keywords: moral dilemmas, moral judgment, decision making, cross cultural, DPHMJ

“. . . but what happens when we are exposed to totally new and can choose the option to leap in and to pull a switch which will
unfamiliar settings where our habits don’t suffice?” redirect the trolley onto a different track and save the five railway
Philip Zimbardo (2007); The Lucifer Effect, p. 6 workers. However, redirected onto the other track, the trolley
will kill one railway worker who would otherwise not have been
INTRODUCTION killed. In an alternative version of the dilemma, the action the
Moral dilemmas have become a standard methodology for protagonist has to perform in order to stop the trolley is different.
research on moral judgment. Moral dilemmas are hypothetical This time, there is no switch but a large stranger who is standing
short stories which describe a situation in which two conflicting on a bridge over the tracks. The protagonist can now choose to
moral reasons are relevant; for instance, the duty not to kill, and push that person with his hands onto the tracks so that the large
the duty to help. By inducing the participants to make a forced body stops the train. The outcome is the same: five individuals
choice between these two reasons, it can be investigated which saved by sacrificing one. However, participants in this task more
reason is given precedence in a particular situation, and which easily consent to pull the switch while they are much more
features of the situation matter for that decision. Accordingly, we reluctant to push the stranger with their own hands. The “action”
assume that this kind of hypothetical “ticking bomb scenarios” that the protagonist of the story can choose to carry out—or
can help to disentangle what determines human moral judgment. not—is termed a moral transgression or moral violation. The
This is, however, only possible if the moral dilemmas are very well choice itself, between the act of committing or omitting to carry
designed and potentially relevant factors are controlled for. The out the moral transgression is a moral judgment. The decision to
aim of this paper is to provide a set of such carefully designed and commit the harm is referred to as an utilitarian moral judgment,
validated moral dilemmas. because it weights costs and benefits, while the decision to refrain
The moral dilemmas commonly used in Cognitive from harm is a deontological moral judgment, because it gives
Neuroscience experiments are based on what Foot (1967) more weight to the “not to kill” principle.
and Thomson (1976) called the “Trolley Problem.” The trolley The influential work of Greene et al. (2001), which intro-
dilemma has two main versions. In the first one, a runaway duced moral dilemmas into Cognitive Neuroscience, has been
trolley is heading for five railway workers who will be killed if followed by many other studies as a way to deepen our under-
the trolley pursues its course. The experimental participant is standing of the role of emotion in moral judgment (for a review,
asked to take the perspective of a protagonist in the story who see Christensen and Gomila, 2012). However, results obtained

www.frontiersin.org July 2014 | Volume 5 | Article 607 | 1

Christensen et al. Moral dilemma validation study

with this methodological approach have been heterogeneous, In what follows we outline the changes we have made regard-
and there is a lack of consensus regarding how to interpret ing (i) the instructions given to the participant (subsection
them. Instructions to the Participant); (ii) the dilemma design, i.e., adjust-
In our opinion, one of the main reasons for this lays in ment of dilemma length, expression style, etc. (subsection Dilemma
the simple fact that the majority of studies have relied on the Design (1)—Formulation), (iii) the dilemma conceptualization,
initial set of moral dilemmas devised by Greene et al. (2001). i.e., thorough adaptation to the conceptual factors of Personal
While this set indisputably provided invaluable evidence about Force, Benefit Recipient, Evitability, and Intentionality (subsection
the neural underpinnings of moral judgment, it was not validated. Dilemma Design (2)—Conceptual Factors), and (iv) the formula-
Thus, conceptual pitfalls and formulation errors have poten- tion of the question eliciting the moral judgment (subsection The
tially remained unchallenged (Christensen and Gomila, 2012). Question Prompting the Moral Judgment). In the end, we have pro-
In fact, one of the key findings that have been reported (i.e., duced 23 dilemmas with two versions each, one personal and one
emotional involvement in moral judgment) might have been due impersonal, 46 dilemmas in total.
to uncontrolled variations in the dilemma formulations, rather
than to the factors supposedly taken into account (i.e., personal INSTRUCTIONS TO THE PARTICIPANT
vs. impersonal versions of the dilemma). As a matter of fact, To increase verisimilitude, we suggest that instructions at the
Greene and colleagues themselves have worded constructive self- beginning of the experiment ideally emphasize that participants
criticism with respect to that initial dilemma set and suggested are going to read short stories about difficult situations as they
using only a subset of the initial dilemmas, however, without vali- are likely to appear in the news or in the radio (for instance:
dating them either (Greene et al., 2009). Still, researchers continue “in the following you will read a series of short stories about dif-
to use this initial set. Here we present our efforts to remedy this ficult interpersonal situations, similar to those that we all see on
situation. the news every day or may read about in a novel”) (Christensen
We have fine-tuned a set of dilemmas methodologically and and Gomila, 2012, p. 14). This may help to put the participants
conceptually (controlling four conceptual factors). The set was “in context” for the task that awaits them. In addition, instruc-
selected from previously used moral dilemma sets: (i) Greene tions could include a remark about the fact that participants will
et al. (2001, 2004) and (ii) Moore et al. (2008) (this set was be offered one possible solution to the situation, and that their
based on Greene et al.’s but optimized). Both sets have been task will be to judge whether the proposed solution is acceptable,
used in a wealth of studies, however, without previous valida- given the information available (such as: “for each of the difficult
tion (e.g., Royzman and Baron, 2002; Koenigs et al., 2007; Moore situations a solution will be proposed. Your task is to judge whether
et al., 2008, 2011a,b). After the dilemma fine-tuning, norming to accept or not this solution”). Indeed, the closure of options
values were obtained for each dilemma: (i) of arousal and valence or alternatives is important. However, in previous dilemma sets,
(to ascertain the differential involvement of emotional processes some dilemmas have included expressions such as “the only way
along the dimensions of the 4 conceptual factors) and (ii) of to avoid [death of more people] is to [action proposal],” while
moral judgment (to confirm that moral judgment is sensitive to other dilemmas did not. Whereas this is important information,
the four factors)1. Finally, in the Supplementary Material of this including that same sentence in all dilemmas could make the
work, we provide the new set in six languages: English, French, reading rather repetitive and result in habituation. On the other
Spanish, German, Danish, and Catalan in order to make it more hand, including it only in some dilemmas could bias participants’
readily available for cross-cultural studies in the field. Please note responses to these dilemmas with respect to the others. Therefore,
that the norming study was carried out with the Spanish dilemma we suggest presenting it only in the general instructions to the
version. We encourage norming studies in the other languages participants.
(and in other cultures).
DILEMMA DESIGN (1)—FORMULATION
Control for formal characteristics of dilemma formulation
DILEMMA “FINE-TUNING”—PROPOSAL OF AN OPTIMIZED
includes:
SET
All dilemmas included in this set involved the decision to carry Word count
out a moral transgression which would result in a better overall Word count across dilemma categories: in the original sets the
numerical outcome. The participant was always the protagonist of dilemmas were rather long. This can entail an excessively long
this action (the moral transgression)2 and all dilemmas involved experimental session, resulting in participant fatigue. In Moore
killing (i.e., all social and other physical harm dilemmas were et al. (2008) an effort was made to control for mean word
eliminated). Furthermore, of the initial 48 dilemmas, 2 were elim- count: the Personal moral dilemmas (PMD) had 168.9 words
inated (the personal and impersonal versions of the cliffhanger and Impersonal moral dilemmas 169.3 (IMD). The maximum
dilemma) due to the unlikely acrobatics they involve. word count of a dilemma was 254 and the minimum was 123.
We shortened the dilemmas removing information that was not
1 Please note that study with a preliminary version of the revised set has strictly necessary and equalized the expression style of personal
recently been published (Christensen et al., 2012).
and impersonal versions of each dilemma. For instance, technical
2 For a detailed description of the dilemmas, see also Moore et al. (2008). For terms and long, non-familiar words were removed. Now the first
clarity it should be said that these 48 dilemmas are made up of 24 different three sentences of each dilemma are almost the same for both ver-
short stories, which have a personal and an impersonal version each. sions of a dilemma (personal and impersonal). For instance, the

Frontiers in Psychology | Emotion Science July 2014 | Volume 5 | Article 607 | 2

Christensen et al. Moral dilemma validation study

English version of the new dilemma set has a mean word count of Trade-off
130 words in the Personal and 135 in the Impersonal moral dilem- Trade-off across dilemmas: previous sets mixed different kinds of
mas. Our maximum number of words in a dilemma is 169 and moral transgressions, like stealing or lying. It is important not to
the minimum 93. See the Supplementary Material for the word mix them with killing, in order to avoid the risk of a non-desired
counts for each translation. carry-over effect between dilemmas. For instance, stealing, lying,
or lack of respect, may elicit less severe judgments when killing
Framing effects is also present in other dilemmas of the set, than when it’s not.
A framing effect consists in that people may judge one and Therefore, all dilemmas now raise the conflict between the option
the same situation differently, just because of the way it is to kill a person in order to save a larger number of people, and
described (Tversky and Kahneman, 1981; Petrinovich et al., the option of doing nothing and letting that larger number of
1993). Specifically, a clear risk of framing effects concerns the people die.
use of “kill” in some dilemmas, but “save” in others. People feel
more inclined to choose inaction when kill is used, and more Number of individuals
inclined toward action when save is emphasized (Petrinovich Number of individuals saved if the moral transgression is carried
and O’Neill, 1996). To avoid this, in all dilemmas the words kill out: in the set there now are the following categories (i) 5–10 peo-
and save are used in the second paragraph where the partici- ple, (ii) 11–50 people, (iii) 100–150 people and (iv) “thousands”
pant is given the information about the proposed action (i.e., of people or “masses” of people. It is an important variable to
the moral transgression) and its consequences. Conversely, the control for. A utilitarian response should become easier as more
words are removed from the question (e.g., Rescue 911 scenario: people are saved. Conversely, if moral judgment is purely deonto-
instead of Is it appropriate for you to kill this injured person logical, the number of people saved is totally irrelevant. This is an
in order to save yourself and everyone else on board? the action interesting question to have as a working hypothesis. Using dif-
verbs throw and keep were used). It is important to highlight ferent amounts of “saved individuals” in the formulations of the
the trade-off between cost (throw someone) and benefit (keep dilemmas allows researchers to explore at which point the posi-
yourself and more people in the air) in the questions of all tive consequences outweigh the transgression required to obtain
dilemmas. This was not accounted for in any of the previous them. For instance, it has been shown that attachment (“close-
dilemma sets. ness of relationship”) to the victim determines moral judgment
more than the number of beneficiaries involved. Still, this ques-
Situational antecedents tion needs further research, once closeness is controlled for (Tassy
In the original dilemma sets, keeping the situational antecedent et al., 2013). In this work, however, no specific analysis of this
used to present the characters constant was not accounted for. variable will be made, as it exceeds the limits of this norming
Thus, in the Personal version of the Nuclear reactor dilemma study.
the situational antecedent could bias the participants’ responses:
you are the inspector of a nuclear power plant that you suspect Information
has not met its safety requirements. The plant foreman and you Information supplied about the certainty of the consequences for
are touring the facility when one of the nuclear fuel rods over- the story character impersonated by the participant: in the Tycoon
heats. . . Later, it is this same foreman you are asked to consider and Nobel Prize dilemmas it said that “if you decide to [action of
to push into the fuel rod assembly. The participant was given the dilemma], nobody will ever find out.” This implies informa-
knowledge about a badly kept nuclear plant, with an in-charge tion about the future which cannot really be given with certainty,
individual who did not bother to make the plant meet the safety while at the same time contrasting with other stories where no
requirements. This makes it easier to sacrifice the plant fore- such commitments about the future are made. This kind of infor-
man to save the city than to sacrifice another, random, innocent mation can bias moral judgment and confuse it with legal pun-
person—which is the option to consider in all other dilemmas. ishment (or its lack). Therefore, this information was removed
Hence, prior information about the state of the power plant was altogether from the dilemmas. Similarly, dilemmas were excluded
removed, so that the foreman has no overt responsibility for the that cannot be understood without the assumption of an extraor-
nuclear accident which is about to happen. Now he is a “random” dinary ability or an unlikely event (such as the Cliffhanger)3.
person to be sacrificed, like in the other dilemmas. The Nobel
Prize dilemma had a similar problem. A situational antecedent DILEMMA DESIGN (2)—CONCEPTUAL FACTORS
made the person in a position to be sacrificed (here, your fellow On the grounds of the literature about moral judgment
researcher) appear a greedy bad person, so that it may be eas- (Christensen and Gomila, 2012), four main factors need to be
ier to sacrifice him than another innocent fellow researcher. The controlled for in moral dilemma formulation: Personal Force,
dilemma was reformulated so that the fellow researcher appeared Benefit Recipient (who gets the benefit), Evitability (whether the
not to know that the potential buyers would use the invention death is avoidable, or not), and Intentionality (whether the harm
as a weapon and only the protagonist explicitly knows it and is is willed and used instrumentally or a side-effect).
now again the only person with the possibility to prevent greater
harm from happening. In total, four dilemmas were modified to 3 We also considered removing the Bike Week Dilemma due to the act of acro-
keep constant the situational antecedents of the characters in the batics that it involves, but finally left it in. However, we encourage researchers
dilemmas. to potentially reconsider this.

www.frontiersin.org July 2014 | Volume 5 | Article 607 | 3

Christensen et al. Moral dilemma validation study

Personal force In the present dilemma set, the Personal dilemmas have been
Initially, Greene et al. (2001, 2004) defined a Personal moral revised accordingly. Personal Moral Dilemmas now require that
dilemma as one in which the proposed moral transgression sat- the agent is directly involved in the production of the harm.
isfied three criteria: (i) the transgression leads to serious bodily Impersonal Moral Dilemmas are those in which the agent is only
harm; (ii) this harm befalls a particular person or group of indirectly involved in the process that results in the harm.
people; and (iii) the harm is not the result of deflecting an exist-
ing threat onto a different party. Subsequently, Cushman et al. Benefit recipient
(2006) remarked that the crucial feature in a personal dilemma Self-interest is a well-known influence in moral judgments
is whether physical contact between the victim and the aggressor (Bloomfield, 2007). People will be more prone to accept an
is involved; a point also emphasized by Abarbanell and Hauser action whose consequences benefit themselves (i.e., the agent her-
(2010), while Waldmann and Dieterich (2007) focused on the self) than one that benefits others, maybe complete strangers.
Locus of Intervention (focus on the victim or on the threat) as the This “Self-Beneficial” vs. “Other Beneficial” contrast has been
difference between personal and impersonal dilemmas. Another introduced more clearly in the revised set. We reformulated the
proposal contended that the difference between Personal and Modified Euthanasia dilemma due to a confound in the trade-off
Impersonal is whether the action is mechanically mediated or not specification. Therefore, as the dilemma had to be an Other-
(Royzman and Baron, 2002; Moore et al., 2008). In more recent beneficial dilemma, now the key secret evidence the soldier could
work, Greene et al. have tried to offer an integrative definition reveal if tortured is the location of a particularly important base
(Greene, 2008; Greene et al., 2009). Specifically, these authors camp (and not the camp of the protagonist’s group).
propose that a Personal moral transgression occurs when (i) the
force that impacts the victim is generated by the agent’s muscles, Evitability
(ii) it cannot be mediated by mechanisms that respond to the This variable regards whether the death produced by the moral
agent’s muscular force by releasing or generating a different kind transgression is described as Avoidable or Inevitable. Would the
of force and applying it to the other person, and (iii) it cannot be person “to be sacrificed” have died anyway (Inevitable harm),
executed with guns, levers, explosions, gravity. . . or not (avoidable harm)? Transgressions that lead to inevitable
However, it seems as if this redefinition is driven by an effort consequences are more likely to be morally acceptable, by the
to keep the interpretation of the initial results, which results in principle of lesser evil (Hauser, 2006; Mikhail, 2007). In the
a circular argument: that “personal” dilemmas induce deonto- dilemma Rescue 911, a technical error in a helicopter puts the
logical judgments by emotional activation, while “impersonal” protagonist in the situation of having to decide to throw off one
ones induce utilitarian judgments by rational calculation. Yet, it is of her patients for the helicopter to lose weight. Without that
not yet clear which aspect of the personal involvement influences sacrifice the helicopter would fall and everybody—including that
moral judgment through emotional activation, nor is it clear one patient—would die. Conversely, the dilemma can also be
which kind of moral relevance emotions elicited by one’s involve- formulated in such a way that the individual to be sacrificed oth-
ment may have in the judgment. Similar considerations apply to erwise would not have been harmed (Avoidable death), such as
the introduction of the distinction between “high-conflict” and in the classical trolley dilemmas, where neither the bystander nor
“low-conflict” dilemmas (Koenigs et al., 2007), which also seem the innocent railway worker on the side track would have been
based on ex-post-facto considerations. harmed if the protagonist had not changed the course of events.
A principled way to clarify this distinction is in terms of the This distinction has now been made more explicit in the dilem-
causal role of the agent in the production of the harm. What mas (for examples of work where this variable was discussed, see
makes a dilemma impersonal is that the agent just initiates a pro- Moore et al., 2008; Huebner et al., 2011).
cess that, through its own dynamics, ends up causing the harm;
while a dilemma is personal when the agent is required not just Intentionality
to start the action, but to carry it out by herself. According to This factor refers to whether the harm is produced instrumentally,
this view, the presence of mediating instruments, by itself, does as something willed, or whether it happens as an unforeseen side-
not make a dilemma personal or impersonal. It depends of the effect, as collateral damage, to an action whose goal is positive.
kind of active involvement of the agent they require and amounts This variable concerns the doctrine of the double effect that has
to a difference in her responsibility of the caused harm, and in been shown to be psychologically relevant (Foot, 1967; Hauser,
the resulting (felt) emotional experience of it. This can account 2006; Mikhail, 2007). Causing harm is more acceptable when it
for the different moral judgments to Personal and Impersonal is produced as collateral damage, than when it is the goal of an
Dilemmas, which are observed despite the fact that the same action. Accordingly, Accidental harm refers to the case where the
consequences occur. The best philosophical explanation of this innocent victim of the dilemma dies as a non-desired side effect
difference is Anders (1962)’s reflection on the mass murders on of the moral transgression that the protagonist carries out to save
the Second World War. He contended that these acts were made others. Conversely, Instrumental harm occurs when the protago-
possible by the technical innovations that reduced the active nist intentionally uses the harm (i.e., the death) of the innocent
involvement of soldiers in the killing to pushing a button to victim as a means (i.e., instrumentally) to save the others.
release a bomb. It is not just that the new arms were of massive The reformulation of the dilemmas and the fine-tuning
destruction, but that their use was easier for us humans. Killing according to this factor is particularly relevant and one of the
with one’s hands is not just slower, but harder. main contributions of this norming paper. In the modified set

Frontiers in Psychology | Emotion Science July 2014 | Volume 5 | Article 607 | 4

Christensen et al. Moral dilemma validation study

of Moore et al. (2008), all Personal dilemmas were Instrumental, To summarize the four factors Personal Force, Benefit Recipient,
while Impersonal dilemmas included six Instrumental and six Evitability, and Intentionality, the illustration in Figure 2 provides
Accidental. The present set now allows a full factorial design a schematic overview over how the four factors are presented to
including intentionality. To introduce Accidental vs. Instrumental the participant during the course of a moral dilemma.
harm in Personal dilemmas attention was paid to key aspects of
the causal chain of the dilemma leading to the proposed salvation THE QUESTION PROMPTING THE MORAL JUDGMENT
of the greatest number of people. First, the exact intention that The formulation of the final question to elicit the moral judg-
the protagonist has in the very moment of committing the moral ment after reading the dilemma has also given rise to some
transgression was identified (does she carry out an action with controversy. The problem of the influence that the type of ques-
the intention to kill or not?). Second, a differentiation was made tion exerts on participant’s moral judgments has been addressed
between whether the harm is directly produced by the protagonist, empirically (e.g., O’Hara et al., 2010). Four question formats
or indirectly triggered by her action (do the positive consequences were used: wrong, inappropriate, forbidden, and blameworthy and
(the salvation of many) follow directly from the victim’s death, found that people judged moral transgressions more severely
or by some other event, an independent mechanism which was when the words “wrong” or “inappropriate” were part of the
triggered by the protagonist’s actions but not directly by her, nor formulation, than when the words “forbidden” or “blamewor-
directly willed by her?). The final point concerned by what means thy” were used. Another study found different behavioral effects
the larger number of people is saved (are they saved directly by following the questions Is it wrong to. . . ? vs. Would you? (Borg
the death of the victim, or for a different reason?). et al., 2006). The question Would you. . . ? resulted in faster RTs in
Following this rationale, for a better comprehension of the judging moral scenarios as compared to judgments of non-moral
Intentionality factor, the moral transgression is divided into a scenarios, while the question Is it wrong to. . . ? did not show any
5-part causal chain. This helps to disentangle the Accidental- differences in RT comparing the moral to the non-moral condi-
Instrumental dichotomy (see Figure 1). The first thing to identify tion. In view of these findings, it seems that deciding what to do
is the action by the protagonist (what exactly does she do?). Second, is not processed in the same way as deciding whether an action
which is the exact intention behind that action (why exactly does is right or wrong, and that in moral dilemmas is the first that
she do it?)? Third, does the victim die by the intervention of some matters.
intermediate (and protagonist-independent) mechanism or is the In recent work, two groups of researchers have addressed the
death directly due to the action of the protagonist (does she kill related issue of whether “what we say is also what we do.” Thus,
directly or by an independent mechanism?)? Fourth, how does the it was found that answering the question Is it acceptable to. . . ? vs.
innocent victim die (how does she die?)? Fifth, how is the larger the question Would you. . . ? resulted in differential response ten-
number of people saved (are they saved due to the death of the dencies (Tassy et al., 2013). However, another study showed that
victim or for some other reason?)? increasing the contextual information available to the participant

FIGURE 1 | Example of the causal chain of the proposed moral victim was literally used instrumentally with the intention to free the trapped
transgression that leads to the salvation. In the Instrumental version of group. Conversely, in the Accidental version of the Iceberg dilemma, the
the Burning Building dilemma the proposed action is “to use the body of the action of the protagonist is “to push the emergency access hatch.” The
victim.” The intention is “use the body to break down burning debris.” The intention behind that action is “to make the oxygen flow to the upper section
victim dies directly by the fire and there is no independent mechanism in of the boat.” The victim dies due to a knock on the head by an independent
between. A larger number of people are saved due to the fact that the mechanism which is the falling down of the hatch. Thus, the victim dies as a
burning debris was eliminated with the victim. The harm to the victim was side-effect of the act of salvation that the protagonist carries out with the
thus used as a means to save others. Said in different words, the body of the intention to get oxygen to the upper section of the boat.

www.frontiersin.org July 2014 | Volume 5 | Article 607 | 5

Christensen et al. Moral dilemma validation study

FIGURE 2 | The four factors in the dilemma set, adapted from is not carried out (Inevitable death, the person would die anyway), or not
Christensen and Gomila (2012), reproduced with permission. (1) (Avoidable death, the person would not die if no action is taken). (4)
Personal Force: the kind of imaginary involvement with the situation: Intentionality: if the action is carried out intentionally with the explicit
Personal, as direct cause, or Impersonal, as an indirect agent in the aim to kill the person as a means to save others, this is Instrumental
process of harm. (2) Benefit Recipient: concerns whether the harm (it explicitly needs the death of that person to save the others). If
protagonist’s life is at stake (Self-Beneficial action), or not the innocent person dies as a non-desired side-effect of the action by
(Other-Beneficial action). (3) Evitability: regards whether the victim would some independent mechanism and not directly by the action of the
die alongside the other individuals in the group if the moral transgression protagonist, the harm is Accidental.

resulted in more coherence between what they said they would There are 24 dilemmas where the death is Avoidable and 22
do and what they actually did (Feldman Hall et al., 2012). In any where it is Inevitable. Finally, there are 18 dilemma scenarios with
case, it is clear that consistency is required. Accidental harm (7 Personal and 11 Impersonal; 10 Self-Beneficial
For the present dilemma set a direct question was used Do and 8 Other-Beneficial; 10 Avoidable and 8 Inevitable) and 28
you [action verb] so that. . . to emphasize the consequences of the with Instrumental harm (16 Personal and 12 Impersonal; 12 Self-
choice made by the agent. Scales (Likert, Visual Analogical. . . ) Beneficial and 16 Other-Beneficial; 14 Avoidable and 14 Inevitable).
were used instead of a dichotomous answer format, as a way to See Table 1 for a summary. Please note that it was not possible
uncover the degree of conflict experienced. to provide the same number (quantity) of dilemmas in each of
the 16 categories because we relied on the materials of the former
SUMMARY: THE REVISED SET set. Refer to our discussion of this matter in the Supplementary
The revised set consists of 46 dilemmas, of which 23 are Personal Material (A) on limitations.
and 23 are Impersonal. As can be observed in Table 1, we main-
tained the original dilemma numbers so that it is easy to compare AROUSAL AND VALENCE NORMING EXPERIMENT
across sets. In 23 of the 46 dilemmas, the protagonist’s life is Peoples’ moral judgment has been shown to be sensitive to the
in danger and the moral violation results in saving not only affective impact of a dilemma in the individual (Moretto et al.,
a greater number of individuals, but also the protagonist her- 2010; Navarrete et al., 2012; Ugazio et al., 2012). However, no
self (Self-Beneficial dilemmas), whereas in the remaining 23, the dilemma set has so far been assessed in terms of the affec-
protagonist’s life is not in danger (Other-Beneficial dilemmas). tive arousal the individual dilemmas elicit in normal population
In turn, there are 11 Personal and 11 Impersonal Self -Beneficial as they are read—i.e., even if no moral judgment is required.
dilemmas, and 12 Personal and 12 Impersonal Other-Beneficial Therefore, data points for affective arousal and valence were
dilemmas. obtained for each dilemma of this set.

Frontiers in Psychology | Emotion Science July 2014 | Volume 5 | Article 607 | 6

Christensen et al. Moral dilemma validation study

Table 1 | Revised dilemmas.

Personal Impersonal

Self-beneficial Other-beneficial Self-beneficial Other-beneficial

Avoidable Inevitable Avoidable Inevitable Avoidable Inevitable Avoidable Inevitable

death death death death death death death death

1 13 25 37 2 14 26 38
3 15 27 39 4 16 28 40
5 17 29 41 6 18 30 42
7 19 31 43 8 20 32 44
9 (21) 33 45 10 (22) 34 46
11 23 35 47 12 24 36 48

The numbers refer to the dilemma number as given in the stimulus set (Supplementary Material). The colors refer to instrumental harm (gray) and accidental harm
(black). We have kept the numbers (dilemma numbers) as in Moore et al. (2008) to facilitate comparisons between the sets. Please note that there is not the same
number of dilemmas in each of the 16 categories. Please see our discussion of this matter in the section Arousal and Valence Norming Experiment on limitations.
The dilemmas 21 and 22 were removed (Cliffhanger dilemmas, see in the section Dilemma Design (1)—Formulation for reference).

We know that peoples’ moral judgments vary as a function

Table 2 | Participant characteristics in terms of emotional sensitivity,
of the four conceptual factors Personal Force, Benefit Recipient,
empathy, and alexithymia.
Evitability, and Intentionality. However, how peoples’ affective
responses (valence and arousal) are modulated by these fac- Instrument Mean SD
tors remains to be established. Besides, because inter-individual
differences in emotional sensitivity and empathy can affect the EIM (Bachorowski and Braaten, 1994) 164.75 18.77757859
subjective experience of arousal, participants in this experiment Emotional empathy (Mehrabian and 46.7 25.71703913
Epstein, 1972)
were assessed on these variables by means of self-report measures.
IRI (Davis, 1983) 55.7 8.095405686
METHODS TAS (Taylor et al., 1985) 18.3 14.78237613
Participants
Sixty-two undergraduate psychology students participated in this
study in exchange for a course credit in one of their degree sub- to run on six PCs [Windows XP SP3 PC (Intel Pentium Dual Core
jects (43 females, 19 males; age range = 18–48 years; m = 21.0, E5400, 2.70 GHz, 4 GB RAM)] and stimuli were displayed on 19
SD = 5.35). Participants completed four self-report measures. screens (with a resolution of 1440 × 900 p; color: 32 bits; refresh
First, the Interpersonal Reactivity Index (IRI) (Davis, 1983), which rate: 60 Hz). Data were analyzed using the statistical package SPSS
has four scales that focus on perspective taking, tendency to iden- v. 18 (www.ibm.com).
tify with fictitious characters, emotional reactions to the negative
experiences of others, and empathic concern for others. Second, Procedure
the Questionnaire of Emotional Empathy (Mehrabian and Epstein, Participants signed up for the experiment in class after having
1972) that conceives empathy as the vicarious emotional response completed the four self-report scales. The day of the experi-
to the perceived emotional experience of others. It explicitly ment, participants provided demographic data regarding gender,
understands empathy as different from Theory of Mind (ToM) age, and level of study. Informed consent was obtained from
and focuses on emotional empathy where high scores indicate a each participant prior to participation in any of the tasks and
high responsiveness to other peoples’ emotional reactions. Third, questionnaire procedures.
the Questionnaire of Emotional Sensitivity (EIM) (Bachorowski Participants were instructed as outlined in the section
and Braaten, 1994), which refers to the intensity with which a per- Instructions to the Participant. Each dilemma was presented in
son experiences emotional states irrespectively of their affective white Arial font, pt 16, on a black screen. By key press, the first
valence. Fourth, participants completed the Toronto Alexithymia paragraph of the dilemma appeared. With the next key press,
Scale (TAS) in which a high score means difficulties in under- the second paragraph appeared4 . Participants read at their own
standing and describing emotional states with words (Taylor et al., pace, advancing from one screen to the other by pressing the
1985). For results on the self-report measures, see Table 2. All space bar. With the third key press, the first two paragraphs of
were native Spanish speakers. the dilemma disappeared and two Likert scales appeared on sub-
sequent screens, the first asking participants to indicate their level
Materials
The forty-six moral dilemmas were arranged to be presented in 4 Please note: in this arousal and valence norming procedure participants did
random order in the stimulus presentation program DirectRT not see the question. This was to avoid confounds between the arousal and
(www.empirisoft.com) v. 2006.2.0.28. The experiment was set up valence judgments and a moral judgment.

www.frontiersin.org July 2014 | Volume 5 | Article 607 | 7

Christensen et al. Moral dilemma validation study

of arousal (1 = not arousing at all; 7 = very arousing) and the sec- Table 3 | RM ANOVA of the RT of the arousal ratings.
ond asking them to indicate the perceived valence of the dilemma
RM ANOVA—Main effects on RT AROUSAL
(1 = very negative; 7 = very positive). The ratings were made
(Personal Force, Benefit Recipient, Evitability, Intentionality)
by means of key press on the number keyboard of the computer.
Four practice dilemmas were added in the beginning of the task. Mean (ms) SE F-test p r
Data from these trials were discarded before data analysis. (1, 61)
The experiment was carried out in a laboratory of the uni-
versity suited for experiments with six individual PCs separated PERSONAL FORCE
in individual booths. Participants carried out the task in groups Personal (PMD) 2564.46 112.96
5.796 0.019 0.36
of 1–6 people. Viewing distance was of approximately 16 inches Impersonal (PMD) 2716.77 123.19
from the screen. The study was approved by the University’s BENEFIT RECIPIENT
Ethics Committee (COBE280213_1388). Self-beneficial 2506.66 119.52
20.783 <0.001 0.88
Other-beneficial 2774.57 115.66
RESULTS EVITABILITY
Factorial Repeated Measure (RM) 2 × 2 × 2 × 2 Analysis of Avoidable 2648.79 116.83
0.085 0.771 ns
Variances (ANOVA) were computed on subjective arousal and Inevitable 2632.44 117.71
valence ratings (Likert scale data), and on the RT of the INTENTIONALITY
arousal ratings. The factors were (1) Personal Force (Personal Accidental 2623.86 118.10
0.258 0.613 ns
vs. Impersonal harm); (2) Benefit Recipient (Self-Beneficial vs. Instrumental 2657.37 119.02
Other-Beneficial); (3) Evitability (Avoidable vs. Inevitable harm);
The values represent miliseconds.
and (4) Intentionality (Accidental vs. Instrumental harm). As
effect sizes we report Pearson’s r, where 0.01 is considered a small
effect size, 0.3 a medium effect and 0.5 a large effect (Cohen, Accidental the dilemma was rated as more arousing than when it
1988). was Instrumental [t(61) = 3.690, p < 0.001, r = 0.43]. For Other-
To rule out any effect of Gender in the results, the above Beneficial Harm, the pattern was reversed, as the Instrumental
ANOVA was computed with the between-subjects factor Gender. Harm dilemmas were more arousing than the Accidental Harm
There was no effect of gender in any of the interactions dilemmas [t(61) = −1.878, p = 0.065, trend effect, r = 0.05].
with the four factors, neither in the arousal ratings: Personal When comparing the Accidental and Instrumental Harm condi-
Force∗ gender: F(1, 60) = 1.47; p = 0.230; Benefit Recipient ∗ gender: tions, we found that Self-Beneficial, Accidental Harm dilemmas
F(1, 60) = 0.774; p = 0.383; Evitability: F(1, 60) = 0.079; p = resulted in higher arousal ratings than when dilemmas were
0.780; Intentionality∗ gender: F(1, 60) = 0.101, p = 752; nor in Other-Beneficial [t(61) = 7.626, p < 0.001, r = 0.49]. The same
the valence ratings: Personal Force ∗ gender: F(1, 60) = 0.004; pattern emerged when the harm was Instrumental; it was
p = 0.949; Benefit Recipient ∗ gender: F(1, 60) = 0.346; p = 0.558; judged as more arousing when it was Self-Beneficial, than when
Evitability: F(1, 60) = 0.019; p = 0.890; Intentionality∗ gender: it was Other-Beneficial [t(61) = 3.494, p = 0.001, r = 0.17].
F(1, 60) = 0.184, p = 0.670, nor in the RT. Therefore, data of If correcting for multiple comparisons using the Bonferroni
female and male participants were aggregated. method, this would mean accepting a new significance level of
α = 0.05/4 → α∗ = 0.0125. This should be taken into account
Arousal when considering the result with the trend effect.
All 16 dilemma categories were rated as being felt as of moderate
to high arousal (range: m = 5.58–6.24; see Table 3). Two of the Valence
four factors showed significant effects on the arousal ratings. First, Descriptive statistics of the valence ratings confirmed that all 16
there was a significant main effect of Personal Force [F(1, 61) = dilemma categories were rated as being of negative valence (range:
6.031; p = 0.017; r = 0.30], PMD being rated as more arousing m = 1.71–2.23; see Table S1).
(m = 5.92; SD = 0.12), than IMD (m = 5.83; SD = 0.12). The There were significant main effects of Personal Force [F(1, 61) =
second main effect was of Benefit Recipient [F(1, 61) = 47.57; p < 28.00; p < 0.001; r = 0.57] and of Benefit Recipient [F(1, 61) =
0.001; r = 0.66], Self-Beneficial Dilemmas being rated as more 31.509; p ≤ 0.001; r = 0.58]. PMD were rated as significantly
arousing (m = 6.02, SD = 0.12) than Other-Beneficial Dilemmas more negative (m = 1.905, SD = 0.065) than IMD (m = 2.054;
(m = 5.70, SD = 0.13). See Figure S3. There were no significant SD = 0.068). Likewise, Self -Beneficial Dilemmas were rated as
main effects of Evitability [F(1, 61) = 0.368; p = 0.546], nor of significantly more negative (m = 1.884, SD = 0.068) than Other-
Intentionality [F(1, 61) = 0.668; p = 0.417]. See Table S1 for the Beneficial Dilemmas (m = 2.075; SD = 0.067). The two other
means and Figure S3 in the Supplementary Material. factors did not show main effects [Evitability F(1, 61) = 1.201;
There was a significant interaction of Benefit p = 0.277; and Intentionality F(1, 61) = 0.135; p = 0.715]. See
Recipient*Intentionality [F(1, 61) = 15.24; p < 0.001; r = 0.44]. Table S1.
This indicates that Intentionality had different effects on partic- There were two significant interactions. The first was Personal
ipants’ ratings of arousal depending on whether the dilemma Force∗ Intentionality [F(1, 61) = 7.695, p = 0.007; r = 0.33]. The
was Self-Beneficial or Other-Beneficial. Figure S4 illustrates the Figure S5 shows that Intentionality had different effects on
results. Paired t-tests showed that when Self-Beneficial Harm was how people rated the valence of PMD and IMD. Paired t-tests

Frontiers in Psychology | Emotion Science July 2014 | Volume 5 | Article 607 | 8

Christensen et al. Moral dilemma validation study

showed that Accidental harm was rated as significantly more Table 4 | Summary table of the regression analysis of arousal ratings
negative than Instrumental harm in Impersonal Moral dilemmas as predictors of the arousal ratings’ RT for each of the 16 dilemma
[t(61) = −2.297, p = 0.025, r = 0.08], while no such difference categories.
was found between Accidental and Instrumental harm for Personal
Variables B SE B R2 B P
Moral dilemmas [t(61) = 1.441, p = 0.155, r = 0.03]. See Figure
S5. If correcting for multiple comparisons using the Bonferroni PMD_Self_Avo_Acc −773.62 176.50 0.243 −0.493 0.000
method, this would mean accepting a new significance level of PMD_Self_Avo_Instr −336.08 134.03 0.095 −0.308 0.015
α = 0.05/4 → α∗ = 0.0125. This should be taken into account PMD_Self_Ine_Acc −181.10 144.65 0.025 −0.160 0.215 (ns)
when considering the result of the first t-test (p = 0.025). PMD_Self_Ine_Instr −692.58 113.55 0.380 −0.619 0.000
The second significant interaction was Benefit PMD_Other_Avo_Acc −130.67 150.71 0.012 −0.111 0.389 (ns)
Recipient ∗ Intentionality [F(1, 61) = 6.041, p = 0.017; r = 0.30]. PMD_Other_Avo_Instr −231.73 143.76 0.042 −0.204 0.112 (ns)
This indicates that intention had different effects on the valence PMD_Other_Ine_Acc −276.63 136.91 0.062 −0.252 0.048
ratings depending on whether the dilemma was Self - or Other- PMD_Other_Ine_instr −495.32 140.80 0.171 −0.414 0.001
Beneficial. Paired t-tests showed that for Self-Beneficial Dilemmas, IMD_Self_Avo_Acc −348.19 129.55 0.107 −0.328 0.009
harm was judged significantly more negative when it was IMD_Self_Avo_Instr −582.35 126.31 0.261 −0.511 0.000
Accidental as compared to Instrumental harm [t(61) = −2.300, IMD_Self_Ine_Acc −572.35 153.15 0.189 −0.435 0.000
p = 0.025, r = 0.08]. No such difference in valence ratings of IMD_Self_Ine_Instr −382.88 174.58 0.074 −0.272 0.032
Accidental and Instrumental harm for Other-Beneficial dilemmas IMD_Other_Avo_Acc −516.66 154.98 0.156 −0.395 0.002
[t(61) = 1.296, p = 0.200, r = 0.03]. See Figure S6. If correcting IMD_Other_Avo_Instr −486.55 150.54 0.148 −0.385 0.002
for multiple comparisons using the Bonferroni method, this IMD_Other_Ine_Acc −140.19 180.26 0.010 −0.100 0.440 (ns)
would mean accepting a new significance level of α = 0.05/4 IMD_Other_Ine_Instr −339.32 146.90 0.082 −0.286 0.024
→ α∗ = 0.0125. This should be taken into account when
considering the result of these t-tests (p = 0.017 and p = 0.025). Abbreviations → IMD, Impersonal Moral Dilemmas; PMD, Personal Moral dilem-
The assessment of valence was only carried out to confirm that mas; Self, Self-Beneficial; Other, Other-Beneficial; Avo, Avoidable; Ine, Inevitable;

all dilemmas were of a strongly negative valence. This has hereby Acc, Accidental; Instr, Instrumental.

been confirmed and no other analysis will be carried out involv-

ing this feature of the dilemmas. All values for both arousal and to the dilemmas in general, and the shorter the RT (negative
valence are available for each dilemma in the excel spreadsheet correlation coefficient).
that accompanies this manuscript (Supplementary Material).
SUMMARY: AROUSAL AND VALENCE NORMING EXPERIMENT
Reaction time For a dilemma to be very negatively arousing (i.e., ratings very
A RM ANOVA was carried out on the RT of the arousal rat- negative in valence and high in arousal), the proposed moral
ings with the factors Personal Force, Benefit Recipient, Evitability, transgression had to be described as up-close and Personal.
and Intentionality. Main Effects were found for Personal Force and Besides, dilemmas where the protagonist’s own life was at stake
Benefit Recipient, no interactions were significant. See Table 3. were perceived as more negatively arousing than those dilemmas
Next, a regression analysis was conducted to ascertain how where other peoples’ lives were at stake. In particular, if the death
much of the variance in the RT of the arousal ratings was of the innocent victim happened accidentally as a non-desired side-
explained by the arousal ratings. This procedure was executed effect, the dilemma was perceived as more negatively arousing,
for each of the 16 dilemma categories. Summary Table 4 shows specifically, if the protagonist’s life was at stake, than if the acci-
that except for four of the categories, the arousal ratings signif- dental death of the victim happened in the intent to save other
icantly explained between 6 and 38% of the variance in the RT. people. In detail:
Figure 3 shows how the overall correlation between the variables
indicates that the more arousing a dilemma was, the faster partic- Affective arousal and valence
ipants indicated their rating. The correlation coefficient between - there were significant main effects of the factors Personal
the mean arousal ratings and the mean RT of arousal ratings was, Force and Benefit Recipient both for arousal and valence rat-
p < 0.001; r = −0.434. ings: Personal and Self-Beneficial dilemmas were perceived as
more arousing and more negative than Impersonal and Other-
Inter-individual differences: emotional sensitivity and empathy Beneficial dilemmas, respectively;
To ensure that the results of our arousal ratings were not driven - there were significant interactions between the two above
by inter-individual differences, participants had been assessed on factors and the factor Intentionality. Intentionality influences
a series of emotion-related questionnaires. Of the four question- perceived arousal in such way that Self-Beneficial dilemmas
naires, the level of empathy measured with the questionnaire by (as compared to Other-Beneficial dilemmas) were rated as
Mehrabian and Epstein had a significant effect on arousal ratings more arousing when harm happened as a non-desired side-
and on arousal rating RT. The overall correlation coefficients for effect (Accidental harm), while Instrumental harm (harm used
arousal ratings and Empathy scores was r = 0.289; p = 0.025 and as a means) was equally arousing in both Self - and Other-
for arousal RT and empathy scores it was r = −0.325; p = 0.011. Beneficial dilemmas. Furthermore, when harm was Personal
The higher the empathy scores, the higher the arousal ratings (up-close and corporal), as compared to Impersonal (distant

www.frontiersin.org July 2014 | Volume 5 | Article 607 | 9

Christensen et al. Moral dilemma validation study

DISCUSSION OF AROUSAL AND VALENCE NORMING EXPERIMENT

While all dilemmas are rated similarly as negative in valence, sig-
nificant differences were found in how they were rated in terms of
felt arousal. This means, first, that at least part of the emotional
involvement in moral judgment of the dilemmas can be due to the
arousal triggered when reading the situational description. And
second, results showed that differences in arousal are due to how
the different conceptual factors are manipulated. Thus, Personal
Force and Self-Beneficial dilemmas give rise to higher arousal rat-
ings than Impersonal and Other-Beneficial ones. Prima facie this
suggests that arousal has something to do with identification of
the experimental participant with the perspective of the main
character in the dilemmatic situation: it’s when one feels more
directly involved in the conflict, because of the action to be carried
out or the consequences for oneself that the action will have, that
one feels more aroused—even without having to make a judg-
ment. However, this prima facie interpretion is too simplistic, for
three reasons.
In the first place, it is clear that Personal Force dilemmas high-
light the personal involvement in physically producing the harm.
FIGURE 3 | Correlation between Arousal ratings and the RT. Color
coding: Personal Moral Dilemmas (PMD; Blue/Red, circles); Impersonal Similarly, Self-Beneficial dilemmas give rise to higher arousal rat-
Moral Dilemmas (IMD; Green/Yellow, squares). Arousal ratings are 1 = Not ings only when the harm produced is Accidental, rather than
arousing, calm; 7 = Very arousing, on the x-axis. RT is in milliseconds (ms) Instrumental. The latter case is one of self-interest: we experience
on the y-axis. The numbers refer to the dilemma numbers in the less conflict when what’s to be done is for our own benefit. Yet,
dilemma set.
it becomes difficult when a benefit cannot be produced without
collateral damage. Third, whereas Self-Beneficial dilemmas take
and abstract), and used as a means (Instrumental harm), longer to be rated (than Other-Beneficial), Personal Force ones
dilemmas were rated as more negative, than if harm was are rated faster than Impersonal ones. Jointly, these results sug-
Impersonal. Conversely, participants found Accidental harm gest that arousal ratings can have several etiologies, and therefore
equally negative when it was Personal (up-close and corporal) cannot be interpreted simply as indication of degree of imaginary
and Impersonal (distant and abstract). involvement with the situation or as a measure of experienced
conflict. Both these causes need to be considered.
Reaction time
RT to a moral judgment task has previously been suggested DILEMMA VALIDATION STUDY—MORAL JUDGMENT
as an indicator of emotional involvement. The more arousing EXPERIMENT
a dilemma was, the faster participants were in making their To validate this moral dilemma set, a moral judgment task was set
rating. up to confirm the 4-factor structure in the dilemmas; i.e., the four
conceptual factors Personal Force, Benefit Recipient, Evitability,
Inter-individual differences and Intentionality.
There was a correlation between inter-individual differences in Furthermore, to explore how the intentionality factor is under-
empathy assessed by means of the Questionnaire of Emotional stood by participants, two versions of the dilemma set were
Empathy (Mehrabian and Epstein, 1972) and the arousal rat- prepared: one version remained as has been described so far, while
ings. It showed that the higher the levels of empathy, the more in the other the question eliciting the moral judgment included an
arousing were the dilemmas to the participant. This makes sense “accidental harm specification” in the accidental harm dilemmas.
because this instrument describes sensitivity to others’ emotional For instance, in the dilemma Burning Building, the question is Do
states. It specifically conceives empathy as the vicarious emo- you put out the fire by activating the emergency system, which will
tional response to the perceived emotional experience of others leave the injured without air, so you and the five other people can
and understands empathy as different from ToM and perspective- escape? The sentence which will leave the injured without air is the
taking, and focuses on emotional empathy where high scores accidental harm specification. This makes it clear to the reader the
indicate a high responsiveness to other peoples’ emotional reac- consequences of the proposed action. The analysis of this variable
tions. However, apart from this correlation between arousal rat- is included here, but in future researchers can choose to leave the
ings and empathy level, no other individual differences had an accidental harm specification out of the question.
effect on perceived arousal (the other variables we assessed were Additional analyses include (i) the analysis by Greene et al.
gender, IRI, emotional sensitivity, alexithymia). We therefore con- (2001, 2004) that gave rise to the Dual Process Hypothesis of
clude that—at least in this sample of Spanish Undergraduates- Moral Judgment (DPHMJ), (ii) an additional analysis of the
the arousal ratings of this dilemma set are rather robust across Intentionality factor, and (iii) an analyses of how interindividual
individual differences. differences influence moral judgment.

Frontiers in Psychology | Emotion Science July 2014 | Volume 5 | Article 607 | 10

Christensen et al. Moral dilemma validation study

METHODS of numbers) of the computer. Four practice dilemmas were added

Participants in the beginning of the task. Data from these trials were discarded
Forty-three undergraduate psychology and educational science before data analysis. The study was approved by the University’s
students participated in this study in exchange for a course credit Ethics Committee (COBE280213_1388).
in one of their degree subjects (30 females and 13 males; age
range = 18–54 years; m = 20.65, SD = 5.52). None of them RESULTS
had seen the dilemmas before. See Table 5 for participant char- A factorial RM 2 × 2 × 2 × 2 ANOVA was computed with
acteristics including self-report measures of (i) the IRI (Davis, the Within-Subject factors Personal Force (PMD vs. IMD),
1983), (ii) the Questionnaire of Emotional Empathy (Mehrabian Benefit Recipient (Self-Beneficial vs. Other Beneficial), Evitability
and Epstein, 1972), (iii) the Questionnaire of Emotional sensitiv- (Avoidable vs. Inevitable harm), and Intentionality (Accidental
ity (EIM) (Bachorowski and Braaten, 1994), (iv) the TAS (Taylor vs. Instrumental harm). Question Type (with vs. without the
et al., 1985), (v) the personality questionnaire Big Five (McCrae Accidental harm specification) was the Between-Subject factor.
and Costa, 1999), and (vi) the Thinking Style Questionnaire, Need As effect sizes we report Pearson’s r, where 0.01 is considered
For Cognition Scale (NFC) (Cacioppo et al., 1984). All participants a small effect size, 0.3 a medium effect and 0.5 a large effect
were native Spanish speakers. (Cohen, 1988).
Subjective ratings: moral judgment
Materials There was no significant main effect of the between-group
Forty-six standard moral dilemmas and four practice dilemmas factor Type of Question (with or without accidental harm
were presented in random order with the stimulus presentation specification) [F(1, 41) = 0.164, p = 0.688] and there were
program DirectRT (www.empirisoft.com) v. 2006.2.0.28. The no significant interactions between the Between-Subjects fac-
experiment was set up to run on six PCs (Windows XP SP3 PC tor Type of Question and the four within-subject factors:
(Intel Pentium Dual Core E5400, 2.70 GHz, 4 GB RAM) and stim- Personal Force∗ Question Type [F(1, 41) = 0.09; p = 0.766; ns];
uli were displayed on 19 screens (with a resolution of 1440 × 900 Benefit Recipient ∗ Question Type [F(1, 41) = 0.296; p = 0.589;
p; color: 32 bits; refresh rate: 60 Hz). ns]; Evitability∗ Question Type [F(1, 41) = 0.010; p = 0.921; ns];
Intentionality∗ Question Type [F(1, 41) = 0.013; p = 0.911; ns].
Procedure This means that the two question formats (with and without the
As in the previous experiment described in the section Arousal Accidental harm specification) are equivalent and do not affect the
and Valence Norming Experiment. Additionally: after the sec- moral judgment a person makes. This means that the accidental-
ond screen, the first two screens disappeared and the question ity of the harm is understood from the narrative without the need
appeared. The question eliciting the moral judgment was “Do to explicitly state it to the individual. Thus, data was aggregated
you [action verb] so that. . . .” A 7-point Likert scale was displayed for subsequent analyses.
below the question with the labels “No, I don’t do it” under the There were significant main effects of all four Within-Subject
number “1” and “Yes, I do it” under the number “7.” Half of the factors: Personal Force [F(1, 41) = 54.97; p < 0.001; r = 0.75];
participants (22 participants) saw the question “Do you [action Benefit Recipient [F(1, 41) = 4.347; p = 0.043; r = 0.31];
verb] so that. . . ,” while the other half (21 participants) saw a ques- Evitability [F(1, 41) = 69.984; p < 0.001; r = 0.79]; and
tion that furthermore involved the Accidental harm specification Intentionality [F(1, 41) = 12.971; p = 0.001; r = 0.49].
in the case of the Accidental harm dilemmas, such as in: “do you Participants were less likely to commit harm in PMD (m = 4.069;
[action verb] which will [mechanism that will lead to the death] so SD = 0.124) than in IMD (m = 4.717; SD = 0.113) and they
that. . . ” (Type of Question). The ratings were made by means of were more likely to commit a moral transgression to save them-
key press on the using the number keys of the keyboard (top row selves (m = 4.508; SD = 0.103), than to save others (m = 4.278;
SD = 0.111). When the suggested harm was Inevitable, people
were more likely to commit it (m = 4.633; SD = 0.124) than
Table 5 | Participant characteristics.
when harm was Avoidable (m = 4.152; SD = 0.103). Finally,
Instrument Factors Mean SD when the death of the victim was Accidental, participants were
more likely to commit the moral transgression (m = 4.549;
EIM (Bachorowski and Braaten, 165 18.53 SD = 0.125) than when it was Instrumental (m = 4.236;
1994) SD = 0.112). See Figures S7A–D.
Emotional empathy (Mehrabian 48.58 23.41 Five of the six possible two-way interactions between the
and Epstein, 1972)
four factors were significant. See Table 6 for a summary of the
IRI (Davis, 1983) 54.60 6.99
means and interaction coefficients. Table 7 shows the t-tests to
TAS (Taylor et al., 1985) 16.58 12.88
break down the interactions. Figure S8 summarizes the interac-
Big Five (McCrae and Costa, Neuroticism 12.30 11.65
tions graphically. If correcting for multiple comparisons using
1999) Extraversion 21.58 9.70
Openness to 15.81 10.76
the Bonferroni method, this would mean accepting a new signifi-
experience cance level of α = 0.05/4 → α∗ = 0.0125 for breaking down each
Agreeableness 13.72 9.84 interaction. This should be taken into account when considering
Consciencousness 22.58 12.63 the result of the t-test in Table 7D (Self-Beneficial Accidental vs.
NFC (Cacioppo et al., 1984) 17.44 18.84 Instrumental harm; p = 0.022).

www.frontiersin.org July 2014 | Volume 5 | Article 607 | 11

Christensen et al. Moral dilemma validation study

Table 6 | Summary table of the interactions (dependent variable: moral judgment, Likert scale rating; range: 1;7).

Factors Descriptives Interaction coefficients

Interactions Factor 1 Factor 2 Mean SD F (1, 41) p r

Personal Force * Beneficiency Personal Self 4.076 0.142

Other 4.061 0.129
18.248 <0.001 0.55
Impersonal Self 4.939 0.139
Other 4.494 0.119

Personal Force * Evitability Personal Avoidable 3.890 0.110

Inevitable 4.248 0.147
8.864 0.008 0.42
Impersonal Avoidable 4.415 0.112
Inevitable 5.018 0.123

Personal Force * Intention Personal Accidental 4.326 0.141

Instrumental 3.812 0.131
14.582 <0.001 0.51
Impersonal Accidental 4.773 0.129
Instrumental 4.660 0.114

Beneficiency * Evitability Self Avoidable 4.222 0.135

Inevitable 4.793 0.146
1.663 0.204 ns
Other Avoidable 4.082 0.110
Inevitable 4.474 0.132

Beneficiency * Intention Self Accidental 4.416 0.137

Instrumental 4.599 0.140
40.202 <0.001 0.70
Other Accidental 4.683 0.146
Instrumental 3.872 0.118

Evitability * Intention Avoidable Accidental 4.410 0.112

Instrumental 3.894 0.112
12.990 <0.001 0.49
Inevitable Accidental 4.689 0.151
Instrumental 4.577 0.121

First, the Benefit Recipient variable had a differential effect on accidental harm, participants were less likely to commit harm if it
the moral judgment for PMD and IMD (Figure S8A). Participants was accidental to save herself, than if it was to save others.
were more likely to commit harm if the harm was carried Fifth, Intentionality also affected how participants judged
out to safe themselves (Self -Beneficial, as compared to Other- Avoidable and Inevitable dilemmas (Evitability factor), (Figure
Beneficial), however, only if the dilemma was Impersonal. If harm S8E). When harm was Avoidable (as compared to Inevitable), par-
was Personal, participants were equally likely to commit the harm ticipants were less likely to commit it when the harm described
both when it was Self - or Other-Beneficial. in the dilemma was Instrumental than when it was Accidental.
Second, also the Evitability variable had a differential effect on However, participants were equally likely to commit harm to
the moral judgment for PMD and IMD (Figure S8B). Participants both Accidental and Instrumental harm dilemmas when the harm
made more deontological responses for PMD in general; how- described in the dilemma was Inevitable.
ever, they were more likely to commit harm when the death of That there was no interaction between Benefit Recipient and
the innocent person was Inevitable (as compared to Avoidable). Evitability means that participants were equally likely to commit
Third, also the Intentionality variable affected how partici- harm, irrespective of whether death was Avoidable or Inevitable
pants judged PMD and IMD (Figure S8C). Again participants for Self- or Other-Beneficial dilemmas.
were overall more likely to make a deontological moral judgment
in PMD than in IMD, however, participants were less likely to Reaction time
commit the moral transgression when harm was Instrumental (as There was one significant main effect [Intentionality: F(1, 41) =
compared to Accidental), but specifically only in the case of PMD. 13.252; p = 0.001; r = 0.49] and one significant interaction
Fourth, the Intentionality variable affected how participants [Intentionality∗ Question Type: F(1, 41) = 13.629; p = 0.001; r =
judged Self - and Other-Beneficial dilemmas (Figure S8D). If the 0.50]. Participants in general needed longer to make moral judg-
proposed harm was Instrumental, participants were less likely ments about actions involving Accidental harm (m = 5803.223;
to commit it when the dilemma involved harm toward Others SD = 424.081) than of actions involving Instrumental harm
(as compared to harm toward the participant herself), while for (m = 5185.185; SD = 394.389). The interaction indicates that

Frontiers in Psychology | Emotion Science July 2014 | Volume 5 | Article 607 | 12

Christensen et al. Moral dilemma validation study

Table 7 | Follow-up t-tests to break down the interactions in the Due to the fact that the only main effect and interactions that
moral judgment task. appear significant in the analysis of the RT data is the factor
that regards the Between-Subject variable Type of Question, this
Mean SE t-test(42) p r
effect was explored more closely. Therefore, the RM ANOVA was
(A) Tests to break down the interaction Personal Force*Benefit computed again, first with the participants in the With condition
Recipient and afterwards with the participants in the Without condition.
PERSONAL MORAL DILEMMAS Again the factor Intentionality was significant in the With con-
Self- 4.076 0.142 dition [F(1, 22) = 21.208; p < 0.001; r = 0.70], but not in the
beneficient Without condition [F(1, 19) = 0.002; p = 0.964]. Hence, the effect
0.134 0.894 ns
Other- 4.061 0.129 was merely driven by the higher number of words in the questions
beneficient in the With condition.
IMPERSONAL MORAL DILEMMAS To ensure that RT was not conditioned by the word count of
Self- 4.939 0.139 the questions in general, a regression was computed with word
beneficient count in the question as a predictor and RT as the dependent
3.535 0.001 0.48
Other- 4.494 0.119 variable. No significant relationship was found (B = −27.695;
beneficient
BSD = 30.711; β = −0.234; p = 0.382). Hence, the word count
(B) Tests to break down the interaction Personal Force*Evitability of the questions did not influence the RT of participants except
PERSONAL MORAL DILEMMAS in this particular case of the Intentionality factor. Apart from this
Avoidable 3.890 0.110 problematic effect, there were no other significant main effects or
−4.742 <0.001 0.59 interactions.
Inevitable 4.248 0.147
IMPERSONAL MORAL DILEMMAS As much research in the field of moral judgment with moral
Avoidable 4.415 0.112 dilemmas suggests a relation between the type of moral judg-
−9.159 <0.001 0.82
Inevitable 5.018 0.123 ment (deontological vs. utilitarian) and RT, this matter was
explored further. First, a curvilinear regression was computed
(C) Tests to break down the interaction Personal Force*Intentionality with Moral Judgment as predictor and the RT as dependent vari-
PERSONAL MORAL DILEMMAS able. The resulting model was significant [F(1, 41) = 11.015; p <
Accidental 4.326 0.141
4.681 <0.001 0.59 0.001; r = 0.46] and moral judgment accounted for 33.9% of the
Instrumental 3.812 0.131 variance in the RT. Both for very deontological (Likert ratings
IMPERSONAL MORAL DILEMMAS toward 1) and very utilitarian moral judgments (Likert ratings
Accidental 4.773 0.129 toward 7) participants were faster than when making a more
1.265 0.213 ns
Instrumental 4.660 0.114
intermediate moral judgment (Likert ratings around 4). See the
(D) Tests to break down the interaction Benefit Recipient* illustration of the relation between moral judgment and RT in
Intentionality Figure 4.
SELF-BENEFICAL DILEMMAS To assess RT as a function of the response given (deontolog-
Accidental 4.416 0.137 ical vs. utilitarian in absolute terms, not in a scale from 1 to 7
−2.397 0.021 0.35
Instrumental 4.610 0.140 as presented above) as in Greene et al. (2001, 2004), the Moral
OTHER-BENEFICAL DILEMMAS Judgment values of the 7-point Likert scale were dichotomized.
Accidental 4.683 0.146 Judgments of values between 1 and 3 were considered “deonto-
5.605 <0.001 0.65
Instrumental 3.872 0.118 logical,” and values between 5 and 7 were considered “utilitarian.”
Values of 4 were discarded. Mean RT was calculated as a func-
(E) Tests to break down the interaction Evitability*Intentionality tion of this re-coding. Subsequently, the ANOVA from Greene
AVOIDABLE HARM et al. (2001, 2004) 2 × 2 (Response Type and Personal Force) was
Accidental 4.411 0.112 carried out. No significant main effects were found [Response
5.853 <0.001 0.67
Instrumental 3.894 0.112 Type: F(1, 42) = 0.402; p = 0.529; Personal Force: F(1, 42) = 0.197;
INEVITABLE HARM
p = 0.659].
Accidental 4.689 0.151
0.977 0.334 ns In previous analyses, the factor Intentionality has been
Instrumental 4.578 0.121
shown to be of key relevance in moral judgment. Therefore,
another 2 × 2 ANOVA with the variables Response Type and
Intentionality was run. There was a significant main effect
Intentionality had a differential effect on RT depending on of Intentionality (p = 0.015) and a significant interaction
the Question Type. The group that had the question with of Response Type∗ Intentionality (p = 0.018), see Table 8 and
the Accidental harm specification, needed significantly longer Figure S9. Breaking down the interaction it was shown that
to respond to Accidental harm (m = 6356.081; SD = 578.441) participants took longer to make a deontological moral
than the group without such specification (m = 5250.365; SD = judgment when harm was then produced accidentally,
620.309). No such difference appeared between the groups than if it was instrumental (p = 0.003). No such difference
for Instrumental harm (m = 5112.582; SD = 537.941 and m = was found for utilitarian moral judgments (p = 0.681), see
5259.065; SD = 576.878, respectively). Figure S9.

www.frontiersin.org July 2014 | Volume 5 | Article 607 | 13

Christensen et al. Moral dilemma validation study

Table 8 | Main Effects and Interactions of the RM ANOVA Question

Type*Intentionality.

(A) RM ANOVA—Main effects on RT MORAL JUDGMENT as a

function of RESPONSE TYPE

Mean SE F (1, 41) p r

QUESTION TYPE
Deontological response 5680.779 427.726
0.005 0.946 ns
Utilitarian response 5661.827 441.793
INTENTIONALITY
Accidental harm 6009.467 449.472
6.499 0.015 0.37
Instrumental harm 5333.139 415.105
INTERACTIONS
Response Type*Intentionality 6.010 0.018 0.65

(B) Tests to break down the interaction

Mean SE t-test(42) p r

FIGURE 4 | Curvilinar relationship between Moral Judgment and RT. DEONTOLOGICAL RESPONSE
Color coding: Personal Moral Dilemmas (Blue/Red, circles); Impersonal Accidental harm 6434.148 571.955
Moral Dilemmas (Green/Yellow, squares). Mean Likert scale responses: 1 =
3.313 0.003 0.46
Instrumental harm 4927.411 393.270
No, I don’t do it, i.e., deontological moral judgment; 7 = Yes, I do it, i.e.,
UTILITARIAN RESPONSE
utilitarian moral judgment. RT is in milliseconds (ms). PMD, Personal Moral
Dilemmas; IMD, Impersonal Moral Dilemmas. Accidental harm 5584.787 424.480
−0.414 0.681 ns
Instrumental harm 5738.867 528.586

Mean Likert scale responses: 1 = No, I don’t do it, i.e., deontological moral judg-
Inter-individual differences: gender ment; 7 = Yes, I do it, i.e., utilitarian moral judgment. RT is in milliseconds (ms).
There was a significant interaction between the factor Benefit
Recipient and the participants’ gender [F(1, 61) = 10.079; p =
0.003; r = 0.37]; male participants were more ready to commit Inter-individual differences: thinking style, personality traits,
a harm in the case of Self -Beneficial dilemmas (m = 5.137; SD = emotional sensitivity
0.215), than female participants (m = 4.235; SD = 0.142). In the To test the influence of inter-individual differences on moral
Other-Beneficial dilemma category, no such gender differences judgment a regression was computed with all of the scores of
were found (males: m = 4.439; SD = 0.203; females: m = 4.208; the questionnaires assessing inter-individual differences in the
SD = 0.133). This effect is reported for the sake of completeness model predicting the mean moral judgment of the participants.
of the scientific record. However, first, we did not specifically con- As shown in Table S2, the resulting regression model was signif-
template this effect, so we did not have equal numbers of male and icant [F(10) = 2.954; p = 0.011; r = 0.47] and explained 50.5%
female participants. Second, we do not aim to make any assump- of the variance in the moral judgments. However, only three of
tions about gender differences based on such preliminary data. the 10 predictor variables were significant: Emotional Sensitivity
There is no sound scientific evidence that supports why there (p = 0.018), and two of the Big Five factors, Agreeableness (p =
should be gender differences in moral judgment, nor of what kind 0.046) and Conscientiousness (p = 0.001). The higher the scores
these may be, nor what should be the evolutionary basis for them. in the EIM, the more deontological were the moral judgments
This is a sensitive issue that deserves thorough investigation that (participants with higher scores in the EIM were less susceptible
goes far beyond the scope of this paper. Therefore, we assume that to commit the proposed harm). For the two factors of the Big Five,
there are no genuine gender differences in moral judgment in par- the pattern was reverse: the higher the scores, the more utilitar-
ticipants of one same culture and have chosen to analyze the data ian were the judgments (participants with higher scores in these
of female and male participants together. two dimensions were more likely to commit the proposed harm).
Two other studies have reported an effect of gender in their However, considering the Beta coefficient, it can be observed that
data (Fumagalli et al., 2009, 2010). However, the dilemma set these effects were—although existent—rather small.
used in these studies was the originally used by Greene et al.
(2001, 2004) which has important methodological shortcomings AROUSAL AND MORAL JUDGMENT
(as pointed out by this paper; for a review see Christensen and In order to determine whether the levels of arousal of the dilem-
Gomila, 2012), which is why ideally such claims on gender dif- mas rated by one group of participants, would be related to the
ferences should really not be made. For such claims to be based moral judgments of a different group of participants, the dataset
on solid grounds a study should be designed controlling variables was transposed and dilemmas treated as cases. A simple regres-
of empathy and other personality factors between genders, and of sion was conducted with the arousal ratings as predictor variable
course, have an equal sample size of each gender. and the moral judgments as dependent variable. The resulting

Frontiers in Psychology | Emotion Science July 2014 | Volume 5 | Article 607 | 14

Christensen et al. Moral dilemma validation study

model was significant [F(1, 44) = 22.613; p < 0.001; r = 0.58], more readily committed by the participants than if the proposed
showing that the level of arousal of a dilemma predicted 33.9% harm would result in using the death of the victim as a means to
of the variance in the moral judgment variable. Figure 5 shows salvation of the others.
that the more arousing a dilemma was, the more likely partici- As regards the interactions between the factors:
pants were to refrain from action (i.e., not committing the moral First, the interaction between Personal Force and Benefit
transgression). See Table S3 for the model parameters. Recipient indicated that participants were equally likely to commit
a moral transgression when the proposed harm involved “bloody
SUMMARY: MORAL JUDGMENT EXPERIMENT hands,” both when the harm would result in salvation of oneself
With this fine-tuned set of moral dilemmas it was confirmed that or of others. However, when the proposed harmful action was
the four factors Personal Force, Benefit Recipient, Evitability, and abstract and distant, participants made a difference in their moral
Intentionality determined participants’ moral judgment: judgment, depending on whether the salvation regarded them-
First, participants tended to exhibit a deontological response selves or others. Abstract harm commission made a utilitarian
style (i.e., they refrained from committing harm) when harm response more likely when it was executed to save themselves.
was described as Personal (as compared to Impersonal), Other- Second, the interaction between Personal Force and
Beneficial (as compared to Self -Beneficial), Avoidable (as Intentionality indicated that harm that happened as a non-
compared to Inevitable), and Instrumental (as compared to desired side-effect of the moral transgression was consented
Accidental). In other words, when harm was abstract and spatially equally in IMD, both when harm was accidental and when it was
and intentionally separated from the agent, participants were instrumental. However, in PMD, if harm was used as a means
more likely to commit this moral transgression than if the harm (instrumentally), this made participants’ moral judgments more
was described as up-close and gave an impression of “bloody deontological than when harm was accidental.
hands.” Third, the interaction between Benefit Recipient and
Second, participants more readily sacrificed the life of another Intentionality indicated that for Self-Beneficient Dilemmas,
person if their own life was at stake than if the moral transgression when harm happened as a non-desired side-effect of the pro-
would merely save other people. Besides, if harm to the victim posed action, participants were less likely to commit the moral
would have happened anyway, irrespective of whether the moral transgression, than when it was instrumental. Conversely, when
transgression was carried out by the agent or not (as in “or one the harm would benefit others, the pattern was reverse: more
person of 5 is killed or they all die”), participants were more likely deontological moral judgments when harm was instrumental,
to incur in the moral transgression. than when it was accidental.
Third, participants more readily committed harm if harm hap- Fourth, the interaction between Personal Force and Evitability
pened as a non-desired side-effect of the action of the agent, it was indicates that for both IMD and PMD, avoidable harm resulted in
more deontological moral judgments than did inevitable harm.
Fifth, the interaction between Evitability and Intentionality
indicates that both when harm to the victim could have been
avoided, harm as a side-effect was more readily consented, than
was the use of harm as a means. For inevitable harm no such dif-
ference between accidental and instrumental harm commission
was found.
Furthermore, we found that the more arousing a dilemma
was, the more likely it was that participants would choose a
deontological response style.
Finally, there was no main effect of Type of Response found by
Greene et al. (2001, 2004), indicating that with this optimized
dilemma set deontological responding is not faster than utilitar-
ian responding. Neither was there an interaction between Type
of Response∗ Personal Force. However, with an additional ANOVA
with the factors Type of Response and Intentionality it was shown
that there was a significant main effect of Intentionality. Yet, more
importantly, there was an interaction between Type of Response
and Intentionality. This indicates that for dilemmas people were
judging deontologically, it took them particularly long to make
that judgment in the case when the proposed harm would result
in accidental harm to the victim.
FIGURE 5 | Relationship between level of arousal of a dilemma and the
moral judgment made to that dilemma. Color/shape coding: Personal
Moral Dilemmas (Blue/Red, circles); Impersonal Moral Dilemmas DISCUSSION OF THE MORAL JUDGMENT EXPERIMENT
(Green/Yellow, squares). Mean Likert scale responses: 1 = No, I don’t do Summing up, results here show that that we are more prone
it, i.e., deontological moral judgment; 7 = Yes, I do it, i.e., utilitarian moral
to behave for our benefit, if the harm will take place in any
judgment. Mean Arousal scale responses: 1 = Not arousing, calm; 7 = Very
arousing.
case and producing the harm is not very demanding. Conversely,
we are going to experience a conflict—indexed by a longer

www.frontiersin.org July 2014 | Volume 5 | Article 607 | 15

Christensen et al. Moral dilemma validation study

response—when we are forced to do the harm ourselves, or to deontological moral judgments (i.e., refraining from harm), while
do harm as collateral damage to benefit others. Moral principles anger induction (an emotion that triggers approach tendencies)
can be broken but only in well-justified situations (when con- resulted in more utilitarian moral judgments (i.e., committing
sequences are “big enough”). It’s not that we are deontological harm). This finding doesn’t fit the Dual Process account either,
or utilitarian thinkers, we are neither: moral judgments are bet- because the study shows how different emotional phenomena
ter viewed from the point of view of casuistics, the particularist trigger both deontological and utilitarian moral judgment ten-
approach to morals that takes the details of each case into account. dencies.
Any small detail may matter to our moral judgment. Results show, Therefore, we propose that a potentially more suitable account
in any case, that rules are not applied algorithmically or in a strict of moral judgment is one that gives a different role to emotions
order (Hauser, 2006). in moral judgment, specifically, to the importance of the arousal
response which is triggered in the individual by the dilemmatic
OVERALL DISCUSSION situation along the way suggested by the Affect Infusion Model
Apart from providing normative values of valence, arousal, moral (AIM) by Forgas (1995). This model posits that (i) arousal prop-
judgment and RT for 46 moral dilemmas5 , the results of this erties of the situation, (i) the motivational features of the emo-
dilemma validation study challenge the DPHMJ proposed by tions triggered by it, and (iii) the associated cognitive appraisal
Greene et al. (2001, 2004). According to this hypothesis, deon- mechanisms, all play a crucial role in every judgment. This model
tological moral judgments (refraining from harm) are fast and also posits that affect infusion is a matter of degree: any judg-
emotion-based, while utilitarian moral judgments (deciding to ment is also dependent on previous knowledge of the individual
commit the harm) are slow as a result of deliberate reason- about the event or situation he or she is about to judge; this
ing processes. The assumptions of the DPHMJ were based on implies that it is dependent on deliberate reasoning as well as on
a reaction time finding where an interaction between the Type the magnitude of the emotional arousal triggered by the event or
of Response given (deontological vs. utilitarian) and the Personal situation.
Force (Personal vs. Impersonal) showed that when harm was See the Supplementary Material for a summary of limitations
consented in a Personal Moral Dilemma (utilitarian response), of the method.
RT was significantly longer than when harm was not consented
(deontological response). No such difference in the response time CONCLUSION
was found for Impersonal Moral Dilemmas. However, in our In this work, we have followed Hauser et al. view of moral dilem-
study, while we also found that higher arousal correlates with mas: “. . . the use of artificial moral dilemmas to explore our moral
deontological judgment (in line with Moretto et al., 2010), we psychology is like the use of theoretical or statistical models with
failed to find the relationship with RT: both deontological and different parameters; parameters can be added or subtracted in
utilitarian decisions can be made equally fast, and both to per- order to determine which parameters contribute most signifi-
sonal and impersonal dilemmas, depending on the other factors cantly to the output” (Hauser et al., 2007). We have tried to
involved. To put it another way, a fast judgment takes place when, control for the variables known to influence moral judgment, in
either a deontological reason guides the judgment, or when utili- order to find out which ones matter most, and how they interact.
tarian considerations clearly dominate. Therefore, while we agree One main result of this work is that, when dilemmas are
that the dilemmas that take longer are those where the experi- validated, Greene’s main effect of personal dilemmas partly dis-
enced conflict is greater, conflict, however, has a more complex appears, for a more complex pattern, which casts doubt on the
etiology. In particular, judgment takes longer when people are view that some moral judgments are the result of a deliberation,
torn between utilitarian considerations of the greater good (sav- while others, the deontological ones, are reached emotionally.
ing many), and the suffering produced in others as an accidental While higher arousal is related to deontological judgments, it is
side-effect. An increased RT is likely to have been caused by rea- not true that deontological judgments are faster than utilitarian
soning processes in order to explore a way to avoid the conflict, in ones. Deontological judgments may take longer than utilitarian
either case. ones if, after taking time to weight the options, and to look for
As a matter of fact, the DPHMJ’s central result concerning a way to minimize the transgression, one cannot find a way to
personal vs. impersonal dilemmas has already been challenged. choose not to violate one’s principles.
McGuire et al. (2009) reanalyzed the data sets from Greene and Research with moral dilemmas holds fascinating possibilities
colleagues and removed what they called “poorly endorsed items” to study the grounding psychological principles of human moral
(those dilemmas not designed carefully enough). After this pro- cognition. Contrary to the criticisms brought up against this
cedure by McGuire et al., the key effect disappeared from the data methodology, and in line with an increasing number of other
(McGuire et al., 2009). Similarly, Ugazio et al. (2012), on their researchers, we believe that it is specifically the artificial nature
part, showed that both deontological and utilitarian responding of moral dilemmas that make this methodology so valuable. In
could actually be triggered by different emotions with different any case, the scenarios described to us in moral dilemmas are
motivational tendencies. In their study, disgust induction (an not more artificial than the stories narrated in novels and movies
emotion that triggers withdrawal tendencies) resulted in more where life-and death-decisions change the course of supposedly
inevitable events. Besides, other abundant channels of informa-
5 Supplementary Material accompanies this manuscript with all data points tion of that kind are the news on TV, radio, in the papers, and on
presented in this work. the internet. They inform us of atrocities that happened around

Frontiers in Psychology | Emotion Science July 2014 | Volume 5 | Article 607 | 16

Christensen et al. Moral dilemma validation study

the corner of our house while we were sleeping, or of heartbreak- Davis, M. H. (1983). Measuring individual-differences in empathy - evidence
ing life-threatening situations that some individual in a war swept for a multidimensional approach. J. Pers. Soc. Psychol. 44, 113–126. doi:
10.1037/0022-3514.44.1.113
country has had to go through. . . Are moral dilemmas really all
Feldman Hall, O., Mobbs, D., Evans, D., Hiscox, L., Navrady, L., and
that unreal and artificial to us? Dalgleish, T. (2012). What we say and what we do: the relationship
between real and hypothetical moral choices. Cognition 123, 434–441. doi:
AUTHOR NOTE 10.1016/j.cognition.2012.02.001
All authors: Human Evolution and Cognition (IFISC-CSIC) and Foot, P. (1967). The Problem of Abortion and the Doctrine of the Double Effect.
Department of Psychology, University of the Balearic Islands, Reprinted in Virtues and Vices and Other Essays in Moral Philosophy (1978).
Oxford: Blackwell.
Carretera de Valldemossa, km. 7.5, Building: Guillem Cifre de Forgas, J. P. (1995). Mood and judgment: the affect infusion model (AIM). Psychol.
Colonya, 07122 Palma, Spain. Nadine K. Gut current affili- Bull. 117, 39–66.
ation: School of Psychology and Neuroscience, University of Fumagalli, M., Ferrucci, R., Mameli, F., Marceglia, S., Mrakic-Sposta, S., Zago, S.,
St Andrews, St Mary‘s Quad, South Street, St Andrews, KY16 et al. (2009). Gender-related differences in moral judgments. Cogn. Process. 11,
9JP, UK; Strathclyde Institute of Pharmacy and Biomedical 219–226. doi: 10.1007/s10339-009-0335-2
Fumagalli, M., Vergari, M., Pasqualetti, P., Marceglia, S., Mameli, F., Ferrucci,
Sciences, University of Strathclyde, 161 Cathedral Street, Glasgow, R., et al. (2010). Brain switches utilitarian behavior: does gender make the
G4 0RE, UK. difference? PLoS ONE 5:e8865. doi: 10.1371/journal.pone.0008865
Greene, J. (2008). “The secret Joke of Kant’s Soul,” in Moral Psychology, Vol. 3, ed
ACKNOWLEDGMENTS W. Sinnott-Armstrong. (Cambridge, MA; London: MIT Press), 35–80.
The study was funded by the research project FFI2010- Greene, J. D., Cushman, F. A., Stewart, L. E., Lowenberg, K., Nystrom, L. E.,
and Cohen, J. D. (2009). Pushing moral buttons: the interaction between per-
20759 (Spanish Government: Ministry of Economy and sonal force and intention in moral judgment. Cognition 111, 364–371. doi:
Competitiveness), and by the Chair of the Three Religions 10.1016/j.cognition.2009.02.001
(Government of the Balearic Islands) of the University of the Greene, J. D., Nystrom, L. E., Engell, A. D., Darley, J. M., and Cohen, J. D. (2004).
Balearic Islands, Spain. Julia F. Christensen and Albert Flexas The neural bases of cognitive conflict and control in moral judgment. Neuron
were supported by FPU PHD scholarships from the Spanish 44, 389–400. doi: 10.1016/j.neuron.2004.09.027
Greene, J. D., Sommerville, R. B., Nystrom, L. E., Darley, J. M., and Cohen, J. D.
Ministry of Education, Culture and Sports (AP2009-2889 and (2001). An fMRI investigation of emotional engagement in moral judgment.
AP2008-02284). Nadine K. Gut was supported by a scholarship Science 293, 2105–2108. doi: 10.1126/science.1062872
of the School of Psychology and Neuroscience, University of St Hauser, M., (ed.). (2006). Moral Minds: How Nature Designed our Universal Sense
Andrews, UK. We want to thank Dr. Camilo José Cela-Conde of Right and Wrong. New York, NY: Ecco/Harper Collins.
Hauser, M., Cushman, F., Young, L., Jin, R. K. X., and Mikhail, J. (2007). A dis-
for help and advice at different stages of this work; and a special
sociation between moral judgments and justications. Mind Lang. 22, 1–21. doi:
thank you goes to Lasse Busck-Nielsen, Françoise Guéry and 10.1111/j.1468-0017.2006.00297.x
Trevor Roberts for help in the language editing process. Huebner, B., Hauser, M. D., and Pettit, P. (2011). How the source, inevitability and
means of bringing about harm interact in folk moral judgments. Mind Lang. 26,
SUPPLEMENTARY MATERIAL 210–233. doi: 10.1111/j.1468-0017.2011.01416.x
The Supplementary Material for this article can be found Koenigs, M., Young, L., Adolphs, R., Tranel, D., Cushman, F., Hauser, M., et al.
(2007). Damage to the prefrontal cortex increases utilitarian moral judgements.
online at: http://www.frontiersin.org/journal/10.3389/fpsyg. Nature 446, 908–911. doi: 10.1038/nature05631
2014.00607/abstract McCrae, R. R., and Costa, P. T. Jr. (1999). “A five-factor theory of personality,” in
Handbook of Personality: Theory and Research, 2nd Edn.,ed L. A. Pervin (New
REFERENCES York, NY: Guilford Press), 139–153.
Abarbanell, L., and Hauser, M. D. (2010). Mayan morality: an exploration of per- McGuire, J., Langdon, R., Coltheart, M., and Mackenzie, C. (2009). A reanalysis of
missible harms. Cognition 115, 207–224. doi: 10.1016/j.cognition.2009.12.007 the personal/impersonal distinction in moral psychology research. J. Exp. Soc.
Anders, G. (1962). Burning Conscience: The Case of the Hiroshima Pilot. New York, Psychol. 45, 577–580. doi: 10.1016/j.jesp.2009.01.002
NY: Monthly Review Press. Mehrabian, A., and Epstein, N. (1972). A measure of emotional empathy. J. Pers.
Bachorowski, J. A., and Braaten, E. B. (1994). Emotional intensity - measurement 40, 525–543.
and theoretical implications. Pers. Individ. Dif. 17, 191–199. doi: 10.1016/0191- Mikhail, J. (2007). Universal moral grammar: theory, evidence and the future.
8869(94)90025-6 Trends Cogn. Sci. 11, 143–152. doi: 10.1016/j.tics.2006.12.007
Bloomfield, P. (2007). Morality and Self-Interest. Oxford: Oxford University Press. Moore, A. B., Clark, B. A., and Kane, M. J. (2008). Who shalt not kill? Individual dif-
doi: 10.1093/acprof:oso/9780195305845.001.0001 ferences in working memory capacity, executive control, and moral judgment.
Borg, J. S., Hynes, C., Van Horn, J., Grafton, S., and Sinnott-Armstrong, W. (2006). Psychol. Sci. 19, 549–557. doi: 10.1111/j.1467-9280.2008.02122.x
Consequences, action, and intention as factors in moral judgments: an fMRI Moore, A. B., Lee, N. Y. L., Clark, B. A. M., and Conway, A. R. A. (2011a). In
investigation. J. Cogn. Neurosci. 18, 803–817. doi: 10.1162/jocn.2006.18.5.803 defense of the personal/impersonal distinction in moral psychology research:
Cacioppo, J. T., Petty, R. E., and Kao, C. F. (1984). The efficient assessment of need cross-cultural validation of the dual process model of moral judgment. [Article].
for cognition. J. Pers. Assess. 48, 306–307. doi: 10.1207/s15327752jpa4803_13 Judgm. Decis. Mak. 6, 186–195.
Christensen, J. F., Flexas, A., de Miguel, P., Cela-Conde, C. J., and Munar, E. Moore, A. B., Stevens, J., and Conway, A. R. A. (2011b). Individual differences in
(2012). Roman Catholic beliefs produce characteristic neural responses to sensitivity to reward and punishment predict moral judgment. [Article]. Pers.
moral dilemmas. Soc. Cogn. Affect. Neurosci. 9, 1–10. doi: 10.1093/scan/nss121 Individ. Dif. 50, 621–625. doi: 10.1016/j.paid.2010.12.006
Christensen, J. F., and Gomila, A. (2012). Moral dilemmas in cognitive neuro- Moretto, G., Làdavas, E., Mattioli, F., and di Pellegrino, G. (2010). A psychophysi-
science of moral decision-making: a principled review. Neurosci. Biobehav. Rev. ological investigation of moral judgment after ventromedial prefrontal damage.
36, 1249–1264. doi: 10.1016/j.neubiorev.2012.02.008 J. Cogn. Neurosci. 22, 1888–1899. doi: 10.1162/jocn.2009.21367
Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences, 2nd Edn. Navarrete, C. D., McDonald, M. M., Mott, M. L., and Asher, B. (2012). Morality:
Hillsdale, NJ: Lawrence Erlbaum Associates Inc. emotion and action in a simulated three-dimensional “trolley problem”.
Cushman, F., Young, L., and Hauser, M. (2006). The role of conscious reasoning Emotion 12, 364–370. doi: 10.1037/a0025561
and intuition in moral judgment: testing three principles of harm. Psychol. Sci. O’Hara, R. E., Sinnott-Armstrong, W., and Sinnott-Armstrong, N. A. (2010).
17, 1082–1089. doi: 10.1111/j.1467-9280.2006.01834.x Wording effects in moral judgments. Judgm. Decis. Mak. 5, 547–554.

www.frontiersin.org July 2014 | Volume 5 | Article 607 | 17

Christensen et al. Moral dilemma validation study

Petrinovich, L., and O’Neill, P. (1996). Influence of wording and framing Waldmann, M. R., and Dieterich, J. H. (2007). Throwing a bomb on a person ver-
effects on moral intuitions. Ethol. Sociobiol. 17, 145–171. doi: 10.1016/0162- sus throwing a person on a bomb - Intervention myopia in moral intuitions.
3095(96)00041-6 Psychol. Sci. 18, 247–253. doi: 10.1111/j.1467-9280.2007.01884.x
Petrinovich, L., O’Neill, P., and Jorgensen, M. (1993). An empirical-study of moral Zimbardo, P. (2007). The Lucifer Effect. New York, NY: Random House Trade
intuitions - toward an evolutionary ethics. J. Pers. Soc. Psychol. 64, 467–478. doi: Paperbacks.
10.1037/0022-3514.64.3.467
Royzman, E., and Baron, J. (2002). The preference of indirect harm. Soc. Justice Res. Conflict of Interest Statement: The authors declare that the research was con-
15, 165–184. doi: 10.1023/A:1019923923537 ducted in the absence of any commercial or financial relationships that could be
Tassy, S., Oullier, O., Mancini, J., and Wicker, B. (2013). Discrepancies between construed as a potential conflict of interest.
judgment and choice of action in moral dilemmas. Front. Psychol. 4:250. doi:
10.3389/fpsyg.2013.00250 Received: 17 April 2014; accepted: 29 May 2014; published online: 01 July 2014.
Taylor, G. J., Ryan, D., and Bagby, R. M. (1985). Toward the development a Citation: Christensen JF, Flexas A, Calabrese M, Gut NK and Gomila A (2014) Moral
new self-report alexithimia scale. Psychother. Psychosom. 44, 191–199. doi: judgment reloaded: a moral dilemma validation study. Front. Psychol. 5:607. doi:
10.1159/000287912 10.3389/fpsyg.2014.00607
Thomson, J. J. (1976). Killing, letting die, and the trolley problem. Monist 59, This article was submitted to Emotion Science, a section of the journal Frontiers in
204–217. doi: 10.5840/monist197659224 Psychology.
Tversky, A., and Kahneman, D. (1981). The framing of decisions and Copyright © 2014 Christensen, Flexas, Calabrese, Gut and Gomila. This is an open-
the psychology of choice. Science 211, 453–458. doi: 10.1126/science. access article distributed under the terms of the Creative Commons Attribution License
7455683 (CC BY). The use, distribution or reproduction in other forums is permitted, provided
Ugazio, G., Lamm, C., and Singer, T. (2012). The role of emotions for moral judg- the original author(s) or licensor are credited and that the original publication in this
ments depends of the type of emotion and moral scenario. Emotion 12, 579–590. journal is cited, in accordance with accepted academic practice. No use, distribution or
doi: 10.1037/a0024611 reproduction is permitted which does not comply with these terms.

Frontiers in Psychology | Emotion Science July 2014 | Volume 5 | Article 607 | 18