13 Apr 2002 14:59 AR AR158-02.tex AR158-02.

SGM LaTeX2e(2001/05/10) P1: GJC

10.1146/annurev.polisci.5.091001.170657

Annu. Rev. Polit. Sci. 2002. 5:31–61

DOI: 10.1146/annurev.polisci.5.091001.170657
Copyright °
c 2002 by Annual Reviews. All rights reserved

EXPERIMENTAL METHODS IN POLITICAL SCIENCE

Rose McDermott
Department of Government, McGraw Hall, Cornell University, Ithaca, New York 14850;
e-mail: rmm21@cornell.edu

Key Words experiments, behavioral economics, political economy, game theory,

Annu. Rev. Polit. Sci. 2002.5:31-61. Downloaded from www.annualreviews.org

individual choice, methodology

by University of Illinois - Chicago on 05/17/12. For personal use only.

■ Abstract This article reviews the use of experiments in political science. The
beginning section offers an overview of experimental design and measures, as well as
threats to internal and external validity, and discusses advantages and disadvantages
to the use of experimentation. The number and placements of experiments in political
science are reviewed. The bulk of the essay is devoted to an examination of what we
have learned from experiments in the behavioral economics, political economy, and
individual choice literatures.

INTRODUCTION

Anyone who takes an antibiotic, confident that illness will remit, is implicitly
trusting in the power and validity of experiments as applied to real-world contexts.
Indeed, the hard sciences, including biology, chemistry, physics, and medicine,
all rely primarily on experimentation to examine and illuminate basic processes.
Psychology embodies a long and distinguished history of experimentation, and
behavioral economics, which involves a great deal of experimentation, has re-
cently gained increasing prominence within the larger field of economics. But
the methodology of experimentation has been slow to garner a following in po-
litical science. Experimentation might easily dovetail with methods more estab-
lished in political science, such as formal modeling, to produce and cumulate
useful knowledge; however, political scientists typically prefer archival work,
case studies, field work, surveys, quantitative analysis, and formal modeling in-
stead. Yet these other methods need not compete with experimentation. Indeed,
the most exciting opportunity for methodological advancement using experimen-
tation lies at the intersection of formal modeling and experimental testing: For-
mal models present hypotheses that are tested, refined, and explored through
experimentation in a reciprocal manner. This process is widely and success-
fully employed within behavioral economics. As yet, however, political science
remains slow to embrace the added value offered by the methodology of
experimentation.

1094-2939/02/0615-0031$14.00 31
 13 Apr 2002 14:59 AR AR158-02.tex AR158-02.SGM LaTeX2e(2001/05/10) P1: GJC

32 MCDERMOTT

This essay addresses the use of experiments and the experimental method in
political science. Following a brief background discussion of the experimental
method, including threats to internal and external validity, relative advantages
and disadvantages, and ethics, the essay concentrates on what we have learned
of substance from experiments in behavioral economics and political science that
should be of interest to mainstream political scientists.
My overall goal is to advocate for the utility of experiments for political science.
I do not argue that experiments are the only, nor the best, form of methodological
inquiry. Rather, I argue that experimentation can be particularly useful under certain
circumstances: when existing methods of inquiry have produced inconsistent or
contradictory results; when empirical validation of formal models is required;
Annu. Rev. Polit. Sci. 2002.5:31-61. Downloaded from www.annualreviews.org

when investigators want to triangulate in on specific processes that have already

been examined in a more general way using other methodologies; and when evi-
by University of Illinois - Chicago on 05/17/12. For personal use only.

dence is needed to support strong causal claims. Experiments can combine with
other methods to provide what Campbell described as a “fish scale model of
omniscience,” whereby each methodological layer serves to illuminate and support
other component parts.

EXPERIMENTAL METHODS

What do we mean by experiments? I take the term to refer primarily to laboratory

studies in which investigators retain control over the recruitment, assignment to
random conditions, treatment, and measurement of subjects. This definition as-
sumes that experimenters take pains to assure that the experimental situation does
not vary in any way other than the intended independent variables in order to assure
the internal validity that allows causal claims.
To be sure, other forms of experimentation exist, including field studies, field
experiments, and even simulation studies. Field studies typically look retrospec-
tively at the effect of naturally occurring events. Although such studies seek to
achieve maximum realism and representative subject samples, the experimenter
has no control whatsoever over the variables of interest. The ideal subject of a field
study is a naturally occurring situation with both “before” and “after” data, such
as students’ test results before and after implementation of an educational reform
program (Walker 1976).
Field experiments take place outside the laboratory but allow the experimenter
to retain some limited control over the central variables. At their best, field ex-
periments can offer a reasonable trade-off between internal and external validity
through increased realism without too much loss of control. Simulations offer a
similar compromise between the concerns of internal and external validity. Some
experiments use people as subjects; other rely solely on computerized models.
But for purposes of the discussion below, laboratory experiments are the gold
standard.
 13 Apr 2002 14:59 AR AR158-02.tex AR158-02.SGM LaTeX2e(2001/05/10) P1: GJC

EXPERIMENTS 33

Experimental Design
Why do we need experiments? We need experiments because they help to reduce
the bias that can exist in less rigorous forms of observation. Experiments reduce the
impact of bias by introducing standardized procedures, measures, and analyses.
Important aspects of experimental design include standardization, randomiza-
tion, between-subjects versus within-subject design, and experimental bias.
1. Standardization remains crucial in experimentation because it ensures that
the same stimuli, procedures, responses, and variables are coded and ana-
lyzed. This reduces the likelihood that extraneous factors, of which the ex-
perimenter might not even be aware, could influence the results in decisive
Annu. Rev. Polit. Sci. 2002.5:31-61. Downloaded from www.annualreviews.org

ways. Standardization requires that the same set of experimental procedures,

by University of Illinois - Chicago on 05/17/12. For personal use only.

or experimental protocol, is administered in the same way to subjects across

conditions; only the independent variable (or variables) of interest is manipu-
lated. This process ensures that the data elicited from subjects are comparable
and are not the result of some extraneous feature in the environment.
2. Randomization refers to the assignment of subjects to experimental condi-
tions. Experimenters assign subjects randomly to ensure that no unrelated
or spurious factors vary consistently within a given population and therefore
bias the results. The idea is that background differences cancel each other
out in the course of random assignment, since each individual is as likely
to be placed in one condition as in another. Thus, no systematic differences
in subjects can bias the results of the study. Many people unfamiliar with
probability might think that random assignment is little more than alternately
assigning Condition A or B as subjects walk in the door. However, the most
elegant and pure way to ensure random assignment of subjects to conditions
is to use a random number table to assign manipulations.
Experimental design can be between-subjects or within-subject. Typi-
cally, one experimental condition is compared to another experimental
condition and then to a control condition. The control condition creates
a baseline against which investigators compare the results of their manipula-
tions. In between-subjects designs, different groups of subjects are randomly
assigned to various experimental or control conditions. In within-subject de-
signs, otherwise known as the A-B-A experimental design strategy, each
person serves as his or her own control by participating across time in both
control and treatment conditions. Subjects begin with a baseline (A), are then
administered a treatment or manipulation condition (B), and are measured
again at baseline (A) once the treatment ends (Zimbardo & Gerrig 1996). The
comparison of before and after measures on the variable of concern inform
the investigator as to the impact, if any, of the treatment on the subject.
3. Placebo effects, in medicine, account for patients whose condition improves
as a result of a fake treatment, such as a sugar pill. These effects can cause
quite powerful changes in outcome based on an individual’s belief that the
 13 Apr 2002 14:59 AR AR158-02.tex AR158-02.SGM LaTeX2e(2001/05/10) P1: GJC

34 MCDERMOTT

treatment will work. Control conditions are important in experimental veri-

fication precisely because they help determine the extent of placebo effects
in an experimental manipulation.
4. Experimental bias. Although experiments seek to maximize experimenter
control over the independent variables in a study, the experimental pro-
cess itself can introduce potential sources of bias. Three important forms
of experimental bias are expectancy effects, experimenter bias, and demand
characteristics.

EXPECTANCY EFFECTS Expectancy effects occur when an experimenter commu-

nicates, usually in a subtle, unconscious way, how he or she wants the subject to
Annu. Rev. Polit. Sci. 2002.5:31-61. Downloaded from www.annualreviews.org

behave or respond (Rosenthal 1966). Results then take the form of a self-fulfilling
by University of Illinois - Chicago on 05/17/12. For personal use only.

prophecy as experimenters create the reactions they hope to elicit with their subtle
signals and not with their controlled manipulation. Ways to overcome this bias
include having various experimenters run some subjects in all conditions, making
the experimenter blind to the subjects’ conditions, designing the experiment to
avoid experimenter involvement (as can be done with computer-generated experi-
ments), or treating the experimenter as a factor or variable in the statistical analysis
at the end of the experiment to determine if any particular experimenter elicited
distinctive responses from the subjects.

EXPERIMENTER BIAS Experimenter bias can overlap with expectancy effects but
is in fact distinct from them theoretically. Many experimental choices originate
from the experimenter’s beliefs and attitudes, and these choices can influence the
design of an experiment in a nonrandom way. Sometimes this may be acceptable,
but concerns arise especially when an investigator remains unaware that his beliefs
have unduly affected the design of the study (Roth 1988).

DEMAND CHARACTERISTICS Demand characteristics are similar to expectancy

biases as well, although the cues emerge from the subject’s interpretation of the
experiment rather than from anything the experimenter says or does directly
(McConahay 1973). A systematic bias can be introduced if the purpose of the
experiment is too obvious. This bias can be exacerbated if subjects, experiencing
evaluation apprehension (Rosenberg 1965), try to make the experimenter like them
by doing what they believe the experimenter wants. Ways to reduce the impact
of this problem on experimental outcomes include using deception to ensure that
subjects cannot determine the relevant hypotheses or tests, evaluating the demand
characteristics of the experiment in the analysis at the end of the study, and using
computer technology to complicate and/or depersonalize the experiment to reduce
the likelihood that subjects can discern its true purpose.

Experimental Measures
Experimental measures strive for reliability as well as internal and external validity.
Reliability and validity are central concepts in all experimental measurement.
 13 Apr 2002 14:59 AR AR158-02.tex AR158-02.SGM LaTeX2e(2001/05/10) P1: GJC

EXPERIMENTS 35

Reliability refers to the extent to which an experimenter tests the same thing
time and time again. A reliable result is one that is easily replicable. Reliability
improves when measures are standardized, when a larger number of measures have
been taken, and when factors that might bias the data are controlled in advance
(Zimbardo & Gerrig 1996).
Experimental measures can take several forms: self-reports, behavioral mea-
sures, physiological measures, and incentives.

SELF-REPORTS, BEHAVIORAL MEASURES, AND PHYSIOLOGICAL MEASURES Self-

reports are usually verbal or written reports of a subject’s responses to a particular
set of stimuli. They can take the form of paper-and-pencil questionnaires, surveys,
Annu. Rev. Polit. Sci. 2002.5:31-61. Downloaded from www.annualreviews.org

or interviews. Some of these data, which might be originally obtained in qualitative

by University of Illinois - Chicago on 05/17/12. For personal use only.

form, can be coded into quantitative categories for later analysis. Behavioral mea-
sures require experimenters to observe the behavior of subjects by, for example,
videotaping them and later examining the tapes for characteristics such as facial
expressions or tendency to dominate in a group. Physiological measures include
such data as heart rate, galvanic skin response, blood pressure, or more extensive
tests such as magnetic resonance imaging (MRI) or positron-emission tomogra-
phy (PET) tests. More intrusive tests, such as those that analyze saliva, urine, or
blood, might also be conducted to determine hormone levels or other variables of
interest.

INCENTIVES Most experiments in psychology do not offer incentives other than

extra credit in a course for participants. Indeed, many introductory psychology
classes require students to participate in experiments. However, many behavioral
economics experiments do offer material incentives, typically money or a lottery
that offers the chance of a cash reward to subjects. There are two kinds of incentives:
those offered to subjects merely for showing up to participate in a study, and those
intrinsic to the experiment itself.

Threats to Internal and External Validity

Campbell & Stanley (1963) describe the classic distinction between internal and
external validity as follows: Internal validity asks, “Did in fact the experimen-
tal treatments make a difference in this specific experimental instance?” External
validity investigates the generalizability of the results by inquiring, “To what pop-
ulations, settings, treatment variables, and measurement variables can this effect
be generalized?” As McConahay (1973) noted, “Generally, with a large number of
specific exceptions, psychologists are more concerned about the internal validity
of a research design while political scientists are more concerned with external
validity.” Walker (1976), bemoaning the fact, went so far as to argue that herein lay
one of the main reasons for the lack of productive communication between the two
disciplines: “The social psychologist, with his reliance on controlled laboratory
experiments, has different interests and concerns than the political scientist, who
is more at home in the field.”
 13 Apr 2002 14:59 AR AR158-02.tex AR158-02.SGM LaTeX2e(2001/05/10) P1: GJC

36 MCDERMOTT

Once an investigator has determined what to investigate and how to assess

and measure the relevant variables, he or she must ensure that the design of the
experimental protocol does not fall prey to one of the flaws that can confound the
results (see Campbell & Stanley 1963 and Campbell & Ross 1968 for the classic
discussion of these confounds).

THREATS TO INTERNAL VALIDITY There are nine potential threats to internal va-
lidity in experimentation:
1. History refers to any event that occurs outside the experimenter’s control in
the time between the measures on the dependent variable. This phenomenon
Annu. Rev. Polit. Sci. 2002.5:31-61. Downloaded from www.annualreviews.org

becomes a concern when there is a lot of time between the measurements on

the dependent variable.
by University of Illinois - Chicago on 05/17/12. For personal use only.

2. Intersession history refers to events that occur inside the study itself, which
are beyond the control of the investigator and may affect the outcome of the
study. Extreme temperature fluctuation, unexpected fire drills, or unknown
preexisting relationships between some subjects might affect one session
of an experiment but not another. These confounds all threaten the internal
validity of the experiment.
3. Maturation refers to the natural needs, growth, and development of indi-
viduals over time. For example, an experiment that relies on deprivation
for motivation depends on maturation effects for thirst to occur. Maturation
processes that work independently of the investigator over the course of an
experiment can bias results as well.
4. Performance effects. Performance can change as a result of experience. Test
performance can be affected by the very act of having taken the test be-
fore. Therefore, pre- and post-tests with exactly the same questions do not
constitute identical assessments because, independent of the intervening ma-
nipulation, taking the first test may influence the answers to the second test
through the natural process of learning and experience.
5. Regression toward the mean. Since all scores represent some combination of
the real score and some random error, subjects who manifest an extreme score
are likely to move closer to the mean on the next measurement, as the random
error fluctuates. Experimenters who specifically pick subjects because they
manifest an extreme score on some dimension, like authoritarianism, are
likely to confound their results through their failure to incorporate regression
effects into their subject selection procedures.
6. Subject self-selection. Subjects who self-select into particular experiments
or conditions are likely to differ in some systematic way from those who are
randomly assigned to a condition.
7. Mortality occurs not only when subjects die but also when they are lost to
follow-up by the investigator. In political science, mortality occurs most often
in experiments that require the same subject to show up more than once, and
 13 Apr 2002 14:59 AR AR158-02.tex AR158-02.SGM LaTeX2e(2001/05/10) P1: GJC

EXPERIMENTS 37

the subject fails to show up for the second part of the experiment. This can also
happen in longitudinal and field studies that examine dynamics over time; as
people move and change their living situations, they can become hard to trace.
Sometimes financial and other incentives can alleviate this problem. From an
ethical perspective, the most important cases of experimental mortality occur
when subjects leave in the middle of an experiment because some aspect of
the experiment has made them uncomfortable. If a study has a high degree
of interexperimental drop-out, the investigator should take pains to ascertain
that the experiment is being conducted in an appropriate and ethical manner.
8. Selection-maturation interaction occurs when subjects are placed into an
experimental condition in a nonrandom manner and some aspect of the group
Annu. Rev. Polit. Sci. 2002.5:31-61. Downloaded from www.annualreviews.org

differs in maturation from the other groups in an important or systematic way.

by University of Illinois - Chicago on 05/17/12. For personal use only.

9. Unreliable measures. If measures are unreliable, if the subject population

shifts, or if some other aspect of the experimental condition is affected in a
nonrandom manner, results can become biased.

THREATS TO EXTERNAL VALIDITY Campbell (1968) outlines the six major threats
to external validity in experimentation:
1. Testing interaction effects. Testing can increase subjects’ sensitivity to the
variables under investigation, which makes it difficult to generalize the results
to a population that has not been pretested.
2. Unrepresentative subject population. What can college sophomores tell us
about real-world decision makers? Sears (1986) argues that college sopho-
mores differ in systematic and marked ways from other people: They are more
self-absorbed; they have less crystallized attitudes, a less clear sense of self,
higher rates of compliance, less stable peer relationships, and stronger cog-
nitive skills. Remarkably, many experimental findings using college sopho-
mores have proved remarkably robust (Roth 1988). However, many people
remain concerned about subject pools. Obviously, the best way to deal with
this problem of external validity would be to sample directly from the pop-
ulations of interest. Etheredge (1978) did this in his extensive study of 126
career foreign service officers at the State Department to examine “how emo-
tional predispositions might shape elite foreign policy thinking.” But often
this is not possible because such people are either too busy or not interested
in participating in experiments. Another strategy against this limitation in-
volves simulations with real or former decision makers using a hypothetical
or past crisis as a stimulus. Such simulations have produced very accurate
results. In one of the most powerful examples of a simulation’s prescient
prediction of a real-world outcome, a Joint War Games Agency of the Joint
Chiefs of Staff conducted a major war game simulation of the conflict in
Vietnam. The goal was to start with current resources as of July of 1965 and
simulate the likely outcome through September of 1966. The results indi-
cated that the United States would not be able to win the conflict in the long
 13 Apr 2002 14:59 AR AR158-02.tex AR158-02.SGM LaTeX2e(2001/05/10) P1: GJC

38 MCDERMOTT

run and was unlikely to do better than stalemate in the short run (as reported
in Burke & Greenstein 1989). Simulations that accurately mimic real-life
problems and resources can engage the same psychological processes that
operate in the real world.
3. Hawthorne effect. The third limitation on external validity involves the so-
called Hawthorne effect (Roethlisberger & Dickson 1939), whereby people
change their behavior merely because they are aware of being observed.
People who know they are in an experiment may behave differently than they
would if they were not in an experiment or were unaware of the experiment.
4. Professional subjects. On large and relatively anonymous college campuses,
a student eager to earn money can participate in many experiments across
Annu. Rev. Polit. Sci. 2002.5:31-61. Downloaded from www.annualreviews.org

departments without any one experimenter realizing how experienced this

by University of Illinois - Chicago on 05/17/12. For personal use only.

subject has become. Overly experienced or jaded subjects may be more likely
to guess the underlying hypothesis or manipulation in an experiment if they
have participated in similar ones in the past.
5. Spurious measures. Some unexpected aspect of the experiment may induce
subjects to give systematically irrelevant responses to particular measures,
which are then understood to be experimental effects.
6. Irrelevant measures. Irrelevant aspects of the experimental condition might
produce results that appear to be experimental effects.

ADVANTAGES AND DISADVANTAGES OF EXPERIMENTS

Since most of the threats to internal and external validity can be anticipated, what
are the benefits that a skilled experimenter might accrue from using experimen-
tation as opposed to other research strategies, such as case study work or formal
models? What problems would still plague the interpretation and utility of his or
her results?

Advantages
The comparative advantages of experiments lie in their high degree of internal va-
lidity. No other methodology can offer the strong support for the causal inferences
that experiments allow. Correlational studies, for example, do not show causa-
tion. Since a laboratory setting allows investigators to control all aspects of the
environment so that only the independent variables differ, any differences on the
dependent variable can be attributed to the manipulation, and thus offer support
for causal inferences. Experiments offer at least five such advantages:
1. Ability to derive causal inferences. “The major advantage of laboratory
experiments is in its [sic] ability to provide us with unambiguous evidence
about causation” (Aronson & Carlsmith 1968). Because of the random-
ization of subjects and the control of the environment, experiments allow
 13 Apr 2002 14:59 AR AR158-02.tex AR158-02.SGM LaTeX2e(2001/05/10) P1: GJC

EXPERIMENTS 39

confidence regarding causal inferences about relationships among the vari-

ables of interest.
2. Experimental control. The experimenter has control over the recruitment,
treatment, and measurement of subjects and variables.
3. Precise measurement. Experimenters design and implement the desired
measures and ensure that they are administered consistently.
4. Ability to explore the details of process. Experiments offer the opportunity
to explore phenomena of interest in great detail. Complex relationships can
be broken down and investigated in smaller units in order to see which part
of the process results in the differences of interest. In addition, experiments
Annu. Rev. Polit. Sci. 2002.5:31-61. Downloaded from www.annualreviews.org

allow particular relationships to be explored in the presence or absence of

other variables, so that the conditions under which certain relationships hold
by University of Illinois - Chicago on 05/17/12. For personal use only.

can be examined as well.

5. Relative economy. Although experiments may be more costly and time-
consuming than some research methodologies, such as formal modeling,
they are certainly more economical than conducting large surveys or field
experiments. Students are a relatively inexpensive and reliable subject pool,
and a large number of them can be run in a semester. Experiments embedded
in larger surveys (Sniderman et al. 1991, Kuklinski et al. 1997) may provide
a more representative sample but would also require a great deal more time
and money to administer.

Disadvantages
Experiments are not always the ideal methodology. Most concerns about their
disadvantages within political science revolve around questions of external validity
and how widely the findings of the laboratory apply to real-world actors and
phenomena. There are four main disadvantages to the use of experiments:
1. Artificial environment. Many experimental settings are artificially sterile and
unrepresentative of the environments in which subjects might normally per-
form the behavior under study. There are at least two important aspects of
this limitation. First, it might be impossible or unethical to create the desired
situation within a laboratory. An experimenter could not study the effects of
a life-threatening illness by causing such disease in a subject. Second, it may
be very hard to simulate many phenomena of interest—an election, a war,
an economic recession, and so on.
2. Unrepresentative subject pools. As noted above, subject pools may be un-
representative of the populations of interest.
3. External validity. For political scientists, questions surrounding external va-
lidity pose the greatest concern with experimentation. What can experiments
tell us about real-world political phenomena? Beyond the nature of the sub-
ject pool, this concern is at least twofold. First, in the laboratory it is difficult
to replicate key conditions that operate on political actors in the real world.
 13 Apr 2002 14:59 AR AR158-02.tex AR158-02.SGM LaTeX2e(2001/05/10) P1: GJC

40 MCDERMOTT

Subjects typically meet only for a short period and focus on a limited task.
Even when money serves as a material incentive, subject engagement may
be low. In the real world, actors have histories and shadows of the future with
each other, they interact around many complex issues over long periods, and
they have genuine strategic and material interests, goals, and incentives at
stake. Can the results of a single-session experiment tell us anything about
such a complicated world?
Second, and related, many aspects of real-world complexity are diffi-
cult to simulate in the laboratory. Cultural norms, relationships of authority,
and the multitask nature of the work itself might invalidate any results that
emerge from an experiment that does not, or cannot, fully incorporate these
Annu. Rev. Polit. Sci. 2002.5:31-61. Downloaded from www.annualreviews.org

features into the environment or manipulation (Walker 1976). In particular,

subjects may behave one way in the relative freedom of an experiment, where
by University of Illinois - Chicago on 05/17/12. For personal use only.

there are no countervailing pressures acting on them, but quite another when
acting within the constrained organizational or bureaucratic environments
in which they work at their political jobs. Material and professional incen-
tives can easily override more natural psychological or ethical concerns that
might manifest themselves more readily in the unconstrained environment
of the laboratory. Failure to mimic or incorporate these constraints into ex-
periments, and difficulty in making these constraints realistic, might restrict
the applicability of experimental results to the real political world.
There are two important things to understand about external validity. First,
external validity is only fully established through replication. Experiments
testing the same model should be conducted on multiple populations using
multiple methods in order to determine the external validity of any given ex-
perimental paradigm. Second, external validity is more closely related to the
realism created within the experiment than to the external trappings of simi-
larity to real-world settings, which is referred to as mundane realism. As long
at the experimental situation engages the subject in an authentic way, exper-
imental realism has been constructed; under these circumstances, mundane
realism may be nice but is hardly required to establish causality. Moreover,
even if the experiment closely approximates real-world conditions, if its sub-
jects fail to engage in an experimentally realistic way, subsequent findings
are useless.
4. Experimenter bias. Experimenter bias, including expectancy effects and de-
mand characteristics, can limit the relevance, generalizability, or accuracy
of certain experimental results.

EXPERIMENTAL ETHICS
As a result of concerns about the ethical treatment of human subjects, the U.S.
Department of Health and Human Services imposes strict guidelines on all research
involving human subjects. Institutional review boards at major research institutions
 13 Apr 2002 14:59 AR AR158-02.tex AR158-02.SGM LaTeX2e(2001/05/10) P1: GJC

EXPERIMENTS 41

oversee the administration of these guidelines and require advance approval on all
experiments with human subjects to ensure their ethical treatment. These boards
have the power to reject proposals that they deem to inadequately protect human
subjects from unnecessary pain and suffering. The job of these boards involves
weighing the risks and benefits of each study for the appropriate balance between
risk to the subject and benefit for science or society. The guidelines include four
important components:
1. Informed consent. Informed consent requires that the experimenter provide
every subject with a disclosure statement prior to the experiment, describing
the experimental procedures, along with expected gains and risks for the
subject. Subjects are told they can leave the experiment at any time without
Annu. Rev. Polit. Sci. 2002.5:31-61. Downloaded from www.annualreviews.org

penalty and are given contact information to report any concerns about their
by University of Illinois - Chicago on 05/17/12. For personal use only.

experience in the experiment to the institutional review board.

2. Risk/gain assessment. Experimenters are required to take every reasonable
precaution to reduce the potential risks to their human subjects.
3. Deception. Some critics argue that any deception is not consistent with in-
formed consent (Korn 1987), and others argue that deception is unethical
and damages the reputation of experimental scientists in the larger society
and is thus unjustified in any circumstance (Baumrind 1985). However, in
some instances, deception may be necessary to ensure that subjects cannot
guess the hypotheses under investigation. Were subjects to be aware of these
hypotheses, their behavior would likely shift as a result of that knowledge,
and results would be biased. When deception is used, institutional review
boards are particularly careful to ensure that it is necessary. In addition, they
may require experimenters to minimize its use, deny the application alto-
gether, or require careful monitoring and reporting on the experiment while
it is running.
4. Debriefing. At the end of an experiment, experimenters should be careful to
tell the subjects as much as possible about the experiment. If deception was
involved, experimenters should explain what the deception was and why it
was deemed necessary for the unbiased collection of the data. In particular,
subjects should be reassured that all their data will be confidential and that
the experimenter will obtain subjects’ written permission before any of their
information is shared publicly.

REVIEW OF EXPERIMENTS RELEVANT

TO POLITICAL SCIENCE
As noted, experiments have not been as widely employed in political science as in
either psychology or behavioral economics. Why have we, as a discipline, been so
slow to adopt experimental methodology? Why have all other social sciences used
experiments to their advantage, and yet political scientists, with few exceptions,
 13 Apr 2002 14:59 AR AR158-02.tex AR158-02.SGM LaTeX2e(2001/05/10) P1: GJC

42 MCDERMOTT

resist incorporating experimental methodology into their own work or trusting

those who do? Where has our field gone methodologically, and why has this
movement been away from experimentation?
There are at least four reasons why political science has not been as receptive
to the use of experiments as other social sciences. First, methodology in political
science has moved toward large-scale multiple regression work. There is nothing
wrong with this; indeed, in behavioral economics, there is a robust subfield devoted
to using experiments in concert with formal models and statistical analysis to
generate, test, and develop original hypotheses. So although experimental and
formal or statistical methods need not be contradictory, the topics studied using
these methods tend to be orthogonal to each other. For example, whereas multiple
Annu. Rev. Polit. Sci. 2002.5:31-61. Downloaded from www.annualreviews.org

regression tends to concentrate on large groups, experimental work often focuses

on small numbers of individuals.
by University of Illinois - Chicago on 05/17/12. For personal use only.

Second, an alternative movement in political science, which tends to eschew

the large-scale regression work, focuses on cultural and social aspects of particular
phenomena. Although constructivists and postmodern scholars pursue some of the
same issues as social psychologists, such as status concerns and the evolution of
norms, most cultural analysts remain disinclined to believe that experimental work
can get at phenomena as complex and multidimensional as political institutions or
cultural and social structures.
The third concern is more practical. Lack of experimental training in political
science at the graduate level means that few students add experimental design to
their arsenal of research tools as a matter of course. As a result, only especially
motivated students will contact an experimenter in a psychology or economics
department to learn the basics of experimental design and procedure. The best way
to learn how to do experiments is to run experiments. When training and experience
are difficult or unavailable, the concentration of experimentalists required to shift
disciplinary culture and practice fails to emerge.
Last, many political scientists, unlike other social scientists, seem to believe that
experimentalists expect experimental work to stand on its own, the way it does in
physics, chemistry, or biology. Unlike in biology, where every aspect of a particular
investigation can take place within a petri dish, most phenomena of interest to
political scientists are complex and involve many different variables. Experiments
can be used very effectively, as they have been in experimental economics, to
provide a middle ground between theory and naturally occurring empirical data.
Reciprocation between formal models and experimental testing has advanced
both theory and method in behavioral economics, and can serve a similar function
in political science as well. In addition, experimentation can be used to triangulate
in on various issues for which other methods have produced either inconsistent
or contradictory results. Finally, experiments can be effective in breaking down
complex processes into isolated pieces, which can then be examined and explored
in detail and in interaction. This process might demonstrate, for instance, that
small changes in seemingly meaningless variables produce huge shifts in outcome,
or that huge shifts in certain ostensibly important factors effect little change in
 13 Apr 2002 14:59 AR AR158-02.tex AR158-02.SGM LaTeX2e(2001/05/10) P1: GJC

EXPERIMENTS 43

outcome. These insights, gained from the experimental procedure, help with theory
development as well as theory testing.
Political science may have significant historical, cultural, or practical reasons
for its lack of affinity to the use of experimentation; nonetheless, the past need
not predict the future in this arena. Experimentation can achieve the same kind of
successful impact in political science that it has had in other fields such as psy-
chology and economics. We can learn from other social sciences that experiments
need not stand on their own, as in biology, in order to be effective and useful in
refining theory, providing evidence, and testing causal claims. Rather, experiments
can dovetail with other methods in order to produce a cumulation of knowledge
and an advancement in both theory and method within political science.
Annu. Rev. Polit. Sci. 2002.5:31-61. Downloaded from www.annualreviews.org

How might we respond to the previous lack of experimental work in political

science? As noted, one of the mistakes that many political scientists make is to
by University of Illinois - Chicago on 05/17/12. For personal use only.

assume that experiments in political science need to mimic those in biology, if not
in substance, at least in process. Yet, this stringent standard is not required; we
need only note that experiments can dovetail with other methods to produce useful
knowledge in order to adequately justify their utility.
In order to illustrate this process in action, the remainder of this essay is devoted
to an explication of some of the experimental literature of relevance to mainstream
political science. Certain experimental literature in behavioral economics and so-
cial psychology is addressed as well, since much of this work is relevant to issues
of concern to political science. This article does not have the space to cover ac-
complishments in all experimental areas of interest to political science. Moreover,
it is only through a sequence of experiments exploring related topics in different
ways and on different populations that cumulative knowledge and external validity
emerge. Therefore, following a brief overview of the presence of experiments in
political science, I concentrate on systematic programs of larger research within
behavioral economics and political science that have produced results with great
relevance to major issues of interest in political science.

Overview
A comprehensive overview of experiments published by established political sci-
entists reveals a total of 105 articles between 1926 and 2000. Only about 57 of them
were published in political science journals. Many more strong articles written by
political scientists, often in collaboration with economists, on political topics have
been published in either psychology or economics journals. I examined 48 articles
that appeared in non–political science journals but were written by political scien-
tists; six or seven individuals, either alone or in collaboration, wrote the majority
of them. The list of 105 articles does not include those published in the now-
defunct Experimental Study of Politics. This journal was founded in 1971 because
many believed that their experimental work had been unfairly rejected from the
established political science journals (McConahay 1973); Experimental Study was
created expressly to redress this difficulty. However, most of the articles published
 13 Apr 2002 14:59 AR AR158-02.tex AR158-02.SGM LaTeX2e(2001/05/10) P1: GJC

44 MCDERMOTT

in this journal were not as experimentally sophisticated as those published in psy-

chology journals (McConahay 1973), and the journal ceased publication in 1975.
A casual perusal of contemporary journals reveals that the same double standard
exists today, with the experimental sophistication of typical articles in psychol-
ogy or economics journals outstripping that of many articles published in political
science journals. In addition, my review excluded some very good experiments
published in edited volumes but not in journals (Iyengar & McGuire 1993, Palfrey
1991, Kagel & Roth 1995).
This overview revealed an increase in the number of published experiments
over time. A single article was published in the 1920s, 5 in the 1950s, 7 in the
1970s, 42 in the 1980s, and 45 in the 1990s. (Note that the increase between the
Annu. Rev. Polit. Sci. 2002.5:31-61. Downloaded from www.annualreviews.org

1970s and the 1980s may have been at least partly due to the demise of the afore-
mentioned Experimental Study of Economics journal in 1975.) Also interesting is
by University of Illinois - Chicago on 05/17/12. For personal use only.

the concentration of experiments in a small number of journals: Public Opinion

Quarterly (POQ) and American Political Science Review (APSR) account for the
most articles, with 21 each. These journals, with very few peripheral exceptions,
were the only place that experiments were published in political science until the
1980s. Political Psychology has published 18 experimental articles since the 1980s.
American Journal of Political Science (AJPS) has published 13 experiments and
Journal of Politics has published 7. These five journals together account for 80 of
the 105 experimental articles published by established political scientists in this
period. Interestingly, the trend appears to be shifting, such that in the 1990s only 2
experimental articles were published in POQ and only 5 in APSR. AJPS captured
the majority of the experimental articles published in the 1990s. Moreover, the re-
maining articles published in the 1990s appeared mostly in one of three additional
journals [Political Behavior (PB); Journal of Risk and Uncertainty; and Games
and Economic Behavior], only one of which (PB) is primarily a political science
journal.
It is quite striking that so few journals account for the majority of published
experimental work in political science. The odds of getting an experimental arti-
cle published drastically improve if it is submitted to one of these five journals.
Although it is possible that political scientists simply do not submit experimental
work to other political science journals by chance, the larger number of articles by
political scientists on political topics published in either psychology or economic
journals (Econometrica and American Economic Review in particular) suggest
otherwise. The relatively low prevalence of experimental work in political sci-
ence journals suggests that only a few are genuinely receptive to this form of
methodological investigation. Indeed, personal communication from the editor of
one major journal indicated that he could “never” put any experiment into print
because he believed that experiments held no relevance whatsoever for political
science. Such entrenched biases are difficult to overcome.
In addition, the bias toward experiments focused primarily on American top-
ics is reflected in the predominance of voting behavior as a primary concern of
experimental work. Out of the 105 articles surveyed, 25 were related to voting
 13 Apr 2002 14:59 AR AR158-02.tex AR158-02.SGM LaTeX2e(2001/05/10) P1: GJC

EXPERIMENTS 45

behavior. The second, third, and fourth most popular topics were bargaining (13),
games (10), and international relations topics (10). Other topics that attracted
experimental interest included committee work (8), experimental bias (6), race
(6), field experiments (5), media (4), leadership (4), and experiments embedded
in surveys (3). Given the high percentage of experimental articles focused on vot-
ing, the question arises whether more experimenters actually focus on voting, or
whether experiments on voting simply find more receptive journals available, and
thus are more likely to be published overall.
Several areas of experimental research are relevant to the concerns of main-
stream political scientists. Rather than examine each experiment individually, I
discuss the major relevant findings in two broad areas of experimental research.
Annu. Rev. Polit. Sci. 2002.5:31-61. Downloaded from www.annualreviews.org

In both of these areas, practitioners have developed sequences of experimental

work on different populations using different measures in order to clarify and
by University of Illinois - Chicago on 05/17/12. For personal use only.

contextualize the generalizability of their results. Such sequences allow investi-

gators to produce more reliable results, as one set of findings spurs another test
that might narrow, reinterpret, or extend previous conclusions. Steady and gradual
progress can thus attack seemingly impenetrable work. This essay concentrates on
behavioral economics, political economy, and psychological studies of individual
choice.

Experiments in Behavioral Economics

Within the past 15 years in particular, behavioral economics has grown exponen-
tially and transformed itself through the systematic use of experimentation. These
experiments often focus on issues of concern to political scientists. However,
behavioral economics, like psychology, embraces some core assumptions about
human nature and experimentation that not all political scientists share: skepticism
toward notions of perfect rationality; emphasis on experimental validation of mode-
ling assumptions; integration of micro level data; and adoption of lessons about
human cognitive processing from experimental social and cognitive psychology
(Laibson 2000). Further, experiments deemed successful in economics share cer-
tain central characteristics: clear instructions; absolutely no deception; stylized,
stripped-down settings; anonymity; cash incentives; experimental tests of formal
models, which can include comparisons with rational models; and, ideally, subse-
quent validation of the findings with supportive field data (Laibson 2000).
Obviously, economic topics have preoccupied political scientists using other
methodologies, such as game theory and rational choice forms of analysis. In
behavioral economics, theoretical development has profited from the interchange
between formal models and experimental tests of such models. Formal models
present hypotheses, which are then tested under experimental conditions. The
experimental findings are used to refine and develop hypotheses to produce new
theoretical models and predictions, which can then be experimentally tested in turn.
Political scientists might benefit from this process as well in order to both establish
empirical validation of formal models and speed the cumulation of knowledge.
 13 Apr 2002 14:59 AR AR158-02.tex AR158-02.SGM LaTeX2e(2001/05/10) P1: GJC

46 MCDERMOTT

Although the venues and contexts for investigation may differ, a great poten-
tial for substantive overlap exists between behavioral economics, psychology, and
political science, which might be exploited for greater cross-disciplinary and in-
terdisciplinary work. Current areas of research in psychology and economics offer
promising opportunities for collaboration with political scientists who share such
interests: social preferences, including investigations of norms, social networks,
altruism, status, and trust; bounded rationality, involving decision making in com-
plex environments; learning and expectation formation; attitudes toward risk; and
cognitive biases (Laibson 2000).
Behavioral economics has concentrated on six main substantive areas since
its experimental work began in the 1930s. These are (Roth 1995) (a) Prisoner’s
Annu. Rev. Polit. Sci. 2002.5:31-61. Downloaded from www.annualreviews.org

Dilemma and public goods issues; (b) problems of coordination and cooperation;
(c) dynamics of bargaining; (d ) experimental markets; (e) auction behavior; and
by University of Illinois - Chicago on 05/17/12. For personal use only.

( f ) individual choice. Some experimental work has also been done in the area
of industrial organization, although the majority of work in this area has been
conducted by social psychologists.

PRISONER’S DILEMMA AND PUBLIC GOODS The Prisoner’s Dilemma game was
developed by Dresher and Flood at the Rand Corporation in 1950 (Flood 1952);
the story was added by Tucker (1950) later (Straffin 1980). The central conundrum
was that although cooperative play was transparently more profitable in the long
run, equilibrium choice favored defection, producing less benefit for each player.
Thus, the game produced a challenging test for equilibrium predictions. It was
ideal for experimental investigation, which could examine alternative predictions
in a controlled setting. In this environment, scholars developed a preference for
repeated-play games. Most of these experiments demonstrated that cooperation
begins early but breaks down over time, such that in multiround repeated games,
players learn to defect earlier and earlier (Selten & Stoecker 1986). In most of
these experiments, subjects know that a better outcome of mutual cooperation
exists, but fear of exploitation makes mutual defection the only stable strategy
over time. Even more interesting are recent findings suggesting that at least some
players hold values independent of the payoffs embedded in the game structure,
such as fairness, altruism, or concern with reputation building (Andreoni & Miller
1993). These findings will no doubt prompt further experimental work to test the
nature and limits of these seemingly noneconomic motivations. Political science
readers will also remember the famous Axelrod testing of Prisoner’s Dilemma
strategies in large-scale computer tournaments (Axelrod 1984). In his simulation,
a “tit for tat” strategy, in which the player begins with cooperation and makes the
same move as the opponent did the previous round, emerged as the most effective
strategy for maximizing payoffs.
Apparently the public goods problem was first presented by Swedish economist
Knut Wicksell in the nineteenth century (Roth 1995). Most experimental work on
the public goods and free rider problem has focused on the conditions under which
it might be most problematic and how its impact can be reduced in those situations.
 13 Apr 2002 14:59 AR AR158-02.tex AR158-02.SGM LaTeX2e(2001/05/10) P1: GJC

EXPERIMENTS 47

Early work using single examples of public goods problems indicated little free
riding (Johansen 1977). Like the work in Prisoner’s Dilemma, experimental work
on public goods problems moved toward repeated plays, demonstrating that in
successive rounds, voluntary contributions decline (Isaac et al. 1985). Future work
will no doubt explore the conditions under which free riding produces the greatest
problems and examine ways in which to ameliorate its effects.

COORDINATION Economists investigating coordination problems experimentally

study how subjects playing games with multiple equilibria decide on one in partic-
ular. In other words, how do players achieve a “meeting of the minds” (Ochs 1995)?
Coordination clearly is influenced by particular aspects of the situation or environ-
Annu. Rev. Polit. Sci. 2002.5:31-61. Downloaded from www.annualreviews.org

ment, some of which are not adequately captured by traditional economic models.
by University of Illinois - Chicago on 05/17/12. For personal use only.

In many cases, coordination problems prevent optimal decision making from a ra-
tional perspective. For example, a factor that should not matter from a traditional
economic perspective may exert a tremendous influence, as when the mere pres-
ence of a dominated strategy affects the equilibrium chosen (Cooper et al. 1990).
In an experiment involving only problems of coordination, with no inherent
conflict of interest, Van Huyck et al. (1990) found that in a repeated game where
outcomes are made public after each round, behavior quickly converges around the
least profitable equilibrium. Obviously, this outcome does not represent economic
rationality in the traditional sense. Crawford (1991) and others have sought to
explain these findings using learning models and other game theory models offered
by evolutionary biologists. Such models suggest that stable equilibria occur when
strategies are not subject to invasion and dominance by new strategies. Crawford
does not argue that the coordination problem itself is evolutionary in nature; rather,
learning within the game allows stable, albeit economically nonrational, equilibria
to emerge and dominate over time.

BARGAINING Experiments on bargaining behavior also examine how individuals

arrive at an agreeable compromise equilibrium, with the added constraint of conflict
of interest between players. According to most game theoretic models, player
preferences and opportunities should determine outcomes of bargaining behavior.
But experimental evidence has shown that bargaining outcomes are affected by
variables that are not incorporated into player preference or opportunity, including
the player’s subjective expectations about the behavior of his opponent, which
appears to be learned through experience (Roth & Schoumaker 1983).
The main debate in the bargaining literature has been between those who con-
centrate on game theoretic predictions and those who are interested in more so-
ciological or cultural considerations. First, important findings in this area include
the significance and frequency of disagreements and costly delays during the bar-
gaining games. Second, many experiments demonstrate clear “deadline effects,”
which support the notion that many agreements are reached very close to the final
deadline for agreement (Roth et al. 1988). Both these sets of findings have spurred
additional work to determine the impact of these factors on the nature of bargaining
 13 Apr 2002 14:59 AR AR158-02.tex AR158-02.SGM LaTeX2e(2001/05/10) P1: GJC

48 MCDERMOTT

(Roth 1995). This topic is discussed below in the political science section on co-
operation.

EXPERIMENTAL MARKETS Experimental markets typically draw on the basic de-

sign established by Chamberlain (1948) to encourage subjects to behave according
to the laws of supply and demand, often with the goal of comparing alternative
types of markets. Many of the important experiments in this area have been con-
ducted by Plott and colleagues in response to specific policy questions surrounding
the potential market effects of instituting particular government regulations (Hong
& Plott 1982, Grether & Plott 1984).
Other work on experimental markets examines how traders use and share infor-
Annu. Rev. Polit. Sci. 2002.5:31-61. Downloaded from www.annualreviews.org

mation. For example, work on asset valuation (Forsythe et al. 1982) demonstrates
that prices tend to converge to a perfect equilibrium after replication. In addition,
by University of Illinois - Chicago on 05/17/12. For personal use only.

information aggregation (Forsythe & Lundholm 1990) has been examined within
the context of experimental markets. Security markets have been examined in this
way as well (Plott & Sunder 1988). Forsythe et al. (1992) used an experimental
market whose ultimate value was tied to a future election outcome to examine how
markets aggregate information. They found that their experimental market prices
did a reasonable job of predicting the outcome of the election. On the basis of these
results, Forsythe et al. (1992) argue that market transactions reduce the impact of
biases such as political opinions on subjects’ pricing decisions.
Provocative work on cross-cultural behavior in bargaining and experimen-
tal market environments suggested that market outcomes converged to equilib-
rium and that there were no payoff differences between subjects in Jerusalem,
Ljubljana, Pittsburgh, and Tokyo (Roth et al. 1991). However, differences that
deviated from equilibrium predictions did occur everywhere in both the agree-
ment and the frequency of disagreement. The experimental procedures employed
made the experimenters confident that observed differences in bargaining behavior
did not result from differences in language or currency; rather, the investigators
tentatively attributed these discrepancies to cultural differences.

AUCTIONS Auction behavior is familiar to anyone who is a fan of, or knows an

addict of, eBay. The most consistent and prominent result of the study of auction
behavior is the “winner’s curse” (Bazerman & Samuelson 1983, Thaler 1992),
wherein the winner of an auction pays much more for the prize than either its
real value or the price he intended to pay at the outset. In addition, the average
bid is often well below the objective value of the auctioned object. Uncertainty
about the true value of the object appears to exacerbate this effect. These findings,
suggestive of persistent mistakes on the part of bidders, are clearly contradictory
to what established economic theory would predict about equilibrium behavior in
auctions (Milgrom & Weber 1982).

INDIVIDUAL CHOICE Experimental work on individual choice has produced im-

portant findings in behavioral economics. This area of work retains the greatest
 13 Apr 2002 14:59 AR AR158-02.tex AR158-02.SGM LaTeX2e(2001/05/10) P1: GJC

EXPERIMENTS 49

overlap between social psychologists and experimental economists in theory,

method, and practice. Research on individual choice encompasses at least three
important topics: preference reversals, judgment under uncertainty, and decision
making under risk.
The topic of preference reversals has proved particularly problematic for eco-
nomists attempting to sustain the accuracy of rational models (Tversky et al.
1990). Early experimental work by psychologists Slovic & Litchenstein (1968;
Litchenstein & Slovic 1971) noted the anomaly that people will put a higher price
on a given lottery when asked to buy or sell it (bids), but when asked to partici-
pate in one, most individuals will choose the other lottery (choice). These results
remain remarkably robust. Preference reversals seem to result from differences
Annu. Rev. Polit. Sci. 2002.5:31-61. Downloaded from www.annualreviews.org

in the way people process information about probabilities and payoffs. Bids ap-
pear to be governed by payoffs, whereas choices tend to be driven by probabilities.
by University of Illinois - Chicago on 05/17/12. For personal use only.

Although these findings remain inconsistent with expected utility and other rational
models of decision making, psychologists readily explain this result as an exam-
ple of the anchoring and adjustment heuristic (Kahneman et al. 1982), whereby
people initially latch onto a value, which can be arbitrary and irrelevant, and fail
to adequately adjust that value to present circumstances in making subsequent
judgments. The first anchor people fix on is the monetary payoff, and then they
insufficiently adjust choices to shifts in probabilities. Preference reversals result
from this inadequate adjustment from the initial monetary anchor.
The second main area of research within the individual choice literature is
judgment under uncertainty (Kahneman et al. 1982). This experimental work ex-
amines how individuals judge the frequency or likelihood of certain outcomes. This
work consistently and robustly demonstrates at least three important judgmental
heuristics that appear to control people’s assessments of frequency: anchoring and
adjustment (described above), representativeness, and availability. The representa-
tiveness heuristic claims that individuals assess frequency based on the similarity
between the judged object or event and the categories to which it might belong. The
availability heuristic argues that people judge likelihood based on salience, i.e., the
ease of retrieval or imagination of the example from memory. All three judgmental
heuristics contradict central assumptions in most expected utility models, which
expect dominance, invariance, and intransitivity to hold sway in judgments about
probabilities.
Decision making under risk has been most closely examined by the same psy-
chologists who conducted the seminal work on judgmental biases (Kahneman &
Tversky 2000). In attempting to develop a descriptively accurate model of choice
as an alternative to expected utility models, Kahneman & Tversky (1979; Tversky
& Kahneman 1992) delineated prospect theory. Prospect theory incorporates two
successive phases: editing and evaluation. In editing, prospects or choices are
framed for a decision maker. Robust experimental evidence indicates that trivial
aspects of framing options can consistently exert profound impacts on the substance
of choice. Specifically, seemingly trivial changes in the method, order, or form in
which options are presented to a decision maker systematically affect the content of
 13 Apr 2002 14:59 AR AR158-02.tex AR158-02.SGM LaTeX2e(2001/05/10) P1: GJC

50 MCDERMOTT

choice. Evaluation itself encompasses two components as well: the value function
and the weighting function. The value function has three central characteristics:
(a) outcomes are judged in relative, not absolute, terms; (b) individuals tend to be
risk-seeking in the domain of losses and risk-averse in the domain of gains; and
(c) people tend to be loss-averse in general. The role of the weighting function
is similar to, but distinct from, that of probability assessments in expected utility
models. First, people have great difficulty incorporating extremes such as impos-
sibility and certainty into their decision-making strategies. Second, individuals
tend to overweight low probabilities while simultaneously underweighting mod-
erate and high probabilities. All these empirical results surrounding the value and
weighting functions contradict the predictions of standard expected utility models.
Annu. Rev. Polit. Sci. 2002.5:31-61. Downloaded from www.annualreviews.org
by University of Illinois - Chicago on 05/17/12. For personal use only.

Experiments in Political Science

As in economics, some political scientists are beginning to use experimentation
to test formal models in a controlled empirical setting. The three main areas of
research in this area are (a) voting and elections, (b) committee and jury decision
making, and (c) problems of coordination and cooperation (Palfrey 1991). This
third area is similar in some theoretical ways to work on coordination in behavioral
economics, but the domains of application differ. Palfrey (1991) argues that these
topics have produced at least three important themes in the relationship between
formal modeling and experimental research in political science. These include
the importance of strategic behavior in studying complex political actions and ac-
tors; the critical significance of incomplete or asymmetric information, especially
as related to issues of reputation, communication, and signaling; and finally, the
importance of explicitly building dynamic models, which are aided by experi-
mental methods and impact problems related to party identification, realignments,
incumbency, and political business cycles.

EXPERIMENTAL VOTING AND ELECTIONS Since Downs’s An Economic Theory of

Democracy, many scholars have tried to examine the foundations of democratic
elections with formal models. Increasingly, these models are being tested experi-
mentally (Palfrey 1991). Plott (1991), for one, tested the spatial model to examine
certain aspects of elections, including voter turnout.
Voter turnout lends itself nicely to experimental investigation. For example,
Palfrey & Rosenthal (1985) argued that according to game theoretic analysis un-
der assumptions of complete information, analysts should expect equilibria of high
turnout, even when the costs of voting are high. Instead, they demonstrated ex-
perimentally that under conditions of uncertainty about the preferences and costs
of others for voting, only voters with very low voting costs will vote in a large
election. In other work on voter turnout, Green and colleagues have attempted to
rehabilitate the use of field experiments begun by Gosnell (1926). In a study on
the effects of canvassing, phone calls, and direct mail on voter turnout, Gerber
 13 Apr 2002 14:59 AR AR158-02.tex AR158-02.SGM LaTeX2e(2001/05/10) P1: GJC

EXPERIMENTS 51

& Green (forthcoming1) found that personal canvassing increased voter turnout,
whereas phone calls appeared to have no impact. Direct mail appeared to have a
slight impact on voter turnout. In addition, they found that asking voters whether
they could be “counted on” to vote increased the impact of personal canvassing.
Other topics that have been investigated experimentally under the rubric of
voting and elections include candidate competition (Plott 1991), retrospective vot-
ing (McKelvey et al. 1987), political competition (Boylan et al. 1991), and voter
information costs.
Lau and Sears have used experiments to examine related topics. Their study
of the evaluation of public figures (Lau et al. 1979) concluded that the so-called
positivity bias often found in survey results is not an artifact of the measurement
Annu. Rev. Polit. Sci. 2002.5:31-61. Downloaded from www.annualreviews.org

process alone but rests on some real bias in assessment. Related work on political
by University of Illinois - Chicago on 05/17/12. For personal use only.

preferences (Sears & Lau 1983) showed that self-interest may result from political
and personal cues in surveys that trigger artifactual results. Finally, these authors
have experimentally explored the nature of political beliefs (Lau et al. 1991).
Political party identification has also been examined experimentally (Cowden
& McDermott 2000). We were intrigued by previous work, using different metho-
dologies, that achieved somewhat contradictory results regarding the long-term
stability of party identification. We designed an experiment that assessed student
subjects’ party identification, among other things, early in the semester. Later,
after participating in one experiment that manipulated the extremity of real candi-
dates in experimental elections, or another in which subjects role-played either the
prosecutor or defender of Clinton in the impeachment hearing, subjects filled out
a second, standard party identification measure. Our results indicated that party
identification, even in a young population that should have had less time to develop
strong associations, showed remarkable stability.
Media effects on candidate evaluation and voting have been another extremely
productive research topic. Some of the best and most imaginative experimentation
has been conducted in the area of media studies and political communication by
Iyengar and colleagues. Their creative studies have demonstrated that television
news influences how viewers weight problems and evaluate candidates (Iyengar
et al. 1982); that television news frames individuals’ explanation of events (Iyengar
1987); that negative advertising reduces voter turnout (Ansolabehere et al. 1994);
and that candidates gain the most by advertising on issues over which they can
claim “ownership” (Ansolabehere & Iyengar 1994). Iyengar continues to advance
the methodology of experimentation itself as well, with recent studies that use new
technology and field strategies to ameliorate some of the traditional criticisms of
external validity problems (Iyengar 2000). These strategies include bringing the
experiments into natural settings by creating living room environments in shopping
malls and asking subjects to watch television in those settings, with experiments
embedded in the programming. Further, Iyengar has begun to use the internet

1
Gerber A, Green D. The Effects of Canvassing, Phone Calls and Direct Mail on Voter
Turnout: A Field Experiment. Unpublished manuscript.
 13 Apr 2002 14:59 AR AR158-02.tex AR158-02.SGM LaTeX2e(2001/05/10) P1: GJC

52 MCDERMOTT

to reach more diverse populations, which increases experimenter access to more

representative samples.
Experimental studies of candidate evaluation by gender have produced some
interesting findings as well. In an evaluation of campaign coverage of senatorial
candidates, Kahn (1992) found that the press presented male and female candidates
in systematically different ways. Such differences appeared to benefit male can-
didates, who were seen as more viable; this may disadvantage female candidates
at the polls. Nevertheless, sex stereotypes sometimes benefit women because they
were judged more frequently than men to be compassionate and honest. Further
work by Kahn (1994) examining both gubernatorial and senatorial candidates
found that voter perceptions were affected by both news coverage and sex stereo-
Annu. Rev. Polit. Sci. 2002.5:31-61. Downloaded from www.annualreviews.org

types. Interestingly, these factors appear to affect incumbents differently from

challengers, and gubernatorial candidates differently from senatorial candidates. In
by University of Illinois - Chicago on 05/17/12. For personal use only.

particular, gender differences in press coverage were more pronounced in the senate
race and for incumbents. This pattern appears to hurt female senatorial candidates.
On the other hand, sex stereotypes produce more positive evaluations of women
and appear to benefit gubernatorial candidates the most. Note that Kahn’s further
experimental testing of her earlier findings allowed her to further refine and condi-
tionalize her results. The findings of Huddy & Terkildsen (1993) on gender stereo-
typing in the perception of candidates are consistent with Kahn’s. They too find that
female candidates are seen in a positive light on traits such as compassion, whereas
men are perceived to be more competent on military issues. Huddy & Terkildsen
suggest that a gender trait approach best explains the differences they find.

COMMITTEE AND JURY DECISION MAKING A second arena of systematic research

in political science considers committee and jury decision making. Experiments
on committee decision making are typically modeled on legislatures in which
results emerge from a combination of bargaining and voting. Much research thus
focuses on how the bargaining process and the voting rules affect the outcome
of committee decision making, especially under different decision rules. Various
scholars have examined committee decision making under majority rule (Fiorina
& Plott 1978, McKelvey & Ordeshook 1979), plurality (Neimi & Frank 1985),
approval voting (Neimi 1984), noncooperative games (Felsenthal et al. 1988),
competitive solutions (McKelvey & Ordeshook 1983), and universalism (Miller
& Oppenheimer 1982). In particular, agenda setting (Levine & Plott 1977, Wilson
1986) and time constraints (Wilson 1986) offer perfect topics for experimental
investigation based on strategic models. Guarnaschelli et al. (2000), among others,
have recently used experimental work in the investigation of jury decision-making
analysis as well.
Work on committee decision making often uses the ultimatum game as an
experimental tool. Typically, two players must divide a sum of money, such as $10.
The procedure requires one player to offer an amount to the second, who can then
accept or reject it. If the second player rejects it, no one gets the money, whereas
if the player accepts it, both players split the money in the percentage agreed. The
 13 Apr 2002 14:59 AR AR158-02.tex AR158-02.SGM LaTeX2e(2001/05/10) P1: GJC

EXPERIMENTS 53

theoretical question investigated is whether something about a subject’s partner

will affect either person’s willingness to bargain.
Much work on committee decision making grew out of observations about
the problems inherent in conventional game theory’s treatment of these issues.
The results of many experimental bargaining games seemed askew (Ostrom 1998)
and players often exhibited consistent behavioral play (Camerer 1997), neither of
which should be true according to traditional models. In particular, self-interest
does not always work or dominate in these ultimatum games. This was also found
in so-called dictator games, where the first player can solely dictate the division of
goods. Specifically, unequal splits tend to be rejected in favor of “fair” splits. This
outcome should not happen under subgame perfect backward induction equilibria,
Annu. Rev. Polit. Sci. 2002.5:31-61. Downloaded from www.annualreviews.org

which would be predicted by expected utility models.

The promise of behavioral game theory rests on its ability to explore various
by University of Illinois - Chicago on 05/17/12. For personal use only.

aspects of this conundrum experimentally. In particular, experimental work can

build on previous anomalous findings from ultimatum games to examine altruism,
inequality aversion, and so-called mind reading (Wilson 2001). Altruism refers to a
consistent desire to help others, even when it might hurt oneself. Numerous models
of altruism (Forsythe et al. 1994, Eckel & Grossman 1996) typically assume that
altruism is an embedded character trait within a given individual. Altruism succeeds
because it gives people a positive feeling about themselves as a result of their
actions. From an evolutionary standpoint, altruism may exist within communities
because it advances the ability of the society to survive and prosper even when
key individuals, such as mothers after childbirth, are too overwhelmed to perform
their normal tasks successfully. Altruism in this sense may amount to little more
than reciprocal selfishness.
Inequality aversion refers to many individuals’ empirical preference for equal
over unequal distributions of goods, even when extreme self-interest is possible,
as in the dictator game. Examinations of this phenomenon explore how the same
individual might act differently in different situations. What constraints control
the extent to which an individual might cooperate in one circumstance but not
in anther? Early indications suggest that meaningful comparisons are important
(Bolton & Ockenfels 2000) and that at least some inequality aversion derives from
concerns surrounding relative status (Fehr & Schmidt 1999).
Mind reading refers to discerning the intentions of others (Rabin 1993; Levine
1998; Falk & Fischbacher, unpublished manuscript2). Unlike altruism, mind read-
ing allows for the emergence of both positive and negative emotions (Frank 1988,
Smith 1998). Once another person’s intentions have been determined through an
empirical process, people will tend to treat a nice person nicely and a mean person
as they deserve.
Wilson (2001) has conducted a series of experiments investigating these phe-
nomena. He demonstrates the ideal experimental procedures by learning from the

2
Falk A, Fischbacher U. 1998. Kindness is the parent of kindness: modeling reciprocity.
Unpublished manuscript.
 13 Apr 2002 14:59 AR AR158-02.tex AR158-02.SGM LaTeX2e(2001/05/10) P1: GJC

54 MCDERMOTT

experiences of previous studies and designing future studies to address past anoma-
lies or to ameliorate procedural difficulties. He finds that beliefs about others are
important and can change over time. These beliefs appear to be contingent on cues
that individuals receive over time about others. In this way, interaction develops
lasting reputations and labels. Wilson’s work suggests that theoretical models of
individual choice might be impaired by their failure to incorporate such seemingly
nonrational factors as altruism, inequality aversion, and mind reading.

COORDINATION AND COOPERATION Work on coordination and cooperation in po-

litical science resembles similar work in behavioral economics, discussed above.
However, applications differ, and work on cooperation in political science can eas-
Annu. Rev. Polit. Sci. 2002.5:31-61. Downloaded from www.annualreviews.org

ily be applied to problems in security as well as political economy. Topics include

alliances, arms races, trade wars, and sanctions. For example, Geva and Skorick
by University of Illinois - Chicago on 05/17/12. For personal use only.

have used experimentation to test their cognitive calculus model of decision mak-
ing in foreign policy (Geva & Skorick 1999, Geva et al. 2000, Geva & Skorick
2000). These authors use experimentation to test the predictions of their model
against actual behavior in a laboratory setting.
Work on coordination and cooperation remains closely tied to work in social
psychology and behavioral economics. Typically, scholars investigate this topic
using noncooperative game theory (Palfrey 1991). Experimentalists seek to provide
data related to certain models and push those models further by presenting evidence
that might either refute or extend the current theoretical claims. Specific results
indicate that communication increases group cooperation. Ostrom and colleagues
(e.g., Ostrom & Walker 1991) have demonstrated that face-to-face communication,
particularly in repeated-play settings involving common pool resources, exerts a
powerful impact on propensity for cooperation.
Palfrey and colleagues have undertaken a systematic program of experimental
research on topics related to coordination and cooperation. In one experiment,
discounted repeated play proved more effective in generating cooperation than a
single shot trial in a public goods game with incomplete information; however,
results depended on the ability to monitor others and on the specific environmental
conditions (Palfrey & Rosenthal 1994). Palfrey and colleagues have concentrated
on the centipede game, in which two players alternately have a chance to take a
larger portion of a continually escalating amount of money (McKelvey & Palfrey
1992, Fey et al. 1996). Once one person takes the money, the game ends. Ac-
cording to game theory predictions under assumptions of complete information,
the first player should take the larger pile in the first round of play. However,
this does not happen in reality. Rather, subjects operating under conditions of un-
certainty and incomplete information about the payoff appear willing to consider
the small possibility that they are playing against an altruistic opponent. Although
the probability increases over time that a player will take the pile of money, the game
typically continues into subsequent rounds. Palfrey has also investigated choice
in other games (McKelvey & Palfrey 1995). This work shows great richness in
its ability to combine formal modeling with experimental testing of such models.
 13 Apr 2002 14:59 AR AR158-02.tex AR158-02.SGM LaTeX2e(2001/05/10) P1: GJC

EXPERIMENTS 55

The combination of methods allows greater confidence in results that point in the
same direction.
Experimental work by Miller and colleagues has explored a variety of topics,
including committees (Miller & Oppenheimer 1982). In work on games, Eavey &
Miller (1984a) demonstrate that when universalist options, which offer “something
for everyone,” exist in legislatures, concerns about fairness go beyond what ex-
pected value expectations would predict. Further, Miller & Oppenheimer (1982)
find that competitive coalitions with a minimum winning coalition occur only
when universal options are unavailable. In work on bargaining, Eavey & Miller
(1984b) show that a bureaucratic monopoly on agenda setting allows bargaining
with a voting body without necessarily imposing the agenda setter’s preferences
Annu. Rev. Polit. Sci. 2002.5:31-61. Downloaded from www.annualreviews.org

on all. They conclude that bureaucratic agenda control in legislative bodies sup-
ports a bargaining model over an imposition one. Although some of this work
by University of Illinois - Chicago on 05/17/12. For personal use only.

(Miller & Oppenheimer 1982, Palfrey & Rosenthal 1994) points out the discrep-
ancies between rational choice theory and the behavior of individuals in the real
world, experiments are used not only to test and critique existing formal models
but also to discover anomalies and challenges that are then incorporated into the
next generation of model development.
Bolton (1991) has used experimentation to investigate how actual bargaining
behavior differs from game theoretic predictions. Bolton & Zwick (1995) demon-
strate that the opportunity to punish an opponent who treats you unfairly presents
a more accurate explanation for deviations from perfect equilibrium solutions than
the existence of anonymity for the subject. Note that although experimental find-
ings may be at odds with some predictions of formal theory, the overall relationship
between game theoretic modeling and experimentation in these exercises is col-
laborative; experiments empirically test formal models and suggest discrepancies
as well as validations, and then formal modelers can attempt to incorporate these
empirical demonstrations into later, more sophisticated models.
In our work on topics related to international relations, we investigate the
impact of factors such as sex, uncertainty, and framing effects on arms races
and aggression. In one experiment involving three rounds of a simulated crisis
(McDermott & Cowden, forthcoming), we find that although uncertainty exerts
no systematic effect on weapons procurement or likelihood of war, men are sig-
nificantly more likely to purchase weapons and engage in aggressive action than
women. In another experiment involving a simulated crisis game (McDermott
et al. 2002), we examine the impact of framing in terms of striving for superiority
or parity with the opponent, two kinds of uncertainty, and the tone of messages
on weapons procurement. We find that embracing the frame of striving for su-
periority does indeed increase weapons procurement on the part of subjects. The
tone of the message exerts a tremendous impact as well; recipients of hostile
messages are much more likely to procure weapons than recipients of friendly
messages. As in our other work, uncertainty appears to have no effect on weapons
procurement. Finally, in more recent work, as yet unanalyzed, we manipulated
the incentive to go to war to further examine the impact of sex differences on
 13 Apr 2002 14:59 AR AR158-02.tex AR158-02.SGM LaTeX2e(2001/05/10) P1: GJC

56 MCDERMOTT

levels of aggression. We plan to expand this paradigm to include other popula-

tions, including military officers, to further explore the impact of factors such as
hormones, including testosterone, and nonverbal gestures on tendencies toward
aggression.

CONCLUSIONS

Experimentation is one of many methods that can be used to examine political

phenomena. Experiments have a long and distinguished history of effective usage in
other disciplines, including hard sciences such as physics and biology, medicine,
Annu. Rev. Polit. Sci. 2002.5:31-61. Downloaded from www.annualreviews.org

and social sciences such as psychology and economics. Unfortunately, experi-

by University of Illinois - Chicago on 05/17/12. For personal use only.

ments have been slower to acquire a dedicated following of practitioners in poli-

tical science, mostly because of concerns about external validity. In many cases,
this concern merely reflects a misunderstanding of the replication requirements
necessary to establish external validity. But this concern may also indicate a fail-
ure to understand the difference between experimental realism, which is essential
and requires the subject to be actively engaged in the process under investigation,
and mundane realism, which refers to inessential trappings of the experimental
situation that increase only the appearance, not the reality, of external validity
(McDermott, under review3).
The primary advantage of experiments is that they offer unparalleled control
over the variables of interest. This is because the experimental method permits the
systematic manipulation of variables in a controlled environment with randomly
assigned subjects. Experiments thus offer the highest degree of internal validity;
experimenters can be pretty confident that outcomes differ on the basis of the
variables manipulated systematically within the experimental conditions. This en-
ables experimenters to make causal arguments about which factors cause certain
outcomes, or contribute to them, and which do not.
Another advantage of experiments results from the scientific rigor built into the
process. Experimenters remain aware of, and retain control over, the independent
variables of interest. Experimenters carefully record results as dependent variables.
Later statistical analysis allows the detailed testing of the relationships between
these variables and any interactions among them. With this process, results that
might not have been obvious to less systematic or large-scale analysis become
prominent. Experiments allow causal inference, precise measurement and control,
and clarity of detail.
Unfortunately, many political scientists assume that experimental results in po-
litical science need to be able to stand on their own, as in biology, and that if they
cannot, they are useless. Nothing could be further from the truth. Experimenta-
tion can readily dovetail with other methodologies to produce systemic bodies of
knowledge. As demonstrated by much of the work in behavioral economics and

3
McDermott R. Experimental methodology in political science. Submitted.
 13 Apr 2002 14:59 AR AR158-02.tex AR158-02.SGM LaTeX2e(2001/05/10) P1: GJC

EXPERIMENTS 57

some of the work in political science, the intersection of formal modeling and
experimental testing is highly productive. Experiments can be, and have been,
effectively used to test formal models, demonstrate unpredicted anomalies in out-
comes that then provoke more sophisticated models, and suggest extensions and
limitations of existing models under particular conditions.
In addition, experiments provide effective methodological help in examining
areas in which other methodologies have produced inconsistent or contradictory
findings, as was the case in our work on party identification. Experiments also
offer clear advantages over other methods in particular areas of investigation, such
as the validation of theories developed by formal modeling, or in further theory
testing and refinement. Experiments offer useful insights in work that investigates
Annu. Rev. Polit. Sci. 2002.5:31-61. Downloaded from www.annualreviews.org

the underlying process of a particular phenomenon as opposed to its outcome.

Finally, invoking multiple methods, including experimentation, in investigating
by University of Illinois - Chicago on 05/17/12. For personal use only.

a phenomenon allows greater confidence in consensual results. In this way, ex-

periments can help in triangulating in on research questions. Indeed, experimen-
tation can serve a useful purpose, as it has in behavioral economics, to advance
knowledge in political science more quickly and systematically and to cumu-
late such knowledge through the process of building on previous experimental
work.

ACKNOWLEDGMENTS
I would like to thank Jonathan Cowden, Margaret Levi, one anonymous reviewer,
the participants in the CBRSS Experimental Methods Conference at Harvard, and
especially Sidney Tarrow for enormous help in writing this review.

The Annual Review of Political Science is online at http://polisci.annualreviews.org

LITERATURE CITED
Andreoni J, Miller J. 1993. Rational coopera- son, Vol. 2. Reading, MA: Addison-Wesley.
tion in the finitely repeated Prisoner’s Rev. ed.
Dilemma: experimental evidence. Econ. J. Axelrod R. 1984. The Evolution of Coopera-
103:570–85 tion. New York: Basic Books
Ansolabehere S, Iyenger S. 1994. Riding the Baumrind D. 1985. Research using intentional
wave and claiming ownership over issues: the deception: ethical issues revisited. Am. Psy-
joint effects of advertising and news coverage chol. 40:165–74
in campaigns. Public Opin. Q. 58:335–57 Bazerman M, Samuelson W. 1983. I won the
Ansolabehere S, Iyengar S, Simon A, Valen- auction but don’t want the prize. J. Confl.
tino N. 1994. Does attack advertising de- Resolut. 27:618–34
mobilize the electorate? Am. Polit. Sci. Rev. Bolton G. 1991. A comparative model of bar-
88:829–38 gaining: theory and evidence. Am. Econ. Rev.
Aronson E, Carlsmith M. 1968. Experimenta- 81:1096–36
tion in social psychology. In The Handbook Bolton G, Ockenfels A. 2000. Measuring moti-
of Social Psychology, ed. G Lindzey, E Aron- vations for the reciprocal responses observed
 13 Apr 2002 14:59 AR AR158-02.tex AR158-02.SGM LaTeX2e(2001/05/10) P1: GJC

58 MCDERMOTT

in a simple dilemma game. Am. Econ. Rev. Eckel C, Grossman P. 1996. Altruism in anony-
90:166–93 mous dictator games. Games Econ. Behav.
Bolton G, Zwick R. 1995. Anonymity versus 16:181–91
punishment in ultimatum bargaining. Games Etheredge L. 1978. A World of Men: The Private
Econ. Behav. 10:95–121 Sources of American Foreign Policy. Cam-
Boylan R, Ledyard J, Lupia A, McKelvey R, bridge, MA: MIT Press
Ordeshook P. 1991. Political competition in Fehr E, Schmidt K. 1999. A theory of fairness,
a model of economic growth: an experimen- competition and cooperation. Q. J. Econ.
tal study. In Laboratory Research in Political 114:817–68
Economy, ed. T Palfrey, pp. 33–68. Ann Ar- Felsenthal D, Rapoport A, Maoz A. 1988. Tacit
bor: Univ. Mich. Press cooperation in three alternative noncoopera-
Burke J, Greenstein F. 1989. How Presidents tive voting games: a new model of sophisti-
Annu. Rev. Polit. Sci. 2002.5:31-61. Downloaded from www.annualreviews.org

Test Reality: Decisions on Vietnam, 1954 and cated behavior under the plurality procedure.
1965. New York: Russell Sage Fdn. Elect. Stud. 7:143–61
by University of Illinois - Chicago on 05/17/12. For personal use only.

Camerer C. 1997. Progress in behavioral game Fey M, McKelvey R, Palfrey T. 1996. An ex-
theory. J. Econ. Persp. 11:167–88 perimental study of a constant-sum centipede
Campbell DT. 1968. Quasi-experimental de- game. Int. J. Game Theory 25:269–87
sign. In International Encyclopedia of the Fiorina M, Plott C. 1978. Committee decisions
Social Sciences, ed. DL Sills, Vol. 5. New under majority rule: an experimental study.
York: Macmillan Am. Polit. Sci. Rev. 72:575–98
Campbell DT, Ross HL. 1968. The Connecticut Flood M. 1952. Some experimental games. Res.
crackdown on speeding: time-series data in Memo. RM-789, RAND Corp., June
quasi-experimental analysis. Law Soc. Rev. Forsythe R, Horowitz J, Savin N, Sefton M.
3:33–53 1994. Fairness in simple bargaining games.
Campbell DT, Stanley JC. 1963. Experimen- Games Econ. Behav. 6:347–69
tal and quasi-experimental designs for re- Forsythe R, Lundholm R. 1990. Information
search on teaching. In Handbook of Research aggregation in an experimental market. Eco-
on Teaching, ed. NL Gage. Chicago: Rand nometrica 58:309–48
McNally Forsythe R, Nelson F, Neumann G, Wright J.
Chamberlain R. 1948. An experimental imper- 1992. Anatomy of an experimental political
fect market. J. Polit. Econ. 56(2):95–108 stock market. Am. Econ. Rev. 82:1142–61
Cooper R, DeJong D, Forsythe R, Ross T. Forsythe R, Palfrey T, Plott C. 1982. Asset
1990. Selection criteria in coordination valuation in an experimental market. Econo-
games: some experimental results. Am. Econ. metrica 50:537–67
Rev. 80:218–33 Frank R. 1988. Passions Within Reason: The
Cowden J, McDermott R. 2000. Short term Strategic Role of Emotions. New York:
forces and partisanship. Polit. Behav. 22: Norton
197–222 Geva N, Mayhar J, Skorick JM. 2000. The
Crawford V. 1991. An “evolutionary” interpre- cognitive calculus of foreign policy deci-
tation of Van Huyck, Battalio and Beil’s ex- sion making: an experimental assessment. J.
perimental results on coordination. Games Confl. Resolut. 44:447–71
Econ. Behav. 3:25–59 Geva N, Skorick JM. 1999. Information in-
Eavey C, Miller G. 1984a. Fairness in majority consistency and the cognitive algebra of for-
rule games with a core. Am. J. Polit. Sci. 28: eign policy decision making. Int. Interact.
570–86 25:333–62
Eavey C, Miller G. 1984b. Bureaucratic agenda Geva N, Skorick JM. 2000. Process and out-
control: imposition or bargaining? Am. Polit. come consequences of simultaneous foreign
Sci. Rev. 78:719–33 policy decisions. Presented at Annu. Meet.
 13 Apr 2002 14:59 AR AR158-02.tex AR158-02.SGM LaTeX2e(2001/05/10) P1: GJC

EXPERIMENTS 59

Int. Soc. Polit. Psychol., July 4–8, Seattle, An experimental examination of sex stereo-
WA types and press patterns in statewide cam-
Gosnell H. 1926. An experiment in the stimula- paigns. Am. J. Polit. Sci. 38:162–95
tion of voting. Am. Polit. Sci. Rev. 20:869–74 Kahneman D, Slovic P, Tversky A, eds.
Grether D, Plott C. 1984. The effects of market 1982. Judgment Under Uncertainty: Heuris-
practices in oligopolistic markets: an exper- tics and Biases. Cambridge, UK: Cambridge
imental examination of the ethyl case. Econ. Univ. Press
Inq. 22:479–507 Kahneman D, Tversky A. 1979. Prospect the-
Guarnaschelli S, McKelvey R, Palfrey T. 2000. ory: an analysis of decision under risk.
An experimental study of jury decision rules. Econometrica 47:263–91
Presented at Exp. Methods Conf., May 11– Kahneman D, Tversky A. 2000. Choices, Val-
12, Cambridge, MA ues and Frames. New York: Cambridge Univ.
Annu. Rev. Polit. Sci. 2002.5:31-61. Downloaded from www.annualreviews.org

Hong J, Plott C. 1982. Rate filing policies for Press/Russell Sage Fdn.
inland water transportation: an experimental Korn J. 1987. Judgments of acceptability of de-
by University of Illinois - Chicago on 05/17/12. For personal use only.

approach. Bell J. Econ. 18:187–97 ception in psychological research. J. Gen.

Huddy L, Terkildsen N. 1993. Gender stereo- Psychol. 114:205–16
types and the perception of male and female Kuklinski J, Sniderman P, Knight K, Piazza
candidates. Am. J. Polit. Sci. 37:119–47 T, Tetlock P, et al. 1997. Racial prejudice
Isaac R, McCue K, Plott C. 1985. Public goods and attitudes toward affirmative action. Am.
provisions in an experimental environment. J. Polit. Sci. 41:402–19
J. Public Econ. 26:51–74 Laibson D. 2000. Untitled. Presented at Exp.
Iyengar S. 1987. Television news and citizens’ Methods Conf., May 11–12, Cambridge, MA
explanations of national affairs. Am. Polit. Lau R, Sears D, Centers R. 1979. The “positiv-
Sci. Rev. 81:815–32 ity bias” in evaluations of public figures: evi-
Iyengar S. 2000. Experimental designs for po- dence against the instrument artifacts. Public
litical communication research: from shop- Opin. Q. 43:347–58
ping malls to the internet. Presented at Exp. Lau R, Smith R, Fiske S. 1991. Political be-
Methods Conf., May 11–12, Cambridge, liefs, policy interpretations, and political per-
MA suasion. J. Polit. 53:644–75
Iyengar S, McGuire W, eds. 1993. Explorations Levine D. 1998. Modeling altruism and spite-
in Political Psychology. Durham, NC: Duke fulness in experiments. Rev. Econ. Dyn.
Univ. Press 1:593–622
Iyengar S, Peters MD, Kinder D. 1982. Ex- Levine M, Plott C. 1977. Agenda influence and
perimental demonstrations of the “not-so- its implications. Va. Law Rev. 63:561–604
minimal” consequences of television news Litchenstein S, Slovic P. 1971. Reversal of pref-
programs. Am. Polit. Sci. Rev. 76:848–58 erences between bids and choices in gam-
Johansen L. 1977. The theory of public goods: bling decisions. J. Exp. Psychol. 89:46–55
misplaced emphasis? J. Public Econ. 7:147– McConahay J. 1973. Experimental research. In
52 Handbook of Political Psychology, ed. JN
Kagel J, Roth A, eds. 1995. The Handbook Knutson, pp. 356–82. San Francisco: Jossey-
of Experimental Economics. Princeton, NJ: Bass. 542 pp.
Princeton Univ. Press McDermott R, Cowden J. 2001. The effects
Kahn K. 1992. Does being male help? An in- of sex and uncertainty in a crisis simulation
vestigation of the effects of candidate gen- game. Int. Interact. 27:353–80
der and campaign coverage on evaluations of McDermott R, Cowden J, Koopman C. 2002.
U.S. senatorial candidates. J. Polit. 54:497– The effects of framing, uncertainty and hos-
517 tile communications in a crisis simulation
Kahn K. 1994. Does gender make a difference? game. Polit. Psychol. In press
 13 Apr 2002 14:59 AR AR158-02.tex AR158-02.SGM LaTeX2e(2001/05/10) P1: GJC

60 MCDERMOTT

McKelvey R, Ordeshook P. 1979. An experi- Plott C. 1991. A comparative analysis of di-

mental test of several theories of committee rect democracy, two-candidate elections, and
decision making under majority rule. In Ap- three-candidate elections in an experimental
plied Game Theory, ed. S Brams, A Schot- environment. In Laboratory Research in Po-
ter, G Schwodiauer. Wurzburg, Ger.: Physica litical Economy, ed. T Palfrey, pp. 11–32.
Verlag Ann Arbor: Univ. Mich. Press
McKelvey R, Ordeshook P. 1983. Some exper- Plott C, Sunder S. 1988. Rational expectations
imental results that fail to support the com- and the aggregation of diverse information
petitive solution. Public Choice 40:281–91 in laboratory security markets. Econometrica
McKelvey R, Ordeshook P, Collier K, Willi- 56:1085–118
ams K. 1987. Retrospective voting: an expe- Rabin M. 1993. Incorporating fairness into
rimental study. Public Choice 53:101–30 game theory and economics. Am. Econ. Rev.
Annu. Rev. Polit. Sci. 2002.5:31-61. Downloaded from www.annualreviews.org

McKelvey R, Palfrey T. 1992. An experimental 83:1281–302

study of the centipede game. Econometrica Roethlisberger FJ, Dickson WJ. 1939. Man-
by University of Illinois - Chicago on 05/17/12. For personal use only.

60:803–36 agement and the Worker. Cambridge, MA:

McKelvey R, Palfrey T. 1995. Quantal response Harvard Univ. Press
equilibria for normal form games. Games Rosenberg MJ. 1965. When dissonance fails:
Econ. Behav. 10:6–38 on eliminating evaluation apprehension from
Milgrom P, Weber R. 1982. Predation, repu- attitude measurement. J. Pers. Soc. Psychol.
tation, and entry deterrence. J. Econ. Theory 1:28–42
27:280–312 Rosenthal R. 1966. Experimental Effects in
Miller G, Oppenheimer J. 1982. Universalism Behavioral Research. New York: Appleton-
in experimental committees. Am. Polit. Sci. Century-Crofts
Rev. 76:561–74 Roth A. 1988. Laboratory experimentation in
Neimi R. 1984. The problem of strategic be- economics: a methodological overview.
havior under approval voting. Am. Polit. Sci. Econ. J. 393:974–1031
Rev. 78:952–58 Roth A. 1995. Introduction. In Kagel & Roth
Neimi R, Frank A. 1985. Sophisticated vot- 1995, pp. 3–109
ing under the plurality procedure: a test of Roth A, Murnighan K, Schoumaker F. 1988.
a new definition. Theory Decision 19:151– The deadline effect in bargaining. Am. Econ.
62 Rev. 78:806–23
Ochs J. 1995. Public goods. In Kagel & Roth Roth A, Prasnikar V, Okuno-Fujiwara M, Za-
1995, pp. 195–253 mir S. 1991. Bargaining and market behavior
Ostrom E. 1998. A behavioral approach to the in Jerusalem, Pittsburgh, and Tokyo: an ex-
rational choice theory of collective action. perimental study. Am. Econ. Rev. 81:1068–
Am. Polit. Sci. Rev. 92:1–22 95
Ostrom E, Walker J. 1991. Communication in a Roth A, Schoumaker F. 1983. Expectations and
commons: cooperation without external en- reputations in bargaining: an experimental
forcement. In Palfrey 1991, pp. 287–322 study. Am. Econ. Rev. 73:362–72
Palfrey T, ed. 1991. Laboratory Research in Sears D. 1986. College sophomores in the lab-
Political Economy. Ann Arbor: Univ. Mich. oratory: influences of a narrow data base on
Press. 323 pp. social psychology’s view of human nature. J.
Palfrey T, Rosenthal H. 1985. Voter participa- Pers. Soc. Psychol. 86:515–30
tion and strategic uncertainty. Am. Polit. Sci. Sears D, Lau R. 1983. Inducing apparently self-
Rev. 79:62–78 interested political preferences. Am. J. Polit.
Palfrey T, Rosenthal H. 1994. Repeated play, Sci. 27:223–52
cooperation and coordination: an experimen- Selten R, Stoecker R. 1986. End behavior in
tal study. Rev. Econ. Stud. 61:545–65 sequences of finite Prisoner’s Dilemma
 13 Apr 2002 14:59 AR AR158-02.tex AR158-02.SGM LaTeX2e(2001/05/10) P1: GJC

EXPERIMENTS 61

supergames: a learning theory approach. J. in prospect theory: cumulative representa-

Econ. Behav. Org. 7:47–70 tion of uncertainty. J. Risk Uncertain. 5:297–
Slovic P, Litchenstein S. 1968. Relative im- 323
portance of probabilities and payoffs in risk Tverksy A, Slovic P, Kahneman D. 1990. The
taking. J. Exp. Psychol. Monogr. Suppl. 78:1– causes of preference reversal. Am. Econ. Rev.
18 80:204–17
Smith V. 1998. The two faces of Adam Smith. Van Huyck J, Battalio R, Beil R. 1990. Tacit
South. Econ. J. 65:1–19 coordination games, strategic uncertainty,
Sniderman P, Piazza T, Tetlock P, Kendrick and coordination failure. Am. Econ. Rev. 80:
A. 1991. The new racism. Am J. Polit. Sci. 234–48
35:423–47 Walker T. 1976. Microanalytic approaches to
Straffin P. 1980. The Prisoner’s Dilemma. political decision-making. Am. Behav. Sci.
Annu. Rev. Polit. Sci. 2002.5:31-61. Downloaded from www.annualreviews.org

UMAP J. 1:102–3 20:93–110

Thaler R. 1992. The Winner’s Curse: Paradoxes Wilson R. 1986. Forward and backward
by University of Illinois - Chicago on 05/17/12. For personal use only.

and Anomalies of Economic Life. Princeton, agenda procedures: committee experiments

NJ: Princeton Univ. Press on structurally induced equilibrium. J. Poli-
Tucker A. 1950. A two-person dilemma. tics 48:390–409
Mimeo, Stanford Univ.; republished 1980. Wilson R. 2001. Untitled. Presented at Exp.
On jargon: the Prisoner’s Dilemma. UMAP Methods Conf., May 11–12, Cambridge, MA
J. 1:1 Zimbardo P, Gerrig R. 1996. Psychology and
Tversky A, Kahneman D. 1992. Advances Life. New York: HarperCollins. 14th ed.
P1: FDS
April 9, 2002 11:17 Annual Reviews AR158-FM

Annual Review of Political Science

Volume 5, 2002

CONTENTS
BARGAINING THEORY AND INTERNATIONAL CONFLICT, Robert Powell 1
EXPERIMENTAL METHODS IN POLITICAL SCIENCE, Rose McDermott 31
Annu. Rev. Polit. Sci. 2002.5:31-61. Downloaded from www.annualreviews.org

POLITICS, POLITICAL SCIENCE, AND URBAN GOVERNANCE:

by University of Illinois - Chicago on 05/17/12. For personal use only.

A LITERATURE AND A LEGACY, Russell D. Murphy 63

DEMOCRACY AND TAXATION, Andrew C. Gould and Peter J. Baker 87
FORECASTING FOR POLICY MAKING IN THE POST–COLD WAR PERIOD,
Stanley A. Feder 111
THE ORIGINS, DEVELOPMENT, AND POSSIBLE DECLINE OF
THE MODERN STATE, Hendrik Spruyt 127
DEMOCRATIC INSTITUTIONS AND REGIME SURVIVAL: PARLIAMENTARY
AND PRESIDENTIAL DEMOCRACIES RECONSIDERED,
José Antonio Cheibub and Fernando Limongi 151
POLITICAL CLEAVAGES AND POST-COMMUNIST POLITICS,
Stephen Whitefield 181
OF WAVES AND RIPPLES: DEMOCRACY AND POLITICAL CHANGE
IN AFRICA IN THE 1990S, Clark C. Gibson 201
HOW CONCEPTUAL PROBLEMS MIGRATE: RATIONAL CHOICE,
INTERPRETATION, AND THE HAZARDS OF PLURALISM, James Johnson 223
THE NEWS MEDIA AS POLITICAL INSTITUTIONS, Michael Schudson 249
THE FIRST DECADE OF POST-COMMUNIST ELECTIONS AND VOTING:
WHAT HAVE WE STUDIED, AND HOW HAVE WE STUDIED IT?,
Joshua A. Tucker 271
CORPORATISM: THE PAST, PRESENT, AND FUTURE OF A CONCEPT,
Oscar Molina and Martin Rhodes 305
LANDMARKS IN THE STUDY OF CONGRESS SINCE 1945, Nelson W. Polsby
and Eric Schickler 333
ELECTORAL AND PARTISAN CYCLES IN ECONOMIC POLICIES AND
OUTCOMES, Robert J. Franzese, Jr. 369
TOWARD A NEW POLITICAL METHODOLOGY: MICROFOUNDATIONS
AND ART, Christopher H. Achen 423

vii
P1: FDS
April 9, 2002 11:17 Annual Reviews AR158-FM

viii CONTENTS

INDEXES
Subject Index 451
Cumulative Index of Contributing Authors, Volumes 1–5 469
Cumulative Index of Chapter Titles, Volumes 1–5 471

ERRATA
An online log of corrections to The Annual Review of Political Science
chapters (if any have yet been occasioned, 1997 to the present) may be
found at http://polisci.annualreviews.org/
Annu. Rev. Polit. Sci. 2002.5:31-61. Downloaded from www.annualreviews.org
by University of Illinois - Chicago on 05/17/12. For personal use only.