Psychological Bulletin © 2009 American Psychological Association

2009, Vol. 135, No. 3, 347–368 0033-2909/09/$12.00 DOI: 10.1037/a0014211

Implicit Measures: A Normative Analysis and Review

Jan De Houwer Sarah Teige-Mocigemba

Ghent University University of Freiburg

Adriaan Spruyt and Agnes Moors

Ghent University

Implicit measures can be defined as outcomes of measurement procedures that are caused in an automatic
manner by psychological attributes. To establish that a measurement outcome is an implicit measure, one
should examine (a) whether the outcome is causally produced by the psychological attribute it was
designed to measure, (b) the nature of the processes by which the attribute causes the outcome, and (c)
whether these processes operate automatically. This normative analysis provides a heuristic framework
for organizing past and future research on implicit measures. The authors illustrate the heuristic function
of their framework by using it to review past research on the 2 implicit measures that are currently most
popular: effects in implicit association tests and affective priming tasks.

Keywords: implicit measures, automaticity, IAT, affective priming

Most psychologists would argue that a full understanding of we first provide a normative analysis of the concept “implicit
the behavior of an individual requires knowledge not only of the measure.” The analysis is normative in the sense that it stipulates
external situation in which the individual is present but also of the the properties that an ideal implicit measure should have. As such,
internal psychological attributes of the individual. Throughout the analysis provides a heuristic framework for reviewing and
the history of psychology, researchers have therefore attempted to evaluating existing research on implicit measures. By examining
measure interindividual differences in the psychological attributes the extent to which a particular implicit measure exhibits these
of people (e.g., Anastasi, 1958; Eysenck & Eysenck, 1985; Mis- normative properties, one can clarify the way in which the measure
chel & Shoda, 1995). During the past decade, a major development is an implicit measure and highlight those issues on which further
in this research has been the introduction of so-called implicit research is required. In the second part of this article, we perform
measures. These measures were originally put forward mainly this exercise with regard to the two types of implicit measures that
within the social psychology literature (e.g., Fazio, Jackson, Dun- are currently most popular: effects in implicit association tests
ton, & Williams, 1995; Greenwald, McGhee, & Schwartz, 1998) (IATs; Greenwald et al., 1998) and affective priming tasks (Fazio
but have since then spread to various other subdisciplines of et al., 1995). Before we present and apply our normative analysis,
psychology, including differential psychology (e.g., Asendorpf, we provide a brief description of these two measures.
Banse, & Mücke, 2002), clinical psychology (e.g., Gemar, Segal, During a typical IAT, participants see stimuli that belong to one
Sagrati, & Kennedy, 2001), consumer psychology (e.g., Maison, of four categories and are asked to categorize each stimulus by
Greenwald, & Bruin, 2004), and health psychology (e.g., Wiers, pressing one of two keys. Two of the four categories are assigned
van Woerden, Smulders, & de Jong, 2002). to the first key, and the two other categories are assigned to the
Despite the widespread use of implicit measures, the actual second key. The core idea underlying the IAT is that categorization
meaning of the term implicit measure is rarely defined. On the
performance should be a function of the degree to which categories
basis of the work of Borsboom (Borsboom, 2006; Borsboom,
that are assigned to the same key are associated in memory. Hence,
Mellenbergh, & van Heerden, 2004) and De Houwer (De Houwer,
by examining which combinations of categories result in the best
2006; De Houwer & Moors, 2007; Moors & De Houwer, 2006),
categorization performance, one should be able to infer which
categories are more closely associated in memory. Take the ex-
ample of a racial IAT designed to measure attitudes toward Black
Jan De Houwer, Adriaan Spruyt, and Agnes Moors, Department of and White individuals (e.g., Mitchell, Nosek, & Banaji, 2003;
Psychology, Ghent University, Ghent, Belgium; Sarah Teige-Mocigemba, Monteith, Voils, & Ashburn-Nardo, 2001). There are four catego-
Department of Psychology, University of Freiburg, Freiburg, Germany. ries of stimuli: stimuli related to Black individuals (e.g., the face of
Portions of this article were presented by Jan De Houwer at the EAPP a Black person), stimuli related to White individuals (e.g., the face
Summer School on Implicit Measures of Personality, 29 July 2007, Berti- of a White person), positive words (e.g., summer), and negative
noro, Italy, and at the GRK 1253 Summer School on Methods of Affective
words (e.g., cancer). In the Black-positive task, participants press
Neuroscience, 6 October 2007, Bronnbach, Germany. Preparation of this
article was supported by Ghent University Grant BOF/GOA2006/001. the first key whenever a Black face or a positive word appears and
Correspondence concerning this article should be addressed to Jan De the second key whenever a White face or a negative word is
Houwer, Department of Psychology, Ghent University, Henri Dunantlaan presented. In the White-positive task, the first key is assigned to
2, B-9000 Ghent, Belgium. E-mail: Jan.DeHouwer@UGent.be White faces and positive words and the second key is assigned to

347
348 DE HOUWER, TEIGE-MOCIGEMBA, SPRUYT, AND MOORS

Black faces and negative words. Participants who are faster on the applies specifically to implicit measures. Because of their norma-
Black-positive than on the White-positive task are assumed to have tive nature, the criteria that we specify in this section set very high
stronger associations in memory between the concepts “Black standards for any type of measurement. It might well be that most
person” and “positive” than between the concepts “White person” psychological measures, implicit or otherwise, do not meet these
and “positive” (or weaker associations between “Black person” standards. Even measures that are not perfect can, however, still be
and “negative” than between “White person” and “negative”). The useful. The normative criteria should therefore not be interpreted
reverse is assumed to be true for someone who is faster on the as minimal conditions that must be met before a measurement
White-positive than on the Black-positive task. Given the addi- outcome can be regarded as an implicit measure. Rather, they are
tional assumption that racial attitudes are a function of the strength ultimate goals. By specifying the characteristics of an ideal im-
of the associations in memory between, on the one hand, the plicit measure, one can use the normative criteria to highlight the
concepts “Black person” and “White person” and, on the other strengths and weaknesses of existing implicit measures and to
hand, the concepts “positive” and “negative,” one can argue that provide direction for future research.
the difference in performance on the Black-positive and White-
positive tasks provides an index of the attitude toward Black What Is a Measure?
persons relative to the attitude toward White persons.
In a typical affective priming task, participants categorize target Given that psychological measures are meant to reveal internal
stimuli as being positive or negative. Each target is preceded by a psychological attributes of individuals, an ideal psychological
prime stimulus. The core idea underlying an affective priming measure should provide an exact index of the extent to which an
measure is that one can estimate the attitude toward the prime individual possesses the psychological attribute that the measure
stimulus by examining how the presence of the prime influences was designed to capture. For instance, an ideal measure of racial
the affective categorization of the target stimuli. For instance, in attitudes should reflect the extent to which a particular individual
order to measure attitudes toward Black and White persons, one likes or dislikes particular racial groups. As pointed out by Bors-
can present on each trial the picture of a Black or a White person boom et al. (2004, p. 1061), “a test is valid for measuring an
as a prime stimulus followed by a positive or a negative target attribute if and only if (a) the attribute exists and (b) variations in
word that participants categorize as being positive or negative in the attribute causally produce variations in the outcomes of the
valence (e.g., Fazio et al., 1995). If Black faces facilitate respond- measurement procedure.” Figure 1A provides a graphical repre-
ing to positive relative to negative target words, this effect would sentation of this statement. When a measurement procedure is
indicate a positive attitude toward Black persons. If Black faces applied to a certain person, a hypothetical attribute within the
facilitate the affective categorization of negative relative to posi- person causes an observable outcome (bottom arrow), which can
tive words, this effect would suggest a negative attitude toward then be used to make an inference about the attribute of the person
Black persons. The attitude toward White persons can be estimated (top arrow). This statement, though at first sight obvious, clarifies
in a similar manner by investigating whether White faces as primes a number of important issues. We discuss these issues in detail and
facilitate responding to positive or negative targets. summarize the main conclusions at the end of this section.
Many other implicit measures have been proposed in recent First, a distinction can be made between a measure in the sense
years. As are IAT effects and affective priming effects, most of these of a procedure and a measure in the sense of the outcome of a
implicit measures are based on performance in speeded reaction time procedure (see also De Houwer, 2006). For instance, the racial IAT
tasks. That is, the psychological attributes of the individual are in- is a measure in the sense of a procedure. It is a set of objective
ferred from the speed or accuracy with which the individual responds guidelines about what someone should do in order to obtain an
to certain stimuli in certain tasks (e.g., De Houwer, 2003a; Nosek & index of racial attitudes (e.g., what stimuli to present in what
Banaji, 2001; see De Houwer, 2003b, for a structural analysis and manner, what instructions to give, and how to register and analyze
review). Other implicit measures, however, focus not on the speed responses). The procedure generates an outcome, namely, a score
of responding but on the content of responses (e.g., Payne, Cheng,
Govorun, & Stewart 2005; Sekaquaptewa, Espinoza, Thompson, A.
Vargas, & von Hippel, 2003; see De Houwer, 2008, for a discus-
sion of dimensions on which implicit measures can differ). In the
first part of this article, we try to determine what all these different PROCEDURE-----PERSON OUTCOME
measures have in common and what it means to say that something
is an implicit measure.

Implicit Measures: A Normative Analysis B.

A normative analysis of the concept “implicit measure” implies

the specification of a set of criteria that an ideal implicit measure PROCEDURE-----PERSON OUTCOME
should meet. Implicit measures are a subclass of all possible
measures of psychological attributes. Hence, an ideal implicit
measure should not only be an ideal measure but should have the Automatic
additional characteristic of being implicit. We therefore first dis-
cuss the normative criteria that any perfect psychological measure Figure 1. A schematic representation of the definition of (A) a measure
should meet. Afterward, we specify the additional criterion that and (B) an implicit measure.
 IMPLICIT MEASURES 349

reflecting the difference in performance on the two IAT tasks (e.g., The statement of Borsboom et al. (2004) has a third important
the Black-positive task and the White-positive task). The outcome implication. It clarifies that validity implies causality. Variations in
is a measure in the sense that it is meant to reflect racial attitudes. a measurement outcome can reveal something about variations in
To avoid confusion, one should always clarify whether the term a psychological attribute only if the attribute somehow causes the
measure is used to refer to a procedure or to an outcome of a outcome. To verify the validity of a measure, we thus need
procedure. We use it to refer only to a measurement outcome and evidence that variations in the to-be-measured attribute indeed
use the term measurement procedure to refer to a procedure used cause variations in the measurement outcome. The most suitable
to generate a measurement outcome. way to obtain such evidence is through experimental research (i.e.,
Second, the claim that a measurement outcome provides a valid research in which the attribute is manipulated experimentally and
measure of a psychological attribute implies the ontological claim the effects of the manipulation on the measurement outcome are
that the attribute exists in some form and influences behavior. examined). This research should reveal not only that variations in
There has been a lot of debate in philosophy about whether it is the attribute cause variations in the measurement outcome but also
possible to substantiate ontological claims. Borsboom et al. (2004) how they do so (see also Wentura & Rothermund, 2007). Knowl-
argued, however, that there simply is no alternative to making edge about the causal mechanisms provides more certainty about the
ontological claims when measuring. It does indeed seem illogical fact that an attribute causes an outcome and allows one to optimize the
to argue that the statement “the outcome is a valid measure of measure in the sense of maximizing the effects of the attribute on
attribute X” and the statement “attribute X does not exist” are both the measurement outcome.
true. If the attribute does not exist or does not cause variation in the Whereas Borsboom et al. (2004) promoted experimental studies
outcome, the outcome cannot be a measure of the attribute. If the as the primary way to study validity, until now, the correlational
outcome is a measure of the attribute, the attribute must exist and approach has dominated validation research. Borsboom et al. ar-
must causally influence the outcome (see Borsboom et al. for a gued that the correlational approach as typically adopted in vali-
more detailed critique of positivist and constructivist views on dation research is suboptimal for the study of the validity of
ontological claims regarding measurement). measures because correlational evidence (a) does not allow for
Ontological assumptions might of course be incorrect or incom- causal inferences and (b) is not directed at examining the relation
plete. To evaluate those assumptions, one can engage not only in between psychological attributes and measurement outcomes.
conceptual analyses but also in empirical research using measures With regard to the first point, there are many well-known reasons
that are assumed to be somehow related to psychological at- why correlations do not allow for causal conclusions. For instance,
tributes. As such, ontological assumptions can also depend on a correlation between two variables might be due not only to a
empirical measurement. Borsboom et al. (2004) nevertheless ar- causal relation between the two but to the presence of a third
gued that ontological assumptions are primordial. There can be no variable that causally influences both other variables. With regard
measurement without ontological assumptions, but there can be to the second point, most correlational validation studies have been
ontological assumptions without measurement (e.g., those based designed to examine how psychological constructs are related to
solely on conceptual considerations). Moreover, as we discuss each other rather than to measurement outcomes. Following the
later, measurement allows for strong conclusions regarding psy- recommendations of Cronbach and Meehl (1955), researchers de-
chological attributes only under very specific conditions. veloped theories (so-called nomological networks) about whether
Regardless of the ontological status that one assigns to assump- a particular target attribute (e.g., intelligence) should or should not
tions about psychological attributes, it is important to realize that be related to other attributes (e.g., general knowledge). A measure
claims about the validity of a measure do imply assumptions about of the target attribute was considered to be valid if it correlated in
the nature of psychological attributes. In psychology, relatively the expected way with measures of other attributes. It would lead
little is known about the attributes that are assumed to underlie us too far afield to discuss all the arguments that Borsboom et al.
behavior. Psychological attributes, such as attitudes, stereotypes, presented against this approach. For the present purpose, it is
and personality traits, cannot be observed directly. Instead, they important to realize that correlational studies about the relations
are inferred indirectly from the observation that there are system- between (measures of) psychological constructs do have important
atic differences in behavior that are not merely a function of limitations for the study of the validity of measures when validity
differences in the external environment. Although most psychol- is defined in terms of the causal impact of a psychological attribute
ogists will agree that there are internal attributes that codetermine on the measurement outcome.
(human) behavior, there is less agreement about what these dif- This fact does not imply that correlational studies are worthless
ferent attributes are and how they should be defined. For instance, and that only experimental studies should be conducted from now
a special issue of the journal Social Cognition (Gawronski, 2007) on. First, correlational results can constrain hypotheses about the
was recently devoted to the question “What is an attitude?” even nature of the psychological attribute that causes variation in a
though the concept “attitude” has been measured in numerous measure. For instance, as the evidence increases that a measure
ways ever since Thurstone (1928). It is important to realize that the correlates in the expected manner with measures of other at-
validity of a measure of psychological attributes can go only as far tributes, it becomes less likely that these correlations are due to a
as the validity of the assumptions about the attributes it is assumed hidden, third factor (see Nosek & Smyth, 2007). As pointed out by
to measure. If these assumptions turn out to be incorrect or an anonymous reviewer, because correlational data are often much
incomplete, the old interpretation of the measure is no longer valid easier to obtain than experimental data, the correlational approach
and claims about the validity of the measure need to be abandoned offers us an efficient way to learn more about the validity of a
or altered. measure.
350 DE HOUWER, TEIGE-MOCIGEMBA, SPRUYT, AND MOORS

Second, experimental research also has limitations. Most prom- psychological attribute causally influences the measurement out-
inently, experiments can provide conclusive information about the come and how it does so; we should also know whether there are
causal properties of psychological attributes only if the imple- other sources of variation and how these sources impact on the
mented manipulations (a) affect the to-be-measured attribute in the measure.
intended manner and (b) do not affect other attributes or processes To conclude, on the basis of the work of Borsboom and col-
that determine performance. When a manipulation does not impact leagues (Borsboom, 2006; Borsboom et al., 2004), we can now
on the attribute that is being measured, the absence of an effect of define a measure as a measurement outcome that is causally
the manipulation on the measure says nothing about the validity of produced by the to-be-measured attribute. The overall quality of
that measure. This fact implies that an experimental approach the measure also depends on whether there are other sources of
makes little sense for measures that capture attributes that are variation. On the basis of these considerations, we argue that an
stable over time and impervious to situational factors. Also, when ideal measure should conform to two normative criteria: It should
a manipulation influences psychological attributes and processes be clear (a) which attributes causally produce the measurement
other than the intended ones, the presence of an effect on the outcome and (b) how these attributes causally produce the mea-
measure provides little information about the validity of that surement outcome. We will refer to the first criterion as the “what
measure. In such cases, it is not clear whether the observed effects criterion” and to the second criterion as the “how criterion” (also
are due to the fact that the measure captures the to-be-measured see De Houwer, Geldof, & De Bruycker, 2005). Both conceptual
attribute or is determined by other attributes and processes. As is analyses and empirical research are required to verify whether a
the case with correlational research, a third factor might thus be measure conforms to these criteria. There needs to be conceptual
responsible for the observed relation between the independent and clarity about the attribute that is assumed to be measured. Not only
dependent variables. correlational but also experimental studies can provide information
The more we know about the determinants of psychological about which attributes actually cause variation in the measurement
attributes and the processes by which attributes influence behavior, outcome and how they do so (also see Wentura & Rothermund,
the more certain we can be that experimental manipulations will 2007).
have only the intended effects. Hence, the merits of an experimen-
tal approach to examining the validity of measures depend on What Is an Implicit Measure?
theoretical and conceptual knowledge. Despite the limitations of
experimental research, Borsboom et al. (2004) convincingly ar- The claim that a measurement outcome is an implicit measure
gued that experimental research should be given a prominent place implies not only that it is a measure (i.e., that it is causally
in validation research. Given that validation research has until now produced by the to-be-measured attribute) but also that it is im-
been dominated by correlational studies, this is an important con- plicit in some sense. De Houwer (De Houwer, 2006; De Houwer
clusion regardless of whether one agrees with Borsboom et al.’s & Moors, 2007) argued that the term implicit can best be under-
evaluation of correlational research. stood as being synonymous with the term automatic. Both terms
In the previous paragraphs, we have argued that claims about the have been used to describe the features of psychological processes
validity of a measure (a) refer to the properties of an outcome of or, more precisely, the conditions under which psychological pro-
a procedure rather than to the procedure itself, (b) imply ontolog- cesses can be operative. For instance, a process can be called
ical assumptions about the causal effect of psychological attributes automatic in the sense that it can operate even when participants do
and thus depend on the validity of these assumptions, and (c) can not have particular goals, a substantial amount of cognitive re-
be examined not only with a correlational approach but with sources, a substantial amount of time, or awareness (of the insti-
experimental studies. A final important point in the work of gating stimulus, the process itself, or the outcome of the process;
Borsboom et al. (2004) is that they distinguished between the see Bargh, 1992; Moors & De Houwer, 2006). From this perspec-
validity of a measure and its overall quality. They argued that a tive, an implicit measure can be defined as a measurement out-
valid measure is not necessarily reliable or predictive of criterion come that is causally produced by the to-be-measured attribute in
variables and could even measure different attributes in different the absence of certain goals, awareness, substantial cognitive re-
groups of respondents (Borsboom et al., p. 1070). This is because sources, or substantial time (De Houwer, 2006; De Houwer &
a measure can be a valid index of a psychological attribute even if Moors, 2007). This definition implies that the processes by which
this attribute is not the only source of variation in the measure. the attribute causes the measurement outcome are automatic in a
Validity implies that the to-be-measured attribute causes variation certain sense of the word, an idea that is graphically represented in
in the measure but does not rule out the possibility that other Figure 1B.
attributes or situational factors are additional sources of variation. We have carefully avoided equating the concepts “implicit” and
When variation in a measurement outcome has multiple sources, “automatic” with one particular feature or set of features. One
one can never be sure that a particular measurement outcome was reason for this is that the different features of automaticity do not
caused by the to-be-measured attribute rather than by other sources always co-occur. For instance, evidence suggests that stereotype
of variance. Also, if the impact of the other sources of variation activation is automatic in that it does not depend on the conscious
is time or context dependent, this will reduce reliability, predictive goal to activate the stereotype or on the presence of processing
validity, and measurement invariance. In light of our aim to resources but is nonautomatic in that it depends on the presence of
specify the normative criteria that an ideal (implicit) measure certain other goals (e.g., Bargh, 1992; Moskowitz, Salomon, &
should meet, these criteria should take into account not only Taylor, 2000). Moreover, different processes can possess different
requirements of validity but the determinants of overall quality. In features of automaticity and thus be automatic in different ways
other words, we need to be sure not only that the to-be-measured (Bargh, 1992). For these reasons, one cannot simply say that a
 IMPLICIT MEASURES 351

process is automatic. It is always necessary to specify in what review past research on implicit measures and highlight the ques-
sense a process can be considered automatic by specifying which tions that need to be addressed in future studies. Because most
automaticity features it possesses and which it does not possess. existing studies have focused on IAT and affective priming effects,
One could, of course, pick out one feature as being the defining we limit our review to these two measures. For each of these two
feature. If this were done, it would mean that a process can be measures, we evaluate whether and in what way they meet the
called automatic or implicit if it can be demonstrated that the three normative criteria of implicit measures: the what criterion,
process can operate under that specific condition. Although this the how criterion, and the implicitness criterion. We do not aspire
approach is potentially useful, at present there is little agreement to describe or even refer to each individual IAT and affective
about what the central defining feature should be. Whereas some priming study that has been conducted in the past (see Klauer &
refer to a certain aspect of awareness (e.g., Greenwald & Banaji, Musch, 2003; Lane, Banaji, Nosek, & Greenwald, 2007; and
1995), others emphasize the lack of control (i.e., the lack of an Nosek, Banaji, & Greenwald, 2007, for more extensive reviews).
impact of goals relating to the process; e.g., Fazio & Olson, 2003). Instead, we summarize the main conclusions that can be reached
In the absence of any convincing arguments to select one of the on the basis of previous research and relate them to the normative
features as being central, the best solution seems to be to always criteria that we put forward in this article. For each conclusion, we
specify the feature or features one is referring to when calling a refer to only a subset of the relevant evidence or, when available,
process automatic or implicit. to papers that provide a review of the literature relevant for that
It is important to point out that evidence regarding the implicitness conclusion.
of a measure does provide some information about the nature of the
underlying psychological attribute (see also De Houwer, 2009). If the
measurement outcome is causally influenced by a psychological
IAT Effects
attribute under a certain set of conditions (e.g., when proximal The What Criterion: What Attributes Cause Variations in
goals are absent), one can conclude that the psychological attribute
IAT Effects?
can be activated and can influence behavior under those condi-
tions. There is, however, not necessarily a one-to-one mapping Associations in memory. In their seminal paper, Greenwald et
between observed measurement effects and the properties of the al. (1998) argued that IAT effects (i.e., the difference in perfor-
underlying psychological attribute. When a measure is not causally mance on the two tasks of an IAT procedure) reflect associations
influenced by the attribute under certain conditions, this could be between concepts (hence the name Implicit Association Test).
because the attribute is not activated under those conditions or Although it is not entirely clear what Greenwald et al. meant by the
because processes by which the attribute influences the measure do term association (see the exchange between Greenwald, Nosek,
not operate under those conditions (see also De Houwer, 2009). Banaji, & Klauer, 2005, and Rothermund, Wentura, & De Houwer,
Likewise, if a variable influences the magnitude of the measure- 2005), they did suggest that psychological attributes, such as
ment outcome, this could be due to its effect on the to-be-measured attitudes and stereotypes, are represented as associations in mem-
attribute or to its effect on other processes that influence the ory (e.g., Greenwald et al., 2002). Hence, the hypothesis that IAT
magnitude of the outcome (see Gawronski, Deutsch, LeBel, & effects reflect associations implies the hypothesis that variations in
Peters, 2008, for a detailed discussion of this issue). Finally, the IAT effects can be caused by psychological attributes such as
implicitness of a measure says little about how the underlying attitudes and stereotypes. Researchers have adopted three ap-
attribute is represented. For instance, it is difficult to determine proaches in examining whether IAT effects do capture attitudes
whether different attributes underlie implicit and explicit measures and stereotypes: experimental, semiexperimental, and correla-
or whether both measures reflect the same attributes under differ- tional. In this section, we present a brief overview of these three
ent conditions (see Nosek & Smyth, 2007; Payne, Burkley, & lines of research.
Stokes, 2008). Therefore, caution is needed when one draws con- The hypothesis that IAT effects are caused by the to-be-
clusions about the properties of psychological attributes on the measured attributes can be examined by experimentally manipu-
basis of empirical measurement results. lating those attributes and examining whether the manipulations
On the basis of these considerations, we can formulate a third influence the IAT effects in the expected manner. Relatively few
normative criterion that should be met before a measure can be studies have adopted this approach. Perhaps the strongest evidence
called an implicit measure: The to-be-measured attribute should for the validity of IAT effects as a measure of attitudes comes from
cause the measurement outcome automatically. This implicitness studies in which novel attitudes were created by pairing previously
criterion implies that (a) it is clearly specified which automaticity unknown stimuli with other, clearly positive or negative stimuli
features one is referring to and (b) there is empirical evidence to (e.g., Olson & Fazio, 2001). The results showed that IAT effects
back up the claim that the measure possesses those automaticity reflected these new attitudes, even when participants were unaware
features (see De Houwer, 2006; De Houwer & Moors, 2007). of the fact that the attitudes resulted from the stimulus pairings.
These results are particularly convincing, because it is difficult to
Implicit Measures: A Review see how the observed effects could have been caused by attributes
or processes other than the newly created attitudes.
Now that we know the normative criteria to which an implicit Other experimental studies focused on the malleability of pre-
measure should conform, we can examine whether each implicit existing attitudes and stereotypes. The results of these studies
measure that has been or will be proposed meets these criteria. showed that IAT effects are sensitive to manipulations of variables
Thus, our analysis provides a heuristic framework not only for past such as the experimental context and instructions (for a review, see
research but for future studies. In this section of the article, we Blair, 2002; Gawronski & Bodenhausen, 2006). For instance, IAT
352 DE HOUWER, TEIGE-MOCIGEMBA, SPRUYT, AND MOORS

measures of racial attitudes indicate that White participants are less A final set of studies used a correlational approach. We can
prejudiced against Black persons (a) when interacting with a Black divide correlational studies in two sets on the basis of the type of
experimenter than when interacting with a White experimenter criterion variable that was used. The first set of studies focused on
(e.g., Lowery, Hardin, & Sinclair, 2001); (b) after seeing movie predictive validity, in that the magnitude of IAT effects was used
clips of Black individuals in a positive compared with a negative as a predictor of a particular behavior thought to be indicative of
situational context (e.g., Wittenbrink, Judd, & Park, 2001); or (c) the to-be-measured attribute. The second set of studies assessed
after seeing pictures of admired Black individuals and disliked convergent validity by examining the relation between IAT effects
White individuals compared with pictures of disliked Blacks and and other measures of the to-be-measured attribute. Recent meta-
admired Whites (e.g., Dasgupta & Greenwald, 2001). In studies on analyses (Greenwald, Poehlman, Uhlmann, & Banaji, in press;
gender stereotypes, Blair, Ma, and Lenton (2001) showed that Hofmann, Gawronski, Gschwendner, Le, & Schmitt, 2005; also
gender stereotypes as measured by IAT effects were less pro- see Lane et al., 2007, and Nosek et al., 2007, for reviews) showed
nounced following counterstereotypic mental imagery but were that IAT effects do tend to correlate in a meaningful manner with
stronger following stereotypic mental imagery. Note, however, such criterion variables. The results of numerous correlational
that there is disagreement about whether these malleability effects studies are thus in line with the idea that IAT effects can capture
provide evidence for the validity of IAT scores. In some cases, the attitudes and stereotypes.
effects on the IAT scores could have been due not to changes in Because of the evidence reviewed above, it is now generally
the to-be-measured attribute but to changes in the extent to which accepted that variations in IAT effects are at least sometimes and
the attribute caused variations in the IAT score (De Houwer, 2009; at least to some extent caused by the attitudes or stereotypes that
Gawronski et al., 2008). For instance, the presence of a Black they were designed to measure. In those cases in which IAT effects
experimenter could lead to an increase in the extent to which can be interpreted as indices of attitudes and stereotypes, there are
participants try to control the outcome of the IAT (see Blair, 2002). restrictions on the manner in which they should be interpreted.
In future research, recently proposed componential accounts, such Most important, IAT effects at best allow for only relative con-
as diffusion model analysis (Klauer, Voss, Schmitz, & Teige- clusions, because they are determined by at least two attitudes or
Mocigemba, 2007) or the quad model (Conrey, Sherman, Gawron- stereotypes. For instance, a racial Black–White IAT effect is
ski, Hugenberg, & Groom, 2005), might help to identify the
determined not only by attitudes toward Black persons but also by
processes that underlie malleability effects.
attitudes toward White persons (Greenwald & Nosek, 2001). Blan-
Most claims about the validity of IAT effects are based on
ton and Jaccard (2006) have argued that IAT scores are also
semiexperimental and correlational data. The most straightforward
relative in that it is impossible to interpret the absolute value and
semiexperimental way of examining the validity of (implicit)
sign of an IAT score. An IAT effect does not reveal whether an
measures is by looking at whether variations in the type of stimuli
individual has positive or negative attitudes as such (e.g., whether
influence the measures in a meaningful manner. For instance, on
Person A likes White people more than Black people). This is so
the basis of normative studies and a priori arguments, one can
because it is not clear what psychological reality corresponds to a
postulate that most people have more favorable attitudes toward
zero score on an IAT. For instance, even if racial attitudes are a
flowers than toward insects. In line with the hypothesis that IAT
direct cause of scores on a racial IAT, it is not certain that a zero
effects can register attitudes, Greenwald et al. (1998) showed that
an IAT designed to measure the attitudes toward flowers and score on the racial IAT means that the person likes Black and
insects indeed revealed more positive attitudes toward flowers than White individuals to the same extent. Because of this fact, the IAT
toward insects. Another popular semiexperimental approach for effect shown by a particular individual can be interpreted only by
testing the validity of (implicit) measures is the so-called known- comparing it with the IAT effects of other persons (e.g., Person A
group approach (e.g., Banse, Seise, & Zerbes, 2001). It involves has more positive attitudes toward White persons or less positive
variations in the type of participants whose reactions are measured. attitudes toward Black persons than does Person B; but see Green-
For instance, one can argue on a priori grounds that White and wald, Nosek, & Sriram, 2006, for a critique of Blanton & Jaccard,
Black individuals should differ in their racial attitudes. 2006).
In support of the validity of the racial IAT, studies have con- There are at least two procedural factors that can bias IAT
firmed that White and Black individuals indeed show different scores and thus complicate the interpretation of the absolute value
racial IAT effects (Nosek, Banaji, & Greenwald, 2002). Although and sign of an IAT score. First, there is evidence suggesting that
results such as these suggest that attributes such as attitudes can IAT effects are determined not only by the attitudes and stereo-
cause variations in IAT effects, their conclusiveness is limited by types concerning the categories (e.g., “Black person,” “White
the semiexperimental design on which these studies are based. person”; De Houwer, 2001) but also by the individual stimuli used
When one divides stimuli or participants into groups on the basis to represent the categories (e.g., a particular Black or White face;
of one particular feature (e.g., valence or group membership), it is Bluemke & Friese, 2006; Govan & Williams, 2004; Mitchell et al.,
difficult to exclude the possibility that the groups differ also with 2003). Because it is often unclear which stimuli are most suitable
regard to other, correlated features. Hence, one cannot be entirely for measuring a particular attitude or stereotype, it is difficult to
confident that observed differences between the groups are due to correct for the impact of this factor. Second, IAT scores are known
the feature that the experimenter used to create the groups. Note, to depend on the order in which the two tasks of the IAT are
however, that the risk of unrecognized confounds also exists in presented. Although measures can be taken to reduce order effects
fully experimental studies. Furthermore, the risk of confounds in (see Nosek, Greenwald, & Banaji, 2005), it is difficult to deter-
semiexperimental studies can be reduced by carefully controlling mine the magnitude of the order effect for a particular individual
for plausible correlated features. and thus to correct the IAT score for these order effects.
 IMPLICIT MEASURES 353

In addition, one should be aware that, as is possible with Salience. A second alternative account was proposed by
virtually all measures of psychological attributes, variations in IAT Rothermund and Wentura (2004), who argued that IAT effects are
effects might be caused by attributes other than attitudes or ste- caused by salience asymmetries. The basic idea is that perfor-
reotypes. As we discuss earlier, such variation could limit the mance during an IAT will be fast when the categories assigned to
overall quality of the IAT as a measure of attitudes and stereo- the same key are similar with regard to their salience. Salience can
types. If IAT effects can be caused by different kinds of attributes be defined as the degree to which a stimulus pops out within a
or situational factors and if it is not clear which effects are caused background of other stimuli. For instance, in a racial IAT, one can
by which factor, the meaning of the effects becomes ambiguous argue that Black faces and negative words are more salient for
(see Fiedler, Messner, & Bluemke, 2006). In the following sec- White participants than are White faces and positive words. Hence,
tions, we examine which other attributes might cause variations in White participants should be faster when Black faces and negative
IAT effects. words are assigned to the same key (as is the case in the White-
Extrapersonal knowledge. It has been argued that variations in positive task) than when Black faces and positive words are
IAT effects can be caused by extrapersonal knowledge (i.e., assigned to the same key (as is the case in the Black-positive task).
knowledge that the individual has but regards as irrelevant for his Rothermund and Wentura reported experimental data (i.e., manip-
or her own responses to objects; see Gawronski, Peters, & LeBel, ulations of salience influence IAT effects), semiexperimental data
2008, for a conceptual analysis). For instance, assume that the (e.g., stimuli that differ only in salience lead to IAT effects), and
racial IAT effect of a particular individual suggests that this correlational data (i.e., IAT effects are related to measures of
individual has a more negative attitude toward Black persons than salience) in support of their hypothesis.
toward White persons. Researchers such as Karpinski and Hilton As was acknowledged by Greenwald et al. (2005), it is thus
(2001) and Olson and Fazio (2004) have argued that this effect beyond any doubt that at least some IAT effects are caused by
does not necessarily reflect the personal attitudes of the individual salience asymmetries (see Kinoshita & Peek-O’Leary, 2006, and
but rather the fact that the individual possesses knowledge of Houben & Wiers, 2006, for more recent evidence). There is still
societal views about Black and White persons. In Western societ- disagreement, however, about the pervasiveness of the impact of
ies, Black persons tend to be regarded in a less favorable manner salience asymmetries (see Rothermund et al., 2005). Also, at the
than are White persons. Even individuals who claim that these conceptual level, there is still uncertainty about how salience
societal views do not correspond with their personal views (e.g., should be measured (e.g., Greenwald et al., 2005) and how it is
Black individuals) might show racial IAT effects suggestive of related to other attributes, such as familiarity and polarity (e.g.,
pro-White attitudes simply because they possess knowledge of the Kinoshita & Peek-O’Leary, 2005, 2006; Proctor & Cho, 2006).
pro-White societal views. Similarity. On the basis of the findings of Lasaga and Garner
The hypothesis that IAT effects can be caused by extrapersonal, (1983), De Houwer et al. (2005) put forward the hypothesis that
societal views has been supported by experiments in which the IAT effects can be caused by similarity at the perceptual level.
manipulation of extrapersonal views led to changes in IAT effects Even before the introduction of the IAT, Lasaga and Garner
(e.g., Han, Olson, & Fazio, 2006; Karpinski & Hilton, 2001). demonstrated that the task of categorizing four stimuli by pressing
Further support came from semiexperimental studies. When one of two keys is easier when perceptually similar stimuli are
groups with diverging personal and societal views were tested, assigned to the same key than when perceptually dissimilar stimuli
IAT effects at least sometimes seemed to be in line with societal are assigned to the same key. Conceptually replicating this study,
views (see data on racial attitudes in Black persons and attitudes De Houwer et al. used four categories, each of which comprised
toward unhealthy but tasty foods such as candy; Olson & Fazio, several stimuli rather than four individual stimuli, and found the
2004; Spruyt, Hermans, De Houwer, Vandekerckhove, & Eelen, same effects (see also Mierke & Klauer, 2003, Experiment 1).
2007). Also, when the IAT procedure is changed in such a way that Because there is no reason to assume that the stimuli used by
it should be less susceptible to the impact of societal views (i.e., by Lasaga and Garner and by De Houwer et al. differed in any
removing error feedback and using more personalized category systematic manner other than with regard to their perceptual fea-
labels), the evidence for the causal role of societal views becomes tures, these studies do strongly suggest that similarity at the per-
weaker (Olson & Fazio, 2004). ceptual level can cause IAT effects.
On the other hand, doubts have been raised about the theoretical In most IAT procedures other than those of Lasaga and Garner
significance and validity of the extrapersonal account of IAT (1983) and De Houwer et al. (2005), stimuli are selected in such a
effects. At the conceptual level, it has been argued that the dis- way that the stimuli of the different categories do not differ
tinction between personal and societal views actually makes little perceptually. Hence, in all likelihood, perceptual similarity does
sense, especially when one considers the automatic effects of these not play a major role in most IAT studies. Nevertheless, the fact
views (Banaji, 2001; Gawronski & Bodenhausen, 2006; Gawron- that perceptual similarity can cause IAT effects led De Houwer et
ski et al., 2008; Nosek & Hansen, 2008a). At the empirical level, al. to formulate the hypothesis that all IAT effects are caused by
correlational studies provided little evidence for a link between some type of similarity. Stimuli and categories can be similar or
IAT effects and measures of societal views (Nosek & Hansen, dissimilar not only with regard to their perceptual features but with
2008a). Furthermore, Nosek and Hansen (2008b) recently obtained regard to other features, such as their (affective) meaning and their
evidence that raises serious doubts about the correct interpretation salience. From this perspective, attitudes and stereotypes can cause
of the experimental and semiexperimental data that were regarded IAT effects because they are a form of semantic similarity (re-
as evidence for the extrapersonal account (Han et al., 2006; Olson gardless of whether they are personally endorsed). Likewise, sa-
& Fazio, 2004). Few arguments remain to support the claim that lience asymmetries can drive IAT effects because they imply
IAT effects are causally influenced by extrapersonal views. similarity with regard to salience. The similarity hypothesis put
354 DE HOUWER, TEIGE-MOCIGEMBA, SPRUYT, AND MOORS

forward by De Houwer et al. therefore encompasses all previously similarity are most salient for an individual in a certain context.
discussed hypotheses and is compatible with the fact that multiple Now that we have discussed various attributes that could cause
attributes can cause IAT effects. Which attributes actually cause variations in IAT effects, we turn to the question of how attributes
the IAT effect should depend on the types of similarity that are can cause variations in IAT effects.
most salient in a given situation (see Medin, Goldstone, & Gent-
ner, 1993). On the negative side, because similarity is uncon- The How Criterion: By Which Processes Do Attributes
strained (everything is similar to everything else in some respect),
Cause Variations in IAT Effects?
the similarity account runs the risk of being unfalsifiable (but see
De Houwer et al., 2005, for a response to this criticism). Random walk model. Brendl, Markman, and Messner (2001)
Cognitive abilities. Finally, correlational studies suggest that introduced an informal (i.e., not mathematically formalized) ran-
IAT effects can be influenced by psychological attributes related to dom walk model in which responding during the IAT is a function
the general cognitive abilities of the individual. First, McFarland of (a) the rate at which evidence is accumulated for a particular
and Crouch (2002) observed a correlation between overall re- response and (b) the response criterion or threshold that the accu-
sponse speed and the magnitude of effects on a variety of IAT mulated evidence must reach before a response can be emitted. In
tasks. If it is assumed that overall response speed is determined to this model, IAT effects can thus be due to factors that influence the
a large extent by the cognitive abilities of the participant, the rate of evidence accumulation or the setting of the response crite-
results of McFarland and Crouch suggest that IAT effects are rion. We explain and evaluate these options consecutively.
determined at least in part by the cognitive ability of the partici- Let us return to the example of a racial IAT. For a participant
pant. Second, IAT effects tend to increase in magnitude when the who likes White persons, a White face not only belongs to the
age of the participants increases (e.g., Hummert, Garstka, O’Brien, category White but also is positively valenced. In the White-
Greenwald, & Mellott, 2002). Because cognitive abilities tend to positive task, both sources of evidence (i.e., the positive valence as
decline with age, this finding also suggests that IAT effects are well as membership in the category White) move the accumulation
determined by general cognitive abilities (see Sherman et al., 2008, process toward the correct response (i.e., the common response for
for evidence supporting this conclusion). Finally, several studies White faces and positive words). In contrast, in the Black-positive
(Back, Schmukle, & Egloff, 2005; McFarland & Crouch, 2002; task, the two sources of evidence move the accumulation process
Mierke & Klauer, 2003) showed that effects in IAT tasks correlate in opposite directions, because White faces and positive words are
even if the tasks are supposed to capture attributes that should not now mapped on different responses. For this reason, the net
be correlated. This finding can be explained if it is assumed that a evidence accumulation rate for White faces should be lower in the
general factor, such as cognitive ability, influences all IAT effects, Black-positive task than in the White-positive task, and this fact
regardless of the attributes they were designed to measure (but see should lead to slower responses in the former task than in the latter.
Klauer et al., 2007). It is important to note, though, that the impact The empirical evidence regarding the role of evidence accumu-
of cognitive abilities depends on how the IAT effects are calcu- lation is mixed. On the one hand, diffusion model analyses per-
lated. The correlations described above seem to be strongest when formed by Klauer et al. (2007) provided support for the hypothesis
the difference in the mean reaction time on the two tasks of an IAT that a process akin to evidence accumulation is one of the sources
is taken as the IAT effect but almost disappear when this differ- of IAT effects. On the other hand, it is unlikely that evidence
ence is standardized (i.e., when mean difference is divided by the accumulation operates according to the principles described by
standard deviation of all reaction times; see Back et al., 2005; Cai, Brendl et al. (2001). In their random walk model, evidence accu-
Sriram, Greenwald, & McFarland, 2004; Mierke & Klauer, 2003, mulation cannot lead to IAT effects for stimuli that are related to
for evidence, and Greenwald, Nosek, & Banaji, 2003, for a rec- only one of the categories, because such stimuli can influence
ommended way of calculating IAT effects). evidence accumulation in only one way. For instance, in a racial
Summary. The available evidence supports the hypothesis that IAT, positive words (e.g., flower) and negative words (e.g., can-
IAT effects at least sometimes and to some extent measure the cer) are typically unrelated to racial groups. Hence, evidence
attributes that they are supposed to measure (i.e., associations in accumulation for these stimuli should be determined only by their
memory such as those that underlie attitudes and stereotypes). valence and should be the same in the White-positive task and the
There is also evidence that they can reflect other attributes, such as Black-positive task. Contrary to this prediction, many studies,
salience, perceptual similarity, and general cognitive skills. The including those reported by Brendl et al. (2001), have shown that
fact that IAT effects can be caused by different types of attributes IAT effects can be observed for stimuli that are related to only one
does complicate the interpretation of these effects (e.g., Fiedler et category (see also Klauer et al., 2007).
al., 2006) and thus the overall quality of the measure (Borsboom et In the random walk model of Brendl et al. (2001), IAT effects
al., 2004). In part, this problem can be solved by learning more could also result from the fact that participants adhere to different
about the conditions under which the various kinds of attributes response criteria in the different tasks of an IAT. For instance, if
influence IAT effects. An important challenge for future research participants for some reason set the response criterion higher in the
is therefore to uncover the variables that determine when a partic- Black-positive task than in the White-positive task of the IAT, this
ular kind of attribute causes variation in IAT effects. We already would lead to slower responses in the former than in the latter task
know that the way in which IAT effects are calculated determines and thus to a racial IAT effect. One problem with this account is
the impact of general cognitive abilities (e.g., Mierke & Klauer, that it does not specify what determines the level of the response
2003). Future studies on this topic might find inspiration in the criterion that participants choose. Brendl et al. argued that partic-
similarity account put forward by De Houwer et al. (2005). Ac- ipants would raise the response criterion if they believed or per-
cording to this account, IAT effects will reflect whatever types of ceived that a task is difficult but did not specify how they would
 IMPLICIT MEASURES 355

arrive at this belief or perception. Is it because of associations in tive priming; see Wentura, 1999). Also, componential approaches,
memory, extrapersonal knowledge, salience asymmetries, or (per- such as the quad model (Conrey et al., 2005), could help isolate the
ceptual) similarity? At the empirical level, the available evidence impact of response activation.
suggests that shifts in the response criterion are at best only one As is the case with the random walk processes discussed in the
source of IAT effects. previous section, several attributes could cause variations in IAT
In an unpublished study, De Houwer (2000) added two catego- effects by means of the response activation mechanism. De Hou-
ries (numbers and nonwords) to a standard flower–insect IAT (see wer et al. (2005) argued that the response activation account fits
Greenwald et al., 1998). If IAT effects are due mainly to a change very well with the idea that IAT effects are driven by different
in response criterion, the effect of the response assignments for kinds of similarity. In fact, the concept “compatibility” can be
flower and insect items (flowers assigned to same key as positive regarded as synonymous with the concept “similarity.” Hence, it
words or insects assigned to same key as positive words) should be can be argued that stimuli activate responses to which they are
as big for the items of the additional categories as for the flower similar in a certain respect. Therefore, all attributes that can be
and insect items themselves. Results showed, however, that the regarded as a particular type of similarity (e.g., with regard to
IAT effect for the additional categories was only marginally sig- meaning, salience, or perceptual form) can cause IAT effects as the
nificant and was significantly smaller than that for the other items. result of the response activation mechanism. General cognitive
Finally, the diffusion model analyses reported by Klauer et al. abilities also could have an effect, because they determine how
(2007) identified shifts in the response criterion as one of several much impact the activated responses have on actual performance.
processes underlying IAT effects. In the next paragraphs, we Differential task switching model. During an IAT, participants
consider a number of additional processes. are instructed to pay attention to two stimulus dimensions in order
Response activation account. De Houwer (2001, 2003b) to categorize the stimuli. For instance, in a racial IAT, they are
pointed out that there are structural similarities between stimulus– asked to respond to faces on the basis of the racial group (Black or
response compatibility tasks, such as the well-known Stroop task White) and to words on the basis of valence (positive or negative).
(see MacLeod, 1991, for a review) and the IAT task. In both tasks, Because faces and words are presented in alternating order, par-
stimulus and response features are compatible on some trials and ticipants constantly need to switch between the tasks of responding
incompatible on other trials. Take the example of the racial IAT. If to the racial features of faces and responding to the valence of
participants are asked to press a first key for positive words and a words. Research on task switching has shown that performance
second key for negative words, the keys become associated with deteriorates as the result of switching between tasks (e.g., Meiran,
positive and negative valence, respectively. In other words, press- Chorev, & Sapir, 2000).
ing the first key becomes a positive response (equivalent to saying Mierke and Klauer (2001, 2003) pointed out that the need to
“good”) and pressing the second key becomes a negative response switch between different tasks depends on which categories are
(equivalent to saying “bad”; see Eder & Rothermund, 2008, for assigned to the same response. Again, take the example of the
evidence supporting this assumption). Hence, for participants racial IAT. When participants who like White persons and dislike
who like White persons but dislike Black persons, stimuli and Black persons are asked to press the first key for White faces and
responses are compatible (in the sense of associated with the positive words and the second key for Black faces and negative
same valence) when they are asked to press the first (positive) key words (White-positive task), they can capitalize on response syn-
for White faces and to press the second (negative) key for Black ergy and simply respond to both faces and words on the basis of
faces (as is the case in the White-positive task). When the same whether they like the presented face or word. Because there is a
participants are asked to press the first (positive) key for Black faces perfect confound between the valence of the faces and the racial
and the second (negative) key for White faces (as is the case in the category of the faces, responding to a face on the basis of its
Black-positive task), the stimuli and responses are always incom- valence or on the basis of its racial group leads to the same
patible. From research on stimulus–response compatibility effects, response. In contrast, when the same individuals are to press the
we know that performance is better when stimuli and responses are first key for Black faces and positive words and the second key for
compatible than when they are incompatible. There is strong evi- White faces and negative words (Black-positive task), they must
dence that such effects are due to processes at the response selection pay attention to the racial feature of the faces, because responding
stage whereby elements of the stimulus activate the incorrect (in case on the basis of the faces’ valence would lead to incorrect re-
of incompatible combinations) or correct (in case of compatible sponses. In the Black-positive task, accurate responding therefore
combinations) response alternative. Because stimulus–response requires task switching. Because task switching leads to perfor-
compatibility varies between the different blocks of an IAT, mance costs (e.g., Rogers & Monsell, 1995), performance will be
De Houwer (2001, 2003b) put forward the hypothesis that IAT less good in the Black-positive task than in the White-positive task.
effects are due to the activation of responses by (relevant or Klauer and colleagues (Klauer & Mierke, 2005; Mierke &
irrelevant features of) the presented stimuli. Klauer, 2001, 2003; see also Klauer et al., 2007) provided strong
Unfortunately, there have been few if any direct tests of this evidence in support of the task switching model of IAT effects.
hypothesis. De Houwer (2001) examined whether IAT effects First, participants who are generally good in switching between tasks
reflect the properties of the individual stimuli or the categories to should generally be less affected by whether the response assignments
which those stimuli belong but did not test whether these effects force them to switch between tasks. Hence, regardless of the attribute
were due to processes at the response selection stage. Neverthe- that an IAT is supposed to measure, these participants should reveal
less, the hypothesis could be tested by using strategies that have smaller IAT effects than do participants who are poor in switching
been applied to demonstrate the role of response selection pro- between tasks. In support of this idea, Mierke and Klauer (2001,
cesses in other stimulus–response compatibility tasks (e.g., nega- 2003) found that effects on different IATs are correlated even when
356 DE HOUWER, TEIGE-MOCIGEMBA, SPRUYT, AND MOORS

those IATs were designed to measure different attributes that should change from trial to trial rather than remain fixed during an entire
not be correlated (e.g., political attitudes and attitudes toward flowers block of trials. Given that recoding processes rely on a consistent
and insects; see Klauer et al., 2007). assignment of categories to response keys over trials (Strayer &
More direct evidence comes from sequential analyses of perfor- Kramer, 1994), Teige-Mocigemba et al. (2008) hypothesized that
mance during the IAT tasks. Because switching between tasks is such a change should impede any kind of strategic recoding and
associated with performance costs, reaction times on any given indeed found evidence for this assumption.
trial should be longer when another dimension was relevant on the Another way of preventing strategic recoding is by avoiding a
previous trial (switch trials) than when the same dimension was perfect confound between stimulus features. For instance, De Houwer
relevant (repetition trials). These differences in reaction times are (2001) presented names of British and non-British (foreign) persons
called task switching costs. In a racial IAT, for instance, responses to British participants. It is important that half of the British and half
on a trial with a face stimulus should be slower when it is preceded of the foreign persons were liked by the participants (e.g., Princess
by a trial with a word than when it is preceded by a trial with Diana, Mahatma Gandhi), whereas the other persons were disliked
another face. If the need for task switching depends on which (Margaret Thatcher, Adolf Hitler). It was unlikely that participants
categories are assigned to the same key, the task switching costs would intentionally decide to respond on the basis of the valence of
should be a function of the category–response assignments. This is the names rather than their category (British or foreign), because in
exactly what Mierke and Klauer (2001, 2003) observed. To ex- half of the cases this approach would have led to an incorrect re-
trapolate their findings to a racial IAT that is completed by sponse. In most IATs, however, there is a perfect confound between
participants who like White persons and dislike Black persons, one valence and category membership (e.g., for a particular person, in a
would expect task switching costs to be smaller in the White- racial IAT, all White faces will be more positive than all Black faces).
positive task than in the Black-positive task. Therefore, one should be aware that participants usually can strategi-
In another set of studies, Klauer and Mierke (2005) found afteref- cally recode standard IAT tasks.
fects indicative of active task switching during the IAT. Let us again Regardless of the exact nature of the processes that underlie
take the example of the racial IAT. When participants who like White differential task switching costs in the IAT, these processes could
persons but dislike Black persons complete the Black-positive task of be responsible for the impact of a variety of attributes on IAT
the racial IAT, they should pay attention to the valence of the words performance. As pointed out by Mierke and Klauer (2001, 2003),
but should avoid paying attention to the valence of the faces. This is participants (intentionally or unintentionally) exploit similarities
so because categorizing stimuli according to valence leads to the between stimuli in an attempt to facilitate task switching in certain
correct response only for words and to the incorrect response for blocks of an IAT. These similarities could be related not only to
faces. On the basis of earlier findings, Klauer and Mierke predicted attitudes or other associations in memory but to salience or per-
that the repeated act of avoiding the evaluation of stimuli should carry ceptual features of the items. Because task switching depends
over to a subsequent task in which the same stimuli had to be heavily on the cognitive abilities of the participant, interindividual
evaluated as being good or bad. In line with this prediction, they found differences in these abilities also should have an important impact
that participants evaluated stimuli (e.g., Black and White faces) more on IAT effects.
slowly following an IAT task in which valence of those stimuli had to Summary. Several proposals have been put forward about the
be ignored (e.g., the Black-positive task for participants who like processes by which attributes can cause variations in IAT effects.
White persons more than Black persons) than after an IAT task in Nevertheless, compared with the number of studies on the relation
which the valence of those stimuli could be used to categorize stimuli between IAT effects and criterion variables (see Greenwald et al., in
fast and correctly (e.g., the White-positive task for those persons). press, and Hofmann et al., 2005, for reviews), relatively little research
Although these data provide strong evidence for the hypothesis has directly examined the role of each of these processes. The avail-
that IAT effects at least in part are due to differential task switch- able evidence provides the strongest support for the involvement of
ing costs, it remains unclear to what extent the differential costs task switching processes, but the exact nature of these processes still
result from a conscious strategy or from automatic processes. In needs to be determined. Moreover, task switching appears to be just
principle, participants may consciously decide to recode certain one of the mechanisms that produce IAT effects (see Klauer et al.,
tasks in the IAT (see De Houwer, 2003a; Rothermund & Wentura, 2007). Hence, there is a clear need for more research on how IAT
2004; Wentura & Rothermund, 2007). In the case of the racial effects come about. This research also can help clarify which at-
IAT, for example, such recoding would involve a conscious inten- tributes influence IAT performance under which conditions. Note,
tion to categorize both faces and words on the basis of valence however, that the what and how criteria do not overlap completely,
when one realizes that such a strategy results in correct (and fast) because one attribute could exert an effect through various processes
responses (e.g., in the White-positive task for people who like and one process could support the effect of various attributes (see
White persons and dislike Black persons). Such a strategic recod- De Houwer et al., 2005).
ing would imply that IAT performance is driven to some extent by
the consciously intentional evaluation of the stimuli. This impli- The Implicitness Criterion: In What Sense Do IAT Effects
cation would raise doubts about whether IAT effects actually Provide an Implicit Measure of Attributes?
provide an implicit measure of attitudes (see below).
Strategic recoding might be prevented in two ways. First, the Above, we argue that the implicitness of a measure refers to the
IAT’s block structure can be eliminated (as recently proposed by conditions under which a psychological attribute causes variations
Rothermund, Teige-Mocigemba, Gast, & Wentura, in press;Teige- in the measure (and thus the conditions under which the measure
Mocigemba, Klauer, & Rothermund, 2008). In the so-called single reflects the psychological attribute). A measure can be called an
block IAT, the assignment of the categories to the responses can implicit measure of a psychological attribute if it is caused by that
 IMPLICIT MEASURES 357

attribute even under conditions that are typically associated with impossible to demonstrate that a process is entirely goal indepen-
automatic processes. In line with Moors and De Houwer (2006; see dent. The best one can do is demonstrate that the process does not
also De Houwer & Moors, 2007), we focus on conditions involv- depend on particular (distal) goals and make those goals explicit
ing the presence of proximal and distal goals, awareness, process- when describing the process as goal independent. Possible distal
ing resources, and time. goals that could be relevant for IAT effects are the goal to respond
The presence of proximal goals. A proximal goal is a goal quickly to stimuli and the goal to make few errors. Apart from
related to the process under study. Proximal goals include the goal preliminary data of Popa-Roch (2008) showing that response-time-
to engage in, stop, alter, or avoid the operation of a process. Hence, based IAT effects decrease in magnitude when the goal to avoid
a process can be automatic in that it operates independently of the errors is removed, we do not know of any studies that examined
proximal goal to engage in, stop, alter, or avoid the operation of whether IAT effects depend on the presence of distal goals.
that process. Processes that operate under those conditions can be The presence of awareness. Although the term implicit is often
called unintentional (in the case of the goal to engage in), uncon- seen as being virtually synonymous with the term unaware (e.g.,
trolled (with regard to the goal to stop, alter, or avoid), or auton- Greenwald & Banaji, 1995), it is rarely made explicit what it
omous (when such processes are independent of all proximal means to say that a measure is unaware. It is important to realize
goals; see De Houwer & Moors, 2007; Moors & De Houwer, that describing a measure as unaware can mean several things
2006). In the case of implicit measures, the processes under study (Bargh, 1992; De Houwer & Moors, 2007). It could point to the
are those by which an attribute of the person causes variations in fact that the to-be-measured attribute causes the IAT effect even
the measure. Hence, the question of whether IAT effects are when participants are unaware of (a) the stimuli that activate the
implicit in the sense of unintentional, uncontrolled, or autonomous attribute (e.g., the attitude object that is presented during the task);
boils down to the question of whether the processes by which the (b) the origins of the attribute itself (e.g., the fact that participants
to-be-measured psychological attribute causes IAT effects operate possess a certain attitude or how they acquired the attitude); (c) the
independently of the goal to engage in, stop, alter, or avoid these fact that the attribute influences performance (e.g., that the out-
processes. In other words, does the attribute still cause IAT effects come reflects a certain attitude); or (d) the manner in which the
(i.e., is the measure still valid) even when the participants (a) do attribute influences performance (e.g., that certain category–
not have the goal to express the attribute in IAT effects, (b) have response assignments lead to better performance than do other
the goal to stop the expression of the attribute in IAT effects, (c) category–response assignments).
have the goal to alter the way in which the attribute is expressed in Can IAT effects actually be unaware in one or more of these
IAT effects, or (d) have the goal to avoid the expression of the four ways? First, IAT effects are obtained by instructing partici-
attribute in IAT effects? pants to categorize the relevant stimuli in certain ways. Therefore,
To the best of our knowledge, only the last two issues have been participants must be aware of the four categories and the stimuli
addressed in research. Whether IAT effects depend on the con- that are presented as instances of these categories. Second, there is
scious goal to alter or avoid the expression of an attribute has been some evidence that IAT effects can register attitudes even when
examined in studies on faking. The results of these studies have participants do not know the origin of those attitudes. As men-
been mixed. Some showed that IAT effects were largely unaf- tioned above, Olson and Fazio (2001) created new attitudes by
fected by instructions to fake a certain attitude (e.g., Asendorpf et pairing neutral objects with liked or disliked objects and found that
al., 2002; Banse et al., 2001; Egloff & Schmukle, 2002; Kim, the IAT could register these attitudes even though participants
2003), whereas others suggested that participants can intentionally were not aware of how the attitudes were created. Note that the
influence IAT effects (e.g., De Houwer, Beckers, & Moors, 2007; participants could be made aware of the attitudes themselves
Fiedler & Bluemke, 2005; Steffens, 2004). The extent to which because they could express these attitudes when asked to do so.
IAT effects can be consciously controlled seems to depend on a Hence, the studies of Olson and Fazio do not demonstrate that IAT
variety of variables, such as how much experience the participants effects can capture unaware attitudes in the sense of attitudes that
have with the IAT (e.g., Fiedler & Bluemke, 2005; Steffens, 2004; participants do not know they possess. Another observation that
see Czellar, 2006; De Houwer et al., 2007; and Schnabel, Banse, might be relevant in this context is that participants are sometimes
& Asendorpf, 2006, for other moderating variables). Hence, the poor in predicting their IAT performance and express surprise
available evidence does not allow for the strong conclusion that when informed about the meaning of their score on certain IATs
IAT effects are implicit in the sense of being always independent (e.g., Mitchell et al., 2003; Nosek, Greenwald, & Banaji, 2007).
of the goals to avoid or alter the expression of the to-be-measured However, it is unclear whether this observation means that the IAT
attribute. Nevertheless, it does seem to be the case that IAT effects picks up attitudes of which the participants are unaware or whether
are more difficult to control than are most traditional (question- the IAT effect reflects other attributes such as extrapersonal
naire) measures (e.g., Steffens, 2004). In this sense, IAT effects knowledge or salience asymmetries. Therefore, at present, there is
can be described as less controllable and thus more implicit than no strong evidence to support the conclusion that IAT effects can
many other measures. Also, the fact that IAT effects can be register attributes of which participants are unaware.
controlled when participants are encouraged to do so does not There is one published study that is relevant for the third and
imply that participants do try to control IAT effects when they do fourth ways in which IAT effects can be considered as unaware.
not receive instructions to do so. Monteith et al. (2001) interviewed White participants about their
The presence of distal goals. Distal goals are goals other than experiences with a racial IAT. Up to 64% of the participants
those related to the process under study. A process can be called noticed that they were faster in the White-positive task than in the
goal independent when its operation does not depend on any Black-positive task. Of the participants who noticed that they were
(proximal or distal) goal. It should be clear that it is difficult if not faster in the White-positive task, 37% attributed this slower per-
358 DE HOUWER, TEIGE-MOCIGEMBA, SPRUYT, AND MOORS

formance to the fact that they apparently had a more negative aware of the fact that the IAT aims to capture the to-be-measured
attitude toward Black persons than toward White persons. These attribute (e.g., racial attitudes) and how it does so (e.g., the differ-
findings were confirmed in two recent unpublished studies show- ence in performance on the White-positive and Black-positive
ing that more than 80% of the participants who took part in a racial tasks of a racial IAT). Our review indicates that the question of
IAT could correctly describe the aim of the IAT (De Houwer & whether IAT effects are actually implicit in some sense of the word
Moors, 2006; Popa-Roch, 2008, p. 118). De Houwer and Moors has largely been neglected in past research. Only the impact of the
moreover found that the percentage of participants who were goals to avoid or alter the expression of attributes has been exam-
aware of the aim of the IAT was twice as large for a racial IAT ined in some detail in studies on faking. Other features of auto-
(80%) as for an IAT designed to measure attitudes toward political maticity (and thus of implicitness) have not been addressed at all
parties (40%). Together, these results strongly suggest that a or have been examined in only a handful of studies. It should be
substantial part of the participants are aware of what IATs are noted that the fact that IAT effects can predict variance in criterion
supposed to measure and have a basic understanding of how IAT variables that cannot be explained on the basis of traditional
effects measure attributes. Hence, IAT effects typically do not (explicit) measures (e.g., see Asendorpf et al., 2002; Hofmann,
seem to be unaware in this sense. Rauch, & Gawronski, 2007) does not provide evidence for the
Many issues remain to be examined. For instance, it is not clear implicitness of the effects. It is not clear whether this incremental
why the percentage of participants who are aware of the aim of an predictive validity is due to the implicit nature of the IAT effects
IAT depends on the categories featured in the IAT (De Houwer & or to the many other differences between IAT effects and tradi-
Moors, 2006). It is also unclear whether awareness of the purpose tional measures.
of an IAT affects the magnitude or predictive validity of the IAT
effects.
Affective Priming Effects
The presence of processing resources. An important feature of
automaticity (and thus of implicitness) is whether a process can The What Criterion: What Attributes Cause Variations in
operate even when processing resources are scarce. This feature is Affective Priming Effects?
examined most often by asking participants to perform a primary
task that depends on the process under study while they perform a Attitudes. It is generally assumed that affective priming effects
secondary task that deploys the available processing resources to a reflect the attitudes that participants have toward the object repre-
certain extent. A process is said to be efficient when the degree of sented by the prime stimuli. For instance, attitudes toward Black
load imposed by the secondary task does not impact on perfor- persons can be estimated by examining the extent to which stimuli
mance on the primary task (Moors & De Houwer, 2006). We know representing Black persons (e.g., photographs of the faces of Black
of only two studies in which the effect of mental load on IAT persons or names typical of Black persons) facilitate responding to
effects was examined. Devine, Plant, Amodio, Harmon-Jones, and positive versus negative targets.1 Whereas the relevant categories
Vance (2002, Study 3) failed to find an effect of a secondary task are made explicit in IAT studies, in affective priming studies, the
on IAT effects. In an unpublished study, Schmitz, Teige, Voss, and categories that the prime stimuli are meant to instantiate are
Klauer (2005) found that an increase in working memory load led typically not made explicit in the instructions. Studies by Olson
to an increase in the magnitude of IAT effects but did not influence and Fazio (2003; see also De Houwer, 2001, 2003a) suggest that,
external correlations with self-reported attitudes. Hence, these because of this, affective priming effects are determined primarily
initial results suggest that the translation of individual attitudes in by the attitudes toward the individual stimuli rather than by the
IAT scores is efficient. However, more research is needed before attitude toward the category of which they are exemplars. The
firm conclusions can be drawn. impact of the category, however, can be amplified by directing
The availability of time. Moors and De Houwer (2006) pointed attention to the category (Olson & Fazio, 2003).
out that the minimal time needed for a process to operate is a The hypothesis that affective priming effects can be caused by
central feature in the concept of automaticity both in its own right attitudes is supported by experimental, semiexperimental, and cor-
and because it can determine several other features. For instance, relational studies. Many studies have confirmed that affective
processes that require very little time to run to completion are most priming effects can pick up novel attitudes that have been created
often difficult to control. In extreme cases, the process might occur by pairing neutral stimuli with other, liked or disliked, stimuli
so quickly that participants cannot become aware of the process or (e.g., De Houwer, Hermans, & Eelen, 1998; see Hermans, Baey-
its input. With regard to the IAT, the impact of time on the validity ens, & Eelen, 2003, for a review), even when participants do not
of IAT effects could be examined by limiting the time that partic- appear to be aware of how the attitudes were acquired (Olson &
ipants have available for responding to each stimulus. As far as we Fazio, 2002). Experiments on the malleability of affective priming
know, such studies have yet to be conducted. effects have shown that these effects can be influenced by a range
Summary. All in all, there is relatively little research about the of variables, such as the nature of the experimental context and
claim that IAT effects provide a measure of psychological at-
tributes that can be qualified as implicit. Although participants 1
Priming procedures have been used to examine attributes other than
seem to have less control over the IAT effects than over many
attitudes (e.g., Wittenbrink, Judd, & Park, 1997; see Wittenbrink, 2007, for
other, more traditional measures, several studies indicate that IAT a review). However, in these procedures, the targets differ not with regard
effects can at least sometimes and to a certain extent be controlled to their affective meaning but with regard to nonaffective, semantic fea-
in a conscious manner. There is evidence showing that IAT effects tures. For instance, to examine the stereotype that women are more likely
are unaware in that they can capture attitudes whose origins are to study art than math, one can present faces of women as primes and ask
unknown, but other studies have demonstrated that participants are participants to decide whether a target word refers to art or math.
 IMPLICIT MEASURES 359

instructions (see Blair, 2002, for a review). As with studies on the (as is the case in affective priming effects) but by nonevaluative
malleability of IAT effects, however, these results provide evi- features, such as semantic meaning (i.e., semantic priming; Lucas,
dence for validity only if it can be demonstrated that the results are 2000), co-occurrence associations (i.e., associative priming; Rat-
caused by changes in the to-be-measured attitudes. cliff, 1988), and even perceptual similarity (e.g., Pecher, Zeelen-
Many semiexperimental studies have shown that stimuli to berg, & Raaijmakers, 1998). Although we do not know any study
which participants should have different attitudes indeed evoke that has examined priming on the basis of the similarity between
different affective priming effects (see Fazio, 2001, and Klauer & the salience of the prime and the target, it seems reasonable to
Musch, 2003, for a review). On the other hand, there are few assume that salience-based priming effects can be observed.
affective priming studies in which the semiexperimental known- It is important to note the fact that priming effects can be based
group approach was adopted. One of these is a study of Otten and on a large variety of attributes does not threaten the claim that
Wentura (1999) in which an affective priming measure of attitudes affective priming effects can be based on attitudes toward the
toward groups revealed that participants preferred the group to primes. In many affective priming studies, a large variety of
which they were (randomly) assigned. primes and targets was used, so that it is unlikely that the evalu-
Finally, affective priming effects have been found to correlate in ative features of the stimuli were confounded with other, noneva-
an expected manner with several kinds of criterion variables, such luative features. Also, affective priming effects have been observed
as real-life behaviors (e.g., Fazio et al., 1995; Spalding & Hardin, even in studies that controlled for a large variety of nonevaluative
1999) and other measures of the attitudes under study (e.g., Deg- features (i.e., semantic meaning, associative links, perceptual similar-
ner, Wentura, Gniewosz, & Noack, 2007; Dunton & Fazio, 1997; ity, familiarity; see Hermans, Smeesters, De Houwer, & Eelen, 2002).
Frings & Wentura, 2003; Spruyt, Hermans, De Houwer, However, when one uses affective priming as a tool for assessing
Vandekerckhove, et al., 2007; Wentura, Kulfanek, & Greve, real-life attitudes, there is often less opportunity to control for
2005). It should be noted, however, that correlations between nonevaluative features of the primes and targets. For instance,
affective priming effects and criterion variables are sometimes when affective priming effects are used to measure racial attitudes,
small or even absent (e.g., Banse, 1999, 2001; Bosson, Swann, & it is possible that, at least for some individuals, Black faces and
Pennebaker, 2000). In part, this fact seems to be related to the negative words are more similar than are Black faces and positive
on-average-limited reliability of affective priming scores. That is, words not only with regard to their valence but also with regard to
repeated administrations (split half or test–retest) of the same their salience. Hence, it is possible that affective priming effects
affective priming measure tend to correlate only to a limited extent for Black faces (e.g., faster responses to negative words preceded
or do not correlate at all (e.g., Banse, 1999, 2001; Bosson et al., by a Black face than to positive words preceded by a Black face)
2000; but see Cunningham, Preacher, & Banaji, 2001). This low do not reflect negative attitudes toward Black persons but the fact
reliability could in part be due to the fact that the relevant category that Black persons are more salient for the participant. It is sur-
(i.e., the attitude object that is being examined) is typically not prising that such risks to the validity of affective priming effects as
made explicit (see Olson & Fazio, 2003; De Houwer, 2009). There a measure of real-life attitudes are rarely acknowledged and have
is accordingly little control over whether or how participants not yet been studied.
process and categorize the prime stimuli, and this lack of control Summary. The claim that affective priming effects can capture
probably results in a large amount of error variance. attitudes is supported mainly by the results of experimental and
Other attributes. Very few studies have examined whether semiexperimental studies with stimuli that evoke different atti-
attributes other than attitudes can cause variations in affective tudes. Evidence from known-group and correlational studies is
priming effects. There is some evidence that affective priming somewhat limited. One should keep in mind that priming effects
effects are less sensitive to extrapersonal knowledge than are IAT can be based not only on evaluative features of the stimuli but on
effects. For instance, Han et al. (2006) showed that an experimen- a range of other features that might sometimes be confounded with
tal manipulation of extrapersonal knowledge had an effect on a evaluative features. This possibility poses a risk to the validity of
traditional IAT measure but did not have one on an affective affective priming effects as a measure of attitudes and should
priming measure or on a personalized IAT measure that was receive more attention in future research.
designed to minimize the impact of extrapersonal views. More
indirect evidence comes from the observation that fewer (White The How Criterion: By Which Processes Do Attitudes
and Black) participants appear to prefer White persons over Black
Cause Variations in Affective Priming Effects?
persons when racial attitudes are assessed by affective priming
effects rather than by a standard racial IAT (Olson & Fazio, 2004; Spreading of activation. The first account of affective (and
see also Spruyt, Hermans, De Houwer, Vandekerckhove, et al., other) priming effects was formulated in terms of activation
2007), but this finding could be related to the lower reliability of spreading through a semantic network (Collins & Loftus, 1975;
the priming measure. Collins & Quillian, 1969). In the network, each concept is repre-
As far as we know, there is little if any evidence regarding the sented by a node. If two concepts are somehow similar in meaning
impact of salience, similarity, or cognitive abilities on affective (e.g., if they share a valence), the nodes representing these con-
priming effects. There is some evidence to suggest that affective cepts are linked by an association through which activation can
priming effects become stronger when the salience of the primes spread. Hence, if a prime stimulus is presented, this will activate
increases (e.g., Klauer, Mierke, & Musch, 2003). Also, it has long not only the corresponding concept node but all other nodes with
been known that priming effects in general (i.e., differences in which it is connected. Assuming that the speed of responding to a
responding to targets as a function of the nature of primes) can be target stimulus depends on the activation level of the concept node
driven not only by the evaluative features of the primes and targets representing the target, a prime stimulus that is affectively related
360 DE HOUWER, TEIGE-MOCIGEMBA, SPRUYT, AND MOORS

to the target stimulus could speed up responding to the target by the important finding that affective priming effects occur in the
preactivating the concept representation of the target in memory evaluation task but not in a semantic categorization task
(see Fazio, 2001, 2007). This spreading-of-activation account of (e.g., De Houwer et al., 2002; see above) is compatible with the
priming effects has dominated thinking about priming so much fact that the valence of the positive and negative primes can induce
that the term priming is often used to refer not to the priming effect a tendency to give positive and negative responses but not a
(i.e., faster responses when targets are presented in the context of tendency to give semantic categorization responses. Other strong
a related prime) but to the process of preactivating representations evidence comes from Wentura (1999), who observed very specific
in memory as the result of spreading of activation. aftereffects of affective priming trials with an incompatible prime
Despite the popularity of this account, research suggests that and target. Responses on the trial after such an incompatible trial
spreading of activation plays at best only a minor role in the were slower when the valence of the correct response corre-
production of affective priming effects. Most important, a sponded to the valence of the incompatible prime on the previous
spreading-of-activation account leads to the prediction that primes trial. This negative priming effect can be explained in the follow-
should facilitate not only the evaluation of affectively related ing manner: When the prime and target differ in valence, the
targets (i.e., responses based on the valence of the targets) but the incorrect response that is activated by the prime needs to be
processing of other (semantic) features of the targets. For instance, inhibited before the correct response can be selected. This inhibi-
if a prime preactivates the concept node of an affectively related tion carries over to the next trial and makes it harder to emit the
target, this preactivation should reduce the time needed to deter- previously inhibited response.
mine the semantic category of the target (e.g., animal or object). What implications does the response activation account have for
Several studies have failed to confirm this prediction. For in- hypotheses about the kinds of attributes that cause variations in
stance, De Houwer, Hermans, Rothermund, and Wentura (2002) affective priming effects? As we have discussed in the context of
failed to find affective priming of semantic categorization re- the response activation account of IAT effects, it is generally
sponses (i.e., does the target refer to an object or a person) but did assumed that stimuli activate those responses to which they are
find strong affective priming of evaluation responses (i.e., is the similar in some respect (e.g., Kornblum & Lee, 1995). Hence, the
target positive or negative), even though the same stimuli were response activation account of affective priming is compatible
presented in the same way in both tasks. More recent studies did with the observation that priming effects can be induced by sim-
find affective priming of semantic categorization responses and ilarity not only with regard to valence but with regard to noneva-
other nonevaluative responses (e.g., naming, lexical decision) but luative features, such as semantic meaning and salience. Because
only under certain conditions (e.g., De Houwer, Hermans, & the impact of response conflicts on performance depends on the
Spruyt, 2001; Spruyt, De Houwer, Hermans, & Eelen, 2007; cognitive abilities of the participants to deal with the conflicts
Spruyt, Hermans, De Houwer, & Eelen, 2002; Wentura, 2000). (e.g., Kane & Engle, 2003), one can predict on the basis of the
Nevertheless, the consensus remains that processes akin to spread- response activation account that the cognitive abilities of partici-
ing of activation play little or no role in standard affective priming pants will determine the magnitude of affective priming effects
tasks (i.e., tasks in which participants are asked to evaluate the (see Klauer & Teige-Mocigemba, 2007, for evidence related to this
targets; see Klauer & Musch, 2003, for a more detailed review of prediction).
the evidence supporting this conclusion). Summary. The available evidence allows for the conclusion
Response activation. The available evidence strongly supports that standard affective priming effects (i.e., those observed in tasks
the hypothesis that affective priming effects in the evaluation task in which participants are asked to evaluate the targets) are due
(i.e., is the target positive or negative) are due to the fact that the mainly to response activation processes. Priming effects by means
prime stimuli activate responses on the basis of their valence. of this mechanism can be caused not only by attitudes but by other
Consider trials on which a positive target (e.g., the word healthy) attributes, including semantic meaning, salience, and cognitive
is presented. Because the target is positive, participants need to abilities.
give a positive response (e.g., say “good”). When the target is
preceded by a positive prime (e.g., a White face for a person who The Implicitness Criterion: In What Sense Do
likes White individuals), the positive valence of the prime will Affective Priming Effects Provide an Implicit Measure
induce a tendency to give a positive response (e.g., say “good”)
of Attributes?
and will thereby facilitate the selection of the positive response
that needs to be given to the target. When the prime is negative The presence of proximal goals. In one of the early studies on
(e.g., a Black face for someone who dislikes Black individuals) affective priming, Hermans, De Houwer, and Eelen (1994, Exper-
and the target is positive, the prime will induce a tendency to give iment 1) observed significant affective priming effects even
a negative response and will thereby slow the selection of the though participants were instructed to ignore the prime stimuli.
correct (positive) response. The response activation account of This result suggested that the effects can occur in the presence of
affective priming thus implies that the prime influences the re- the goal to avoid an impact of the primes on performance (see also
sponse selection process, whereas a spreading-of-activation ac- Klauer & Musch, 2003). In more recent studies, Teige-Mocigemba
count implies that the prime influences the processing of the target and Klauer (2008; also see Klauer & Teige-Mocigemba, 2007) did
itself. find evidence that participants can consciously control affective
Many studies have found evidence for the assumption that priming effects. In some conditions, participants were promised an
affective priming effects arise at the response selection stage (see extra monetary reward for fast and accurate responses to targets
Klauer & Musch, 2003, for an extensive review, and Klauer, following specific primes. In other conditions, participants were
Musch, & Eder, 2005, for a more recent discussion). For instance, explicitly instructed to fake certain attitudes. The affective priming
 IMPLICIT MEASURES 361

effects that were targeted by these instructions were found to be the primes influences their responses (see Wittenbrink, 2007). It
eliminated. The findings are remarkable, because the stimulus remains to be seen, however, whether participants are aware of the
onset asynchrony (SOA) between prime and target was short (275 impact of the primes when the primes are presented supraliminally.
ms) and responses had to be emitted within a window of 800 ms. The presence of processing resources. Hermans, Crombez,
Degner (in press) also found evidence for successful control of and Eelen (2000) asked participants to perform an affective prim-
affective priming effects but could eliminate control by imposing ing task while they recited a series of digits. They found that the
a response deadline of 600 ms. Although it is now clear that magnitude of the affective priming effect was unaffected by the
participants can consciously control affective priming effects, degree of mental load imposed by the secondary task. This finding
more research is needed about the conditions under which control suggests that the translation of the attitude in the priming effect
is possible. is relatively independent of available processing resources and is
The presence of distal goals. In a standard affective priming thus efficient. Klauer and Teige-Mocigemba (2007) replicated this
task, participants are asked to evaluate the targets as good or bad; finding for participants who had larger-than-average working
doing so requires them to adopt the goal to evaluate stimuli. This memory capacity (as measured on memory span tasks). For par-
goal is distal in that it does not refer to the processes by which the ticipants who had smaller-than-average working memory capacity,
attitude toward the prime causes variations in affective priming however, the priming effect became larger with increases in mental
effects. Nevertheless, it is possible that the processes underlying load. The latter finding is in line with the idea that participants
affective priming operate only when participants have the distal engage in effortful processes in an attempt to minimize the impact
goal to evaluate stimuli. Many studies have shown, however, that of the primes on responding. When very few processing resources
affective priming effects (faster responses when prime and target are available (e.g., when participants who have smaller-than-
have the same valence than when they differ in valence) can also average working memory capacity are tested under high mental
be found in tasks that do not require the participants to adopt the load), these effortful processes can no longer operate and result in
goal to evaluate stimuli. For instance, affective priming effects stronger affective priming effects.
have (under certain conditions) been observed when participants If the findings of Klauer and Teige-Mocigemba can be con-
are required to read or name the target (e.g., Bargh, Chaiken, firmed, they would thus offer support for two conclusions: First,
Raymond, & Hymes, 1996; De Houwer et al., 2001; Spruyt et al., the observation of priming effects even when mental load is high
2002), to determine the lexical status (e.g., Wentura, 2000) or suggests that the processes by which the attitude toward the prime
semantic category of the target (e.g., Spruyt, De Houwer, et al., influences responding to the target can be automatic in the sense of
2007), or to compare the prime and target with regard to a efficient. Second, the increase in priming effects when fewer
nonaffective feature, such as color (e.g., Klauer & Musch, 2002). processing resources are available suggests that the effect of the
Note, however, that this evidence is not entirely conclusive, be- prime on responding can be controlled to a certain extent, provided
cause there never was a direct test of whether participants (implic- that sufficient processing resources are available. Note that the
itly) adopted the goal to evaluate stimuli. It would be good to results of Klauer and Teige-Mocigemba do not reveal whether
assess this question in future studies, because it is possible that participants have conscious control of priming effects. In principle,
participants adopt the goal to evaluate stimuli even when it is not it is possible that the effortful processes involved in controlling the
required by the task. magnitude of the priming effect are activated unconsciously.
The presence of awareness. As we discussed earlier, a measure Whether control is conscious needs to be examined in studies in
can be denoted as unaware in that it measures an attribute even when which participants are asked to report their conscious goals while
participants are unaware of (a) the stimuli that activate the attribute, they perform the task or in which conscious goals are manipulated
(b) the origins of the attribute itself, (c) the fact that the attribute (e.g., via faking instructions).
influences performance, or (d) the manner in which the attribute The availability of time. There is ample evidence showing that
influences performance. Evidence suggests that at least some af- the processes by which the attitude toward the prime produces
fective priming effects can be classified as unaware in the first two affective priming effects can operate very quickly and tend to
respects. First, several studies (e.g., Abrams, Klinger, & Green- dissipate very quickly over time. For instance, Klauer, Rossnagel,
wald, 2002; Draine & Greenwald, 1998; Klauer, Eder, Greenwald, and Musch (1997; see also Hermans, De Houwer, & Eelen, 2001,
& Abrams, 2007) have revealed affective priming effects even and Spruyt, Hermans, De Houwer, Vandromme, & Eelen, 2007)
when the primes were presented subliminally (i.e., when partici- found affective priming effects when the onset of the prime oc-
pants were not aware of the presentations of the primes). Second, curred 100 ms before (i.e., SOA of 100 ms) and even simulta-
when novel attitudes are created in the lab, they can lead to neously with (i.e., SOA ⫽ 0 ms) the onset of the target. Whereas
affective priming effects even when participants are not aware of reliable affective priming effects have been observed with SOAs
how the attitudes were acquired (e.g., Olson & Fazio, 2002). Note, up to 300 ms (e.g., Fazio, Sanbonmatsu, Powell, & Kardes, 1986;
however, that this fact does not imply that affective priming effects Hermans et al., 1994) and when the prime was presented 100 ms
can register attributes of which participants are not aware. This after the target (SOA ⫽ ⫺100 ms; e.g., Fockenberg, Koole, &
issue remains to be examined. Semin, 2006), there have been very few if any reports of reliable
We do not know of any study that examined whether partici- affective priming with SOAs larger than 300 ms or smaller than
pants were aware of the fact that their attitudes toward the prime ⫺100 ms (see Klauer & Musch, 2003; for a detailed account of
stimuli influenced their performance or of the way in which the why and how SOA influences priming effects, see Klauer, Teige-
attitudes influenced performance. Of course, in studies on sublim- Mocigemba, & Spruyt, in press). Given that participants need
inal affective priming, participants are not aware of the prime about 600 ms to evaluate the valence of the target (e.g., Hermans
stimuli and thus cannot be aware of the fact that the attitude toward et al., 1994), one can conclude that the prime has an impact on
362 DE HOUWER, TEIGE-MOCIGEMBA, SPRUYT, AND MOORS

responses to the target within a time frame starting at about 500 ms these measures, namely, scores on the Thematic Apperception Test
after prime onset (in case of an SOA ⫽ ⫺100 ms) and ending at (TAT; Morgan & Murray, 1935). The TAT is a projective test in
900 ms after prime onset (in case of an SOA ⫽ 300 ms). If it is which participants are asked to describe pictures of socially am-
assumed that response execution takes about 200 ms, these esti- biguous scenes. On the basis of the content of their responses,
mates can be reduced to 300 ms and 700 ms, respectively. scores can be derived that are assumed to reveal implicit motives,
Summary. Evidence suggests that affective priming effects can such as the need for achievement (e.g., McClelland, Koestner, &
be implicit in that they are based on fast, relatively efficient Weinberger, 1989). We choose this test because it differs substan-
processes (but see Klauer & Teige-Mocigemba, 2007) that can tially from the IAT and the affective priming task and because it
operate even when participants are unaware of the prime stimuli was developed long before the term implicit measure was intro-
and the origins of the attitude toward the primes. The distal goal to duced. As such, it allows us to illustrate the width of application of
evaluate stimuli in the environment does not seem to be necessary our normative analysis.
for affective priming effects to occur. Although there is some From the perspective of the normative analysis, most of the
evidence that certain proximal and distal goals can modulate research on the TAT has been directed at verifying the what
affective priming effects, the evidence on this specific issue is criterion but little or no research has looked at the how and
sparse. implicitness criteria. Most TAT studies were correlational in na-
ture and were aimed at assessing whether TAT scores indeed
Implications reflect implicit motives (see Lilienfeld, Wood, & Garb, 2000, and
McClelland et al., 1989, for opposing views). Very little attention
So far, we have (a) specified the normative criteria that an ideal has been given to verifying the how criterion (i.e., to examining the
implicit measure should meet and (b) examined the extent to which causal nature of the processes by which implicit motives influence
IAT and affective priming effects meet the normative criteria. In the stories that participants produce in response to TAT pictures).
this third section, we make explicit some of the implications of our The only exception of which we are aware is the work of Tuer-
work. We first discuss implications for future research on the linckx, De Boeck, and Lens (2002), who formulated and tested
validation and development of implicit measures. Afterward, we three simple theories about the processes underlying responses
address implications for the use of implicit measures as a tool in during the TAT. In doing so, they produced important new insights
research and psychological practice. into the reliability and construct validity of the measure. The study
of Tuerlinckx et al. is a perfect illustration of Borsboom et al.’s
Implications for the Validation and Development of (2004) argument that examining the processes underlying a mea-
Implicit Measures sure is an essential part of validating a measure.
To the best of our knowledge, there has been little research
One of the main virtues of the normative analysis presented in about whether TAT scores meet the implicitness criterion (i.e.,
this article is that it clarifies what researchers should aim for when about whether the processes underlying the scores are automatic
developing implicit measures. When the issues that have already in a certain manner). It is generally assumed that participants are
been examined are compared with those that should be examined not aware of the psychological attributes that TAT scores reflect
according to the normative analysis, it becomes apparent what still (McClelland et al., 1989), but there are few empirical data about
needs to be done. In short, the normative analysis can guide future this. We also do not know of any research on the impact of
research. Our review of the literature on IAT effects and affective proximal or distal goals, processing resources, or time on TAT
priming effects indeed revealed many important caveats in our scores. Such research is necessary before TAT scores can be
knowledge about these measures. With regard to the what crite- described as implicit measures, and it could reveal important
rion, more research should be directed not only at uncovering information about how these scores come about. We would also
which psychological attributes causally influence IAT and affec- like to highlight that, from the perspective of our normative anal-
tive priming effects but at understanding the variables that deter- ysis, TAT scores could in principle qualify as implicit measures.
mine the relative impact of those attributes. With regard to the how Neither the fact that the TAT was introduced before the term
criterion, there is still debate about the processes underlying IAT implicit measures came into use, nor the fact that TAT scores are
and affective priming effects. It seems to be the case that IAT and derived from the content rather than the speed of responses (see
affective priming effects can be produced by several processes. Payne et al., 2005, for a measure that is based on the content of
The relative contributions of these processes and the variables responses and that is generally considered to be implicit), is
determining their impact have hardly been studied (see Conrey et relevant for deciding whether a measure is implicit. The only thing
al., 2005, and Klauer et al., 2007, for exceptions). With regard to that counts is whether there is empirical evidence to support the
the implicitness criterion, much of the work still needs to be done. conclusion that the processes underlying TAT scores possess fea-
This is a surprising conclusion, given that implicitness is exactly tures of automaticity.
the feature that is supposed to set apart implicit measures from
other measures. Implications for Using Implicit Measures as a Tool
The normative analysis can guide not only future research on
IAT and affective priming effects but research on other implicit Many researchers and practitioners would probably prefer not to
measures that have already been proposed or that will be proposed wait for future improvements of implicit measures but would like
in the future. It would lead us too far afield to discuss the impli- to know now whether and how they should use existing implicit
cations of the normative analysis separately for each implicit measures as a tool for understanding human behavior. The argu-
measure that is currently available. We will discuss only one of ments and evidence that we present in this article clearly show that
 IMPLICIT MEASURES 363

the available implicit measures are not perfect. For most measures, best available measures and interpret them in ways that are sup-
it is not entirely clear what they measure, what processes produce ported by the available evidence. Identifying the imperfections of
the measure, and whether those processes are automatic in a a measure should, however, provide the impetus and direction for
certain manner. This does not mean, however, that the existing studying the measure further and improving it where possible. The
measures should not be used. On the contrary, many studies have normative criteria that were put forward in this article facilitate the
demonstrated the usefulness of implicit measures. detection of imperfections and gaps in our knowledge. As such,
Most important, it has been demonstrated that implicit measures they can be of great value for the further development of implicit
are at least sometimes related to behavioral variance that is not measures of psychological attributes.
related to traditional, explicit measures. The evidence for this
incremental predictive validity is strongest for IAT effects (see General Discussion
Greenwald et al., in press, for a review). Hence, IAT effects can
already provide new and unique insights into behavior. Unlike Implicit measures of attitudes, stereotypes, and other psycho-
most other currently available implicit measures, IAT effects are logical attributes have become popular in research disciplines as
reliable enough to be used as a measure of individual differences diverse as social, personality, clinical, consumer, and health psy-
(e.g., Bosson et al., 2000; Cunningham et al., 2001). Also, software chology. Despite their widespread use, there is still much confu-
and guidelines for implementing the IAT are readily available sion about what implicit measures actually are. On the basis of the
(e.g., Lane et al., 2007). work of Borsboom (Borsboom, 2006; Borsboom et al., 2004) and
Nevertheless, we do advise some degree of caution when inter- De Houwer (De Houwer, 2006; De Houwer & Moors, 2007), an
preting IAT effects and other currently available implicit mea- implicit measure can be defined as the outcome of a measurement
sures, especially at the level of a single individual. As with most procedure that results from automatic processes by which the
behavior, the responses from which implicit measures are derived to-be-measured attribute causally determines the outcome (see
are determined by a variety of factors. It is therefore risky to Figure 1B). From this definition, we have derived three normative
interpret an implicit measure as a pure index of one particular criteria that an ideal implicit measure should meet: (a) The what
psychological attribute. One should also avoid drawing conclu- criterion stipulates that we should know the attributes that causally
sions about the implicitness of a measure in the absence of detailed produce variation in the measure. (b) The how criterion requires
empirical evidence. Because the different features of automaticity that the processes by which the to-be-measured attribute causes
do not necessarily co-occur, each automaticity feature needs to be variations in the measure are known. (c) The implicitness criterion
examined separately. The general scientific principle of conver- entails that the processes underlying a measure should be auto-
gence can be followed in an attempt to overcome these problems. matic. For each implicit measure, one can examine the extent to
A conclusion can be drawn with greater confidence when different which it meets the three normative criteria.
implicit measures support that conclusion. The normative analysis put forward in this article provides a
As our knowledge of implicit measures increases, less caution heuristic framework for past and future research on implicit mea-
will be needed when interpreting these measures. The more we sures. We have used this framework to review the literature on the
know about the different psychological attributes that influence an two currently most popular implicit measures: IAT effects and
implicit measure (what criterion), the processes by which psycho- affective priming effects. By doing so, we have clarified what is
logical attributes produce the measure (how criterion), and the already known about these measures and, perhaps more important,
automaticity of the underlying processes (implicitness criterion), what needs to be examined in future studies.
the more confident we can be in deciding what a particular mea- Acceptance of our normative analysis and heuristic framework
sure actually means. Hence, by verifying whether measures meet depends on acceptance of the definition of the concept “implicit
the what, how, and implicitness criteria, we can gradually increase measures,” from which the analysis and framework were derived.
the overall quality of implicit measures as tools for studying As is the case for all definitions, the definition of the concept
human behavior. “implicit measure” is a matter of convention and thus to a certain
We want to point out that implicit measures are not the only extent arbitrary. We cannot guarantee that everyone will agree
measures that need to be interpreted with caution. The what and with our definition, but we do believe that the work of Borsboom
how criteria apply to all measures, implicit or otherwise. There is et al. (2004) and De Houwer (De Houwer, 2006; De Houwer &
probably not a single measure of psychological attributes that is Moors, 2007) provides a solid conceptual basis for our definition.
perfect, in that it fully meets both criteria. It is not entirely clear At the very least, it has the merit of being explicit. As such, our
what is captured by many traditional measures. Self-report mea- definition provides the conceptual basis for clarifying disagree-
sures, for instance, are known to be susceptible to the effects of ments about the meaning of the term implicit measures and thus
many extraneous factors (e.g., social desirability, the precise word- about the normative criteria that an ideal implicit measure should
ing of items, the sequence in which items are presented; see meet.
Schwarz, 1999, 2007, for a discussion of some of these factors). In line with the arguments of Borsboom (Borsboom, 2006;
Also, little is known about the processes by which psychological Borsboom et al., 2004), we have argued that experimental studies
attributes can influence self-reports. should be crucial in validation research. Experiments are the gold
Just as traditional measures have proven to be useful despite standard for establishing whether a psychological attribute causes
these imperfections, implicit measures can provide added value variation in a measurement outcome and how it does so. Validation
despite the caveats regarding our knowledge about these measures. research is theoretical research. It should be directed at testing
The fact that a measure does not meet the normative criteria should theories about which attributes causally determine measurement
not necessarily stop us from using it. We should always use the outcomes in which ways. Correlational studies can inform the
364 DE HOUWER, TEIGE-MOCIGEMBA, SPRUYT, AND MOORS

construction and evaluation of such theories, especially when they sures with different real-life behaviors in terms of the extent to
are conducted in a systematic manner (see Nosek & Smyth, 2007, which those measures and behaviors are influenced by the same
for an example of such an approach). attribute under the same set of conditions. In line with the idea of
Until now, we have largely ignored one piece of correlational transfer-appropriate processing (e.g., Roediger, 1990), one could
evidence: the reliability of a measure. Often, reliability is consid- argue that the more similar a measure is to a behavior in this
ered to be a necessary condition for validity. This is, however, respect, the more the measure will be able to predict the behavior.
not entirely true. As argued by Borsboom et al. (2004) and Tuer- For instance, real-life, attitude-driven behavior that occurs when
linckx et al. (2002), a measure can be valid (i.e., caused by the people do not have the conscious goal to evaluate stimuli in the
to-be-measured attribute) even when it is not reliable. Such a environment (e.g., buying products under time pressure) might be
situation can, for instance, arise when the underlying psychologi- related most to measurement outcomes that occur in the absence of
cal attribute does not remain stable over time or context. The a conscious evaluation goal. This approach entails that one should
presence of reliability also provides little information about what it study in detail not only the conditions under which the attribute
is that the measure captures. Reliability is, however, an important influences the measure but also the conditions under which the
determinant of the overall quality of a measure (e.g., Borsboom et attribute influences the to-be-predicted behavior.
al., 2004). For instance, when the aim is to predict future behavior, Given the quantity and complexity of the research involved in
a measure is required that remains stable over time. Hence, it is verifying whether and in what sense a measure can be regarded as
important to continue to examine the reliability of measures. an implicit measure, one might choose to adopt an apparently more
We should also examine whether a measure is influenced by simple, pragmatic approach in which various measures are simply
attributes other than the to-be-measured attribute. The extent to related to various behaviors without much consideration for con-
which a measurement outcome can be used to make an inference ceptual or theoretical issues. Measures that predict a particular
about a specific attribute of the person depends not only on behavior can be considered useful even if it is not known what
whether that attribute causes variation in the outcome but on attribute the measures actually capture, how they do so, or which
whether other attributes cause variation in the outcome. Put dif- features of automaticity apply. On the one hand, we do agree that
ferently, interpreting a measure as indicative of an attribute re- the practical use of implicit measures should not await a full
quires not only that the attribute is a cause of the outcome but that evaluation in terms of the three normative criteria put forward in
it is the only systematic cause of the outcome. If other attributes this article. Despite the important gaps in our knowledge about
can cause variations in the outcome independent of the to-be- implicit measures, these measures could help researchers predict
measured attribute, one can never be sure whether a certain out- and understand certain behaviors. On the other hand, a purely
come reflects the to-be-measured attribute or another one (e.g., pragmatic approach does have serious limitations. First, in the
Fiedler et al., 2006). Research on the what and how criteria thus absence of strong empirical evidence, one should refrain from
should not be restricted to the to-be-measured attribute but should making statements about how a measure should be interpreted,
examine whether and when other attributes can cause variations in how it works, or whether it is implicit. Second, without a basic
the outcome. It is also important to realize that empirical research level of theoretical understanding of the measures, there is little
will not suffice to determine what a measure actually captures. ground for predicting when a measure will be related to which kind
Detailed conceptual analyses should be undertaken to examine the of behavior. Progress in obtaining evidence for relations between
ontological status of the psychological attributes that are measured. measures and behavior will thus proceed slowly and haphazardly.
The upper limit of what a measure can tell is determined by what Likewise, there will be little guidance for attempts to improve the
is known about the attribute that the measure is assumed to quality of the measures. In the end, a purely pragmatic approach
capture. might be less efficient than a conscientious conceptual and theo-
Studies on the implicitness criterion also involve extensive and retical approach to understanding implicit measures of psycholog-
complicated research. Which features of automaticity apply to ical attributes. We hope that the normative analysis put forward in
each implicit measure must be examined empirically. One could this article will be of help to researchers who choose to adopt this
argue that only some of the automaticity features are truly relevant difficult but necessary approach.
for determining the implicitness of measures. We do not commit to
a position on this point, but we do offer the conceptual tools for References
making possible a debate about what features of automaticity are Abrams, R. L., Klinger, M. R., & Greenwald, A. G. (2002). Subliminal
central for implicit measures. Moors and De Houwer (2006) re- words activate semantic categories (not automated motor responses).
cently defined in detail the various features of automaticity. Ap- Psychonomic Bulletin & Review, 9, 100 –106.
plying this analysis in the context of implicit measures (see also Anastasi, A. (1958). Differential psychology (3rd ed.). New York: Mac-
De Houwer & Moors, 2007) makes it clear in what ways a measure millan.
can be implicit. This process allows researchers to specify what Asendorpf, J. B., Banse, R., & Mücke, D. (2002). Double dissociation
they mean when they say that a measure is implicit and to debate between implicit and explicit personality self-concept: The case of shy
the merits of the various possible features of automaticity and behavior. Journal of Personality and Social Psychology, 83, 380 –393.
Back, M. D., Schmukle, S. C., & Egloff, B. (2005). Measuring task-
implicitness.
switching ability in the Implicit Association Test. Experimental Psychol-
Rather than try to select one feature or set of features as defining ogy, 52, 167–179.
for the implicitness of measures, one could examine as many Banaji, M. R. (2001). Implicit attitudes can be measured. In H. L. Roediger
features as possible in an attempt to understand more fully the & J. S. Nairne (Eds.), The nature of remembering: Essays in honor of
conditions under which the to-be-measured attribute causally in- Robert G. Crowder (pp. 117–150). Washington, DC: American Psycho-
fluences the measure. One could then try to match different mea- logical Association.
 IMPLICIT MEASURES 365

Banse, R. (1999). Automatic evaluation of self and significant others: allows for a differentiation of automatic prejudice. Basic and Applied
Affective priming in close relationships. Journal of Social and Personal Social Psychology, 29, 245–256.
Relationships, 16, 805– 824. De Houwer, J. (2000). [Testing the response criterion account of the
Banse, R. (2001). Affective priming with liked and disliked persons: Prime Implicit Association Test]. Unpublished raw data.
visibility determines congruency and incongruency effects. Cognition & De Houwer, J. (2001). A structural and process analysis of the Implicit
Emotion, 15, 501–520. Association Test. Journal of Experimental Social Psychology, 37, 443–
Banse, R., Seise, J., & Zerbes, N. (2001). Implicit attitudes toward homo- 451.
sexuality: Reliability, validity, and controllability of the IAT. Zeitschrift De Houwer, J. (2003a). The extrinsic affective Simon task. Experimental
für Experimentelle Psychologie, 48, 145–160. Psychology, 50, 77– 85.
Bargh, J. A. (1992). The ecology of automaticity. Toward establishing the De Houwer, J. (2003b). A structural analysis of indirect measures of
conditions needed to produce automatic processing effects. American attitudes. In J. Musch & K. C. Klauer (Eds.), The psychology of evalu-
Journal of Psychology, 105, 181–199. ation: Affective processes in cognition and emotion (pp. 219 –244).
Bargh, J. A., Chaiken, S., Raymond, P., & Hymes, C. (1996). The auto- Mahwah, NJ: Erlbaum.
matic evaluation effect: Unconditional automatic attitude activation with De Houwer, J. (2006). What are implicit measures and why are we using
a pronunciation task. Journal of Experimental Social Psychology, 32, them. In R. W. Wiers & A. W. Stacy (Eds.), The handbook of implicit
104 –128. cognition and addiction (pp. 11–28). Thousand Oaks, CA: Sage.
Blair, I. V. (2002). The malleability of automatic stereotypes and prejudice. De Houwer, J. (2009). Comparing measures of attitudes at the procedural
Personality and Social Psychology Review, 6, 242–261. and functional level. In R. Petty, R. H. Fazio, & P. Brinol (Eds.),
Blair, I. V., Ma, J. E., & Lenton, A. P. (2001). Imagining stereotypes away: Attitudes: Insights from the new implicit measures. New York: Psychol-
The moderation of implicit stereotypes through mental imagery. Journal ogy Press.
of Personality and Social Psychology, 81, 828 – 841. De Houwer, J. (in press). Implicit measures: Similarities and differences. In
Blanton, H., & Jaccard, J. (2006). Arbitrary metrics in psychology. Amer- B. Gawronski & B. K. Payne (Eds.), Handbook of implicit social
ican Psychologist, 61, 27– 41. cognition. New York: Guilford.
Bluemke, M., & Friese, M. (2006). Do features of stimuli influence IAT De Houwer, J., Beckers, T., & Moors, A. (2007). Novel attitudes can be
effects? Journal of Experimental Social Psychology, 42, 163–176. faked on the Implicit Association Test. Journal of Experimental Social
Borsboom, D. (2006). The attack of the psychometricians. Psychometrika, Psychology, 43, 972–978.
71, 425– 440. De Houwer, J., Geldof, T., & De Bruycker, E. (2005). The Implicit
Borsboom, D., Mellenbergh, G. J., & van Heerden, J. (2004). The concept Association Test as a general measure of similarity. Canadian Journal of
of validity. Psychological Review, 111, 1061–1071. Experimental Psychology, 59, 228 –239.
Bosson, J. K., Swann, W. B., & Pennebaker, J. W. (2000). Stalking the De Houwer, J., Hermans, D., & Eelen, P. (1998). Affective and identity
perfect measure of implicit self-esteem: The blind men and the elephant priming with episodically associated stimuli. Cognition & Emotion, 12,
revisited? Journal of Personality and Social Psychology, 79, 631– 643. 145–169.
Brendl, C. M., Markman, A. B., & Messner, C. (2001). How do indirect De Houwer, J., Hermans, D., Rothermund, K., & Wentura, D. (2002).
measures of evaluation work? Evaluating the inference of prejudice in Affective priming of semantic categorization responses. Cognition &
the Implicit Association Test. Journal of Personality and Social Psy- Emotion, 16, 643– 666.
chology, 81, 760 –773. De Houwer, J., Hermans, D., & Spruyt, A. (2001). Affective priming of
Cai, H. J., Sriram, N., Greenwald, A. G., & McFarland, S. G. (2004). The pronunciation responses: Effects of target degradation. Journal of Ex-
Implicit Association Test’s D measure can minimize a cognitive skill perimental Social Psychology, 37, 85–91.
confound: Comment on McFarland and Crouch (2002). Social Cogni- De Houwer, J., & Moors, A. (2006). [Awareness of the aim of the Implicit
tion, 22, 673– 684. Association Test]. Unpublished raw data.
Collins, A. M., & Loftus, E. F. (1975). A spreading activation theory of De Houwer, J., & Moors, A. (2007). How to define and examine the
semantic processing. Psychological Review, 82, 407– 428. implicitness of implicit measures. In B. Wittenbrink & N. Schwarz
Collins, A. M., & Quillian, M. R. (1969). Retrieval from semantic memory. (Eds.), Implicit measures of attitudes: Procedures and controversies (pp.
Journal of Verbal Learning and Verbal Behavior, 8, 240 –248. 179 –194). New York: Guilford Press.
Conrey, F. R., Sherman, J. W., Gawronski, B., Hugenberg, K., & Groom, Devine, P. G., Plant, E. A., Amodio, D. M., Harmon-Jones, E., & Vance,
C. (2005). Separating multiple processes in implicit social cognition: S. L. (2002). The regulation of explicit and implicit race bias: The role
The quad model of implicit task performance. Journal of Personality of motivations to respond without prejudice. Journal of Personality and
and Social Psychology, 89, 469 – 487. Social Psychology, 82, 835– 848.
Cronbach, L. J., & Meehl, P. E. (1955). Construct validity in psychological Draine, S. C., & Greenwald, A. G. (1998). Replicable unconscious seman-
tests. Psychological Bulletin, 52, 281–302. tic priming. Journal of Experimental Psychology: General, 127, 286 –
Cunningham, W. A., Preacher, K. J., & Banaji, M. R. (2001). Implicit 303.
attitude measures: Consistency, stability, and convergent validity. Psy- Dunton, B. C., & Fazio, R. H. (1997). An individual difference measure of
chological Science, 12, 163–170. motivation to control prejudiced reactions. Personality and Social Psy-
Czellar, S. (2006). Self-presentational effects in the Implicit Association chology Bulletin, 23, 316 –326.
Test. Journal of Consumer Psychology, 16, 92–100. Eder, A., & Rothermund, K. (2008). When do motor behaviors (mis)match
Dasgupta, N., & Greenwald, A. G. (2001). On the malleability of automatic affective stimuli? An evaluative coding view of approach and avoidance
attitudes: Combating automatic prejudice with images of admired and reactions. Journal of Experimental Psychology: General, 137, 262–281.
disliked individuals. Journal of Personality and Social Psychology, 81, Egloff, B., & Schmukle, S. C. (2002). Predictive validity of an Implicit
800 – 814. Association Test for assessing anxiety. Journal of Personality and Social
Degner, J. (in press). On the (un)controllability of affective priming: Psychology, 83, 1441–1455.
Strategic manipulation is feasible but can possibly be prevented. Cog- Eysenck, H. J., & Eysenck, M. W. (1985). Personality and individual
nition & Emotion. differences: A natural science approach. New York: Plenum.
Degner, J., Wentura, D., Gniewosz, B., & Noack, P. (2007). Hostility- Fazio, R. H. (2001). On the automatic activation of associated evaluations:
related prejudice against Turks in adolescents: Masked affective priming An overview. Cognition & Emotion, 15, 115–141.
366 DE HOUWER, TEIGE-MOCIGEMBA, SPRUYT, AND MOORS

Fazio, R. H. (2007). Attitudes as object-evaluation associations of varying ciation Test: Comment on Rothermund and Wentura (2004). Journal of
strength. Social Cognition, 25, 603– 637. Experimental Psychology: General, 134, 420 – 425.
Fazio, R. H., Jackson, J. R., Dunton, B. C., & Williams, C. J. (1995). Greenwald, A. G., Nosek, B. A., & Sriram, N. (2006). Consequential
Variability in automatic activation as an unobtrusive measure of racial validity of the Implicit Association Test: Comment on Blanton and
attitudes: A bona fide pipeline? Journal of Personality and Social Jaccard (2006). American Psychologist, 61, 56 – 61.
Psychology, 69, 1013–1027. Greenwald, A. G., Poehlman, T. A., Uhlmann, E., & Banaji, M. R. (in
Fazio, R. H., & Olson, M. A. (2003). Implicit measures in social cognition press). Understanding and using the Implicit Association Test: III.
research: Their meaning and use. Annual Review of Psychology, 54, Meta-analysis of predictive validity. Journal of Personality and Social
297–327. Psychology.
Fazio, R. H., Sanbonmatsu, D. M., Powell, M. C., & Kardes, F. R. (1986). Han, H. A., Olson, M. A., & Fazio, R. H. (2006). The influence of
On the automatic activation of attitudes. Journal of Personality and experimentally created extrapersonal associations on the Implicit Asso-
Social Psychology, 50, 229 –238. ciation Test. Journal of Experimental Social Psychology, 42, 259 –272.
Fiedler, K., & Bluemke, M. (2005). Faking the IAT: Aided and unaided Hermans, D., Baeyens, F., & Eelen, P. (2003). On the acquisition and
response control on the Implicit Association Tests. Basic and Applied activation of evaluative information in memory: The study of evaluative
Social Psychology, 27, 307–316. learning. In J. Musch & K. C. Klauer (Eds.), The psychology of evalu-
Fiedler, K., Messner, C., & Bluemke, M. (2006). Unresolved problems ation: Affective processes in cognition and emotion (pp. 139 –168).
with the “I,” the “A” and the “T”: Logical and psychometric critique of Mahwah, NJ: Erlbaum.
the Implicit Association Test (IAT). European Review of Social Psy- Hermans, D., Crombez, G., & Eelen, P. (2000). Automatic attitude acti-
chology, 17, 74 –147. vation and efficiency: The fourth horseman of automaticity. Psycho-
Fockenberg, D. A., Koole, S. L., & Semin, G. R. (2006). Backward logica Belgica, 40, 3–22.
affective priming: Even when the prime is late, people still evaluate. Hermans, D., De Houwer, J., & Eelen, P. (1994). The affective priming
Journal of Experimental Social Psychology, 42, 799 – 806. effect: Automatic activation of evaluative information in memory. Cog-
Frings, C., & Wentura, D. (2003). Who is watching “Big Brother”? TV nition & Emotion, 8, 515–533.
consumption predicted by masked affective priming. European Journal Hermans, D., De Houwer, J., & Eelen, P. (2001). A time course analysis of
the affective priming effect. Cognition & Emotion, 15, 143–165.
of Social Psychology, 33, 779 –791.
Hermans, D., Smeesters, D., De Houwer, J., & Eelen, P. (2002). Affective
Gawronski, B. (Ed.). (2007). What is an attitude? [Special issue]. Social
priming for associatively unrelated primes and targets. Psychologica
Cognition, 25(5).
Belgica, 42, 191–212.
Gawronski, B., & Bodenhausen, G. V. (2006). Associative and proposi-
Hofmann, W., Gawronski, B., Gschwendner, T., Le, H., & Schmitt, M.
tional processes in evaluation: An integrative review of implicit and
(2005). A meta-analysis on the correlation between the Implicit Asso-
explicit attitude change. Psychological Bulletin, 132, 692–731.
ciation Test and explicit self-report measures. Personality and Social
Gawronski, B., Deutsch, R., LeBel, E. P., & Peters, K. R. (2008). Response
Psychology Bulletin, 31, 1369 –1385.
interference as a mechanism underlying implicit measures: Some traps
Hofmann, W., Rauch, W., & Gawronski, B. (2007). And deplete us not into
and gaps in the assessment of mental associations with experimental
temptation: Automatic attitudes, dietary restraint, and self-regulatory
paradigms. European Journal of Psychological Assessment, 24, 218 –
resources as determinants of eating behavior. Journal of Experimental
225.
Social Psychology, 43, 497–504.
Gawronksi, B., Peters, K. R., & LeBel, E. P. (2008). What makes mental
Houben, K., & Wiers, R. W. (2006). A test of the salience asymmetry
associations personal or extra-personal? Conceptual issues in the meth-
interpretation of the Alcohol-IAT. Experimental Psychology, 53, 292–
odological debate about implicit attitude measures. Social and Person-
300.
ality Psychology Compass, 2, 1002–1023. Hummert, M. L., Garstka, T. A., O’Brien, L. T., Greenwald, A. G., &
Gemar, M. C., Segal, Z. V., Sagrati, S., & Kennedy, S. J. (2001). Mood- Mellott, D. S. (2002). Using the Implicit Association Test to measure
induced changes on the Implicit Association Test in recovered depressed age differences in implicit social cognitions. Psychology and Aging, 17,
patients. Journal of Abnormal Psychology, 110, 282–289. 482– 495.
Govan, C. L., & Williams, K. D. (2004). Changing the affective valence of Kane, M. J., & Engle, R. W. (2003). Working memory capacity and the
the stimulus items influences the IAT by redefining the category labels. control of attention: The contributions of goal neglect, response com-
Journal of Experimental Social Psychology, 40, 357–365. petition, and task set to Stroop interference. Journal of Experimental
Greenwald, A. G., & Banaji, M. R. (1995). Implicit social cognition: Psychology: General, 132, 47–70.
Attitudes, self-esteem, and stereotypes. Psychological Review, 102, Karpinski, A., & Hilton, J. L. (2001). Attitudes and the Implicit Associa-
4 –27. tion Test. Journal of Personality and Social Psychology, 81, 774 –778.
Greenwald, A. G., Banaji, M. R., Rudman, L. A., Farnham, S. D., Nosek, Kim, D. Y. (2003). Voluntary controllability of the Implicit Association
B. A., & Mellott, D. S. (2002). A unified theory of implicit attitudes, Test (IAT). Social Psychology Quarterly, 66, 83–96.
stereotypes, self-esteem, and self-concept. Psychological Review, 109, Kinoshita, S., & Peek-O’Leary, M. (2005). Does the compatibility effect in
3–25. the race Implicit Association Test reflect familiarity or affect? Psy-
Greenwald, A. G., McGhee, D. E., & Schwartz, J. L. K. (1998). Measuring chonomic Bulletin & Review, 12, 442– 452.
individual differences in implicit cognition: The Implicit Association Kinoshita, S., & Peek-O’Leary, M. (2006). Two bases of the compatibility
Test. Journal of Personality and Social Psychology, 74, 1464 –1480. effect in the Implicit Association Test (IAT). Quarterly Journal of
Greenwald, A. G., & Nosek, B. A. (2001). Health of the Implicit Associ- Experimental Psychology, 56, 2102–2120.
ation Test at age 3. Zeitschrift für Experimentelle Psychologie, 48, Klauer, K. C., Eder, A. B., Greenwald, A. G., & Abrams, R. L. (2007).
85–93. Priming of semantic classifications by novel subliminal prime words.
Greenwald, A. G., Nosek, B. A., & Banaji, M. R. (2003). Understanding Consciousness and Cognition, 16, 63– 83.
and using the Implicit Association Test: I. An improved scoring algo- Klauer, K. C., & Mierke, J. (2005). Task-set inertia, attitude accessibility,
rithm. Journal of Personality and Social Psychology, 85, 197–216. and compatibility-order effects: New evidence for a task-set switching
Greenwald, A. G., Nosek, B. A., Banaji, M. R., & Klauer, K. C. (2005). account of the Implicit Association Test effect. Personality and Social
Validity of the salience asymmetry interpretation of the Implicit Asso- Psychology Bulletin, 31, 208 –217.
 IMPLICIT MEASURES 367

Klauer, K. C., Mierke, J., & Musch, J. (2003). The positivity proportion Mischel, W., & Shoda, Y. (1995). A cognitive–affective system theory of
effect: A list context effect in masked affective priming. Memory & personality: Reconceptualizing situations, dispositions, dynamics, and
Cognition, 31, 953–967. invariance in personality structure. Psychological Review, 102, 246 –
Klauer, K. C., & Musch, J. (2002). Goal-dependent and goal-independent 268.
effects of irrelevant evaluations. Personality and Social Psychology Mitchell, J. P., Nosek, B. A., Banaji, M. R. (2003). Contextual variations
Bulletin, 28, 802– 814. in implicit evaluation. Journal of Experimental Psychology: General,
Klauer, K. C., & Musch, J. (2003). Affective priming: Findings and 132, 455– 469.
theories. In J. Musch & K. C. Klauer (Eds.), The psychology of evalu- Monteith, M. J., Voils, C. I., & Ashburn-Nardo, L. (2001). Taking a look
ation: Affective processes in cognition and emotion (pp. 7–50). Mahwah, underground: Detecting, interpreting, and reacting to implicit racial bias.
NJ: Erlbaum. Social Cognition, 19, 395– 417.
Klauer, K. C., Musch, J., & Eder, A. (2005). Priming of semantic classi- Moors, A., & De Houwer, J. (2006). Automaticity: A conceptual and
fications: Late and response-related, or earlier and more central? Psy- theoretical analysis. Psychological Bulletin, 132, 297–326.
chonomic Bulletin & Review, 12, 897–903. Morgan, C. D., & Murray, H. A. (1935). A method of investigating
Klauer, K. C., Rossnagel, R., & Musch, J. (1997). List– context effects in fantasies: The Thematic Apperception Test. Archives of Neurology and
evaluative priming. Journal of Experimental Psychology: Learning, Psychiatry, 34, 289 –306.
Memory, and Cognition, 23, 246 –255. Moskowitz, G. B., Salomon, A. R., & Taylor, C. M. (2000). Preconsciously
Klauer, K. C., & Teige-Mocigemba, S. (2007). Controllability and resource controlling stereotyping: Implicitly activated egalitarian goals prevent
dependence in automatic evaluation. Journal of Experimental Social the activation of stereotypes. Social Cognition, 18, 151–177.
Psychology, 43, 648 – 655. Nosek, B. A., & Banaji, M. R. (2001). The Go/No-Go Association Task.
Klauer, K. C., Teige-Mocigemba, S., & Spruyt, A. (in press). Contrast Social Cognition, 19, 625– 666.
effects in spontaneous evaluations: A psychophysical account. Journal Nosek, B. A., Banaji, M. R., & Greenwald, A. G. (2002). Harvesting
of Personality and Social Psychology. implicit group attitudes and beliefs from a demonstration website. Group
Klauer, K. C., Voss, A., Schmitz, F., & Teige-Mocigemba, S. (2007). Dynamics, 6, 101–115.
Process components of the Implicit Association Test: A diffusion-model Nosek, B. A., Greenwald, A. G., & Banaji, M. R. (2005). Understanding
analysis. Journal of Personality and Social Psychology, 93, 353–368. and using the Implicit Association Test: II. Method variables and con-
Kornblum, S., & Lee, J.-W. (1995). Stimulus–response compatibility with struct validity. Personality and Social Psychology Bulletin, 31, 166 –
relevant and irrelevant stimulus dimensions that do and do not overlap 180.
with the response. Journal of Experimental Psychology: Human Per- Nosek, B. A., Greenwald, A. G., & Banaji, M. R. (2007). The Implicit
ception and Performance, 21, 855– 875. Association Test at age 7: A methodological and conceptual review. In
Lane, K. A., Banaji, M. R., Nosek, B. A., & Greenwald, A. G. (2007). J. A. Bargh (Ed.), Social psychology and the unconscious: The automa-
Understanding and using the Implicit Association Test: What we know ticity of higher mental processes (pp. 265–292). New York: Psychology
(so far) about the method. In B. Wittenbrink & N. Schwarz (Eds.), Press.
Implicit measures of attitudes (pp. 59 –102). New York: Guilford Press. Nosek, B. A., & Hansen, J. J. (2008a). The associations in our heads belong
Lasaga, M. I., & Garner, W. R. (1983). Effect of line orientation on various to us: Searching for attitudes and knowledge in implicit evaluation.
information-processing tasks. Journal of Experimental Psychology: Hu- Cognition & Emotion, 22, 553–594.
man Perception and Performance, 9, 215–225. Nosek, B. A., & Hansen, J. J. (2008b). Personalizing the Implicit Associ-
Lilienfeld, S. O., Wood, J. M., & Garb, H. N. (2000). The scientific status ation Test increases explicit evaluation of the target concepts. European
of projective techniques. Psychological Science in the Public Interest, 1, Journal of Psychological Assessment.
27– 66. Nosek, B. A., & Smyth, F. L. (2007). A multitrait–multimethod validation
Lowery, B. S., Hardin, C. D., & Sinclair, S. (2001). Social influence effects of the Implicit Association Test: Implicit and explicit attitudes are
on automatic racial prejudice. Journal of Personality and Social Psy- related but distinct constructs. Experimental Psychology, 54, 14 –29.
chology, 81, 842– 855. Olson, M. A., & Fazio, R. H. (2001). Implicit attitude formation through
Lucas, M. (2000). Semantic priming without association: A meta-analytic classical conditioning. Psychological Science, 12, 413– 417.
review Psychonomic Bulletin & Review, 7, 618 – 630. Olson, M. A., & Fazio, R. H. (2002). Implicit acquisition and manifestation
MacLeod, C. M. (1991). Half a century of research on the Stroop effect: An of classically conditioned attitudes. Social Cognition, 20, 89 –104.
integrative review. Psychological Bulletin, 109, 163–203. Olson, M. A., & Fazio, R. H. (2003). Relations between implicit measures
Maison, D., Greenwald, A. G., & Bruin, R. H. (2004). Predictive validity of prejudice: What are we measuring? Psychological Science, 14, 636 –
of the Implicit Association Test in studies of brands, consumer attitudes 639.
and behavior. Journal of Consumer Psychology, 14, 405– 416. Olson, M. A., & Fazio, R. H. (2004). Reducing the influence of extraper-
McClelland, D. C., Koestner, R., & Weinberger, J. (1989). How do sonal associations on the Implicit Association Test: Personalizing the
self-attributed and implicit motives differ? Psychological Bulletin, 96, IAT. Journal of Personality and Social Psychology, 86, 653– 667.
690 –702. Otten, S., & Wentura, D. (1999). About the impact of automaticity in the
McFarland, S. G., & Crouch, Z. (2002). A cognitive skill confound on the minimal group paradigm: Evidence from affective priming tasks. Euro-
Implicit Association Test. Social Cognition, 20, 483–510. pean Journal of Social Psychology, 29, 1049 –1071.
Medin, D. L., Goldstone, R. L., & Gentner, D. (1993). Respects for Payne, B. K., Burkley, M. A., & Stokes, M. B. (2008). Why do implicit and
similarity. Psychological Review, 100, 254 –278. explicit attitude tests diverge? The role of structural fit. Journal of
Meiran, N., Chorev, Z., & Sapir, A. (2000). Component processes in task Personality and Social Psychology, 94, 16 –31.
switching. Cognitive Psychology, 41, 211–253. Payne, B. K., Cheng, C. M., Govorun, O., & Stewart, B. (2005). An inkblot
Mierke, J., & Klauer, K. C. (2001). Implicit association measurement with for attitudes: Affect misattribution as implicit measurement. Journal of
the IAT: Evidence for effects of executive control processes. Zeitschrift Personality and Social Psychology, 89, 277–293.
für Experimentelle Psychologie, 48, 107–122. Pecher, D., Zeelenberg, R., & Raaijmakers, J. G. W. (1998). Does pizza
Mierke, J., & Klauer, K. C. (2003). Method-specific variance in the prime coin? Perceptual priming in lexical decision and pronunciation.
Implicit Association Test. Journal of Personality and Social Psychology, Journal of Memory and Language, 38, 401– 418.
85, 1180 –1192. Popa-Roch, M.-A. (2008). Validation expérimentale d’une mesure implic-
368 DE HOUWER, TEIGE-MOCIGEMBA, SPRUYT, AND MOORS

ite du prejudge: Le test d’associations implicates. Unpublished doctoral tion of consumer choice behavior on the basis of affective priming in the
dissertation, Université Pierre Mendès, Grenoble, France. picture-picture naming task. Journal of Experimental Social Psychology,
Proctor, R. W., & Cho, Y. S. (2006). Polarity correspondence: A general 43, 599 – 610.
principle for performance of speeded binary classification tasks. Psy- Spruyt, A., Hermans, D., De Houwer, J., Vandromme, H., & Eelen, P.
chological Bulletin, 132, 416 – 442. (2007). On the nature of the affective priming effect: Effects of stimulus
Ratcliff, R. (1988). A retrieval theory of priming in memory. Psychological onset asynchrony and relatedness proportion in naming and evaluative
Review, 95, 385– 408. categorization. Memory & Cognition, 35, 95–106.
Roediger, H. L. (1990). Implicit memory: Retention without remembering. Steffens, M. (2004). Is the Implicit Association Test immune to faking?
American Psychologist, 45, 1043–1056. Experimental Psychology, 51, 165–179.
Rogers, R., & Monsell, S. (1995). Costs of a predictable switch between Strayer, D. L., & Kramer, A. F. (1994). Strategies and automaticity: I.
simple cognitive tasks. Journal of Experimental Psychology: General, Basic findings and conceptual framework. Journal of Experimental
124, 207–231. Psychology: Learning, Memory, and Cognition, 20, 318 –341.
Rothermund, K., Teige-Mocigemba, S., Gast, A., & Wentura, D. (in press). Teige-Mocigemba, S., & Klauer, K. C. (2008). “Automatic” evaluation?
Minimizing the influence of recoding processes in the Implicit Associ- Strategic effects on affective priming. Journal of Experimental Social
ation Test: The Recoding–Free Implicit Association Test (IAT–RF). Psychology, 44, 1414 –1417.
Quarterly Journal of Experimental Psychology. Teige-Mocigemba, S., Klauer, K. C., & Rothermund, K. (2008). Minimiz-
Rothermund, K., & Wentura, D. (2004). Underlying processes in the ing method-specific variance in the IAT: A single block IAT. European
Implicit Association Test (IAT): Dissociating salience from associa- Journal of Psychological Assessment, 24, 237–245.
tions. Journal of Experimental Psychology: General, 133, 139 –165. Thurstone, L. L. (1928). Attitudes can be measured. American Journal of
Rothermund, K., Wentura, D., & De Houwer, J. (2005). Validity of the Sociology, 33, 529 –554.
salience asymmetry account of the IAT: Reply to Greenwald, Nosek, Tuerlinckx, F., De Boeck, P., & Lens, W. (2002). Measuring needs with the
Banaji, and Klauer (2005). Journal of Experimental Psychology: Gen- Thematic Apperception Test: A psychometric study. Journal of Person-
eral, 134, 426 – 430. ality and Social Psychology, 82, 448 – 461.
Schmitz, F., Teige, S., Voss, A., & Klauer, K. C. (2005, June). Working Wentura, D. (1999). Activation and inhibition of affective information:
memory load in the IAT. Paper presented at the Fifth Workshop on Evidence for negative priming in the evaluation task. Cognition &
Implicit Representations and Personality, Berlin, Germany. Emotion, 13, 65–91.
Schnabel, K., Banse, R., & Asendorpf, J. B. (2006). Employing automatic Wentura, D. (2000). Dissociative affective and associative priming effects
approach and avoidance tendencies for the assessment of implicit per- in the lexical decision task: Yes versus no responses to word targets
sonality self-concept: The Implicit Association Procedure (IAP). Exper- reveal evaluative judgment tendencies. Journal of Experimental Psy-
imental Psychology, 53, 69 –76. chology: Learning, Memory, and Cognition, 26, 456 – 469.
Schwarz, N. (1999). Self-reports: How the questions shape the answers. Wentura, D., Kulfanek, M., & Greve, W. (2005). Masked affective priming
American Psychologist, 54, 93–105. by name letters: Evidence for a correspondence of explicit and implicit
Schwarz, N. (2007). Cognitive aspects of survey methodology. Applied self-esteem. Journal of Experimental Social Psychology, 41, 654 – 663.
Cognitive Psychology, 21, 277–287. Wentura, D., & Rothermund, K. (2007). Paradigms we live by: A plea for
Sekaquaptewa, D., Espinoza, P., Thompson, M., Vargas, P., & von Hippel, more basic research on the IAT. In B. Wittenbrink & N. Schwarz (Eds.),
W. (2003). Stereotypic explanatory bias: Implicit stereotyping as a Implicit measures of attitudes (pp. 195–215). New York: Guilford Press.
predictor of discrimination. Journal of Experimental Social Psychology, Wiers, R. W., van Woerden, N., Smulders, F. T. Y., & de Jong, P. J.
39, 75– 82. (2002). Implicit and explicit alcohol-related cognitions in heavy and
Sherman, J. W., Gawronski, B., Gonsalkorale, K., Hugenberg, K., Allen, light drinkers. Journal of Abnormal Psychology, 111, 648 – 658.
T. J., & Groom, C. J. (2008). The self-regulation of automatic associa- Wittenbrink, B. (2007). Measuring attitudes through priming. In B. Wit-
tions and behavioral impulses. Psychological Review, 115, 314 –335. tenbrink & N. Schwarz (Eds.), Implicit measures of attitudes (pp. 17–
Spalding, L. R., & Hardin, C. D. (1999). Unconscious unease and self- 58). New York: Guilford Press.
handicapping: Behavioral consequences of individual differences in Wittenbrink, B., Judd, C. M., & Park, B. (1997). Evidence for racial
implicit and explicit self-esteem. Psychological Science, 10(6), 535– prejudice at the implicit level and its relationship with questionnaire
539. measures. Journal of Personality and Social Psychology, 72, 262–274.
Spruyt, A., De Houwer, J., Hermans, D., & Eelen, P. (2007). Affective Wittenbrink, B., Judd, C. M., & Park, B. (2001). Spontaneous prejudice in
priming of non-affective semantic categorization responses. Experimen- context: Variability in automatically activated attitudes. Journal of Per-
tal Psychology, 54, 44 –53. sonality and Social Psychology, 81, 815– 827.
Spruyt, A., Hermans, D., De Houwer, J., & Eelen, P. (2002). On the nature
of the affective priming effect: Affective priming of naming responses.
Social Cognition, 20, 225–254. Received May 19, 2008
Spruyt, A., Hermans, D., De Houwer, J., Vandekerckhove, J., & Eelen, P. Revision received September 3, 2008
(2007). On the predictive validity of indirect attitude measures: Predic- Accepted September 8, 2008 䡲