INTERNATIONAL JOURNAL OF SELECTION AND ASSESSMENT VOLUME 11 NUMBERS 2/3 JUNE/SEPTEMBER 2003

Applicant and Recruiter Reactions to

New Technology in Selection:
A Critical Review and Agenda for
Future Research
Neil Anderson
University of Amsterdam

This paper presents a narrative review of recent research into applicant and recruiter
reactions to new technology in employee selection. Different aspects of the use of new
technology are noted including computer-based testing, Internet-based recruitment and
candidate assessment, telephone-based and video-based interviews, video-based situa-
tional judgment tests, and virtual reality scenarios. It is argued that an appropriate way to
conceptualize these advances is as ‘technical innovations’ as defined in the creativity and
innovation research in Industrial, Work, and Organizational (IWO) psychology.
Applicant reactions research is reviewed thematically, and studies into three main themes
are discussed: Applicant preferences and reactions, equivalence, and adverse impact.
Following Bartram (2001), an amphibian-monarchistic analogy is employed at several
stages in the review. Four major criticisms of the extant applicant reactions research base
are noted: its atheoretical orientation, a short-termist concentration upon reactions level
outcomes, an over-reliance on students as surrogates, and a patchiness of coverage of
crucial research questions. The second part of this paper explores neglected issues of
recruiter adoption of new technology for employee selection. Again drawing from
advances in the innovation and creativity literatures, this section explores likely
antecedent factors at the individual and organizational levels of analysis. A general
model of recruiter adoption of new technology is posited as a framework for future
research in this area. For both applicant and recruiter reactions further research is called
for and implications for practice are noted throughout.

Introduction equivalence, amongst other important considerations

(e.g. Anderson, Born, and Cunnigham-Snell 2001; Harris

I n recent years there has been a rapid growth in research

into applicant reactions to new technology in employee
selection procedures. Not surprisingly, given the increasing
1999, 2000; Highhouse and Hoffman, 2001; Hough and
Oswald, 2000; Lievens and Harris, 2003). Although it has
primarily been the growth in the use of computer-based
use of more advanced technology by organizations to testing procedures and Internet-based platforms for the
deliver recruitment and assessment methods to applicant delivery of company web sites for recruitment and
pools, this has spurred the interest of researchers and prescreening tests which has driven these developments,
practitioners alike in the effects that new technology other forms of new technology delivery of selection
may be having upon applicant reactions, fairness, and methods have also been evident including telephone-based
interviews, video-based situational judgment tests, and
virtual reality technology.
Address for correspondence and reprints: Neil Anderson, Department An appropriate way to conceptualize such advances, it is
of Work and Organizational Psychology, University of Amsterdam,
argued, is that they represent ‘technical innovations’
Roetersstraat 15, 1018 WB Amsterdam, The Netherlands. Email:
nanderson@fmg.uva.nl (Damanpour, 1990) as defined in the wider creativity and

r Blackwell Publishing Ltd 2003, 9600 Garsington Road, Oxford OX4 2DQ, UK and
350 Main Street, Malden, MA 02148, USA. 121
122 NEIL ANDERSON

innovation literatures in IWO psychology (e.g. West and tions over the willingness of different recruiters to consider
Farr, 1990). Regrettably the selection and innovation adopting newly emergent technological solutions for staff
literatures have remained almost entirely separate, with selection, their likely attitudes to different levels of
each sub-field having developed in notable isolation from technological complexity, their beliefs over the importance
the other. This has been particularly unfortunate in the case of the impact of company recruitment brochures and web
of technological innovation in selection, it is argued, as site designs, adoption rates of new technology in selection
advances in understanding of technical innovation pro- by different sectors of organizations likely to be competing
cesses in organizations could usefully have been applied to for staff with scarce skills or abilities, are all clearly
the introduction, adoption, and adaption of new technol- important issues for pragmatic research. Yet, this ‘side of
ogy in employee recruitment and selection. This paper the equation’ in new technology research has been
therefore attempts to draw links and parallels between neglected by recent efforts among personnel psychologists,
these two disparate sub-fields in IWO psychology. Parti- and as a result our understanding of these questions lags
cular emphasis is given to definitions of innovation, significantly behind our current state-of-knowledge into
whether all forms of new technology in selection would applicant reactions. The second part of this paper therefore
meet the criteria for being described as ‘innovations’, and highlights the importance of a balanced future research
perhaps more innovatively in the final section of this paper, agenda which includes both applicant and recruiter
to developing and explicating a general model predicting reactions to new technology. Drawing from the now
recruiter adoption of new technology in selection. mature field of innovation and creativity in the workplace
several key directions for future research are proposed as
crucial outstanding issues for concern to which future
Plethora and Paucity
research needs to be addressed.
Overviewing developments in studies into applicant reac-
tions to new technology in selection, the bulk of the
Technological Innovation in Selection: Definitions
research effort has actually been very recent, largely driven
and Trends
by the exponential growth in the use of the Internet and
other computer-based testing batteries. Consequently, A cursory review of the journals, books, and other
much of the research reviewed in this paper has been literature published over recent years reveals a range of
published only over the last decade or so. It is therefore examples of ‘new technology’ being used for employee
timely to overview advances generally in research into new recruitment and applicant assessment (e.g. Bartram, 2001;
technology in staff recruitment and selection, to flag up Lievens, Van Dam and Anderson, 2002). Of course, it is not
important findings and themes of enquiry, and to highlight immediately apparent how the term ‘new technology’
promising directions for future research. Compared with should be defined (see below), but for the present review a
this plethora of recent research into applicant reactions to broad and encompassing interpretation of this term was
new technology, there has been a striking paucity of used in order to include as many aspects of the proliferation
research into recruiter reactions to such advances. The of emergent technology in selection as possible. Our review
second part of this paper therefore contrasts the rapid covered all major journals (Journal of Applied Psychology,
advances being made in our understanding of applicant Personnel Psychology, International Journal of Selection
reactions to new technology against the virtually unre- and Assessment, Academy of Management Journal, Hu-
searched range of questions which still remain over likely man Performance, etc.), several earlier special issues of
recruiter reactions to, and adoption of, new technological journals (e.g. International Journal of Selection and
platforms. As an initial effort to structure the research Assessment, 1995, 3 (2)), numerous authored and edited
agenda in this regard, again the field of innovation in books, unpublished technical and consultancy reports, and
organizations is argued to be a valuable and potentially the published conference proceedings of international
synergistic topic area from which we can draw for future conferences where there has been a spate of recent
research into recruiter reactions (e.g. West, 2002). Indeed, symposia addressing particularly applicant reactions to
the adoption of innovations by organizations has been a Internet-based testing (e.g. Baron and Austin, 2000; Harris,
longstanding theme for research in the innovation litera- Paajanen and Blunt, 2003, Derous, 2003). In addition,
ture and research in this area has uncovered a number of researchers active in this field were contacted with a
robust factors which influence willingness to adopt and request for offprints and in-press or in-progress papers. As
adapt to new workplace practices (see for instance King mentioned previously, this area is relatively newly estab-
and Anderson, 2002 for a comprehensive review). A lished in personnel psychology, a point vividly reflected in
recurrent theme in the innovation literature has been the the recency of the publication dates of research cited in the
adoption of new technology by organizations and so it is present paper. Most developments have occurred since the
entirely appropriate to consider this issue with respect to early 1990s onwards, and in relation to applicant reactions
the adoption of new technology by recruiters for staff there is a clear and encouraging pattern of exponential
selection and assessment procedures. For example, ques- growth in the number and range of papers published.

International Journal of Selection and Assessment r Blackwell Publishing Ltd 2003

APPLICANT AND RECRUITER REACTIONS 123

Despite this proliferation of research interest, the moot definition argues that an innovation must be new to the
point of how one is to define exactly what is, and what is ‘relevant unit of adoption’, what has been termed relative
not, ‘new technology’ in employee selection has received as opposed to absolute novelty in the innovation literature
little attention in the literature. A valuable starting point is (Anderson and King, 1993; King and Anderson, 2002). In
to refer to Damanpour’s (1990) characterization of other words, that the perception of the innovation, in this
technical innovations which he states include new pro- context the use of new technology for selection, must be
ducts, services or technological processes directly related to new only to the section which is now adopting it, usually
the primary work activity of the section or organization. In HRM departments responsible for applicant screening.
relation to the HRM function, this would clearly include This is an important distinction since absolute novelty
the use of new technology in selection. However, there is requires that the process or technology is not in usage
undoubtedly considerable variation between organizations anywhere else currently, that is, that it is unique. Hence, the
and HRM departments in their level of technical sophis- adoption and use of computer-based testing by an HRM
tication (Bartram, 2001; 2002). For some highly techno- section which had in the past only used pen-and-paper tests
logically-sophisticated organizations themselves involved would be deemed to be innovative in its use of new
in Internet solutions consultancy, possibly even the most technology. Third, the innovation must necessarily demon-
advanced of our current platforms for delivery of assess- strate some benefits at the individual (recruiter job
ment methods in staff selection would be perceived as performance), group (HRM section), or organizational
outmoded, archaic software which by no means would level of analysis. Thus, a failed attempt to introduce new
qualify as ‘new’ technology. For other organizations where technology into the screening process would clearly not
recruitment has been traditionally carried out via hard meet this criterion as neither would the introduction of new
copy paper-based vitas and candidate assessment has technology which performed worse in comparison with
involved only a series of unstructured face-to-face inter- existing, traditional methods. It is important to note,
views, the prospect of conducting simple structured nonetheless, that the distinction between ‘absolute’ and
telephone-based interviews would be viewed as being ‘relative’ novelty implied in West and Farr’s definition
radically innovative. From an applied perspective, there- clearly has sensible limitations. It would not be reasonable
fore, it is likely that recruiters will vary considerably in to argue that the very last HRM department in an industrial
what they would accept to fit the criteria of being ‘new’ and sector to introduce Internet-based testing was being
‘technological’ in selection contexts. innovative, far from it. Merely following the trend long
Conversely, across the extant literature a common behind comparable departments or organizations would be
understanding and metric appears to have been adopted at such a low point on a relative novelty scale as to make
such that self-evidently new technology as the Internet, this conformity behavior rather than innovative behavior
computer-based delivery of assessment instruments, and (see also, West, 2002). Additionally, innovation is most
CD-ROM platforms for scenario presentations to appli- appropriately conceptualized as a cyclical, ongoing, and
cants would undoubtedly fulfill the criteria. Indeed the iterative set of events and processes of re-structuring
present issue of this journal reports a number of studies into (Anderson, De Dreu and Nijstad, in press). That is, it is
such applications. Before moving on to consider the not a single, outcome event, or in personnel psychology
existing research it is however important to clarify what terminology, ‘criterion measure’. Previous innovations spur
precisely is meant by the term ‘new technology’. West and changes in work role and work team functioning that
Farr (1990) define innovation as follows: are likely to be facilitative of a future propensity to
innovate at either of these levels-of-analysis (Anderson
y the intentional introduction and application within a et al., op. cit.).
role, group or organization of ideas, processes, products or Although as Nicholson (1990) points out, these criteria
procedures, new to the relevant unit of adoption, designed and points of semantic debate simply push the definitional
to significantly benefit the individual, the group, the boundaries back a step so that one then has to define what
organization or wider society (p. 9) is meant by ‘intentional’, ‘beneficial’, and even ‘new’, the
West and Farr formulation is helpful. Moreover, as
Their definition is constructive and valuable in several Amabile (1983) has argued, such definitions force
regards. First, West and Farr emphasize that innovation researchers to attend to what study participants themselves
must confer an intended benefit at one or more levels of perceive and rate to be innovative practice, rather than to
analysis – the job role, work group or wider organization. impose upon them an isolated, irrelevant and amorphous
Most applications of new technology in selection will meet terminology. Finally, these aspects of the innovation
this criterion as they have typically been medium- to large- literature are valuable in that they highlight the dangers
scale innovation projects intended to transfer existing or of adopting a so-called ‘pro-innovation bias’ (King and
modified predictor methods onto a new platform for Anderson, 2002), that is the unverified assumption that
delivery to applicant pools. Second, but somewhat more new technology solutions are better than existing, tradi-
problematic concerning new technology in selection, their tional methods. Little research, if any, in the personnel

r Blackwell Publishing Ltd 2003 Volume 11 Numbers 2/3 June/September 2003

124 NEIL ANDERSON

selection field appears to have critically evaluated whether Applicant Reactions: Three Major Research
more highly advanced technological forms of delivering Themes
selection methods actually perform better than traditional
modes of delivery, as discussed later in this paper. Overview
If one accepts the West and Farr (1990) definition, which
This section presents a themed, targeted narrative review of
has indeed been widely the case in the innovation literature,
the main research findings into applicant reactions to new
an overview of the reported incidents and uses of new
technology in selection. It does not attempt to provide a
technology in selection over the last decade or so would
complete, detailed review of all primary studies into every
most probably include all of the following: Computer (PC)-
aspect of applicant reactions or every permutation of use of
based testing, Internet-based testing, telephone-based
the new technology (see above) as this would be beyond the
assessment procedures, computer-based realistic job pre-
scope of a single paper. In addition, several comprehensive
views and situational judgment tests, multimedia simula-
reviews of candidate reactions to selection procedures in
tion tests, and virtual reality immersion testing. In many of
general have been published elsewhere recently (e.g. Ryan
these applications, as noted earlier, it has been rapid
and Ployhart, 2000; and, Anderson, Born and Cunning-
advances in the power and cost-effectiveness of stand-alone
ham-Snell, 2001), as have several reviews of applicant
personal computers and their use in connection with the
reactions to specific aspects of new technology in selection
Internet which has catapulted them into becoming viable
(see Alkhadher, Anderson and Clarke, 1994; Bartram,
tools for use in selection procedures (see Bartram, 2002;
1994 for reviews of computer-based testing; Harris, 2002
Harris and De War, 2001, for instance). Standing in
on Internet-based GMA testing; and, Lievens and Harris,
relation to this list of new technology is the range of uses to
2003 for a comprehensive review of Internet-based
which it has been put in selection, which includes the
recruitment and testing). Three main themes are identified:
following: Vacancy advertisements, company and job
information provision, prescreening methods (recruitment 1. Applicant preferences and reactions
short listing, biodata forms, candidate self-assessment, 2. Equivalence
and so forth), candidate assessment methods (interviews, 3. Adverse impact
cognitive ability tests, personality tests, situational
judgment tests, and so forth), feedback reports to Under each theme research findings are reviewed
applicants and to communicate outcome decisions, and broadly in chronological order as the methods would
finally, data collection for monitoring of adverse impact occur in organizational selection practices – i.e. recruit-
and test validation amongst other purposes. Both lists ment, applicant prescreening methods (application forms,
are not intended to be exhaustive however, but they curriculum vitas, biodata forms), and candidate assessment
are representative of the apparently major forms and methods (interviews, cognitive ability tests, personality
uses of new technology in selection reported in the inventories, other forms of testing, and assessment centers).
literature. The review focuses on key, replicated themes of findings
In some areas, for instance the use of Internet-based rather than on an exhaustive review of all published studies
recruitments and tests of general mental ability (GMA), across all areas of enquiry, therefore. Of necessity, some
developments have been particularly rapid and substantial judgment calls had to be made by the present author over
research is already evident (for a comprehensive and up-to- what these key themes of findings in the present research
date review see Lievens and Harris, 2003); in other areas base were, but in all cases these were identified where (a)
the popularity of using the technology in practice appears several independent studies had produced similar or
to have outstripped the pace of developments in the identical findings, thus permitting some reliance upon the
validatory research base (e.g. concerns over use of the particular finding, and (b) where individual study meth-
Internet for recruitment with regards potential adverse odologies were sufficiently robust as to warrant some
impact and inequalities of access); and in still other areas weight being placed upon their findings, especially where
the new technology appears to be only being used by the multiple studies had used somewhat different but related
most selection-sophisticated organizations who can afford and robust methodologies. In fact, applying these condi-
the high start-up and maintenance costs (e.g. simulator tions, several of the emergent findings in applicant
training for pilot and police driver selection). Thus, there reactions to new technology in selection did not meet one
are considerable differences in both the uptake of new or other of them and were thus omitted from the present
technology for delivery of different predictor methods, and review.
simultaneously, in the maturity of the research-base to
support and verify such use in practice. Whilst this is not
1. Applicant Preferences and Reactions
surprising given the emerging nature of much of the
hardware and software involved, this disparity in usage Research has only very recently begun to explore applicant
and validation remains a challenge for future research in reactions in recruitment phases particularly to Internet-
this area. based company information and application procedures.

International Journal of Selection and Assessment r Blackwell Publishing Ltd 2003

APPLICANT AND RECRUITER REACTIONS 125

The popularity of company web sites certainly cannot be expectations of, and attitudes toward, organizational
denied with one study in the USA reporting that 88 percent selection procedures.
of Global 500 companies having recruitment sections to Applicant reactions to computer-based testing, that is
their company web sites (iLogos, 2001 cited in Lievens tests administered to applicants using personal computers
and Harris, 2003). Further, another US survey found or a networked internal computer system as opposed to
that across nine larger American companies 16 percent of Internet-based testing, have been a topic for research
all subsequent hires originally applied through the interest in the selection literature for a relatively longer
company’s Internet recruitment sites (Maher and period of time (e.g. Alkhadher et al., 1994; Bartram, 1994).
Silverman, 2002). Thus, it would be premature to state Overviewing the now quite voluminous number of studies
that web-based recruitment has completely replaced conducted in this area, the findings can at best be described
traditional, paper-based application procedures, or even as varied and diffuse. On the one hand, a string of studies
for that matter has become the majority route for attracting report positive applicant reactions to sitting computer-
in applications. But certainly amongst larger, multinational based tests as opposed to their pen-and-paper counterparts
organizations there has been a marked trend toward (e.g. Mathisen, Evans, Meyers and Kogan, 1985; Burke,
Internet-based recruitment over the last decade. Integral Normand and Raju, 1987; Davis and Cowles, 1989; Arvey,
to this trend is undoubtedly the likelihood that applicants Strickland, Drauden and Martin, 1990; Schmitt, Gilliland,
appreciate the possibility of being able to log-on from Landis and Devine, 1993). On the other, several studies
home to an organization’s web site to download informa- have either failed to find more positive reactions (Wiech-
tion directly. Thus, it is not clear at the present time mann and Ryan, 2003) or have reported negative reactions
whether this trend is largely being driven by organizational by applicants (e.g. Martin and Nagao, 1989; Meir and
preferences, applicant preferences, or a combination of Lambert, 1991). Of all of these, the recent study by
both. In contrast to this developing use in industry, research Wiechmann and Ryan (2003) reveals the importance of
into applicant reactions to Internet-based recruitment using carefully designed, experimental manipulations of
has lagged behind. Only a couple of published and test-taking conditions to participants. Simple, post-test
in-press studies can be located which examine in any detail only reactions designs may also fail to account for other
the reactions of potential applicants to company web important moderator variables such as applicant mood,
sites (see also, Salgado, this issue). However, these exposure to computer-based tests in the past, computer
studies do show encouragingly positive findings, although anxiety, and so forth (Wiechmann and Ryan). Thus, the
both were conducted using university students as most probable explanation for these conflicting findings is
participants. Rozelle and Landis (in press) compared that the design of the test itself together with moderator
traditional forms of recruitment (college visits, company variables such as how it was administered, treatment of
brochures, etc.) against Internet recruitment sites. applicants during the selection process generally, and
They found that students perceived the latter as more whether there existed an opportunity to receive feedback
realistic but also less formal compared against traditional on test performance, influenced testee ratings of the
methods for student selection. Similarly, Zusman and computer-based testing session. Although some research
Landis (in press) reported that undergraduate students into this mode of delivery of both cognitive ability and
rated Internet-based postings for job placements more personality tests is on-going, much of the research attention
favorably than more traditional documentary sources. has turned toward Internet-based delivery of such testing
Whilst these are encouraging findings it is likely that sessions (Bartram, 2002; Harris, 2002).
students as younger applicants to the labor market will rate A flurry of studies has very recently appeared examining
Internet-based recruitment media more favorably than applicant reactions to Internet-based testing procedures.
older, more experienced, and possibly less computer- Indeed, the use of the Internet for both recruitment and
literate applicants. Note also that these studies concerned testing functions appears to hold out considerable promise
student recruitment (Rozelle and Landis) and job place- as a technology which may transform these aspects of
ment recruitment for students (Zusman and Landis) selection in future (Buchanan and Smith, 1999; Bartram,
respectively so it would be unwise to generalize from these 2001; Mead, 2001; Stanton and Rogelberg, 2001; Paronto
exploratory findings to other sectors of labor market et al., 2003; Derous, 2003). But it has only been in the last
recruitment. Care should even be taken in generalizing four years or so that such developments have taken place
these findings to recruitment for permanent job vacancies and it appears at the moment that the research base is
in industry (e.g. Lievens, van Dam and Anderson, 2002; struggling to keep pace with the speed with which Internet-
Anderson, Lievens, Van Dam and Ryan, in press). based testing is being adopted by organizations interna-
Undoubtedly, future research is called for which extends tionally (Lievens and Harris, 2003). So far, almost
these initial findings to investigate reactions of other types universally positive applicant reactions have been reported
of applicants, to differing formats of web site design, for Internet-based testing batteries. Mead (2001), and
follow-up procedures for applying online, and the longer- Reynolds, Sinar and McClough (2000) both found more
term impact of initial reactions of applicants upon later positive candidate reactions to Internet-based personality

r Blackwell Publishing Ltd 2003 Volume 11 Numbers 2/3 June/September 2003

126 NEIL ANDERSON

tests than to traditional, pen-and-paper forms of adminis- reactions to a screening scenario test, similar in content to
tration. For cognitive ability testing via the Internet, Baron an SJT, across three conditions, interactive voice response
and Austin (2000) report positive reactions to GMA tests by telephone, telephone-based interview screening, and
administered over the Internet. In a large-scale analysis face-to-face interview screening. Most positive responses
Reynolds and Lin (submitted) also report favorable were to the final condition, that is the face-to-face
reactions by applicants to a range of tests being adminis- interview. Chan and Schmitt (1997) investigated whether
tered by Internet platforms, and so across this spate of differences in video-based versus paper-and-pencil SJTs
recent studies, not one has yet reported negative reactions were moderated by the ethnic origin of applicants. As
by applicants to web-based testing administration. Indeed, hypothesized, they found overall that face validity was
Internet-based test administration holds advantages for rated higher for the video-based condition, but also that
both the organization (e.g. cost savings, remote adminis- face validity differences and test performance differences
tration, data collection and monitoring) and the applicant between whites and blacks were significantly smaller for
alike (e.g. no need to physically attend the organization in the video-based SJT (see comments under Adverse Impact,
the case of remote administration, freedom to choose when below).
the test is taken, etc.), and so it is most likely that Internet- Finally, although not an empirical study into applicant
based testing will become substantially more popular in reactions, Aguinis, Henle and Beaty (2001) usefully
future years. Internet-based selection procedures can also review potential applications for use of virtual reality
have a positive impact on perceptions of the organization, technology (or VRT) in selection and assessment. They
that is company image, amongst potential applicants note that VRT incurs very high set-up costs compared with
(Reynolds and Sinar, 2001). At present, it is the larger, other modes of test or SJT delivery and so is probably only
multinational organizations recruiting larger numbers of viable for use in high-demand, high-risk jobs such as pilot
employees that have adopted Internet-based recruitment and other military personnel selection and training. Certain
and testing, especially organizations recruiting from a knowledge, skills, abilities, and other characteristics
nationally or internationally mobile pool of applicants (KSAOs) are more suitable for assessment via VRT
(Bartram, 2002; Lievens and Harris, 2003). In future, we methods, it is argued, including manual dexterity, certain
are likely to see this technology proliferate downwards types of job knowledge, and cognitive and psychomotor
with Internet-based procedures being adopted by medium- functions.
sized and smaller organizations and with access to the
World Wide Web becoming increasingly widespread
2. Equivalence
internationally (Bartram, 2001; Harris, 2002; see also
Van de Ven et al., 1989 for a review of innovation With regard to recruitment there has been a notable
dispersion). shortage of research comparing the equivalence of tradi-
Applicant reactions to different modes of conducting tional forms of sourcing applications against new technol-
employment interviews have been examined by a few ogy sources. Perhaps this is to some extent due to
studies (Straus, Miles and Levesque, 2001). Straus et al. methodological difficulties inherent in designing lab and
(2001) evaluated student reactions to mock interviews field studies to examine equivalence with sufficient
conducted in face-to-face, videoconference, or telephone- robustness. Whatever the reasons, existing studies appear
based conditions. Although the sample size was small to have limited their focus to that of applicant reactions
(n 5 59) and these were simulated employment interviews to new technology rather than to encompass wider
with MBA students, applicants rated the videoconference questions of whether outcome equivalence does indeed
condition significantly less favorably than face-to-face or exist. The most central question is whether use of new
telephone-based interviews. Other research into recruiter technology produces the same quantity and quality of
perceptions (Silvester, Anderson, Haddleton, Cunning- applicants for an organization, of course. Related to this is
ham-Snell and Gibb, 2000; Silvester and Anderson, 2003) the question of whether new technology impacts upon
shows that interviewer reactions to information presented applicants’ decisions to self-select-out in any different
by interviewees may differ across modes of interview manner to more traditional forms of recruitment (Breaugh
presentation, in this case face-to-face versus telephone- and Starke, 2000; Dineen, Ash and Noe, 2002), in addition
based interviews. This is considered below in the following to the issue of potential adverse impact associated with this.
part of this paper. Given the increasingly prevalent use of the Internet for staff
A handful of studies have been conducted into applicant recruitment this is a real shortcoming in our understanding.
reactions to computerized job simulations and situational In contrast a number of questions associated with the
judgment tests (SJTs) using new technology, in this case equivalence of pen-and-paper and both computer-based
with more mixed findings. Shotland, Alliger and Sales tests and Internet-based tests have been addressed by
(1998) report positive reactions to a computerized job several primary studies. Indeed, for computer-based tests
simulation developed to select insurance claims personnel. there has been a large number of studies that have
Conversely, Paronto, Bauer and Truxillo (2003) compared examined the equivalence of particular pen-and-paper

International Journal of Selection and Assessment r Blackwell Publishing Ltd 2003

APPLICANT AND RECRUITER REACTIONS 127

cognitive tests and personality inventories with computer- new technology is that of the potential for new modes of
based formats of the same measures (for major reviews and predictor method delivery to influence adverse impact in
meta-analyses see Mead and Drasgow, 1993; Alkhadher selection. Most interestingly, research suggests that this
et al., 1994; Bartram, 1994; Finger and Ones, 1999; Harris influence may be either beneficial or detrimental, or both,
and De War, 2001). Generally, the results have been depending upon how the technology is used and at what
positive and even the findings of the few studies which do stages in the recruitment and selection process. Indeed, the
report significant between-form differences may be attri- findings are actually bimodal in that on the one hand new
butable to sampling errors within individual studies (Finger technological approaches have been argued to exacerbate
and Ones, 1999). As Harris (2002) also points out, of potential for adverse impact against minority groups,
greater import than simple questions of between-form whereas on the other, several studies unequivocally reveal
measurement equivalence is whether predictive validities the potential for new technologies to ameliorate bias
or adverse impact ratios differ (see below). For Internet- against minorities. In general this bimodal trend in the
based tests, the present review failed to locate any papers research findings splits conveniently between recruitment
published as yet into this question either for tests of on one side and later stage selection methods on the other.
cognitive ability or personality inventories. Internal, With regard to adverse impact in recruitment warning
proprietary research may well be being conducted by bells have been sounded by several authors of late (Sharf,
commercial test publishers into this question, but as yet it 2000; Bartram, 2001; Harris, 2002; Lievens and Harris,
does not seen to have appeared in the public domain. Given 2003). This stems from differences between minority
the growth in the use of the Internet as a platform for test groups in being able to easily access the Internet via
delivery to applicants this is an area in which more research personal computers in the home, or so-called ‘e-loaded’
is sorely called for. adverse impact (Sharf, 2000). Harris (2002) quotes most
A few studies have been published into the equivalence recent statistics in relation to household access to the
of face-to-face, telephone-based, and video-based inter- Internet for homes in the USA. Internet usage percentages
views. Silvester et al. (2000) and Silvester and Anderson (as at September, 2001) were as follows: Whites, 60
(2003) compared face-to-face and telephone-based inter- percent; Asians, 60 percent; Blacks, 40 percent; and,
views. In both studies they failed to find acceptable Hispanics, 32 percent. Thus, households of White or Asian
equivalence between each mode of delivery both in the ownership were almost twice as likely to have access to the
response patterns of interviewees across conditions and the Internet than for Hispanic occupants. In other countries it
attributions made by interviewers from these responses. is likely that these socio-economic differences in Internet
Recruiters rated applicants less leniently in the telephone- access will be similar in their effects, although of course the
based condition and also attributed candidates’ responses prevalence of Internet usage will be different as will the
to structured questions less positively in this condition than minority groups involved (see Bartram, 2001). Whatever
in face-to-face interviews. Straus et al. (2001) found the these international differences, the implication is identical:
opposite, however, that interviewers rated candidates more minority groups that are concentrated in more socio-
favorably in the telephone-based condition than face-to- economically deprived areas will have substantially lower
face, but again equivalence was not demonstrated. It would levels of access to the Internet via home computers. As
seem that equivalence cannot be taken for granted in new Sharf (2000) and Bartram (2001) point out, this has
technological administrations of selection methods even important knock-on effects to the proportions of minority
where this ‘new’ technology is more basic in nature, such as group members who will be able to access easily Internet-
telephone-based interview procedures. based recruitment sites. It also holds the potential for legal
To summarize, the question of equivalence has received challenges against organizations using this technology but,
some research attention across different predictors with in so doing, who may be unwittingly operating an anti-
rather mixed findings. In terms of computer-based testing diversity policy through differential access to the Internet
the findings are generally positive, for Internet-based tests based upon socio-economic disadvantage. Harris (2002) in
there is a notable lack of published work, and for a landmark study based upon a huge sample of applicants
interviews the findings are contradictory but not in favor for customer service jobs (total n 5 1.3 million) did not,
of equivalence. None of these studies has moved on to however, find percentage differences between traditional
examine more demanding questions of differences between sources of recruitment referral and Internet sourced
forms in the predictive validity of alternative modes of applications. Further, another study (Sinar and Reynolds,
delivery in comparing traditional and innovative modes of 2001) does suggest that once logged-on to Internet
presentation to applicants. recruitment sites individuals from racial minorities tend
in general to react more favorably. Combined, these two
studies suggest that there may be few or no conversion-rate
3. Adverse Impact
differences into actual applications between majority and
The third, and in many ways most challenging issue to have minority group members once logged-on to organizational
received attention in research into applicant reactions to recruitment web sites. If so, the first imperative appears to

r Blackwell Publishing Ltd 2003 Volume 11 Numbers 2/3 June/September 2003

128 NEIL ANDERSON

be for organizations to ensure that sufficient proportions of was not aware of the candidates ethnic origin. In this case,
minorities obtain access to their recruitment web sites in therefore, the lack of visual cues available to the
the first place. interviewer conducting telephone-based interviews may
However, this initial imperative opens-up a whole series have mediated against adverse impact, an unintentional
of consequent issues and research questions. Some types of but beneficial outcome from the use of technological
organization are likely to have better resources and innovations in interviewing (see also Silvester and Ander-
knowledge in order to be able to cope with the potential son, 2003). Visual cues were found to have a rather
of e-loaded adverse impact. For instance, organizations different effect by Chan and Schmitt (1997). They
with a long experience of dealing with unfair discrimina- compared Black-White differences on a situational judg-
tion professionally (e.g. state sector bodies), those with ment test administered to subjects in both pen-and-paper
internal expertise on tap (e.g. military divisions with access and video-based conditions. It was found that ethnic
to personnel psychologists), and those with sufficient slack differences in test performance were significantly smaller in
resources to buy-in consultancy expertise (e.g. financial the video-based condition than in the traditional pen-and-
sector employers and multinational companies). Whether paper format. So, to summarize, the present position
these organizations are the most attractive as potential regarding adverse impact and new technology in selection
employers to applicants from minority backgrounds is a can best be described as mixed; concerns over Internet
question for empirical research. This leads onto another access for ethnic minority groups have been voiced but
vexed issue: self-selection by minority group members against this there seem to be no conversion rate differences
themselves causing possible range restriction in the between Whites and ethnic minorities once logged-on to
proportions of applicants received by an organization. company recruitment sites. Finally, there is some initial
Schmidt (2002) provides a seminal discussion of this very evidence that the use of new technology for test and
point in relation to general cognitive ability (GCA) or what interview administration may reduce adverse impact, a
has been termed elsewhere general mental ability (GMA), highly beneficial potential effect that is surely worthy of
(see also, Kehoe, 2002). Currently, at least in the US, it has further research attention.
been argued that the lower proportions of minority group
members who have access to the Internet is the cause for
Critique: Kissing Princes to Find Frogs
concern (cf. Sharf, 2000). I am indebted to the comments of
one of the anonymous reviewers who noted that this may In critically reviewing this body of research into candidate
be ameliorated or exacerbated by self-selection effects by reactions to new forms of technology in selection one is left
minority group members themselves. No published study with both a sense of excitement over an area which is newly
appears so far to have investigated between-group pre- emerging and vibrant, but also a sense of disappointment at
ference differences for using Internet-based application the intellectual rigor and contribution to wider under-
procedures as opposed to traditional documentary meth- standing that these studies combine to make. Should we be
ods. Additionally, restriction of range effects call for further that surprised if applicants presented with better designed
research attention in this regard. For instance, if minority web sites react more positively to poorly designed ones? Or
group applicants entering an organization’s recruitment that reactions to computer-based and Internet-based tests
procedure are comparatively highly qualified and able, this are generally favorable if we limit our subject pool only to
introduces bias in favor of the minority applicant pool. undergraduate students who have been brought up using
This seems to be a likely scenario with regard to Internet- computers as part of their everyday lives? Or that
based recruitment and testing presently as applicants from applicants prefer to sit Internet-based tests in the comfort
minority groups are likely to be comparatively better of their own homes as opposed to having to attend an
educated, open to using new technology, possess a more organization’s offices for group testing sessions? These
informed network of contacts to be aware of organiza- comments are intentionally critical (and perhaps unfairly
tional web sites, and be generally advantaged compared so), but overviewing this newly established research field
against even the majority group. These possible explana- one is forced to conclude that it now runs the risk of
tions may account for the following findings, that new imploding into a quasi-science of empirically proving
technology can reduce minority-majority group differences rather common sensical observations. We need to take
in selection procedures. steps at the present juncture to prevent this from happening
Counter to the concerns over e-loaded adverse impact as this would be to deny the wider field of selection research
stand the results of several studies showing that selection important on-going findings which have the potential to
methods being administered by new technology can reduce add significantly to our understanding of applicant and
or even negate minority-majority group differences and recruiter reactions. Ostensibly these refer to new technol-
recruiter discrimination. First, Silvester et al. (2000) ogy, although it can be persuasively argued that such
suggest that telephone-based interviews may have reduced findings are likely to have wider generality to reactions to
opportunities for recruiters to give less favorable ratings to other impending changes in employee selection. How can
ethnic minority applicants simply because the interviewer the sub-field of reactions research make quantum leaps

International Journal of Selection and Assessment r Blackwell Publishing Ltd 2003

APPLICANT AND RECRUITER REACTIONS 129

forward in the foreseeable future and in so doing avoid cally, and generally from more attention to underlying
becoming encircled in a worthy but dull cul-de-sac of theoretical propositions and hypothesis formulation as to
routine endeavor to quantify fairly obvious reactive out- why applicants may react in the ways they are subsequently
comes? Research needs to move beyond immediate level found to do.
reactions outcomes, to incorporate appropriate and Second, applicant reactions research can be criticized for
competing theoretical rationales, and to move away from its short-sightedness in terms of the type and longevity of
an excessive reliance upon students as surrogate samples to outcomes it has chosen to address. The majority of studies
include other candidate groups applying for actual have only considered immediate-level reactions outcomes
vacancies in the field. and have largely ignored longer-term and potentially more
First, much of the research has been rather atheoretical permanent effects and outcomes. We can draw a parallel
in its orientation, conceptualization, and design. Research here with the training evaluation research and models of
has in general simply addressed rather descriptive level ‘levels of outcomes’ proposed as useful heuristics in this
questions, such as applicant favorability reactions or research (e.g. Goldstein, 1997). Why limit our research
between-forms equivalence, rather than being based upon questions to the level of immediate, preferential and
articulated theoretical formulations or even postulated reactions outcomes by applicants to new technology in
models of interactions between relevant variables (see also selection? Rather, important longer-term questions of how
Lievens and Harris, 2003). This has led to dust-bowl applicant reactions impact upon their intentions to remain
empiricism. Indeed, it would at present be difficult to in the selection procedure, organizational commitment,
counter this criticism by pointing to a raft of studies in and most importantly, applicant decision making as a result
which a clearly articulated theory or model is postulated of their exposure to different methods in selection need to
and empirically tested. This may again be due to the fact be addressed by future studies (Anderson, 2001; Ryan and
that this area is extremely recent in its emergence, but at Ployhart, 2000). This necessarily implies the use of
times it does appear that studies are largely opportunistic longitudinal research designs capable of tracking applicant
efforts designed to address purely pragmatic and immedi- reactions over time in selection procedures, and I would
ate questions. The how much of applicant reactions concur fully with Lievens and Harris (2003) who call
appears to have dominated any deeper empirical questions strongly for the need to use such designs in future.
of the why of applicant reactions. This is regrettable as the Third, research so far has fallen foul of the charge of
field currently lacks any guiding theoretical frameworks or becoming a ‘science of the sophomore’. That is, the use of
even specified models which attempt to explain why undergraduate student samples as surrogates for real-life
applicants react in the ways that they are found to do. A applicant pools can be criticized as falling back on a
consequence of this largely atheoretical orientation has convenience sample most readily accessible to researchers.
been a lack of inclusion of antecedent variables in applicant One is forced to doubt the generalizability of the findings
reactions studies, an important omission. Such variables as from undergraduate samples acting as surrogates, espe-
past exposure to computers, past experiences of new cially in relation to new technology in ‘selection’. Subjects
technology in selection, computerphobic reactions, and are highly educated, have been habitually exposed to
openness to new experience as a Five Factor Model computers on a daily basis, and in some of the extant
personality variable are all likely to influence applicant studies are applying for mock vacancies not for actual job
reactions, yet have only been included in a couple of studies vacancies at a time when many may be only in their first or
to date (see Wiechmann and Ryan (2003) for instance). second years of undergraduate studies. Given the prolific
Another useful theoretical framework which may be drawn growth in the use of some aspects of new technology by
from is noted by Lievens and Harris (2003). They also organizations for recruitment and selection, one is forced
criticize this area for not basing research questions to question rather acerbically this recourse to laboratory-
sufficiently on existing theory and point to organizational based experiments using student samples. This tactic would
privacy theory and organizational justice theory as two have been more defensible if researchers had been testing
related theoretical orientations. Given the relevance of postulated theoretical models which demanded the control
both theoretical formulations to the impact of new and manipulation of certain key variables. As noted above,
technological forms in recruitment and selection, both of however, this has not been the case. Would we likely expect
these frameworks do indeed appear prima facie to offer older workers, manual workers, those having never used
valuable psychological grounding for future research. To computers in either the work or home lives, or senior board
conclude, it is reasonable to challenge the field to become directors whose subordinate staff typically use computers
more theoretically sophisticated in its recourse to appro- for them to react in identical ways to young, highly
priate theoretical formulations and applied models. Ulti- educated college students? Other research would suggest
mately our understanding of applicant reactions will not. Czaja and Sharit (1998) for instance found signifi-
benefit immeasurably from a more theoretical point of cantly less positive attitudes toward computer-based work
departure being taken by future studies, from alternative tasks among older workers than amongst younger adults.
theoretical frameworks and models being tested empiri- The likelihood is then that our existing findings over-

r Blackwell Publishing Ltd 2003 Volume 11 Numbers 2/3 June/September 2003

130 NEIL ANDERSON

estimate the favorability of reactions to new technology in ments as one delved deeper into the studies (aka Finding
selection, and as can be seen from this review, the majority frogs). In offering this critique of the existing research,
of studies have indeed found positive results. Research in these comments are intended to be challenging but
future should examine the reactions of different sectors of throughout to offer constructive points on which we may
the labor market to different types of new technology in improve the robustness and coverage of this newly
selection, and in the absence of needing to control key emerging field of research.
variables for imperative theoretical manipulations, to use
field study designs of actual recruitment and selection
scenarios for real-life job vacancies. Recruiter Reactions: Toward a Research
A fourth point of criticism and a direction for future Agenda
research is the patchiness of coverage of research in this
area alluded to earlier. For some issues there has been a Based upon surveys into Internet-based recruitment and
spate of recent research (e.g. reactions to Internet-based testing use by organizations in the USA (e.g. Cober et al.,
recruitment and testing), whereas in others our research 2000; Cappelli, 2001), and anecdotal accounts by
base lags woefully behind the rush to use new technology researchers in this area internationally (e.g. Bartram,
by practitioners in the field (e.g. adverse impact of GMA 2002; Harris, 2002), there has been a headlong rush by
measures delivered to applicants via the Internet). Indeed, larger organizations to use this technology as part of their
the use of new technology by recruiters, most notably those recruitment procedures. For other technological innova-
in large multinational organizations, has in some cases tions in selection method delivery, however, there is scant
forged well ahead of the validatory research base generated survey evidence to indicate just how popular and in
to date by selection psychologists. This is most striking widespread usage these methods are. Thus, for many of
with regard to research into the implications of new the other manifestations of new technology in selection we
technology for adverse impact. It appears that scholars are presently do not know precisely how popular these are
struggling to keep pace with the sheer rapidity of change in amongst employer organizations and recruiters alike. It
some aspects of the use of new technology by selection seems likely for many of these applications, including
practitioners, and therefore, that research will need to telephone- and video-based interviews, video-based SJTs,
catch-back some of this ground over the coming years. In multimedia simulation tests, and virtual reality immersion
addition to adverse impact, another pressing challenge for testing, that only a handful of the most technologically
researchers in this area is the exponential growth in the use sophisticated and well-resourced HRM departments are
of Internet-based platforms for recruitment and assessment using them at present (including military applications,
by major organizations. If, as cited by Dineen et al. (2002), selection for commercial airline pilots, and so forth). In
90 percent of large corporations in the USA use net-based addition, light has not been shed upon cross-national
recruitment (Cappelli, 2001; Martin, 1998) and 12 differences in the use of these different technologies by
percent now use online screening methods (Cober et al., organizations as most of the existing research has
2000), the exposure of applicants at least in the US to originated in the USA which may well be ahead of other
Internet modes of delivery has already been pretty countries in Europe and the rest of the world in terms of
extensive. Questions of faking-good, impression manage- organizational take-up rates for new technology in selec-
ment, and practice effects over the Internet therefore tion. Regardless of the need for ongoing research to
become prevalent, although again the huge growth in use elucidate these questions, it seems reasonable at least in
by practitioners appears to have outstripped the ability of this narrative review to highlight an apparent split between
the research base to respond so speedily to such important the exponential growth in Internet-based recruitment usage
issues. In some years time it will be most interesting to look which has occurred in recent years and the somewhat
back upon this review and to compare how the coverage of isolated examples of organizations using other technologi-
research on these key issues has developed and to review cal innovations for candidate assessment and evaluation.
how the research base has progressed and expanded to Why has this been the case? What factors may be
redress some of these points of constructive criticism. influencing organizations and recruiters in their take-up of
These four criticisms are therefore put forward as the new technology in selection? What may be the trajectory of
reactions of one researcher, the author, to existing research the innovation processes involved for organizations adopt-
in this area (which arrived on his desk in both paper-based ing such technological solutions to employee resourcing
and computer-based formats). Bartram (2001, p. 265) cites needs? And how best can future research in IWO psychology
Lawrence (1999) who, when interviewing one recruiter conceptualize, model, and empirically investigate these
forced to sift through many vitas before finding a strong processes of innovation adoption by recruiters? These are
one, likened it to ‘kissing frogs before you find a prince’. the key questions that concern this section of this paper.
The present author’s expectations of this field of research Past research into recruiter reactions to technological
were very high before beginning this review (aka Kissing innovations in selection is somewhat scarce. Bartram
princes), but in some respects this resulted in disappoint- (2001) usefully reviews a series of surveys carried out by

International Journal of Selection and Assessment r Blackwell Publishing Ltd 2003

APPLICANT AND RECRUITER REACTIONS 131

professional and recruiter associations in the UK which levels of variables are proposed as likely predictors of
also suggests a phenomenal growth in the use of Internet- recruiter adoption: (1) Organizational level variables,
based recruitment methods. However, the literature search which are postulated as ‘necessary conditions’ for adop-
carried out for the present paper could locate no published tion, and (2) individual level variables associated with the
research by IWO psychologists which has examined recruiter and their job role, which are proposed as
recruiter attitudes to, expectations of, or adoption of new ‘sufficient conditions’ over and above pre-existing neces-
technology in selection. This is surprising for at least two sary conditions.
main reasons. First, there has been a proliferation of very
recent research into applicant reactions, as noted in the first
Organizational Level Variables: Necessary
section of this paper. Second, recruiters will usually be the
Conditions
client for consultancy organizations in IWO psychology
keen to sell new technology in selection into HRM Several comprehensive reviews of primary studies and one
departments in industry and governmental sectors. It is quantitative meta-analytical summary into antecedent
therefore surprising that we have not witnessed more factors at the organizational level-of-analysis found to be
research that takes the recruiter’s perspective to technolo- predictive of innovation in organizations have been
gical innovation in this area, especially since the dominant published (major reviews include, Zaltman, Duncan and
perspective in selection research in general over the years Holbek, 1973; Van de Ven, Angle and Poole, 1989; West,
has been that of the organization and the recruiter (Ryan 2001; and King and Anderson, 2002; see Damanpour,
and Ployhart, 2000; Anderson et al., 2001; in press). 1991 for a meta-analytic integration). The overwhelming
Drawing from the findings of antecedent factors volume of evidence now shows that organizations with
research in the innovation and creativity literatures, organic strategic plans, flat organizational structures,
Figure 1 presents a general model of recruiter reactions democratic leadership styles, a culture and climate which
to, and adoption of, new technology in selection. Two is supportive of flexibility and change, less rule-governed

Organizational Level Variables: Individual Level Variables: Outcome

‘Necessary Conditions’ ‘Sufficient Conditions’

Organizational Strategy and Job and Task Characteristics

Resources
• Autonomy, and job discretion
• Mechanistic-organic strategic type (‘slack innovation’)
• Culture and climate for • Performance of existing systems
technological innovation (‘distress innovation’)
• Budget funds and resources • Performance reward and
• Leadership style management procedures
• Present and likely future
workloads

• Recruiter adoption of new

technology in selection

HR Department Strategy and Recruiter Characteristics and

Resources Educational Background

• Culture and climate for • Competence with new

technological innovation technology versus
• Leadership style ‘computerphobia’
• Budget funds and resources • Personality
• Exposure to new technology via • Motivation and desire to succeed
external contacts and networks • Propensity to innovate
• Climate for excellence in task • Rule-governed versus rule
performance challenging behavior
• Educational background and
past experiences with new
technology

Moderator variables

• Labor market conditions

• Applicant characteristics,
exposure to new technology, and
likely reactions
• Selection ratios across different
levels of organizational
recruitment

With acknowledgements to earlier models of applicant decision making in selection, especially Ryan and Ployhart (2000), and Anderson et al. (2001).

Figure 1: A General Model of Recruiter Adoption of New Technology in Selection

r Blackwell Publishing Ltd 2003 Volume 11 Numbers 2/3 June/September 2003

132 NEIL ANDERSON

working practices, and sufficient resources to fund research de Ven et al., 1999; Anderson et al., in press). An example
and development efforts, are unsurprisingly, more likely to of the former in the context of recruitment is where an HR
be innovative in terms of both their product mix and their practitioner has relatively free time in between large-scale
internal business processes (Damanpour, 1990). By exten- recruitment drives and has available sufficient resources to
sion, Figure 1 posits that HR departments possessing the consider implementing a new company web site, for
same contextual antecedent factors will be more supportive instance. Distress innovation may occur where the recruit-
of innovation adoption by recruiters working within the ment system manifestly fails to cope with the large numbers
department. Without delving in detail into the innovation of applications received and the selector is confronted with
research, there is now accumulated a large body of studies the fall-out from this highly visible failure. In both cases,
to back these rather sensible (possibly even tautological) the model posits that suitable and sufficient job and task
propositions (see West, 2002 for a recent discussion). characteristics, which are in-keeping with the research
At this level-of-analysis these variables are hypothesized findings into work role innovation, will need to be present
as ‘necessary conditions’ as opposed to ‘sufficient condi- as antecedents of technical innovation adoption.
tions’ underlying the adoption of technical innovations by Second, recruiter personal characteristics are also likely
recruiters (Damanpour, 1990). That is, that the presence of to substantively influence their propensity to innovate in
these antecedent factors is a sine qua non for even the their job role. Considerable research into work role
potential for innovation adoption to occur in recruiter innovation suggests that personal characteristics predictive
behavior. Put another way, if the organization habitually of more innovative on-the-job behavior include openness
stifles new ideas, fails to support innovation attempts, and to experience, task mastery, moderate levels of anxiety,
fails to provide basic funds and personnel resources to be high motivation to perform well, independence, above
able to even consider introducing technological innovation, average cognitive ability, tolerance of ambiguity, self-
then recruiters will continue to use traditional methods. confidence, and unconventionality (reviews include, Ama-
However, the presence of these factors is not, in itself, bile, 1983; West, 2001; King and Anderson, 2002;
sufficient for the adoption of technological innovation by Anderson, De Dreu and Nijstad, in press). Although these
recruiters. In addition to these organizational-level ante- findings are as one might expect, the research in this area
cedent factors, Figure 1, again in common with validated has generally found moderately strong effect sizes for these
models of individual innovation, posits that other indivi- characteristics upon work role innovation. In addition,
dual-level variables will constitute ‘sufficient conditions’ Figure 1 proposes further variables as antecedent factors,
for eventual innovation adoption. namely recruiter mastery of new technology as opposed to
‘computerphobia’ (Torkzadeh and Angulo, 1992; Chua,
Chen and Wong, 1999), their educational background, and
Individual Level Variables: Sufficient Conditions
previous experiences with new technology in selection, all
The model proposes that a further set of variables at the posited to be positively related with innovation adoption.
individual level-of-analysis will need to be present for
technical innovation adoption by recruiters. These are
Moderator Variables
specified as a series of sufficient conditions related to their
job and task characteristics on one hand, and to recruiter Finally, in common with recent models of applicant
personal characteristics on the other. Again in-keeping with decision making in selection (Ryan and Ployhart, 2000;
major findings in the innovation research, this time Anderson et al., 2001), Figure 1 proposes that various
regarding individual work role innovation and creativity, moderator variables will influence final decisions as to
Figure 1 summarizes the range of variables which might whether to adopt new technology in the process. These
affect the adoption process at this level. include labor market conditions (difficult recruitment
First, several job characteristics found to be associated markets being likely to be positively associated with
with propensity to work role innovation are specified. attempts to introduce technical innovations), applicant
These include autonomy and job discretion (e.g. West, characteristics (applicants being well-versed in new tech-
1990), the perceived performance of existing recruitment nology, more highly educated, and having access to the
and assessment procedures, reward structures for imple- Internet for instance, being positively associated with
menting improved ways of doing things in the job role adopt decisions by recruiters), and selection ratios across
(Amabile, 1983), and present and probable future work- the range of job roles being recruited for (high selection
loads for the recruiter. A consistent finding in the ratios being positively associated with technological
innovation literatures is that individuals tend either to adoption).
innovate in response to having time, energy and resources This model is therefore proposed as a general heuristic to
to be able to do so (so-called ‘slack innovation’: Kanter, explain and account for recruiter reactions to, and
1983; Van de Ven, Polley, Garud and Venkataraman, adoption of, technological innovation in staff selection.
1999), or alternatively, in response to crisis situations and One contribution the model makes is to provide an initial
overload (so-called ‘distress innovation’: Kanter, 1983; Van structure for much-needed future research into recruiter

International Journal of Selection and Assessment r Blackwell Publishing Ltd 2003

APPLICANT AND RECRUITER REACTIONS 133

adoption decisions, about which presently we know next to many of the research questions into applicant reactions,
nothing. Further, the model attempts to draw some synergy including how applicant preferences and reactions impact
from the volume of studies conducted in the innovation upon their intention to apply, how favorably they rate
field in IWO psychology into organizational- and indivi- different technological media, their impressions of the
dual-level antecedents predictive of work role innovation organization as a potential employer, and so on. But, as
and to apply these findings directly to recruiter adoption of noted by Lievens and Harris (2003), lacking has been any
new technology. Given the present author’s earlier criticism coherent theoretical orientation across this newly estab-
that applicant reactions research has been too atheoretical lished body of applicant reactions studies. Whilst it is
in design, it is fitting that this model is put forward as one undoubtedly useful to know something about reactions
capable of being empirically tested by researchers active in level outcomes, it would surely be far more useful to
this area. Perhaps it too will fail to live up to the ‘frogs and understand how these reactions impact longer-term upon
princes’ fable noted by Bartram (2001), but my intention is applicant expectations of later stages in the selection
to provide an initial, general heuristic model which is process, their motivation to remain as a candidate, the
capable of guiding future research studies into recruiter previews they form of the job and the organization, and
reactions. other aspects of more sustained, expectational and
attitudinal outcomes. Research in this area would benefit
from exposure to developments in studies into applicant
Concluding Comments decision making where such outcome variables are being
included in both models and empirical investigations (Ryan
This paper has attempted to advance our understanding of and Ployhart, 2000; Anderson, 2001). Another issue which
the reactions research in selection in three ways: (1) to warrants attention by researchers active in this area is an
provide a critical, narrative overview of important and over-reliance upon undergraduate students as surrogate
emergent themes of research into applicant reactions to samples for actual applicants to organizational selection
new technology in selection, (2) to draw synergistically procedures. It may be that the reactions of students are
from disparate research into innovation and creativity in generalizable to other populations active in the job market,
order to re-conceptualize the use of new technology in but students are likely to be much more versed in their use
selection as technical innovation, and, (3) to propose of computers and the Internet than other types of
directions for future research into both applicant and applicants. In addition to concerns over ecological validity,
recruiter reactions in this field. As part of the third one is forced to question whether surrogate samples have
objective, a general model predicting recruiter adoption really been necessary given the prevalence of organiza-
of new technology based upon organizational- and tional use of some technological advances for actual
individual-level variables found to be important antece- selection procedures.
dents in the innovation literature is proposed as a frame- Research into recruiter reactions can at present best be
work to generate and guide future research in this area. described as being rather conspicuous solely by its absence.
Research into applicant and recruiter reactions is in a This stands not only as a critical weakness in our
highly anomalous state currently. Compared generally with understanding of the effects of new technology in selection,
research into selection and assessment, research in this area but also as an aberration from the wider field of research in
is commendably candidate-focused, embryonic in its state selection and assessment. Historically selection research
of maturity, strikingly patchy in both its band-width and has emphasized the recruiter and the organization’s
fidelity of coverage of important issues, disappointingly perspective perhaps too heavily (see Ryan and Ployhart,
atheoretical, short-termist in its myopic concentration 2000; Anderson et al., 2001 for instance) to the neglect of
upon only reactions level outcomes, restricted by an applicant perspectives, reactions, and decision-making
unwarranted reliance upon university students as surrogate processes. In stark contrast, the emerging body of research
samples, impressively cutting-edge in its rapid attention to in new technology has focused almost exclusively upon
growth in Internet-based recruitment and testing, and applicant reactions to the detriment of addressing recruiter
uniquely promising in its potential to address the inevit- perceptions and reactions. The propensity of recruiters and
ability of new technology being increasingly used for their organizations to adopt new technology for staff
assessment purposes in the future. In ironic contrast to the selection has received little or no attention from researchers
historic concentration of research in selection upon and yet this is without doubt an important, appropriate and
recruiter and organizational decision making, research viable topic for research within IWO psychology. Presently
into the impact of new technology has been driven almost too little is known about the environmental drivers for
exclusively by concerns over applicant reactions and adopting technical innovations by recruiters, their reac-
perceptions. We currently know next to nothing about tions to such technical change, and the performance of new
recruiter reactions to, expectations of, and willingness to systems in comparison to the old ones they replaced.
adopt different types of new technology for selection. Perhaps one explanation for this is that the whole field of
Pragmatic concerns certainly appear to have underpinned reactions research is so new that the most immediate and

r Blackwell Publishing Ltd 2003 Volume 11 Numbers 2/3 June/September 2003

134 NEIL ANDERSON

pressing questions have been those relating to applicant Anderson, N., De Dreu, C.K.W. and Nijstad, B. (in press). The
reactions. So regrettably, questions into recruiter reactions routinization of innovation research: A constructively critical
have not yet made it onto the agenda in this fast-moving review of the state-of-the-science. Journal of Organizational
Behavior, in press.
topic area. Even for applicant reactions it is true that Anderson, N., Lievens, F., Van Dam, K. and Ryan, AM. (in press)
published studies into some aspects of the new technology, Future perspectives on employee selection: Key directions for
such as Internet-based testing, have only begun to appear research and practice. Applied Psychology: An International
over the last three or four years. Despite the highly Review, in press.
contemporaneous nature of these issues, it is important Arvey, R.D., Strickland, W., Drauden, G. and Martin, C. (1990)
Motivational components of test taking. Personnel Psychology,
that research in future addresses a balanced agenda which 43, 695–716.
includes both applicant and recruiter reactions and Baron, H. and Austin, J. (2000). Measuring Ability via the Internet:
perspectives. Opportunities and Issues. Paper presented at the Annual
To conclude, new technology reactions research is at an Conference of the Society for Industrial and Organizational
early stage of development and in many areas somewhat Psychology, New Orleans, LA.
Bartram, D. (1994) Computer-based assessment. In C.L. Cooper
lags behind the sheer pace of change in organization usage and I.T. Robertson (eds.) International Review of Industrial and
of increasingly widespread aspects of new technology. Organizational Psychology (pp. 31–69). Chichester: John Wiley
There is real evidence of some ‘young princes’ of themes of & Sons Ltd.
research here which hopefully will mature in the coming Bartram, D. (2001) Internet recruitment and selection: Kissing frogs
years, but also evidence of some bedeviling, methodologi- to find princes. International Journal of Selection and Assess-
ment, 8, 261–274.
cal tadpoles which need to be screened out of future study Bartram, D., (2002). Testing on the Internet: Issues, Challenges and
designs in this area. Whilst research into applicant Opportunities. Paper presented at the International Conference on
reactions has made a promising start toward investigating Computer-based Testing and the Internet, Winchester, England.
many pertinent issues and rapidly developing questions, Breaugh, J. and Starke, M. (2000) Research on employee recruit-
psychologically-based research into recruiter reactions is ment: So many studies, so many remaining questions. Journal of
Management, 26, 405–434.
almost non-existent. As organizations and applicants Buchanan, T. and Smith, J.L. (1999a) Using the Internet for
become increasingly accustomed to using such forms of psychological research: Personality testing on the World Wide
new technology in selection it can only be hoped that the Web. British Journal of Psychology, 90, 125–144.
research base in IWO psychology will expand, diversify, Burke, M.J., Normand, J. and Raju, N.S. (1987) Examinee attitudes
and mature to keep pace with these inevitable advances. toward computer-administered ability testing. Computers in
Human Behavior, 3, 95–107.
Cappelli, P. (2001) Making the most of online recruiting. Harvard
Business Review, March, 139–146.
Acknowledgement Chua, D., and Schmitt, N. (1997) Video-based versus paper-and-
pencil method of assessment in situational judgment tests:
I wish to thank Chockalingam Viswesvaran, Jesús Salgado, Subgroup differences in test performance and face validity
and the two anonymous reviewers for their helpful perceptions. Journal of Applied Psychology, 82, 143–159.
Chua, S.L., Chen, D. and Wong, A.F.L. (1999) Computer anxiety
comments on earlier versions of this paper.
and its correlates: A meta-analysis. Computers in Human
Behavior, 15, 609–623.
Cober, R.T., Brown, D.J., Blumental, A.J., Doverspike, D. and Levy,
References P. (2000) The quest for the qualified job surfer: It’s time the
public sector catches the wave. Public Personnel Management,
Aguinis, H., Henle, C.A. and Beaty, J.C. Jr (2001) Virtual reality 29, 479–494.
technology: A new tool for personnel selection. International Czaja, S. and Sharit, J. (1998) Age differences in attitudes toward
Journal of Selection and Assessment, 9, 70–83. computers. Journals of Gerontology: Series B: Psychological
Alkhadher, O., Anderson, N. and Clarke, D. (1994) Computer- Sciences & Social Sciences, 53, 329–340.
based testing: A review of recent developments in research and Damanpour, F. (1990) Innovation effectiveness, adoption and
practice. European Work and Organizational Psychologist, 4, organizational performance. In M.A. West and J.L. Farr (eds),
169–187. Innovation and Creativity at Work: Psychological and Organizational
Amabile, T.M. (1983) The Social Psychology of Creativity. New Strategies (pp. 125–142). Chichester: John Wiley & Sons Ltd.
York: Springer-Verlag. Damanpour, F. (1991) Organizational innovation: A meta-analysis
Anderson, N. (2001) Towards a theory of socialization impact: of effects of determinants and maderators. Academy of
Selection as pre-entry socialization. International Journal of Management Journal, 34, 555–590.
Selection and Assessment, 9, 84–91. Damanpour, F. and Gopalakrishnan, S. (2001) The dynamics of the
Anderson, N. and King, N. (1993) Innovation in organizations. In adoption of product and process innovation in organizations.
C.L. Cooper and I.T. Robertson (eds), International Review of Journal of Management Studies, 381, 45–65.
Industrial and Organizational Psychology (pp. 1–34). Chiche- Davis, C. and Cowles, M. (1989) Automated psychological testing:
ster: John Wiley & Sons. Methods of administration, need for approval, and measures of
Anderson, N., Born, M. and Cunningham-Snell, N. (2001) anxiety. Educational and Psychological Measurement, 49,
Recruitment and selection: Applicant perspectives and out- 311–321.
comes. In N. Anderson, D.S. Ones, H.K. Sinangil and C. Derous, E. (2003). Applicant Reactions Toward Recruitment and
Viswesvaran (eds), Handbook of Industrial, Work & Organiza- Selection. Symposium to the European Association of Work and
tional Psychology Vol. I, pp. 200–218). London: Sage. Organizational Psychology Conference, Lisbon, 2003, May.

International Journal of Selection and Assessment r Blackwell Publishing Ltd 2003

APPLICANT AND RECRUITER REACTIONS 135

Dineen, B.R., Ash, S.R. and Noe, R.A. (2002) A web of applicant Mead, A.D. and Drasgow, F. (1993) Equivalence of computerized
attraction: Person-organization fit in the context of web-based and paper-and-pencil cognitive ability tests: A meta-analysis.
recruitment. Journal of Applied Psychology, 87, 723–734. Psychological Bulletin, 114, 449–458.
Finger, M.S. and Ones, D.S. (1999) Psychometric equivalence of the Meier, S.T. and Lambert, M.E. (1991) Psychometric properties and
computer and booklet forms of the MMPI: A meta-analysis. correlates of three computer aversion scales. Behavior Research,
Psychological Assessment, 11, 58–66. Methods, Instruments and Computers, 231, 9–15.
Goldstein, I.L. (1997) Interrelationships between the foundations Nicholson, N. (1990) Organizational innovation in context:
for selection and training systems. In N. Anderson and P. Herriot Culture, interpretation, and application. In M.A. West and
(eds), International Handbook of Selection and Assessment J.L. Farr (eds), Innovation and Creativity at Work: Psycholo-
(pp. 529–542). Chichester: John Wiley and Sons Ltd. gical and Organizational Strategies. Chichester: John Wiley &
Harris, M.M. (1999) Practice network: I-O psychology.com-the Sons Ltd.
Internet and I-O psychology. The Industrial-Organizational Paronto, M.E., Bauer, T.N. and Truxillo, D.M.(2003). Applicant
Psychologist, 36, 89–93. reactions to three screening methods: What do candidates
Harris, M.M. (2000) The Internet and industrial/organizational prefer? Paper to the Annual Conference of the Society for
psychology: Practice and research perspectives. Journal of e- Industrial and Organizational Psychology, 2003, April.
Commerce and Psychology, 1, 4–23. Reynolds, D.H. and Sinar, E. (2001) Applicant reactions to Internet-
Harris, M.M. (2002). Patrolling the Information Highway: Creat- based selection techniques. In F. Oswald (Chair). Computers 5 -
ing and maintaining a safe, legal, and fair environment for test- Good?: How Test-User and Test-Taker Perceptions Affect
takers. Paper presented at the International Conference on Technology-Based Employment Testing. Symposium conducted
Computer-based Testing and the Internet, Winchester, England. at the 16th Annual Conference of the Society for Industrial and
Harris, M.M. and DeWar, K. (2001). Understanding and using Organizational Psychology, San Diego, CA.
Web-based Recruiting and Screening Tools: Key criteria, current Reynolds, D.H., Sinar, E.F. and McClough, A.C. (2000) Evaluation
trends, and future directions. Workshop presented at the Annual of a Internet-based selection procedure. In N.J. Mondragon
Conference of the Society for Industrial and Organizational (Chair), Beyond the Demo: The Empirical Nature of Technol-
Psychology, San Diego, CA. ogy-Based Assessments. Symposium presented at the 15th
Harris, M.M., Paajanen, G. and Blunt, M. (2003). Internet Annual Conference of the Society for Industrial and Organiza-
Recruitment: How does it Compare to other Sources? Sympo- tional Psychology, New Orleans, LA.
sium to the Annual Conference of the Society for Industrial and Rozelle, A.L. and Landis, R.S. (in press). An examination of the
Organizational Psychology, 2003, April. relationship between use of the Internet as a recruitment source
Highhouse, S. and Hoffman, J.R. (2001) Organizational attraction and student attitudes. Computers in Human Behavior.
and job choice. In C.L. Cooper and I.T. Robertson (eds), Ryan, A.M. and Ployhart, R.E. (2000) Applicants’ perceptions of
International Review of Industrial and Organizational Psychol- selection procedures and decisions: A critical review and agenda
ogy (pp. 37–64). Chicester: John Wiley & Sons Ltd. for the future. Journal of Management, 26, 565–606.
Hough, L.M. and Oswald, F.L. (2000) Personnel Selection: Looking Schmidt, F.L. (2002) The role of general cognitive ability and job
toward the future- remembering the past. Annual Review of performance: Why there cannot be a debate. Human Perfor-
Psychology, 51, 631–664. mance, 15, 187–210.
International Journal of Selection and Assessment, (1995), Special Schmitt, N., Gilliland, S.W., Landis, R.S. and Devine, D. (1993)
Issue on computer-based testing, guest editor: D. Bartram, 3, (2). Computer-based testing applied to selection of secretarial
Kehoe, J.F. (2002) General mental ability and selection in private applicants. Personnel Psychology, 49, 149–165.
sector organizations: A commentary. Human Performance, 15, Sharf, J. (2000) As if ‘g-loaded’ adverse impact isn’t bad enough,
97–106. Internet recruiters can expect to be accused of ‘e-loaded’ impact.
King, N. and Anderson, N. (2002) Managing Innovation and The Industrial-Organizational Psychologist, 38, 156.
Change: A Critical Guide for Organizations. London: Thompson. Shotland, A., Alliger, G.M. and Sales, T. (1998) Face validity in the
Lievens, F. and Harris, M.M. (2003) Research on Internet context of personnel selection: A multimedia approach. Inter-
recruitment and testing: Current status and future directions. national Journal of Selection and Assessment, 6, 124–130.
In C.L. Cooper and I.T Robertson (eds), International Review of Silvester, J. and Anderson, N. (2003) Technology and discourse: A
Industrial and Organizational Psychology. Chichester: John comparison of face-to-face and telephone employment inter-
Wiley & Sons Ltd. views. International Journal of Selection and Assessment, 11,
Lievens, F., van Dam, K. and Anderson, N. (2002) Recent trends and 206–214.
challenges in personnel selection. Personnel Review, 31, 580–601. Silvester, J., Anderson, N., Haddleton, E., Cunningham-Snell, N.
Maher, K. and Silverman, R.E. (2002) Online job sites yield few and Gibb, A. (2000) A cross-modal comparison of telephone and
jobs, users complain. The Wall Street Journal, January, 2, face-to-face interviews in graduate recruitment. International
A1–A13. Journal of Selection and Assessment, 8, 16–21.
Martin, C.L. and Nagao, D.H. (1989) Some effects of computerized Sinar, E.F. and Reynolds, D.H. (2001) Applicant reactions to
interviewing on applicant response. Journal of Applied Psychol- Internet-based selection techniques. In F.L. Oswald (Chair),
ogy, 74, 72–80. Computers 5 Good? How Test-User and Test-Taker Perceptions
Martin, J. (1998) Changing jobs? Try the net. Fortune, 137, Affect Technology-Based Employment Testing. Symposium
205–208. presented at the 16th Annual Conference of the Society
Mathisen, K.S., Evans, F.J., Meyers, K. and Kogan, L. (1985) for Industrial and Organizational Psychology, San Diego,
Human factors influencing patient-computer interaction. Com- CA.
puters in Human Behavior, 1, 163–170. Stanton, J.M. and Rogelberg, S.G. (2001) Challenges and Obstacles
Mead, A.D. (2001) How well does web-based testing work? Results in Conducting Employment Testing via the Internet. In F.L.
of a survey of users of NetAssess. In F.L. Oswald (Chair), Oswald (Chair), Computers 5 Good? How Test-User and Test-
Computers 5 Good? How Test-User and Test-Taker Perceptions Taker Perceptions Affect Technology-Based Employment Test-
Affect Technology-Based Employment Testing. Symposium ing. Symposium presented at the 16th Annual Conference of the
presented at the 16th Annual Conference of the Society for Society for Industrial and Organizational Psychology, San
Industrial and Organizational Psychology, San Diego, CA. Diego, CA.

r Blackwell Publishing Ltd 2003 Volume 11 Numbers 2/3 June/September 2003

136 NEIL ANDERSON

Straus, S.G., Miles, J.A. and Levesque, L.L. (2001) The effects of West, M.A. (2002) Sparkling fountains or stagnant ponds: An
videoconference, telephone, and face-to-face media on inter- integrative model of creativity and innovation implementation
viewer and applicant judgments in employment interviews. within groups. Applied Psychology: An International Review,
Journal of Management, 27, 363–381. 51, 355–386.
Torkzadeh, G. and Angulo, I.E. (1992) The concept and correlates West, M.A. and Farr, J.L. (1990) (Eds.) Innovation and Creativity at
of computer anxiety. Behaviour and Information Technology, Work: Psychological and Organizational Strategies. Chichester:
11, 99–108. Wiley.
Van de Ven, A., Angle, H.L. and Poole, M. (eds) (1989) Research on Wiechmann, D. and Ryan, A.M. (2003) Reactions to Computerized
the Management of Innovation: The Minnesota Studies. New Testing in Selection Contexts. International Journal of Selection
York: Harper and Row. and Assessment, 11.
Van de Ven, A., Andrew, H., Polley, D.E., Garud, R. and Zaltman, G., Duncan, R. and Holbeck, J. (1973) Innovations and
Venkataraman, S. (1999) The Innovation Journey. New York: Organizations. New York: Wiley.
Oxford University Press. Zusman, R.R. and Landis, R. (in press). Applicant preferences for
West, M.A. (2001) The human team: Basic motivations Web-based versus traditional job postings. Computers in Human
and innovations. In N. Anderson, D.S. Ones, H.K. Sinagil and Behavior.
C. Viswesvaran (eds). Handbook of Industrial, Work and
Organizational Psychology Vol. I and II. London/New York:
Sage.

International Journal of Selection and Assessment r Blackwell Publishing Ltd 2003