Journal of Attention Disorders

Journal ofDuPaul
10.1177/1087054705278762
Handler, Attention
/ Assessment
Disorders of
/ November
ADHD 2005 Volume 9 Number 2
November 2005 402-412
© 2005 Sage Publications
10.1177/1087054705278762
http://jad.sagepub.com
hosted at

Assessment of ADHD: Differences Across http://online.sagepub.com

Psychology Specialty Areas

Marcie W. Handler
The May Institute

George J. DuPaul
Lehigh University

Child psychologists are frequently involved in the assessment of ADHD symptoms among school-aged youth. There is limited
information regarding the extent to which psychologists adhere to recommended assessment practices and whether differ-
ences exist in assessment strategies among psychologists from different specialty areas (clinical, counseling, and school) and/
or who practice in different settings (university, school, or outpatient clinic). A 3 (specialty area) × 3 (employment setting)
between-groups design is used wherein 230 child psychologists completed surveys regarding diagnostic practice. Psycholo-
gists differ in adherence with Diagnostic and Statistical Manual of Mental Disorders (4th ed., text revision) diagnostic criteria,
use of clinical interviews, and type of behavior observation. Only 15% of psychologists report using multiple methods consis-
tent with recommended standards of best practice. Differences between groups of psychologists indicate that the diagnosis of
ADHD in children is influenced by the type of psychologist conducting the evaluation and the setting in which the evaluation is
conducted. (J. of Att. Dis. 2005;9(2), 402-412)

Keywords: assessment of ADHD; diagnosis of ADHD; psychology specialty area

A DHD entails persistent problems with inattention,

impulsivity, and hyperactivity that are inconsistent
with one’s developmental level and result in clinically
chronicity, and impairment as outlined by standardized
diagnostic criteria (i.e., DSM-IV-TR; American Academy
of Pediatrics, 2000; Dulcan & Benson, 1997; Goldman,
significant impairment across at least two settings (Amer- Genel, Bezman, & Slanetz, 1998; Hoff, Doepke, & Lan-
ican Psychiatric Association, 2000). Given that ADHD is dau, 2002). Typically, recommended assessment methods
a relatively common behavior disorder, children may be include clinical interviews with the parent, teacher, and
referred to many different types of psychologists (e.g., child; standardized behavior rating scales completed by
clinical, counseling, or school psychologists) and physi- the parent and teacher; review of school information; and
cians (e.g., pediatricians, family practitioners, or psychia- clinical observations of the child’s behavior (when possi-
trists). To diagnose a child with ADHD, Diagnostic and ble, in the classroom and in less structured situations;
Statistical Manual of Mental Disorders (4th ed., text revi- Barkley, 1998; Dulcan & Benson, 1997; DuPaul &
sion; DSM-IV-TR; American Psychiatric Association, Stoner, 2003; Hoff et al., 2002; Zentall & Javorsky, 1996).
2000) criteria requires that a clinician determine not only Despite ongoing discussions about best practice, the
the presence of symptoms but also their severity relative National Institutes of Health (NIH) Consensus Develop-
to same-aged peers, chronicity, pervasiveness, age of ment Panel concluded that “there is a wide variation
onset, and the degree of impairment that results from the among types of practitioners with respect to frequency of
symptoms. diagnosis of AD/HD” (NIH, 1998, p. 9). Thus, given the
Psychologists, psychiatrists, and pediatricians agree growing number of children being diagnosed with
that “best practice” when assessing ADHD symptoms in ADHD, researchers have become interested in examining
children occurs when a clinician uses a multimethod
approach across multiple sources and settings to obtain Address correspondence to George J. DuPaul, Lehigh University, 111
information about symptom presence, pervasiveness, Research Drive, Bethlehem, PA 18015; gjd3@lehigh.edu

402
Downloaded from jad.sagepub.com at PENNSYLVANIA STATE UNIV on September 15, 2016
 Handler, DuPaul / Assessment of ADHD 403

how clinicians actually determine the presence or absence Therefore, the purpose of the present study was to
of ADHD in children. For example, studies with physi- determine how frequently psychologists report using var-
cians indicate that many medical professionals are not ious assessment methods or techniques when they evalu-
using appropriate methods to assess children suspected of ate children suspected of having ADHD and compare
having ADHD. National surveys examining the assess- these practices with the recommended best practice for
ment practices of pediatricians have found that only 20% diagnosing ADHD in children. Furthermore, this investi-
of general practitioners, 24% of family practitioners, and gation examined whether the frequency of methods used
36% of subspecialists had based their definitions of varied across different types of psychologists working in
ADHD on the DSM-IV-TR standardized criteria (Cope- various settings and what demographic factors may
land, Wolraich, Lindgren, Milich, & Woolson, 1987; influence diagnostic practices.
Wolraich et al., 1990). Furthermore, a more recent study
of 116 general and family physicians who diagnosed
Method
ADHD themselves rather than referring their patients to
other clinicians reported infrequent usage of appropriate
measures, such as behavior rating scales (Moser &
Participants
Kallail, 1995). Participants included a sample of clinical and counsel-
National survey studies with psychologists have ing psychologists selected from the most recent Ameri-
revealed that a higher percentage of psychologists than can Psychological Association (APA) membership com-
physicians tend to use recommended methods when eval- puter database. The selected sample included practicing
uating a child for ADHD. For example, although the pri- members of the APA who earned their degrees in clinical
mary methods used continue to be parent interviews and and counseling psychology and who were members of
behavioral observations, studies have found that 78% to Divisions 12 (clinical) and 17 (counseling) as of March
87% of psychologists report using parent or teacher rating 1999. The sample was stratified by gender and geograph-
scales and that approximately 72% to 81% include a ical region. Although the overall sample of clinical and
review of academic records or personally conduct aca- counseling psychologists was randomly selected from the
demic assessments (Brown, Keene, & Middleton, 1994; APA database, the sample of counseling psychologists
Hennigen, 1997). However, approximately 50% of psy- working in schools was not randomly selected. Given that
chologists sampled reported that they continue to employ only 52 counseling psychologists who were members of
projective personality assessments and drawing tests Division 17 indicated that they worked in schools, the
despite evidence suggesting that these methods typically entire sample was provided by APA.
do not provide useful or reliable information during an A random sample of school psychologists was selected
evaluation for ADHD. from the October 1998 database of the National Associa-
In addition, clinical and school psychologists appear to tion of School Psychologists (NASP), which consists of
differ with respect to the use of specific assessment meth- approximately 21,000 members to ensure that both doc-
ods (Brown et al., 1994; Hennigen, 1997; Rosenberg & toral and subdoctoral level psychologists were solicited.
Beck, 1986). For example, most clinical psychologists This sample consisted of psychologists who earned their
tend to rely on behavioral observations in the office, degree in school psychology and was stratified by gender
whereas most school psychologists include behavioral and geographical region.
observations in the child’s classroom (Brown et al., 1994; Based on power analyses, approximately 300 to 375
Hennigen, 1997). In addition to behavioral observations, participants were recruited from each of the three profes-
a higher percentage of school than clinical psychologists sional groups and 250 to 500 from each of the three types
tended to use standardized tests, drawing tests, rating of settings. Given that most clinical and counseling psy-
scales, and neuropsychological tests (Brown et al., 1994) chologists and most psychologists who work in outpa-
as well as academic record reviews (Hennigen, 1997). tient settings do not work with children, a larger sample of
Thus, there is evidence to suggest that differences exist in potential participants were recruited from these groups
diagnostic practices between psychologists who vary in than from other groups.
their training areas. Differences in diagnostic practices
among psychology specialty areas may indicate that the Design
diagnosis of ADHD varies according to the type of psy-
This investigation employed a 3 (specialty area) × 3
chologist conducting the evaluation. In turn, many chil-
(primary work setting) between-groups factorial design.
dren may be unnecessarily diagnosed with ADHD or not
The independent variables consisted of three levels of
receive the treatment they need.

Downloaded from jad.sagepub.com at PENNSYLVANIA STATE UNIV on September 15, 2016

404 Journal of Attention Disorders / November 2005

specialty area that included clinical, counseling, and who commonly assess children suspected of having
school psychology and three levels of work setting that ADHD. The team consisted of psychologists affiliated
included university, school, and outpatient settings (pri- with Lehigh University in Pennsylvania (school and
vate practice, hospital, clinic, community mental health counseling psychologists who currently work or have
center, or counseling center). Demographic factors that worked in academic, school, and outpatient settings) and
psychologists reported also were analyzed to determine The May Institute in Massachusetts (clinical, counseling,
whether these factors accounted for a significant amount and school psychologists who work in school and outpa-
of the variance between the frequencies reported by tient settings). Team members were asked to review the
different types of psychologists. survey and make suggestions so that the final question-
naire was appropriate to different types of psychologists,
Measures easy to understand, and would take participants no more
1 than 15 min to complete.
A survey was created that was modeled after previous
Questions about psychologists’ use of methods consis-
surveys examining diagnostic procedures among pedia-
tent with best practice were not directly asked in the sur-
tricians and psychologists (e.g., Copeland et al., 1987;
vey. Instead, a definition for best practice in the diagnosis
Masone, 1989). It was available to participants in written
of children suspected of having ADHD was based on rec-
and computerized formats with the latter accessible
ommendations provided by experts in psychology, psy-
through the World Wide Web on the Internet. Questions
chiatry, and pediatrics. Specifically, in 1998, the Ameri-
were presented in the following areas: (a) demographic
can Medical Association (AMA) Council on Scientific
information, (b) the frequency of diagnostic methods
Affairs published a report that included recommendations
used, and (c) the frequency with which alternative
to physicians for the diagnosis of ADHD that were
diagnoses are ruled out.
adopted by the AMA as policy (Goldman et al., 1998).
Demographic information requested of each partici-
The American Academy of Child and Adolescent Psychi-
pant included his or her gender, training area, primary
atry published an official action in 1997 describing rec-
work setting, highest type of professional degree, years of
ommended practice parameters for psychiatrists based on
experience since completing a training program, approxi-
a “review of scientific literature and clinical consensus”
mate number of children one works with per week,
(Dulcan & Benson, 1997, p. 1311). Similar recommenda-
approximate number of children suspected of having
tions were incorporated in the American Academy of
ADHD that were evaluated in the past 12 months, and the
Pediatrics’ (2000) guidelines for the assessment of
average amount of time spent on an evaluation of a child
ADHD. Likewise, experts in the field of psychology have
suspected of ADHD. For each question, participants were
made similar recommendations about best practices in
asked to check the option that best answered the item.
the assessment of ADHD to psychologists (Barkley,
Given that clinicians may apply different practices
1998; DuPaul & Stoner, 2003). A multimethod approach
(e.g., including self-report measures) or weigh the credi-
has been recommended that includes an interview with
bility of informants differently when evaluating adoles-
the parent(s) and child; information from the school about
cents, respondents were asked to report how they evaluate
the child’s behavior, cognitive functioning, and academic
only children suspected of ADHD. Children was defined
achievement; standardized rating scales completed by the
for participants as youngsters between the ages of 5 and
parent(s) and the teacher(s); and clinical observations of
11.
the child in the classroom and less structured situations.
Questions focusing on the frequency of methods used
Experts also agree that clinicians should not regularly use
by psychologists were answered using a series of 5-point
laboratory measures given their poor ecological validity
Likert-type scales to determine the degree to which a spe-
and their questionable predictive validity (Cantwell,
cific method is used as part of the diagnostic process. Rat-
1996; Dulcan & Benson, 1997; DuPaul, Anastopoulos,
ings were categorized on the questionnaire as follows: 1 =
Shelton, Guevremont, & Metevia, 1992; Hoff et al.,
rarely or never (0% to 20% of the time), 2 = sometimes
2002).
(21% to 40% of the time), 3 = about half the time (41% to
Thus, based on this literature review, the criteria for
60%), 4 = frequently (61% to 80% of the time), and 5 =
best practice were determined to be answers of frequently
very often or nearly always (81% and 100% of the time).
or very often on questions pertaining to parent interviews,
The same Likert-type scale was used to answer questions
information about school behavior and academic
about how frequently psychologists rule out other
achievement, child interviews, rating scales completed by
disorders.
the parent and teacher, and observations of the child dur-
A pilot version of the questionnaire was distributed to a
ing natural school or home situations. Furthermore,
team of clinical, counseling and school psychologists

Downloaded from jad.sagepub.com at PENNSYLVANIA STATE UNIV on September 15, 2016

 Handler, DuPaul / Assessment of ADHD 405

answers of rarely or sometimes on an item related to labo- participate in the study, 48.7% were not eligible and
ratory measures were considered consistent with best 17.8% were not willing to participate. Overall, 35.3%
practice. (n = 258) of the total eligible sample returned the survey.
Surveys were considered usable (i.e., complete) if 80% of
Procedure the questions were answered and the psychologist indi-
cated that he or she had evaluated at least one child sus-
Participants were randomly selected from APA and
pected of having ADHD in the past 12 months. A usable
NASP membership databases. Potential participants were
response rate of 31.5% (N = 230) was considered accept-
initially sent a prenotification letter asking them to return
able for analysis given that power levels for the analyses
an enclosed postcard indicating whether they (a) were
discussed below were determined to be above .80.
interested in completing the questionnaire through the
U.S. Postal Service mail, (b) were interested in complet-
ing the questionnaire through electronic mail, (c) did not
Demographic Information
work with children ages 5 to 11, or (d) did not wish to Table 1 summarizes responses that participants pro-
participate. vided to questions about demographic information. With
If respondents chose to complete the survey using the respect to training area, 41.7% responded that they were
postal service, they were mailed a packet that included a trained in clinical psychology, 22.6% in counseling psy-
cover sheet, a questionnaire, and a self-addressed chology, and 35.7% in school psychology, with the over-
stamped envelope. No names were placed on the survey to all sample distributed equally between males and
ensure confidentiality. females. Most respondents worked in private practice or
If participants chose to respond to the survey through other outpatient settings; however, more than 45% of the
electronic mail, the first author provided them, through sample worked in school or university settings. An over-
regular or electronic mail, with a Web site address where whelming majority of the sample (88.7%) had attained a
they could access the survey. To maintain confidentiality Ph.D., Psy.D., or Ed.D. as their highest degree, and only
but retain means of tracking people, participants who 11.3% had attained a M.A., M.S., M.Ed., M.A. plus 30, or
responded via electronic mail were required to enter an specialist degree. Thus, when this information was
assigned number given to them when they were mailed included in future analyses, the sample was divided into
the Web site address before the survey could be sent doctoral and subdoctoral degree levels. Approximately
electronically. 60% of the participants reported evaluating between 1
Three weeks following the prenotification letter, a and 20 children for ADHD in the past 12 months. When
follow-up postcard was mailed to people who did not asked to estimate the average amount of time spent on an
respond to the initial mailing reminding them to return ADHD evaluation, approximately 75% of the sample
their response postcard. About 3 to 4 weeks after partici- reported that they spend more than 90 min on each
pants were sent the survey, either through the general mail evaluation.
or through electronic mail, a follow-up postcard or elec- Prior to conducting the primary analyses, a multi-
tronic mail message was sent to participants who had not variate analysis of variance (MANOVA) was completed
completed the survey reminding them to complete the using all 31 dependent variables to determine if there
questionnaire as soon as possible. If surveys were not were differences in the diagnostic practices between two
obtained within 2 weeks following this reminder, a cover groups of psychologists working in outpatient settings.
letter, follow-up survey, and self-addressed envelope Participants who indicated that their primary work setting
were mailed to the remaining participants and to those was in an outpatient hospital, clinic, or counseling center
who never returned the initial response postcard. The were compared to psychologists working in private prac-
entire data collection period lasted approximately 10 tices. Using an alpha level of .05, the effect for setting was
weeks. not statistically significant, Wilks’s lambda (Λ) = .411,
F(31, 36) = 1.67, p = .07. Thus, responses from partici-
pants working in outpatient hospital, clinic, or counseling
Results
centers and participants working in private practices were
combined into a single group referred to as “outpatient”
Respondent Information for all subsequent analyses.
Nearly half (48.4%) of the total potential sample of A one-way repeated-measures MANOVA was con-
participants (N = 1022) responded to the prenotification ducted to determine the degree to which there were differ-
letter using the enclosed response postcard. Although ences in the use of various assessment methods by the
33.5% of those who returned postcards were willing to total sample of psychologists. The results of this analysis

Downloaded from jad.sagepub.com at PENNSYLVANIA STATE UNIV on September 15, 2016

406 Journal of Attention Disorders / November 2005

Table 1 4.76, SD = .67), child interviews (M = 4.73, SD = .71),

Percentage of Psychologists Endorsing Responses to DSM-IV criteria (M = 4.65, SD = .77), and standardized
Demographic Questions rating scales (M = 4.61, SD = .91). No other pairwise
Demographic Possible
comparisons were statistically significant.
Information Response n % Overall, 93.1% of psychologists reported using the
DSM-IV criteria for ADHD frequently (4) or very often
Training area Clinical psychology 96 41.7 (5) when diagnosing children suspected of having
Counseling psychology 52 22.6
ADHD. Although 86.8% of psychologists reported that
School psychology 82 35.7
Primary Academia 37 16.1 they frequently or very often use the DSM-IV criteria as a
work setting School and school system 69 30.0 guideline for diagnosis, only 60.7% of psychologists
Private practice 98 42.6 responded that they frequently or very often strictly
Outpatient hospital, clinic, 26 11.3 adhere to the DSM-IV criteria for ADHD. In addition,
or counseling center 92.5% of psychologists reported that they frequently or
Gender of Female 114 49.6
participant Male 116 50.4
very often use parent interviews.
Highest degree Psy.D 14 6.1 Although 95.3% of psychologists reported that they
attained Ph.D. 174 75.7 frequently or very often obtain information about school
Ed.D. 16 7.0 behavior and academic achievement, only 63.9%
M.A. plus 30 or specialist degree 24 10.4 reported that they frequently or very often obtain this
M.A., M.S., or M.Ed. 2 0.9
information through teacher interviews. In contrast,
Years of 1 to 5 years 22 9.6
experience 6 to 10 years 47 20.4 school records and psychoeducational reports were
since 11 to 15 years 47 20.4 reported to be used frequently or very often by 76.7% and
completing a 16 to 20 years 54 23.5 70.3% of psychologists, respectively.
training More than 20 years 58 25.2 Approximately 91.6% of psychologists reported that
program they frequently or very often use child interviews, with
Approximate 1 to 10 142 61.7
number of 11 to 20 48 20.9
more using informal (66.7%) than semistructured inter-
children seen 21 to 30 26 11.3 views (49.3%). In addition, 91.2% of psychologists
per week 31 to 40 6 2.6 reported that they frequently or very often use standard-
More than 40 7 3.0 ized rating scales, with 82.1% reporting that they fre-
Approximate 1 to 10 86 37.4 quently or very often use rating scales completed by both
number of 11 to 20 52 22.6
the parent and the teacher. Finally, 92.1% of psycholo-
children 21 to 30 32 13.9
evaluated for 31 to 40 20 8.7 gists reported that they frequently or very often use direct
ADHD in the 41 to 50 13 5.7 observations of the child during the assessment session.
past year More than 50 20 8.7 The percentage who reported frequently or very often
More than 100 7 3.0 using observations during natural situations and analogue
Average amount Less than 30 min 3 1.3 situations was 37.9% and 36%, respectively.
of time spent 30 to 60 min 15 6.5
on an ADHD 61 to 90 min 39 17.0
evaluation More than 90 min 172 74.8 Differences Across Specialty Areas and Settings
Note: For some demographic questions, percentages will sum to less A 3 (specialty area) × 3 (employment setting)
than 100% because participants failed to respond to the question. MANOVA was conducted using the seven assessment
techniques as primary dependent variables. The interac-
tion between specialty area and setting was not statisti-
indicated that the difference across all primary categories cally significant, Wilks’s lambda (Λ) = .854, F(21, 426) =
of methods used (N = 163) was statistically significant, 1.14, p = .30, observed power = .82. However, the main
F(3, 523) = 188.85, p < .001, observed power > .99. effects for specialty area, Λ = .809, F(14, 296) = 2.37, p =
Pairwise comparisons using Scheffe’s (1953) procedure .004, observed power = .98, and setting, Λ = .699, F(14,
were calculated to determine where differences existed 296) = 4.14, p < .001, observed power > .99, were statisti-
between means for the primary categories of measures. cally significant.
Psychologists reported using laboratory measures (M = Subsequent univariate 3 (specialty area) × 3 (primary
2.33, SD = 1.59) significantly less frequently than direct work setting) ANOVAs were conducted for each of the
observations of the child (M = 4.84, SD = .50), school seven assessment measures. To correct for Type I error, a
information (M = 4.82, SD = .60), parent interviews (M = Bonferroni adjustment was used to derive an alpha level

Downloaded from jad.sagepub.com at PENNSYLVANIA STATE UNIV on September 15, 2016

 Handler, DuPaul / Assessment of ADHD 407

2
of .007 for all univariate analyses. Eta squared (η ) was Table 2
used to measure the magnitude of treatment effect size, Relationship Between Type of Psychologist and
with a small effect size = .001, medium effect size = .06, Percentage of Psychologists Meeting the Original
and large effect size ≥ .14. Significant F ratios on ANOVA and Revised Criteria for Best Practice
tests were followed by pairwise comparisons using Percentage of
Scheffe’s procedure to delineate between-group Percentage of Psychologists
differences. Psychologists Who Met Best
For three of the seven assessment procedures, a signifi- Meeting Who Met Practice Criteria
cant main effect for setting was obtained. Setting effects Best Practice Best Practice Within Type of
Criteria Criteria Criteria Specialty Area or Setting
were found for the frequency of using DSM-IV criteria,
F(2, 179) = 5.18, p = .007, η2 = .06, parent interviews, Original criteria for best practice
F(2, 189) = 24.88, p < .001, η2 = .21, and child interviews, Total 33 15.3
F(2, 183) = 7.79, p = .001, η2 = .08. The average fre- Specialty area
quency with which psychologists reported using the Clinical 14 42.4 15.1
DSM-IV criteria was significantly less for psychologists Counseling 2 6.1 4.1
working in schools (M = 4.32, SD = 1.11) than in outpa- School 17 51.5 23.0
Work setting
tient settings (M = 4.73, SD = .61, p < .05). In similar fash-
University 5 15.2 15.6
ion, the average frequency with which psychologists School 20 60.6 31.7
reported using interviews with parents or primary care- Clinic 8 24.2 7.1
givers was significantly less for psychologists working in Revised criteria for best practice
schools (M = 4.26, SD = 1.02) than in outpatient (M =
Total 72 33.3
4.95, SD = .21, p < .01) and academic settings (M = 4.92,
Specialty area
SD = .28, p < .01). Finally, the average frequency with Clinical 38 52.8 40.9
which psychologists reported using interviews with chil- Counseling 6 8.3 12.2
dren was significantly less for psychologists working in School 28 38.9 37.8
schools (M = 4.35, SD = 1.05) than in outpatient settings Work setting
(M = 4.88, SD = .51, p < .01). University 10 13.9 31.3
School 23 31.9 36.5
For the frequency of using standardized behavior rat- Clinic 39 54.2 32.2
ing scales, the main effect for specialty area was statisti-
cally significant, F(2, 185) = 7.48, p = .001, η2 = .07. The
average frequency with which counseling psychologists
psychologists, and 4.1% of counseling psychologists
reported using standardized rating scales (M = 4.11, SD =
reported using methods consistent with these criteria.
1.29) was significantly less than clinical psychologists
There was a significant association between the percent-
(M = 4.64, SD = .87, p < .05) and school psychologists
age of psychologists meeting the criteria for best practice
(M = 4.75, SD = .64, p < .01). In fact, only 78.9% of coun-
and their specialty areas, η (2, N = 216) = 8.13, p = .017.
2

seling psychologists, as compared to 93.3% of clinical

Of those who met the criteria for best practice (N = 33),
and 95.5% of school psychologists, reported that they fre-
51.5% were school psychologists, 42.4% were clinical
quently or very often used rating scales. No significant
psychologists, and 6.1% were counseling psychologists.
interaction or main effects were found for the remaining
With respect to setting, 31.7% of psychologists work-
three assessment methods (use of school information,
ing in schools, 15.6% of psychologists working in acade-
behavioral observations, and laboratory measures).
mia, and 7.1% of psychologists working in outpatient set-
tings reported using methods reflecting best practice.
Comparison With Recommended Assessment Again, there was a significant association between the
Protocol percentage of psychologists meeting the criteria for best
Table 2 presents the percentage of psychologists practice and their primary work settings, η2(2, N = 216) =
reporting methods consistent with the criteria for best 20.22, p < .001. Of those who met the criteria for best
practice and the percentage of psychologists meeting this practice (N = 33), 60.6% worked in schools, 24.2%
criteria within each specialty area and work setting. Over- worked in outpatient settings, and 15.2% worked in
all, 15.3% of all sampled psychologists reported using academia.
methods that met best practice. With respect to specialty Further analysis indicated that a critical method affect-
area, 23.0% of school psychologists, 15.1% of clinical ing the use of best practice by psychologists was the use

Downloaded from jad.sagepub.com at PENNSYLVANIA STATE UNIV on September 15, 2016

408 Journal of Attention Disorders / November 2005

of direct observations in natural situations of the child Impact of Demographic Variables

suspected of having ADHD. A 3 (specialty area) × 3 (pri-
A multivariate multiple regression analysis was con-
mary work setting) ANOVA (N = 227) on the use of direct
ducted using the frequency of use for the seven assess-
observations of children in natural situations yielded sig-
ment methods as the dependent variables. Independent
nificant main effects for specialty area, F(2, 218) = 9.43,
(i.e., predictor) variables for each analysis included gen-
p < .001, η2 = .08, and primary work setting, F(2, 218) =
der, the highest degree attained (doctoral or subdoctoral),
48.46, p < .001, η2 = .31. On average, direct observations
years of experience, number of children seen per week,
of children in natural situations were used by school psy-
number of children evaluated for ADHD in the past year,
chologists (M = 3.46, SD = 1.53) more frequently than by
and average time spent conducting an ADHD evaluation.
clinical psychologists (M = 2.38, SD = 1.44, p < .001) and
This regression analysis (N= 159) was found to be signifi-
counseling psychologists (M = 2.61, SD = 1.46, p = .001).
cant, Λ = .246, F(7, 146) = 64.02, p < .001.
Similarly, this method was reportedly used by psycholo-
Given that the multivariate regression analysis was sig-
gists working in schools (M = 4.07, SD = 1.22) signifi-
nificant, separate multiple regression analyses were con-
cantly more often than those working in academic set-
ducted for each of the seven primary dependent catego-
tings (M = 3.00, SD = 1.55), and both groups reported
ries. Significant prediction effects for demographic
using this method more often than psychologists working
variables were found for the frequency of using parent
in outpatient settings (M = 2.07, SD = 1.22, p < .001) for
interviews, F(6, 189) = 3.70, p = .002, R2 = .077, child
all pairwise comparisons.
interviews, F(6, 180) = 3.27, p = .005, R2 = .068, standard-
Given the significant differences in psychologists’ use
ized rating scales, F(6, 183) = 3.75, p = .002, R2 = .080,
of direct observations of the child during natural situa-
and laboratory measures, F(6, 215) = 4.72, p < .001, R2 =
tions based on specialty area and work setting and the fact
.092. Specifically, the more time spent on an evaluation
that only 37.9% of all psychologists reported frequently
and the higher the degree, the more frequently psycholo-
or very often using this assessment method, the criteria
gists reported using interviews with parents. Similarly,
for best practice were revised. Specifically, answers of
the higher the degree attained, the more frequently psy-
frequently or very often on the item related to direct obser-
chologists reported using interviews with children. In
vations of the child in general rather than observations
contrast, greater use of rating scales was predicted by
during natural school or home situations were considered
more time spent on an evaluation and fewer years of expe-
consistent with revised criteria. All other criteria for best
rience. Finally, the more evaluations for ADHD con-
practice remained the same.
ducted in the past year and the fewer number of children
Approximately 33.3% of all sampled psychologists
seen per week, the more frequently psychologists
reported using methods that met this revised criteria for
reported using laboratory measures.
best practice (see Table 2). With respect to specialty area,
40.9% of clinical psychologists, 12.2% of counseling
psychologists, and 37.8% of school psychologists Discussion
reported using methods reflecting the less stringent best
practice. There was a significant association between the On average, psychologists in this sample reported fre-
percentage of psychologists meeting the modified criteria quently using the DSM-IV criteria, an interview with the
for best practice and their specialty areas, χ2(2, N = 216) = parent, an interview with the child, standardized rating
12.85, p = .002. Of those who met the revised criteria for scales, information about school behavior, and direct
best practice (N = 72), 52.8% were clinical psychologists, observations of the child as part of the assessment of
38.9% were school psychologists, and 8.3% were ADHD. Similar to previous findings by Hennigen (1997),
counseling psychologists. Rosenberg and Beck (1986), and Thompson (1998), psy-
With respect to setting, 31.3% of psychologists work- chologists in this study reported using clinical laboratory
ing in academia, 36.5% of psychologists working in measures (e.g., continuous performance tests) infre-
schools, and 32.2% of psychologists working in outpa- quently and significantly less often than other types of
tient settings reported using methods reflecting the modi- methods.
fied best practice. There was no significant association This study was the first of its kind to ask psychologists
between the percentage of psychologists meeting the cri- to report their adherence to diagnostic criteria. Previous
teria for best practice and their primary work settings, national surveys on the diagnostic practices of physicians
χ2(2, N = 216) = .414, p = .813. indicated that only 20% to 36% of pediatricians had based

Downloaded from jad.sagepub.com at PENNSYLVANIA STATE UNIV on September 15, 2016

 Handler, DuPaul / Assessment of ADHD 409

their definitions of ADHD on the DSM-IV standardized gists working in schools. Despite this, only 23% of school
criteria when they diagnosed children with ADHD psychologists and 31% of psychologists working in
(Copeland et al., 1987; Wolraich et al., 1990). Similarly, schools reported methods consistent with best practice.
Setterberg et al. (1991) found that only 59% of child psy- Results indicate that the critical method affecting the use
chiatrists reported often using the precise DSM-IV criteria of best practice was the use of direct observations during
when diagnosing children with disruptive behavior disor- natural situations. Similar to past studies (Brown et al.,
ders. In contrast, 61% of psychologists in this sample 1994; Hennigen, 1997), psychologists working in
reported frequently or very often strictly adhering to the schools and clinicians trained in school psychology used
DSM-IV criteria for ADHD. This may be related to the this particular diagnostic method more often than other
fact that most training programs in psychology include groups. However, psychologists working in schools was
information on the use of the DSM-IV and ADHD, the only group, on average, to report frequently (i.e., an
whereas medical training programs may not include spe- average rating of more than 4) using direct observations
cific training on the DSM-IV or ADHD (Copeland et al., of children during natural situations. It is not surprising
1987; Wolraich et al., 1990). that it appears that access to children in natural situations
Approximately 95% of psychologists reported that strongly influences one’s use of this diagnostic method.
they frequently or very often obtain information about Even when the criteria for best practice was made less
school behavior and academic achievement. Although rigorous to include any observations of children rather
only 64% reported frequently or very often using teacher than observations specifically during natural situations,
interviews, a large percentage of psychologists in this only 33% of all psychologists surveyed reported using
study reported frequently using rating scales completed methods consistent with best practice when diagnosing
by teachers (85%) or school records (77%). Thus, more ADHD in children. The largest percentages of clinicians
psychologists appear to be using standardized rating meeting the criteria for best practice were clinical psy-
scales completed by teachers and school records rather chologists and psychologists working in outpatient set-
than teacher interviews when collecting information tings. Despite this, only 41% of clinical psychologists and
about children’s school-related behaviors (Hennigen, 32% of psychologists working in outpatient settings who
1997). This may be because rating scales provide infor- participated in this study reported methods consistent
mation about symptom severity, allow for normative with these standards of best practice.
comparisons, and require less time than teacher Thus, access to direct observations during natural situ-
interviews. ations must not be the only factor that influenced psychol-
The use of rating scales differed across psychology ogists’ use of best practice when diagnosing ADHD in
specialty areas. In contrast to Thompson (1998), who children. Results suggest that inclusion of rating scales
found no differences between clinical psychologists and completed by both the parent and teacher and the use of
counselors, on average, counseling psychologists in this laboratory measures influenced counseling psycholo-
study reported using standardized rating scales signifi- gists’ use of methods consistent with best practice. Like-
cantly less often than clinical and school psychologists. wise, best practice by psychologists working in schools
Similar to Hennigen (1997), no significant differences appeared to be influenced by their use of parent and child
were found between clinical and school psychologists. interviews. Furthermore, although the average use and
Thus, an increase in the use of rating scales throughout the percentage of psychologists who reported at least fre-
the years may reflect an overall improvement in clinical quently using each measure consistent with best practice
and school psychology training programs’ emphasis on was high, the percentage of psychologists including all of
the need for standardized instruments and normative these diagnostic methods was low. This indicates that in
information. It appears, however, that counseling psy- addition to individual factors, using a multimethod
chology programs may place less emphasis on these approach to diagnose ADHD may influence
measures than other training programs in psychology. psychologists’ use of methods consistent with best
practice.
Use of Best Practice
Influence of Demographic Factors
Only 15% of psychologists surveyed in this study
reported using methods consistent with the criteria for Results indicated that the more time psychologists
best practice when diagnosing children with ADHD. The spent on an ADHD evaluation and the higher the degree
largest percentages of clinicians who met the criteria for attained, the more frequently they reported using inter-
best practice were school psychologists and psycholo- views with parents when diagnosing children with

Downloaded from jad.sagepub.com at PENNSYLVANIA STATE UNIV on September 15, 2016

410 Journal of Attention Disorders / November 2005

ADHD. Similarly, the higher the degree attained, the psychologists to be trained in diagnostic assessment
more frequently psychologists reported using interviews procedures.
with children. The more time psychologists spent on an Differences that emerged between types of psycholo-
ADHD evaluation and the fewer years of experience they gists in their use of methods consistent with standards of
had working in psychology, the more frequently they best practice suggest that training programs need to teach
reported use of standardized rating scales. In contrast, the empirically supported methods of assessing children for
fewer number of children seen per week but the more ADHD. In particular, counseling psychologists reported
ADHD evaluations conducted in the past year, the more using assessment methods, such as rating scales, strict
frequently psychologists reported using laboratory adherence to the DSM-IV criteria, and multiple methods
measures. consistent with best practice, less often than other types of
Findings related to this research question need to be psychologists. Findings from this study and others (e.g.,
interpreted cautiously. Despite the significant findings Thompson, 1998) suggest that graduate programs in
obtained in the multiple regression analyses, demo- counseling psychology need to improve the way they
graphic variables only accounted for a small proportion educate trainees in diagnostic methods for them to be suf-
of the total variance (i.e., small to medium treatment ficiently prepared to evaluate children suspected of
effect sizes). Furthermore, it is not clear that these vari- ADHD.
ables predict the frequency with which diagnostic mea- Although psychologists working in schools have the
sures are used. For example, the positive correlation most access to direct observations of children and fre-
between time spent on an evaluation and the use of stan- quently use this method, only 32% of those sampled
dardized rating scales may indicate that psychologists reported using multiple diagnostic methods consistent
who choose to use rating scales will, in turn, need to with best practice. Findings suggest that clinicians who
devote more time to an ADHD evaluation. Similarly, it currently work in schools may need continuing education
may be the combined use of diagnostic methods (e.g., rat- courses about other diagnostic methods (e.g., conducting
ing scales) and demographic factors (e.g., time spent on parent interviews and how to differentially diagnose psy-
the evaluation) that predicts adherence with best practice. chological disorders) to conduct more comprehensive
evaluations for ADHD.
Implications for Training and Practice Given that few psychologists in this study reported
using methods consistent with best practice, the obvious
The findings from the current study provide several
question becomes “Why are psychologists not adhering
implications relevant to the training of psychologists.
to best practice?” A larger percentage of psychologists
Although 93% of psychologists reported that they fre-
working in schools as compared to other settings are
quently or very often use the DSM-IV criteria for ADHD,
using methods consistent with best practice, in part,
there was high variability across different types of psy-
because they have access to direct observations of chil-
chologists. Likewise, 1 out of every 3 psychologists
dren’s behaviors. Given this access, they may have the
reported that they did not frequently strictly adhere to the
best opportunity, with proper training in other necessary
criteria for ADHD. The DSM-IV manual reminds clini-
diagnostic methods (e.g., parent interviews, child inter-
cians that a “lack of familiarity with DSM-IV or exces-
views, and differential diagnosis), to diagnose children
sively flexible and idiosyncratic application of DSM-IV
with ADHD. However, they are often prevented from
criteria or conventions substantially reduces its utility as a
doing so by school districts because of liability concerns
common language for communication” (American Psy-
or beliefs by school personnel that psychological diagno-
chiatric Association, 1994, p. xxiii). That is not to say that
ses should be given by doctoral-level clinical psycholo-
clinicians are expected to always strictly adhere to the cri-
gists or physicians.
teria, but without frequent use of a standardized criteria,
There are obvious limitations that prevent other types
one cannot be assured that different types of psycholo-
of psychologists from observing children during natural
gists are using the same definition of ADHD. Therefore,
situations. For example, psychologists in outpatient set-
training programs in clinical, counseling, and school psy-
tings are less likely to obtain financial reimbursement
chology may need to increase their emphasis on the
from insurance companies for this service or have easy
importance of using standardized diagnostic criteria
access to natural situations in schools or homes. If direct
when conducting an ADHD evaluation. Although other
observations are not conducted in natural situations, psy-
types of assessment (e.g., functional behavioral assess-
chologists working in outpatient settings could request
ment) may be more useful for developing treatment pro-
information obtained through direct observations from
grams, results indicate that it is still necessary for

Downloaded from jad.sagepub.com at PENNSYLVANIA STATE UNIV on September 15, 2016

 Handler, DuPaul / Assessment of ADHD 411

those who work in the schools to make their evaluations percentages of clinicians were ineligible to participate
more comprehensive and consistent with standards of because they did not evaluate children for ADHD
best practice. (Thompson, 1998). However, there is a potential bias in
In addition, there is some evidence to suggest that psy- generalizing to all psychologists based on this low
chologists who have more than 90 min to devote to an response rate. It is possible that those psychologists who
ADHD evaluation will be more likely to use some of the chose to respond to the survey had a vested interest in par-
methods consistent with best practice (e.g., interviews ticipating given that they work with children and conduct
with parents and standardized rating scales) than are ADHD evaluations. If this is true, results based on this
those who devote less time to an evaluation. Thus, psy- selective sample may overestimate psychologists’
chologists may not allocate nor be given sufficient time to reported diagnostic practices.
conduct evaluations that include comprehensive diagnos- The number of clinical and counseling psychologists
tic methods consistent with best practice. working in an academic setting who were willing to par-
Past research suggests that when clinicians do not use ticipate in this study was too small to generalize findings
assessment methods consistent with best practice, chil- beyond this sample. It may be that clinical and counseling
dren tend to be misdiagnosed. Some studies have found psychologists do not frequently maintain employment in
prevalence rates to be inflated when multiple informants academic settings and simultaneously work with children
were not used. For example, one study found that rates of (which was part of the inclusion criteria for participation
pure ADHD (i.e., no comorbid diagnoses) in a clinic sam- in this study). Other studies also have had difficulty
ple ranged from 12% to 36% when behavior rating scales recruiting psychologists who work in academic settings
were completed by parents or teachers respectively but but continue to assess children (Hennigen, 1997). Like-
only 6% when both types of rating scales were employed wise, more clinical and counseling psychologists had to
(Cohen, Riccio, & Gonzalez, 1994). Likewise, children be recruited because they were less likely to work with
tend to be overdiagnosed when evaluations do not include children than school psychologists. Although sampling
multiple methods. For example, Cotugno (1993) found procedures were similar to past studies (Rosenberg &
that when comprehensive evaluations consisting of inter- Beck, 1986), the method used to obtain potential partici-
views, rating scales, measures of intelligence and aca- pants may have been too restrictive in that clinical and
demic achievement, and the differential diagnosis of counseling psychologists were specifically recruited
other disorders were conducted at community health cen- from Divisions 12 and 17 of APA, respectively, to obtain a
ters, only 40% of children who had been previously diag- more homogeneous sample.
nosed maintained ADHD as a primary or secondary diag-
nosis. Given that most psychologists’ reported practices
Conclusion
in this study did not reflect a multimethod approach con-
sistent with best practice, practitioners may need to ques- A larger percentage of psychologists in this study
tion whether children previously diagnosed with ADHD reported frequently adhering to standardized diagnostic
have received an accurate diagnosis. criteria and using behavior rating scales when conducting
an evaluation for ADHD than was previously reported by
Limitations pediatricians and psychiatrists. Likewise, a larger per-
Given that information was collected through a self- centage of psychologists reported using rating scales than
report measure, it is possible that estimates obtained has been reported in previous studies with psychologists.
through self-reports do not accurately reflect psycholo- Although reported averages suggest that psychologists
gists’ actual diagnostic practices. However, if the ten- frequently use these important diagnostic methods, at
dency is to overestimate one’s behavior on self-report best, only 1 in every 3 psychologists reported using a
measures, then psychologists’ use of various assessment multimethod approach necessary to meet the standards of
measures and methods consistent with best practice may best practice that have been proposed by psychologists,
be worse than reported. psychiatrists, and pediatricians. Furthermore, there was
Several items related to the present study limit the variability in the reported diagnostic practices across dif-
generalizability of the results. A usable response rate of ferent types of psychologists based on their specialty area
31.5% was nearly identical to the response rates obtained and primary work setting. Thus, although psychologists
in similar national studies (Hennigen, 1997; Rosenberg & may be the most qualified professionals to diagnose chil-
Beck, 1986), and other researchers have found that large dren with ADHD relative to other types of clinicians,

Downloaded from jad.sagepub.com at PENNSYLVANIA STATE UNIV on September 15, 2016

412 Journal of Attention Disorders / November 2005

results of this study suggest the need for dramatic in children and adolescents. The Journal of the American Medical
improvements to enhance the consistent use of diagnostic Association, 279, 1100-1108.
Hennigen, L. M. (1997). Clinical assessment practices for the diagno-
practices relative to standards of best practice. sis of attention deficit disorders in children. Dissertation Abstracts
International, 58 (10), 5645B. (UMI No. AAM98-11700)
Hoff, K. E., Doepke, K., & Landau, S. (2002). Best practices in the
Note assessment of children with attention-deficit/hyperactivity disor-
der: Linking assessment and intervention. In A. Thomas & J.
1. A copy of the survey is available from the first author on request.
Grimes (Eds.), Best practices in school psychology IV (pp. 1129-
1150). Bethesda, MD: National Association of School
Psychologists.
References Masone, V. (1989). A survey of diagnostic criteria used by consulting
psychologists to identify learning-disabled adults. Dissertation
American Academy of Pediatrics. (2000). Clinical practice guideline:
Abstracts International, 50, 1947.
Diagnosis and evaluation of the child with attention-deficit/hyper-
Moser, S. E., & Kallail, K. J. (1995). Attention-deficit hyperactivity
activity disorder. Pediatrics, 105, 109-133.
disorder: Management by family physicians. Archives of Family
American Psychiatric Association. (1994). Diagnostic and statistical
Medicine, 4, 241-244.
manual for mental disorders (4th ed.). Washington, DC: Author.
National Institutes of Health. (1998). Diagnosis and treatment of atten-
American Psychiatric Association. (2000). Diagnostic and statistical
tion deficit hyperactivity disorder. NIH Consensus Statement
manual for mental disorders (4th ed., text revision). Washington,
Online, 16(2), 1-37.
DC: Author.
Rosenberg, R. P., & Beck, S. (1986). Preferred assessment methods
Barkley, R. A. (1998). Attention deficit hyperactivity disorders: A
and treatment modalities for hyperactive children among clinical
handbook for diagnosis and treatment (2nd ed.). New York:
child and school psychologists. Journal of Clinical Child Psychol-
Guilford.
ogy, 15, 142-147.
Brown, G. W., Keene, R., & Middleton, H. (1994). A comparison of
Scheffe, H. (1953). A method for judging all contrasts in the analysis
evaluation methods and treatment modalities for attention deficit
of variance. Biometrika, 40, 87-104.
disorder by school psychologists and other practitioners. Seattle,
Setterberg, S. R., Ernst, M., Rao, U., Campbell, M., Carlson, G.,
WA: National Association of School Psychologists. (ERIC Docu-
Shaffer, D., et al. (1991). Child psychiatrists’ views of DSM-III-R:
ment Reproduction Service No. ED 378717)
A survey of usage and opinions. Journal of the American Academy
Cantwell, D. P. (1996). Attention deficit disorder: A review of the past
of Child and Adolescent Psychiatry, 30, 652-658.
10 years. Journal of the American Academy of Child and Adoles-
Thompson, R. W. (1998). A statewide survey of professionals’ opin-
cent Psychiatry, 35, 978-987.
ions and practices concerning the assessment of attention deficit/
Cohen, M. J., Riccio, C. A., & Gonzalez, J. J. (1994). Methodological
hyperactitivy disorder in children. Dissertation Abstracts Interna-
differences in the diagnosis of attention-deficit hyperactivity disor-
tional, 59, 0739.
der: Impact on prevalence. Journal of Emotional and Behavioral
Wolraich, M. L., Lindgren, S., Stromquist, A., Milich, R., Davis, C., &
Disorders, 2, 31-38.
Watson, D. (1990). Stimulant medication use by primary care phy-
Copeland, L., Wolraich, M., Lindgren, S., Milich, R., & Woolson, R.
sicians in the treatment of attention deficit hyperactivity disorder.
(1987). Pediatricians’ reported practices in the assessment and
Pediatrics, 86, 95-101.
treatment of attention deficit disorders. Journal of Developmental
Zentall, S. S., & Javorsky, J. (1996). Functional and clinical assess-
and Behavioral Pediatrics, 8, 191-197.
ment of AD/HD: Implications of the DSM-IV in the schools. Jour-
Cotugno, A. J. (1993). The diagnosis of attention deficit hyperactivity
nal of Psychoeducational Assessment, 22-41.
disorder (AD/HD) in community mental health centers: Where and
when. Psychology in the Schools, 30, 338-344.
Dulcan, M. K., & Benson, R. S. (1997). Summary of practice parame-
ters for the assessment and treatment of children, adolescents, and Marcie W. Handler, Ph.D., is a psychologist with the May Institute in
adults with AD/HD. Journal of the American Academy of Child Norwood, Massachusetts. She is a graduate of the doctoral program in
and Adolescent Psychiatry, 36, 1311-1317. School Psychology at Lehigh University in Bethlehem, Pennsylvania.
DuPaul, G. J., Anastopoulos, A. D., Shelton, T. L., Guevremont, D. C., Her primary research interest is promoting positive behavior support
& Metevia, L. (1992). Multimethod assessment of attention-deficit interventions in school settings.
hyperactivity disorder: The diagnostic utility of clinic-based tests.
Journal of Clinical Child Psychology, 21, 394-402. George J. DuPaul, Ph.D., is a professor of school psychology and
DuPaul, G. J., & Stoner, G. (2003). AD/HD in the schools: Assessment associate chair of the Department of Education and Human Services
and intervention strategies (2nd ed.). New York: Guilford. at Lehigh University. His primary research interests are school-based
Goldman, L. S., Genel, M., Bezman, R. J., & Slanetz, P. J. (1998). interventions for children and adolescents with ADHD and assessment
Diagnosis and treatment of attention-deficit/hyperactivity disorder of college students with ADHD.

Downloaded from jad.sagepub.com at PENNSYLVANIA STATE UNIV on September 15, 2016