Schizophrenia Bulletin vol. 34 no. 3 pp.

523–537, 2008
doi:10.1093/schbul/sbm114
Advance Access publication on October 25, 2007

Cognitive Behavior Therapy for Schizophrenia: Effect Sizes, Clinical Models, and
Methodological Rigor

Til Wykes1,2, Craig Steel2, Brian Everitt3, and the group allocation are likely to have inflated effect sizes.

Downloaded from http://schizophreniabulletin.oxfordjournals.org/ at Universitaetsbibliothek Muenchen on July 4, 2014

Nicholas Tarrier4 Evidence considered for psychological treatment guidance
2
Department of Psychology and; 3Department of Biostatistics and should take into account specific methodological detail.
Computing, Institute of Psychiatry, King’s College London, De
Crespigny Park, London SE5 8AF, UK; 4Division of Clinical Key words: CBTp/schizophrenia/meta-analysis/trials/
Psychology, School of Psychological Sciences, University of symptoms/functioning
Manchester

Background: Guidance in the United States and United Introduction

Kingdom has included cognitive behavior therapy for psy-
chosis (CBTp) as a preferred therapy. But recent advances Cognitive behavior therapy (CBT) has been accepted as
have widened the CBTp targets to other symptoms and a treatment for affective disorders for a number of years
have different methods of provision, eg, in groups. Aim: and has been fully integrated into services since the 1980s.
To explore the effect sizes of current CBTp trials including However, despite the case studies by Beck1 and Shapiro
targeted and nontargeted symptoms, modes of action, and Ravenette2 in the 1950s,’ specific symptom interven-
and effect of methodological rigor. Method: Thirty-four tions for schizophrenia did not appear until much later. Psy-
CBTp trials with data in the public domain were used as chotherapy for schizophreniaintheform ofpsychodynamic
source data for a meta-analysis and investigation of the therapies had been discredited and during the period of
effects of trial methodology using the Clinical Trial Assess- deinstitutionalization symptoms were treated merely as
ment Measure (CTAM). Results: There were overall ben- behaviors to be extinguished (eg, Liberman et al3 and
eficial effects for the target symptom (33 studies; effect Meichenbaum and Cameron4). However, despite earlier
size 5 0.400 [95% confidence interval {CI} 5 0.252, optimism neither medication nor behavioral treatments
0.548]) as well as significant effects for positive symptoms successfully extinguished symptoms which were either pres-
(32 studies), negative symptoms (23 studies), functioning ent sporadically or remained continuously despite adequate
(15 studies), mood (13 studies), and social anxiety (2 stud- treatment. Theoretical underpinnings such as the stress-
ies) with effects ranging from 0.35 to 0.44. However, there vulnerability models were developed to understand not
was no effect on hopelessness. Improvements in one domain only the development of the disorder but also its mainte-
were correlated with improvements in others. Trials in nance. These also began to be informed by research on
which raters were aware of group allocation had an inflated expressed emotion (eg, Brown et al5 and Butzlaff and
effect size of approximately 50%–100%. But rigorous Hooley6) and so began to include social and psychological
CBTp studies showed benefit (estimated effect size 5 markers as well as biological ones. The difficulty in identi-
0.223; 95% CI 5 0.017, 0.428) although the lower end fyingrigorousandunambiguouspsychosocialmarkersmay
of the CI should be noted. Secondary outcomes (eg, nega- have hampered further development of this area.
tive symptoms) were also affected such that in the group of CBT for affective disorders became accepted in the
methodologically adequate studies the effect sizes were not health services through government guidelines (eg, UK
significant. Conclusions: As in other meta-analyses, CBTp National Institute for Health and Clinical Excellence),
had beneficial effect on positive symptoms. However, psy- but it also increased its theoretical research base. It
chological treatment trials that make no attempt to mask was inevitable that eventually some of the developed
techniques would be used for people with a diagnoses
of schizophrenia. The first controlled studies on cognitive
1
To whom correspondence should be addressed; tel: 44-20-7848- behavior therapy for psychosis (CBTp) emerged in the
0596, fax: 44-20-7848-5006, e-mail: t.wykes@iop.kcl.ac.uk. early 1990s in the United Kingdom, and this treatment
Ó 2007 The Authors
This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/
by-nc/2.0/uk/) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.
523
T. Wykes et al.

has developed and included some of the theoretical some combination therapies to reduce harm for those
underpinnings of CBT for other disorders. However, who have a dual diagnosis and substance misuse.26 All
unlike CBT for other disorders, which have its roots in these symptoms have more recently been recognized as
Beck Philadelphia Institute, CBTp developed indepen- those which hamper recovery and affect quality of life.
dently perhaps because the main research bases for the Guidance decisions are mainly based on evidence from
2 types of CBT were separated by the Atlantic Ocean. randomized controlled trials (RCTs), and despite recent
There is much speculation and argument about why criticisms of their appropriateness in mental health,27,28
CBTp emerged first in the United Kingdom, but it RCTs remain the gold standard by which all treatments
may be that the different service structures in the United are judged.29–31 Such trials reflect the scientific method of
Kingdom within which clinical psychologists worked demonstrating the value of a new treatment in compar-
were more encouraging for nonmedical approaches to ison to an appropriate control group by minimizing all
drug-resistant psychotic symptoms. The implementation possible sources of bias which could render unsafe the

Downloaded from http://schizophreniabulletin.oxfordjournals.org/ at Universitaetsbibliothek Muenchen on July 4, 2014

of CBTp was against a tide of skepticism about the de- conclusion that the new treatment is beneficial.29 There
velopment of psychotherapy for people with psychosis in are established conventions concerning how clinical trials
both countries and specific evidence of poor outcomes.7 should be carried out and analyzed29,32 and these have
There was also optimism about the likelihood of im- been formalized in the Consolidated Standards for
proved medication now somewhat tempered by the Clin- Reporting of Trials (CONSORT) statement.33,34
ical Antipsychotic Trials in Intervension Effectiveness8 Despite these guidelines, there is still considerable var-
and Cost Utility in the Latest Antipsychotic Drugs in iability and room for improvement in rigor. Thornley and
Schizophrenia Study trials9 and the positive evidence Adams35 reported that in 2000 (mainly drug) trials, meth-
on behavioral approaches to psychosocial rehabilitation odological rigor had not improved to a minimal standard
which may have contributed to resistance about the de- over a period of 35 years. Such rigor is known to affect
velopment of this form of treatment. However, although the estimation of treatment effects. For instance, poorer
the practice of CBTp was enthusiastically grasped by quality masking of allocation of treatments has been
UK clinical psychologists, it has now also developed shown to be associated with up to 40% increased estimate
in the United States with studies approaching the provi- of benefit in circulatory and digestive diseases, mental
sion of CBTp differently, eg, by more group-based health, obstetrics, and childbirth.36,37
approaches. Again, service structures and the availability This raises the question of whether differences in meth-
of skilled clinical psychology staff probably contribute to odology will also inflate the effects of psychological treat-
these variations. Within the United Kingdom, psycholog- ment trials. Marshall et al38 report that in 150 nondrug
ical treatments are typically individualized and based trials, one-third of the claims that treatment was superior
upon an idiosyncratic case formulation (see Tarrier to control would not have been made if published scales
and Calam10 and Tarrier11). In the United States, the had been used in the assessment. It is of course possible
recent move to more evidence-based practice has been that other aspects of the design and method could also
to counter the past standard practice of providing psy- affect the results of psychological treatment trials. But
chological treatment from a nonsystematic, nonmanual- even though meta-analyses have had a direct effect on
ized perspective. Thus, even if the actual intervention treatment guidance, methodological rigor of individual
techniques used were similar in the United States and trials is hardly referred to,13,18–23 and the strong claim
the United Kingdom, their strategic application could has been made that the quality of trials included must
differ. be investigated in order to ensure that meta-analyses pro-
Reviews of studies of CBTp have suggested that they vide valid estimates of the true effects of treatment.39
are useful for the treatment of schizophrenia.12–23 The Aspects of psychological therapy, in addition to meth-
next step is therefore the incorporation of these treat- odology, may also affect the treatment effect, in partic-
ments into services. In the United Kingdom, the National ular the content. There have been no investigations of
Institute for Clinical Excellence included cognitive be- the different elements of treatment in a direct head-to-
havior therapy (CBTp) in its preferred list of treatments head comparison, but there may be more general influ-
for schizophrenia.24 UK National Health Services are ences that could be investigated in a meta-analysis.
now implementing this guidance because patients in CBTp in schizophrenia varies in its emphasis on cognitive
UK services have the right to expect that this treatment and/or behavioral dimensions of therapy and at the
will be available. This therapy has also been considered in extreme end of the continuum merges with some form
the Schizophrenia Patient Outcome Report Team guid- of psychodynamic treatments. Clinical emphasis in any
ance in the United States and has been recommended.25 model is also dependent on the services in which it is pro-
There are now more published studies available, and vided and the background professional training of the
there has been an expansion of the likely symptom targets therapists which differs from country to country. There
for CBTp to include, in addition to positive symptoms, may therefore be particular differences to explore in stud-
negative symptoms, depression, and anxiety, and also ies between countries. The superiority of any model has
524
 Assessing Quality of CBTp Trials

never been investigated, and as we move from efficacy to For example, Kemp et al41 was excluded because its aim
effectiveness studies this is an important consideration. and outcome were medication compliance and its method
Since the publication of the latest meta-analysis, more was motivational interviewing rather than more con-
data from newer CBTp trials have become available. The ventional CBT. Two studies were excluded because the
aim of this article is to look in more detail at this authors did not make the data available that was neces-
expanded data set with its extended list of outcomes sary for inclusion in the meta-analysis, leaving a total
and investigate the effects of different methodological sample of 34 studies.
attributes as well as clinical models in the calculation
Effect Size Calculation
of benefit. In particular, we will estimate the possible in-
flation of effect size due to lack of masking of participants Effect sizes of the treatment trials were calculated from
to groups because it has been identified as a potent meth- the following equation.19
odological variable in the inflation of the treatment effect. Effect size = ðMt  Mc Þ=SDc

Downloaded from http://schizophreniabulletin.oxfordjournals.org/ at Universitaetsbibliothek Muenchen on July 4, 2014

Methods where Mt is the mean of the CBTp group at posttreat-
ment, Mc is the mean of the control group at posttreat-
Trial Inclusion ment, and this is divided by the SD of the control group of
Multiple systematic searches of Embase, Medline, Cur- participants at posttreatment. Outcome was considered
rent Contents, Web of Science, PsychInfo, and the in relation to positive symptoms, negative symptoms,
Cochrane Collaborative Register of Trials were per- functioning, mood, hopelessness, and social anxiety
formed using the following search terms either as key where such data were available, and the measures chosen
terms or as key words: to reflect these outcomes in the calculation of the effect
(SCHIZO* or SCHIZOPHRENIA or SCHIZO- size was a continuous one. With respect to positive symp-
AFFECTIVE DISORDER) AND (COGNITIVE toms, this was, in most cases, a summary score from a re-
THERAPY or COGNITIVE BEHAVIOUR THER- liable measure such as the Positive and Negative
APY or COGNITIVE BEHAVIOR THERAPY) Syndrome Scale (PANSS) or Brief Psychiatric Rating
AND (RANDOM or RANDOMISED CONTROL Scale, but other measures such as Psychotic Symptoms
TRIAL or CLINICAL TRIAL). Rating Scales were also used when these were unavailable
One hundred forty-one publications were identified or when the authors had powered their study on a specific
which were hand searched including their reference lists. continuous measure. The control group was considered
In addition, the reference lists from reviews and meta- to be the TAU group or a control adjunct treatment
analyses12–23 were hand searched. This produced a total that had been hypothesized to be inactive for the main
of 35 articles that were judged potentially eligible for this outcome. All effect sizes were recalculated from the study
review and read independently by 2 of the authors. In data except in one case in which the data were insuffi-
addition, using our own knowledge of work in this ciently reported and so the effect size reported in a previ-
area through conference presentations (2004–2006), the ous meta-analysis was used.19 The main consideration
Beck psychosis networks, and familiarity with appropri- was the effect after treatment; so we have not included
ate research groups around the globe, one further trial any follow-up data.
was added that had been submitted.40 This study has sub-
sequently been published. The criteria to retain a publica- Measures
tion were Features Used to Assess Quality of Trial Reports. A list of
relevant features were extracted from the CONSORT

the studies sample had to contain a majority of people guidelines34 that are the current convention for describ-
with a diagnosis of schizophrenia; ing clinical trials in major medical journals, eg, Lancet,

all patients received standard psychiatric care (TAU, British Medical Journal, Journal of the American Medical
treatment as usual) including appropriate medication; Association, and Archives of General Psychiatry. Expert

in the experimental group, CBT was an adjunct to opinion from psychologists, psychiatrists, statisticians,
TAU; and methodologists was then sought on this checklist.

there was a control group; These opinions provided face validity. Individual features

there was an allocation procedure; were differentially weighted based on previous data on

CBT treatment was targeted at one of the following methodological characteristics that can influence out-
(positive or negative symptoms of psychosis, function- come (eg, Thornley and Adams35, Moher et al36,
ing, mood, hopelessness/suicidality, or social anxiety). Marshall et al38, Chalmers et al42, Jadad et al43, Juni
et al44, Juni et al45, Kazdin and Bass46, and Sterne
Trials were excluded if they were uncontrolled, tested et al47). The resulting list had 15 items grouped into 6
forms of psychotherapy other than CBT, or whose out- areas of trial design: sample size and recruitment method,
comes did not include symptoms at the end of treatment. allocation to treatment, assessment of outcome, control
525
T. Wykes et al.

groups, description of treatments, and analysis (see treat analysis) because it maintains the benefits of
below). Items were weighted in score depending on the randomization.32 Some commonly used procedures to
importance highlighted by previous methodology articles accommodate missing data were not considered to be ap-
and meta-analyses. For example, the total item score for propriate, eg, missing value substitution by ‘‘last obser-
allocation to treatment group was higher than the section vation carried forward’’ because of its underlying
on descriptions of active treatment. assumptions and likely optimism about the precision
We did not want to confuse the rating of methodolog- of the extracted effect size.32
ical quality with the quality of the report39,48; so issues
were clarified with the trial researchers, and where neces- Reliability of CTAM. Independent ratings by 2 of the
sary data or information were requested that were not authors of an initial 18 studies showed good blind inter-
available within the published report. Subsequently, we rater agreement of 0.96. The scale showed adequate in-
made the ratings of methodological quality of each indi- ternal consistency (Cronbach a = .697). The total score

Downloaded from http://schizophreniabulletin.oxfordjournals.org/ at Universitaetsbibliothek Muenchen on July 4, 2014

vidual trial available to the trial researchers and asked for and the scores for the 6 domains were used as the out-
their comments. come measures. The accuracy of the CTAM scores
The methodology areas included within the rating scale was confirmed by the first author of each specific study
(Clinical Trial Assessment Measure: CTAM) are de- in 27 cases. The authors of 6 studies did not comment,
scribed below. A more detailed account with specific and one was not contactable.
scores is given in Tarrier and Wykes.49
Sample Characteristics Validity of CTAM. Face validity from experts and the
Recruitment via volunteers or referrals of ‘‘suitable’’ CONSORT statement have already been described. In
patients by clinicians will not produce as representative addition, concurrent validity was assessed within an ini-
a sample as a geographic or epidemiological cohort. In tial 22 studies by correlation with the scores from 3 other
addition, we have included that sample size should be scales devised to assess methodological rigor.42,43,52 The
based on adequate power calculations. correlations of these scales with the CTAM scores on the
Allocation to Treatment corpus of CBTp data were as follows: CTAM and Jadad,
The process of random allocation needs to be appro- q = .960, P < .001; CTAM and Chalmers, q = .93,
priate and clearly described,50 and the sequence is con- P < .001; and CTAM and Brown, q = .80, P < .001.
cealed from the research team because this has been This indicates that CTAM had excellent concurrent val-
shown to be associated with larger treatment effects.51 idity. Predictive validity will be tested in the relationship
Assessment of Outcome between CTAM scores and effect sizes. It might be
Standardized assessment methods should be used38 expected that less rigorous studies (ie, more potential
and collected independently of treatment by assessors for bias) will produce larger effect sizes.
who were unaware of treatment allocation (normally
called blinded or masked assessment). Rating of the Emphasis of the Clinical
Control Groups Model. Descriptions of the clinical characteristics of
A control treatment that includes standard psychiatric the underlying model and clinical techniques included
care or TAU is a prerequisite. The use of a control treat- in each study were extracted from the written reports,
ment that includes another psychological treatment, such and all identifiers were removed so that the rater could
as supportive counselling or a placebo treatment (eg, remain masked to the identity of the study. An indepen-
befriending), is desirable to allow the estimation of the dent qualified CBTp psychologist ranked the studies on
nonspecific effects of treatment to be assessed. the basis of least to most behavioral. This was defined on
Description of Treatments the basis of whether there was more emphasis on issues
Treatments should be described so that they can be in- in the past, eg, historical variables, which might have an
dependently replicated; this would be aided by a manual impact on the future (least behavioral), or whether there
or protocol. Assessment of adherence to the treatment was a focus on the here and now and the interpretation of
protocol or some method of treatment quality assessment current events (more behavioral).
should also be carried out. This is an important aspect of
psychological treatments that will not be covered by more Results
generic rating scales of trial quality.
Analysis The Trials
The most acceptable methods for statistical analysis Thirty-four studies of CBTp were identified through pub-
were judged (by an experienced trial statistician) as being lished records, conference presentations, and networks of
appropriate to the data and the trial design. For instance, CBTp specialists throughout the world. Data were avail-
the results should be analyzed on an analyzed-as- able either in their published form or, where necessary,
randomized (sometimes referred to as an intention-to- were made available to the authors.
526
 Assessing Quality of CBTp Trials

Table 1. Number of Studies Providing Specific Outcomes (Numbers in Parenthesis are Those Studies That Specifically Targeted that
Outcome)

Positive Negative
Symptom Symptom Functioning Mood Hopelessness Social Anxiety Total

Individual CBTp 27 (24) 19 (1) 12 (2) 12 (0) 3 (0) 0 (0) 27

Group CBTp 5 (4) 4 (1) 3 (0) 3 (0) 1 (0) 2 (2) 7
Total 32 (28) 23 (2) 15 (2) 15 (0) 4 (0) 2 (2) 34

Note: CBTp, cognitive behavior therapy for psychosis.

Downloaded from http://schizophreniabulletin.oxfordjournals.org/ at Universitaetsbibliothek Muenchen on July 4, 2014

Twenty-five studies involved the treatment of chronic Trials with larger sample sizes have better CTAM scores
patients in the community, 7 studies of acutely ill which probably reflects the likelihood of larger invest-
patients, 1 of chronic inpatients, and 1 contained a mixed ments by funding organizations to be in higher quality tri-
population. Twenty studies were from the United als (Spearman q = .596, P < .001).
Kingdom; 5 from the United States; 2 each from
Germany, Australia, and the Netherlands; and 1 each Clinical Emphasis
from Canada, Italy, and Israel. The average number of The 34 trials differed in their clinical emphasis with some
participants in each trial was 58.2 (range 11–353), and representing schema-dependent therapy and one even
the median proportion of those lost to follow-up assess- reporting an emphasis on the ‘‘links between current
ment was 14.5% (range 0%–45%). Seven studies (21%) symptoms and earlier real life events.’’ At the other
had a dropout rate higher than 25%, which is the level end of the scale studies reported an emphasis on be-
above which many statisticians would question the valid- havioral homework, relaxation, and the development
ity of the study findings. of behavioral coping strategies. There was, however, con-
The studies had varied targets of intervention. Most siderable overlap. Behavioral rehearsal, eg, and the rever-
focused on positive symptoms (one study concentrated sal of avoidance (or safety behaviors) were common to
on reducing the powerfulness of the voice53), although many of the approaches irrespective of their stated em-
2 targeted negative symptoms, 2 targeted functioning, phasis. The clinical emphasis as ranked by an indepen-
and 2 targeted social anxiety. Twenty-seven studies dent CBTp specialist is given in table 2. Although not
were individual CBTp, while 7 studies were group significant, group studies were more likely to have
CBTp. All studies have continuous measures of outcome a more behavioral emphasis than individual studies
for the target symptom. (t = 1.6, df = 32, 95% CI = 14.8 to 1.9).
Table 1 shows the total number of studies reporting an
outcome with the number of studies whose intervention
Clinical Model and Trial Quality
was targeted at a specific symptom shown in parenthesis.
Table 2 shows a summary of the 34 studies included in the There was no significant association between the empha-
analyses including the methodological quality ratings. sis of the clinical model and methodological rigor of the
trials as measured by the CTAM total score (q = .19,
Trial Quality P = .28). Neither the total score nor the domain scores
for CTAM were associated with clinical emphasis.
The maximum score for the CTAM is 100, and in this
sample of trials, the mean score was 61.2 (SD 18.1) with
a median of 56 and a range of 27–100 (see table 2). There Study Origin and Trial Quality
was some variability in methodology. All, except 4 studies, Those studies originating outside the United Kingdom
had random allocation, and 5 studies did not have inde- had smaller sample sizes. In fact, only 2 non-UK studies
pendent assessments of outcome measures (but they had a sample size that was above 27 participants per
were different studies). However, few reports adequately group, which is the smallest sample size likely to identify
described the process of assessor blinding or verified blind- a modest clinically significant effect.46 Overall trial qual-
ing at the end of the study. More than half of the studies did ity was significantly higher in UK trials (mean CTAM
not use a statistical method that was judged to take satis- score: UK 67.6, non-UK 52.1; t = 2.69, 95% CI =
factory account of dropouts from assessment. Methodo- 27.4 to 3.8). The differences in method, apart from
logical criteria have evolved over time with later trials sample size, were (1) in the way the sample was drawn—
showing higher CTAM scores (Spearman correlation = non-UK sites mainly relying on convenience samples, (2)
.393, P = .02) and an increased quality in allocation policy the quality of random allocation, and (3) the quality of
(ie, independence from the research team and true random control group. There were no differences in the target
allocation) (Spearman correlation = 0.463, P = .006). symptom effect sizes or the clinical emphasis of the model
527
 Table 2. Included Studies and Their Characteristics
528

T. Wykes et al.
Positive Treatment
United Individual/ Primary Symptom Total CTAM Sample Allocation Assessment Control Analysis Description Clinical
Trial Year Kingdom Group Aim Effect Size (Max 100) (Max 10) (Max 16) (Max 32) (Max 16) (Max 15) (Max 11) Model

Milton et al54 1978 Y I P 0.78 52 0 13 26 10 0 3 25

Tarrier et al55 1993 Y I P 0.35 49 2 10 16 16 5 0 32
Garety et al56 1994 Y I P 0.55 39 2 0 16 6 9 6 14.5
Bentall et al57 1994 Y I P 0.29 53 2 10 23 10 5 3 11
Drury et al58 1996 Y I P 0.93 53 2 13 16 10 9 3 23
Kuipers et al59 1997 Y I P 0.37 63 7 16 13 6 15 6 4.5
Tarrier et al60 1998 Y I P 0.73 96 10 16 32 16 11 11 32
Daniels61 1998 N G N 0.64 42 2 0 26 6 0 8 34
Levine et al62 1998 N G P 2.36 56 0 10 26 6 11 3 32
Pinto et al63 1999 N I P 0.99 44 2 10 16 10 0 6 28
Haddock et al64 1999 Y I P 0.49 56 2 10 26 10 5 3 24
Halperin et al65 2000 N G SA n/a 27 2 10 6 6 0 3 8.5
Sensky et al66 2001 Y I P 0.14 81 7 16 26 10 11 11 2.5
Bailer et al67 2001 N I N 0.34 38 2 0 16 6 11 3 20
Barrowclough et al26 2001 Y I P 0.26 80 10 16 29 6 11 8 14.5
Lewis et al68 2002 Y I P 0.12 100 10 16 32 16 15 11 20
Turkington et al69 2002 Y I P 0.23 77 10 16 26 6 11 8 14.5
Durham et al70 2002 Y I P 0.32 84 7 16 29 16 5 11 20
Hall and Tarrier71 2002 Y I P 0.88 41 2 16 6 6 5 6 29.5
Valmaggia et al72 2002 N I P 0.32 62 2 13 26 10 11 0 8.5
Granholm et al73 2002 N I F 0.62 40 2 10 16 6 0 6 14.5
Gumley et al74 2003 Y I P 0.19 53 10 10 16 6 5 6 6
Rector et al75 2003 N I P 0.28 55 2 13 26 6 5 3 7
Jolley et al76 2003 Y I P 0.10 75 5 16 26 6 11 11 2.5
Kingsep et al77 2003 N G SA n/a 56 2 0 26 6 11 11 29.5
Trower et al53 2004 Y I CH 1.75 71 7 16 26 6 5 6 1
Wiersma et al78 2004 N I P 0.65 54 7 16 16 6 6 3 26
Bechdolf et al79 2004 N G P 0.02 67 7 16 26 10 5 3 20
Startup et al80 2005 Y I P 0.44 64 10 16 16 6 5 11 45
Cather et al81 2005 N I P 0.04 56 2 16 26 6 0 11 27
Granholm et al82 2005 N I F 0.07 87 10 13 32 6 15 11 14.5
Wykes et al83 2005 Y G P 0.02 79 10 16 26 6 15 6 20
Gaudiano and 2006 N I P 0.47 45 2 14 3 6 15 6 10
Herbert84
Barrowclough et al40 2006 Y G P 0.04 87 10 16 29 6 15 11 14.5

Note: CTAM, Clinical Trial Assessment Measure; primary aim: P, positive; N, negative; SA, social anxiety; CH, command hallucinations; F, functioning; n/a, not applicable;
y, yes; N, no.

Downloaded from http://schizophreniabulletin.oxfordjournals.org/ at Universitaetsbibliothek Muenchen on July 4, 2014

 Assessing Quality of CBTp Trials

Table 3. Results of Meta-analyses

Mean Weighted 95% Confidence Heterogeneity Test (df), No. of Sample

Effect Size Interval Significance Level Studies Size

Target symptom 0.400 0.252, 0.548 74.1 (32), significant at the 5% level 33 1964
Positive symptoms 0.372 0.228, 0.516 61.7 (31), significant at the 5% level 32 1918
Negative symptoms 0.437 0.171, 0.704 118.1 (22), significant at the 5% level 23 1268
Functioning 0.378 0.154, 0.602 36.7 (14), significant at the 5% level 15 867
Mood 0.363 0.079, 0.647 52.7 (12) significant at the 5% level 15 953
Hopelessness 0.190 0.547, 0.166 10.0 (3), not significant 4 431
Social anxiety 0.353 n/a n/a 2 61

Downloaded from http://schizophreniabulletin.oxfordjournals.org/ at Universitaetsbibliothek Muenchen on July 4, 2014

between studies originating inside and outside the United reported all outcomes, and not all trials targeted partic-
Kingdom although it was clear that the UK studies ular outcomes; so the initial analysis reports the results of
spanned the whole continuum of emphasis whereas a meta-analysis of the indicated target for the study. In
non-UK sites tended to have a more behavioral the 34 trials, only 33 CBTp trials reported this outcome
emphasis. for the symptom that the trial was targeting. Because
Several meta-analyses were conducted on this corpus there were only 2 studies for the assessment of social anx-
of data in relation to the various outcomes as well as iety, neither heterogeneity nor CIs are provided.
the targeted outcomes. The outcome under investigation In terms of the relationships between different out-
for each meta-analysis was the total symptom score on an comes, there are significant correlations between
appropriate scale, eg, PANSS positive symptom score, improvements in positive symptoms and a worsening
PANSS negative symptom score, Global Assessment of in hopelessness (R = 0.978, P = .022, N = 4), improve-
Functioning for functioning, Calgary Depression Scale ments in negative symptoms (R = 0.830, P < .001,
for depression, Brief Social Phobia Scale for social anx- N = 23), and some evidence of improved functioning,
iety, and Beck Hopeless Scale for hopelessness/suicidality but this did not quite reach significance (R = 0.510,
(see Method for decisions on choice of outcomes). P = .052, 2 tailed, N = 15). Improvements in negative
symptoms were related to improvements in functioning
Meta-analyses of All CBTp Trials in Relation to Various (R = 0.656, P = .021, N = 12). There is also a positive re-
Symptom and Functioning Outcomes lationship between improvements in functioning and
Six separate meta-analyses were carried out on the data improvements in mood (R = 0.954, P = .003, N = 6).
using a random-effects model (see Everitt and Pickles32 Theseresults suggest thatthere isa relationship between dif-
and Fleiss85) which was applied to the effect sizes ferent outcomes in that targeting one outcome may have
obtained from the CBTp trials which reported the various positive (and sometimes detrimental) effects on others.
outcomes
Individual and Group CBTp: Comparison of Outcome in

target symptoms (as specified by the research team in Relation to the Target Symptom
their publication),
To address this question, a random-effects model was ap-

positive symptoms,
plied separately to the individual CBTp studies (number

negative symptoms,
of studies = 26, total number of participants = 1565) and

functioning,
to the group CBTp studies (number of studies = 7, total

mood,
number of participants = 399) in relation to the outcome

hopelessness.
for the target symptom. For the individual CBTp studies,
the estimated effect size was 0.415 (SE = 0.08); the corre-
Heterogeneity of effect sizes was assumed (and later
sponding figure for the group CBTp studies was 0.386
confirmed for many analyses, see table 3); so a random-
(SE = 0.20). The estimated 95% CI for the difference is
effects (RE) model rather than a fixed-effects model was
(0.384, 0.442) suggesting that there is no evidence of any
adopted32,85 (see table 3). Even in the few instances where
difference in effect size between individual and group CBTp.
the test for homogeneity of effect sizes was not signifi-
cant, the RE model was still used because of concerns
about the test’s lack of power. The results of the individ- Exploration of Heterogeneity of Outcomes Using
ual meta-analyses are given in table 3. The table also pro- Individual CBTp Aimed at Positive Symptoms
vides the estimated overall effect size, the 95% confidence There was heterogeneity of effect size noted in the meta-
intervals (CIs), and the usual chi-square test of the homo- analyses reported above. The purpose of the following
geneity of the effects. A significant chi-square value indi- analyses is to investigate this heterogeneity in more detail
cates heterogeneity in the effect sizes. Not all trials with specific reference to only those studies that indicated
529
T. Wykes et al.

Downloaded from http://schizophreniabulletin.oxfordjournals.org/ at Universitaetsbibliothek Muenchen on July 4, 2014

Fig. 1. Forest Plot of the Effect Sizes for the Trials Shown in Table 2.

that the target symptom was positive symptoms. We also Relationship Between Methodological Quality, Clinical
chose those that provided CBTp in a more traditional ap- Emphasis, and Effect Size. To investigate the various
proach by an individual therapist. This has 2 bene- relationships, a weighted analysis is necessary because
fits—the studies are homogenous in terms of their the estimated effect sizes clearly have different precisions
mode of treatment provision and treatment target but and any unweighted analysis ignores this feature of the
also report on CBT because it was originally developed data. The weight applied to a study was the reciprocal
as a therapy aimed at residual symptoms. This gave a large of the sum of the estimated between study variance
sample because the majority of studies were individual and the estimated variance of the effect size for the study
CBTp aimed at reducing positive symptoms (number of (see Everitt86). The former is found from the random-
studies = 24, total number of participants = 1450). These effects model used in the meta-analysis (see above),
studies were therefore investigated in more detail to un- and the latter is approximated by the sum of the sample
derstand whether methodological rigor (or lack of it) and sizes for the experimental and control groups divided by
clinical emphasis affected the estimate of effectiveness. the product of these sample sizes (see Fleiss85). Because
the Trower et al53 trial had a distinct focus of intervention
Meta-analysis of Individual CBTp Trials Aimed at Positive (command hallucinations), the results of some analyses
Symptoms. A random-effects model was applied to the were repeated to check the effects of this study on the out-
effect sizes obtained from the 24 individual CBTp trials come of the analysis.
aimed at positive symptoms, leading to an estimated
overall effect size of 0.399 with a 95% CI of 0.243, Relationship of CTAM and Effect size. The simple cor-
0.556. The usual chi-square test of the homogeneity of relation was significant whether or not Trower et al53
the trials took the value 40.0 with 23 df, which is signif- study was excluded (Spearman q = .485, P < .02).
icant at the 5% level. There is evidence of heterogeneity of However, in a weighted effect size analysis including
effect size in the 24 studies. A forest plot (see Everitt86) of the Trower et al53 study, the estimated regression coeffi-
the effect sizes and associated 95% CIs ordered by CTAM cient of effect size on CTAM (found from a weighted
score is shown in figure 1. regression—see above) was .0056 with 95% CI
530
 Assessing Quality of CBTp Trials

Downloaded from http://schizophreniabulletin.oxfordjournals.org/ at Universitaetsbibliothek Muenchen on July 4, 2014

Fig. 3. Funnel Plot.

Masked and Unmasked Assessment. One of the main

Fig. 2. A Bubbleplot of Effect Size Against Clinical Trial
Assessment Measure (CTAM) Score (Radii of the Circles Represent
questions of interest in this study was whether masking
the SE of the Effect Size). of the assessor to the treatment group allocation can
lead to different effect sizes because this has been shown
for a number of medication studies. There is also some
[0.0152, 0.004]. A bubbleplot (see Everitt86) of effect indication of a relationship because the assessment do-
size against CTAM in which the radii of the circles main includes group masking which was found in the
represent the SE of the effect size of a study is shown results above. To address the specific question of mask-
in figure 2. The tendency is, as expected, for smaller ing, a random-effects model was applied separately to the
CTAM scores to be associated with larger effect sizes, 14 masked studies and to the 10 unmasked studies. For
but the relationship is weak suggesting that other factors the masked studies, the estimated effect size was 0.307
not included in the CTAM scale also affect the trial (95% CI = 0.087, 0.527); the corresponding figure for
results. the unmasked studies was 0.492 (95% CI = 0.312,
0.672). There is a tendency for the unmasked studies
Relationship Between Domains of CTAM and Effect to be overoptimistic about the effects of CBTp, with
Size. The estimated regression coefficients for the var- effect sizes of 50%–100% higher than those found in
ious domains of CTAM and effect size and their esti- masked studies. Even more telling perhaps is the
mated CIs are given in table 4. In these analyses, none lower end of the 95% CI (0.087, 0.527) for the masked
of the regression coefficients are significant. However, studies.
when the Trower et al53 study was excluded, the relation-
ship of the assessment domain and effect size became sig-
Relationship Between Clinical Emphasis and Effect
nificant (the estimated regression coefficient was .017
Size. The analyses were carried out excluding the
(95% CI = 0.032, 0.002). All but one domain is in
Trower et al53 study because the clinical emphasis in
the expected direction of higher effect sizes with poorer
this study of so different. The estimated regression
methodology within a particular domain.
coefficient of .014 (0.0004, 0.028) was significant and
suggests that more behavioral treatments produce greater
Table 4. Relationship Between Methodological Rigor and Effect effect sizes.
Size
Publication Bias. Possible publication bias was investi-
Regression 95% Confidence gated by a funnel plot (effect size against precision) (see
Domain Coefficient Interval figure 3). The absence of studies in the left-hand corner
Sample .018 (0.065, 0.039) of this plot is usually taken as an indication of possible
Allocate .002 (0.058, 0.062) publication bias. The current plot does not appear to
Assess .014 (0.035, 0.007) indicate any evidence of a worrying publication bias
Control .013 (0.056, 0.030) and so suggests that the estimated effect size found
Analysis .020 (0.057, 0.017) from the random-effects model applied to the 24 studies
Treat .002 (0.049, 0.045)
is realistic.
531
T. Wykes et al.

Table 5. Effect Sizes by Methodological Quality

Mean Weighted 95% Confidence Heterogeneity Test (df), No. of Sample 95% Confidence
Effect Size Interval Significance Level Studies Size Interval of Difference

Target symptom
High CTAM 0.223 0.017, 0.428 27.73 (11), significant at the 5% level 12 1124 0.038, 0.584
Low CTAM 0.534 0.343, 0.725 35.35 (20), significant at the 5% level 21 840
Positive symptom
High CTAM 0.222 0.016, 0.427 27.83 (11), significant at the 5% level 12 1124 0.002, 0.532
Low CTAM 0.487 0.311, 0.664 27.35 (19), not significant 20 794
Negative symptom
High CTAM 0.206 0.104, 0.516 28.33 (8), significant at the 5% level 9 631 0.100, 0.908
Low CTAM 0.610 0.200, 1.020 83.33 (13), significant at the 5% level 14 637

Downloaded from http://schizophreniabulletin.oxfordjournals.org/ at Universitaetsbibliothek Muenchen on July 4, 2014

Functioning
High CTAM 0.147 0.172, 0.466 8.49 (4), not significant 5 347 0.058, 0.782
Low CTAM 0.509 0.221, 0.797 23.04 (9), significant at the 5% level 10 520
Mood
High CTAM 0.084 0.154, 0.322 9.21 (5), not significant 6 685 0.048, 1.144
Low CTAM 0.680 0.174, 1.186 32.96 (8), significant at the 5% level 9 268

Note: CTAM, Clinical Trial Assessment Measure.

Relationship Between Methodological Quality and Effect The measure of trial quality, CTAM, detected method-
Size in Each of the Outcome Domains ological variability even in studies that had already
Because there was some relationship between methodo- passed some methodological inclusion criteria. In fact,
logical quality and effect size, the outcomes shown in ta- in other assessments of quality, these studies would prob-
ble 3 were investigated in terms of the relationship ably have been considered ‘‘good research evidence.’’25
between studies where the methodology by current stand- The methodology scores were not bimodal, and there
ards might be considered adequate. Because there was no was also variability in items scoring highly among the tri-
specific domain that was poor in all the studies, a cutoff als. Although there was variability, there is some evidence
score for the CTAM total of 65 was taken to indicate ad- that trials improved their quality and their size over time
equate methodology. This produced 12 studies with ad- with larger more rigorously controlled trials appearing
equate methodology and 22 with poorer methodology. more recently. Similar variation has been noted in
The results of the meta-analyses in each of these groups CBT and suicidal behavior and therapies involving vir-
are shown in table 5. For each symptom area, the effect tual reality although the trials investigated here were
size is larger for the low CTAM studies. This difference is of higher quality and were more likely to be large enough
significant for the target symptom and for assessments of to identify clinically significant effects.87,88
mood, and the CIs for the difference is highly skewed for As well as variability in the methodologies adopted
all the other measures. The CIs for the weighted effect there is also variability in the types of therapy offered un-
sizes in higher CTAM scoring studies are also not signif- der the single umbrella of CBTp. We tried to capture this
icant for negative symptoms, functioning, and mood. in our ranking of emphasis, but even if the emphasis was
However, even when the more stringent criterion is similar, the ingredients used both within a particular
used to define the groups, there are still modest effect study and certainly within a particular participant are
sizes for positive symptoms and the targeted symptom. likely to be variable (see Tarrier and Wykes for a more
detailed discussion49). Our independent rating of clinical
emphasis was not linked to methodological rigor. As
expected, studies from the United Kingdom had higher
Discussion
methodology scores because this site had developed
What Variability Is There Between Studies? the CBTp study models which then led to the larger
This is the largest review of CBTp trials containing 20 and more rigorous studies. Other sites are now building
more trials than the most comprehensive meta-analysis21 from initial pilot and feasibility studies to more signifi-
and 475 more participants. The field has certainly now cant RCTs.
matured so that there are now nearly 6 times as many
patients as in the first reported meta-analysis.19 The av- Is CBTp Effective?
erage number of participants in each trial is relatively The obvious further question is, for what outcome? Be-
small (average 66), but this is the same as for drug trials cause CBTp was designed to specifically target positive
in schizophrenia over the past 35 years.35 symptoms, we have used this data set to provide the
532
 Assessing Quality of CBTp Trials

answer on efficacy and the simple answer to this question we would reduce the heterogeneity in the effect sizes, but
is yes it has modest effects. This is the conclusion of other again although the effects were modest (0.399) there was
meta-analyses in this area.12–16,18–23 The current meta- still significant heterogeneity.
analysis was methodologically rigorous, for instance, In our subsample, the trials with poorer methodology,
we carried out assessments based on weighted effect sizes as measured by CTAM, tended to have larger effect sizes,
that earlier meta-analyses failed to do (eg, Rector and although the relationship is relatively weak. Only about
Beck18 and Gould et al19), included all eligible trials 60% of meta-analyses carry out quality assessments of
that others failed to do (eg, Pilling et al13), and considered methodology and even fewer (50%) actually link these tri-
methodological variability that others failed to do (eg, al quality assessments to the interpretation of the findings
Pfammatter et al20). Although the overall effect size on in meta-analyses89 or specifically to psychological treat-
positive symptoms was lower than assessed by others ment studies. The measure we derived for the quality as-
(eg, Rector and Beck18 and Gould et al19), it was in sessment is reliable and has evidence of both internal and

Downloaded from http://schizophreniabulletin.oxfordjournals.org/ at Universitaetsbibliothek Muenchen on July 4, 2014

line with recent evaluations using a smaller data set external validity, and although there may be further
but similar rigorous methods21 (effect size = 0.35, no developments it is clear that the CTAM list provides
studies = 15). Even when a stringent methodological cri- a start for improving the methodological rigor specifi-
terion is adopted a modest significant positive effect size cally of psychological treatment trials.
remains. Some assessments of trial quality have been found se-
But perhaps more interesting is that this meta-analysis riously to affect the assessment of treatment effectiveness
shows that CBTp may have an effect on other outcomes in other areas of treatment.90 However, in our assess-
even if these were not the specific targets of the therapy. ments, no specific domain dominated. This may be be-
The effects for all outcomes were modest, and with the cause of the heterogeneity of the trial methodologies
exception of hopelessness (4 studies) the effects were themselves91 suggesting that some studies are particularly
significant overall although these results are more spec- adept at one aspect of methodological rigor and not
ulative because they seem to be affected by the method- others. This is hopeful because it suggests that with
ological rigor of the study. However, for hopelessness, enough effort all aspects could be improved and high lev-
not only did this outcome showed homogeneity of effect els of methodological rigor would be possible. Method-
size but also 3 out of the 4 studies showed negative ef- ological quality also affected the effect sizes for the other
fects suggesting that current CBTp approaches are not outcomes investigated making most not significant. So
beneficial for this particular outcome and may even be potential methodological quality variation can (and
detrimental. did) lead to bias and reduction in precision of the esti-
There are also relationships between outcomes such mates of the therapy’s effectiveness.90–92 These results
that some outcome improvements seem to be correlated are no different to those that have been found for
suggesting that irrespective of the actual target, CBTp drug and other medical treatment, and the results
has wider beneficial impacts. The mode of transmission reported here are salutary for all psychiatric studies.
of these benefits and the timing of such overlapping ben- The most influential individual methodological vari-
efits is not clear and would require further prospective able was masked assessment, which is known to be dif-
investigation using individual level data. ficult to carry out in psychological treatment trials.
The mode of provision of CBTp also seems to have lit- When masked assessment was attempted (we do not
tle effect with group studies showing the same modest know how successful this was), then there was nearly
effects. This may make them more cost-effective for par- a 60% reduction in the effect size. This compares with
ticular outcomes and is a mode of presentation that is the 34% inflation of effect size reported in other studies.39
more highly developed in the United States. However, In order to ensure true randomization, the random allo-
there are fewer studies testing group therapy and further cation sequence should be irreversibly administered and
work in this area is merited to investigate how much should be concealed to the individuals in charge of
improvement is necessary and to reduce the heterogeneity enrolment. Knowledge of possible upcoming allocation
of the effect sizes. Work by both Wykes et al83 and may permit selective allocation of patients through
Barrowclough et al40 on the clustering of effects within some form of manipulation. This might result in those
groups provides an analytic strategy for identifying fac- more amenable or more likely to benefit from the treat-
tors that predict good outcomes and therefore the likeli- ment being allocated differentially between groups. The
hood of increased efficacy of group treatment. effects on subsequent measures are also essential to
consider. It is obvious that assessors may well be biased
in their assessment, particularly their expectation of pos-
Can We Account for Heterogeneity in Effect Size? itive changes if they know the group allocation. This is
We had thought that by choosing a more homogenous a problem not only in the assessment of psychological
sample of studies, ie, those that provided CBTp individ- treatments but in many other fields. In a recent assess-
ually and which specifically targeted positive symptoms, ment based in Denmark, the majority of the trials
533
T. Wykes et al.

reviewed had inadequate concealment irrespective of ter into a trial and the pool from which they were
whether they used data in the trial protocol or the trial recruited can greatly affect the ease of treatment and po-
publication.93 tentially the treatment response. Developments in back-
Other meta-analyses have suggested that CBTp for ground mental health services over time and the
positive symptoms is more effective for acute populations differences between available services between different
compared with chronic ones, but these conclusions may locations, such as the United Kingdom and United
also be tempered by overall trial quality. For instance, in States, can also make comparisons difficult. This is
the studies identified by Zimmerman et al,21 the acute true both between different studies and within a study be-
studies had only 1 in 3 with adequate methodology (as tween experimental and control groups when both re-
defined in this article) but for the chronic group 5 out ceive standard care or TAU. The nature of TAU can
of 10 had adequate methodology. These methodology be extremely varied. An investigation of this was beyond
quality differences are likely to have an impact on the ef- the scope of this review and must wait until there are

Downloaded from http://schizophreniabulletin.oxfordjournals.org/ at Universitaetsbibliothek Muenchen on July 4, 2014

fect size calculations, as was seen in table 5. However, larger numbers of studies. We hoped to capture some
even with the reduction in effect size, the results of the of the differences by looking at country of origin, but
meta-analysis on positive symptoms indicates that for this is not a very clean variable. Participant willingness
this increased corpus of trials there is a benefit for to enter a trial may also be affected by the nature and
CBTp on the positive symptoms of schizophrenia al- accessibility of alternative care from standard services.
though the lower end of the CI suggests only marginal These are issues which might be expected to potentially
benefits. The general results therefore support other influence trial results but are rarely considered or
published meta-analyses and systematic reviews that assessed other than in the folklore of the academic
CBTp should be included in guidance on treatments community.
for schizophrenia. While still supporting the use of CBTp in schizophre-
There was a trend for clinical models of CBTp that nia we wish to caution against exaggerated claims of the
emphasize more behavioral aspects of treatment to pro- magnitude of treatment benefit. Many of the studies in-
duce larger effect sizes, and this relationship could not be volved in our meta-analysis were carried out by or under
accounted for by differences in methodological rigor. So, the supervision of the experts in CBTp. There have been
studies from all schools of therapy were carried out as some studies showing that after specialist training
well (or as badly) as each other. Our assessment of the CBTp can be of value in local service either by training
level of behavioral emphasis was based only on the pub- psychiatrists or nurses in key skills. But in order to op-
lished information on the type of therapy approach. It is timize the treatment effects in local services, it may be
of course quite possible that every client in any study re- necessary to provide specific training in the clinical
ceived a different amount of the therapy and therefore models that are most successful. There is some indica-
a mixture of the behavioral and more cognitive aspects tion of a bias toward the use of behavioral features
of the treatment. Assessments of fidelity measure if the although the specific successful ingredients of the un-
therapy complies with what is expected of CBTp but derlying models have not yet been investigated. How ef-
rarely do they measure whether all aspects of the therapy fective novel treatments are disseminated into the wider
manual were received. If some patients received less of mental health service is a challenge that sometimes
a ‘‘dose’’ of therapy, it might be expected that this would results in unanticipated difficulties.94 The hope that
lead to lower clinical effectiveness (see Tarrier and CBTp would become widely available through brief
Wykes49 for a full discussion of the validity issues in training to less well-qualified and skilled staff is not
CBT studies). In the future, trials of CBTp or any psy- borne out by experience. But the cost of CBTp being
chological treatment need to include some measure of delivered by well-trained and experienced psychologists
the ‘‘effective dose’’ of a specific therapy. This is certainly should not preclude the adoption of CBTp into mental
not the same as measures for medication therapies which health services. Even if group treatments were an alter-
are simple and generally only refer to the number of days native and perhaps cost-effective, these too require ex-
of treatment at a specified level. A sophisticated but perienced and trained personnel to carry them out. The
simple-to-apply measure for psychological treatments fact that organ transplants can only be carried out by
needs to include aspects of the therapy process—eg, how experienced and skilled surgeons is not advanced as
many specific CBTp sessions has the person received. a reason to restrict transplant surgery. In mental health
This might be gleaned from a simple process measure col- services, cheap and ineffective alternatives should not be
lected by the therapist and later tested for reliability. a viable option to useful, valued, and effective but more
In addition, we were not able to assess adequately some expensive ones.
of the external validation factors, such as treatment gen- Despite the current acceptance of CBTp in treatment
eralization, because most studies recruited patients from guidance and in some service settings—particularly the
a convenience sample (ie, through referrals from other United Kingdom—we still need more information if
agencies) and were relatively small. How participants en- we are to provide CBTp effectively and efficiently. The
534
 Assessing Quality of CBTp Trials

concern of CBTp specialists is that research funders will 8. Meltzer HY, Bobo WV. Interpreting the efficacy findings in
assume that, once the therapy has been accepted, it the CATIE study: what clinicians should know. CNS Spectr.
2006;11:14–24.
requires no further investment; so we have set out
9. Jones PB, Barnes TR, Davies L, et al. Randomized controlled
some key features of a future CBTp research program. trial of the effect on quality of life of second- vs first-genera-
It should include tion antipsychotic drugs in schizophrenia: Cost Utility of the
Latest Antipsychotic Drugs in Schizophrenia Study (CUt-

methodologically rigorous efficacy trials of well- LASS 1). Arch Gen Psychiatry. 2006;63:1079–1087.
defined treatment programs, 10. Tarrier N, Calam R. New developments in cognitive-

measures of treatment process which allow an estimate behavioural case formulation. Epidemiological, systemic and
social context: an integrative approach. Cogn Behav Psy-
of the dose of treatment that is more sophisticated than
chother. 2002;30:311–328.
‘‘number of sessions,’’
11. Tarrier N. Case Formulation in Cognitive Behaviour Therapy:

outcome measures that are acceptable not only to the

Downloaded from http://schizophreniabulletin.oxfordjournals.org/ at Universitaetsbibliothek Muenchen on July 4, 2014

The Treatment of Challenging and Complex Clinical Cases.
clinical and academic community but also to the London, UK: Routledge; 2006.
patients themselves, 12. Cormac I, Jones C, Campbell C, Silveira da Mota Neto J.

studies of effectiveness with different models, staff Cognitive behaviour therapy for schizophrenia (Cochrane
training methods, and background service provision, Review). Cochrane Library. Issue 1Oxford, UK: Update Soft-

long-term follow-up studies of the durability of treat- ware; 2003.
ment effects. 13. Pilling S, Bebbington P, Kuipers E, Garety P, Orbach G,
Morgan C. Psychological treatments in schizophrenia:
I. Meta-analysis of family intervention and cognitive behav-
This program will produce a refinement of treatment iour therapy. Psychol Med. 2002;32:763–782.
that has been tested using rigorous methods and is there- 14. Thornicroft G, Susser E. Evidence-based psychotherapeutic
fore likely to offer the most benefit to people with con- interventions in the community care of schizophrenia. Br J
tinuing positive symptoms of schizophrenia. Psychiatry. 2001;178:2–4.
15. NHS Centre for Reviews and Dissemination Psychosocial
interventions for schizophrenia. Eff Health Care. 6. 2000. pp. 1–8.
Acknowledgments 16. Dickerson FB. Cognitive behavioral psychotherapy for
schizophrenia: a review of recent empirical studies. Schizophr
We would wish to thank the authors of all participating Res. 2000;43:71–90.
studies for their help and assistance and also 17. Dickerson FB. Update on cognitive behavioural psychother-
G. Haddock, N. Rector, A. Gumley, B. Gaudiano, apy for schizophrenia: review of recent studies. J Cogn Psy-
and L. Valmaggia for providing further then chother. 2004;18:189–205.
unpublished data; R. Gould and K. Mueser for details 18. Rector NA, Beck AT. Cognitive behavioral therapy for
of their meta-analysis; G. Dunn and T. Johnson for schizophrenia: an empirical review. J Nerv Ment Dis.
comments on design and statistics; and E. Peters for 2001;189:278–287.
rating the clinical emphasis of the models of therapy. 19. Gould RA, Mueser KT, Bolton E, Mays V, Goff D. Cogni-
Both Wykes and Tarrier have carried out CBTp trials. tive therapy for psychosis in schizophrenia: an effect size anal-
ysis. Schizophr Res. 2001;48:335–342.
20. Pfammatter M, Junghan U, Brenner H. Efficacy of psycho-
References logical therapy in schizophrenia: conclusions from meta-
analyses. Schizophr Bull. 2006;32:S64–S80.
1. Beck AT. Successful out-patient psychotherapy of a chronic 21. Zimmermann G, Favrod J, Trieu V, Pomini V. The effect
schizophrenic with a delusion based on borrowed guilt. Psy- of cognitive behavioural treatment on schizophrenia
chiatry. 1952;15:205–212. spectrum disorders: a meta-analysis. Schizophr Res. 2005;
2. Shapiro MB, Ravenette AT. A preliminary experiment on 77:1–9.
paranoid delusions. J Ment Sci. 1959;105:295–312. 22. Sensky T. The effectiveness of cognitive therapy for schizo-
3. Liberman RP, Teigen J, Patterson R, Baker V. Reducing de- phrenia: what can we learn from the meta-analyses. Psy-
lusional speech in chronic paranoid schizophrenics. J Appl chother Psychosom. 2005;74:131–135.
Behav Anal. 1973;6:57–64. 23. Gaudiano BA. Is symptomatic improvement in clinical trials
4. Meichenbaum D, Cameron R. Training schizophrenics to of cognitive-behavioral therapy for psychosis clinically signif-
talk to themselves: a means of developing attentional control. icant? J Psychiatr Pract. 2006;12:11–23.
Behav Ther. 1973;4:515–534. 24. NICE Schizophrenia: Core interventions in the Treatment and
5. Brown GW, Birley JLT, Wing JK. Influence of family life on Management of Schizophrenia in Primary and Secondary Care.
course of schizophrenic disorders—replication. Br J Psychia- Available at: http:\\www.nice.org.uk. Accessed October 19,
try. 1972;121:241–258. 2007.
6. Butzlaff RL, Hooley JM. Expressed emotion and psychiatric 25. Lehman AF, Kreyenbuhl J, Buchanan R, et al. The Schizo-
relapse: a meta-analysis. Arch Gen Psychiatry. 1998;5: phrenia Patient Outcomes Research Team (PORT): updated
547–552. treatment recommendations 2003. Schizophr Bull. 2004;30:
7. Mueser KT, Berenbaum H. Psychodynamic treatment of 193–217.
schizophrenia: is there a future? Psychol Med. 1990;20: 26. Barrowclough C, Haddock G, Tarrier N, et al. Randomized
253–262. controlled trial of motivational interviewing, cognitive

535
T. Wykes et al.

behaviour therapy, and family intervention of patients with 47. Sterne JAC, Juni P, Schulz KF, Altman DG, Barlet C, Eggar
co-morbid schizophrenia and substance use disorder. Am J M. Statistical methods for assessing the influence of study
Psychiatry. 2001;158:1706–1713. characteristics on treatment effects in meta-epidemiological
27. Richardson A, Baker M, Burns T, Lilford RJ, Muijen M. research. Stat Med. 2002;21:1513–1524.
Reflections on methodological issues in mental health re- 48. Huwiler-Muntener K, Juni P, Junker C, Eggar M. Quality of
search. J Ment Health. 2000;9:463–470. reporting of randomised trials as a measure of methodologic
28. Slade M, Priebe S. Are randomised controlled trials the only quality. JAMA. 2002;287:2801–2804.
gold that glitters? Br J Psychiatry. 2001;179:286–287. 49. Tarrier N, Wykes T. Is there evidence that cognitive behaviour
29. Pocock SJ. Clinical trials with multiple outcomes: a statistical therapy is an effective treatment for schizophrenia? A cau-
perspective on their design, analysis, and interpretation. Con- tious or cautionary tale? Behav Res Ther. 2004;42:1377–1401.
trol Clin Trials. 1997;18:530–545. 50. Schulz KF, Altman DG, Moher D. Allocation concealment
30. Doll R. Controlled trials: the 1948 watershed. BMJ. in clinical trials. JAMA. 2002;288:2406–2407.
1998;317:1217–1220. 51. Altman DG, Schulz KF, Moher D, et al. The revised CON-

Downloaded from http://schizophreniabulletin.oxfordjournals.org/ at Universitaetsbibliothek Muenchen on July 4, 2014

31. Everitt BS, Wessely S. Clinical Trials in Psychiatry. Oxford, SORT statement for reporting randomised trials: explanation
UK: Oxford University Press; 2003. and elaboration. Ann Intern Med. 2001;134:663–694.
32. Everitt BS, Pickles A. Statistical Aspects of the Design and 52. Brown SA. Measurement of quality of primary studies for
Analysis of Clinical Trials. London, UK: Imperial College meta-analysis. Nurs Res. 1991;40:352–355.
Press; 2000. 53. Trower P, Birchwood M, Meaden A, Byrne S, Nelson A, Ross
33. Begg C, Cho M, Eastwood S, et al. Improving the quality of K. Cognitive therapy for command hallucinations: randomised
reporting of randomized controlled trials: the CONSORT controlled trial. Br J Psychiatry. 2004;184:312–320.
statement. JAMA. 1996;276:637–639. 54. Milton F, Patwa VK, Hafner RJ. Confrontation vs belief
34. Moher D, Schultz K, Altman D. The CONSORT statement: re- modification in persistently deluded patients. Br J Med Psy-
vised recommendations for improving the quality of reports of chol. 1978;51:127–130.
parallel group randomisation. JAMA. 2001;285:1987–1991. 55. Tarrier N, Beckett R, Harwood S, Baker A, Yusupoff L,
Ugarteburu I. A controlled trial of two cognitive behavioural
35. Thornley B, Adams C. Content and quality of 2000 con-
methods of treating drug-resistant residual psychotic symp-
trolled trials in schizophrenia over 50 years. BMJ.
toms in schizophrenic patients: I. Outcome. Br J Psychiatry.
1998;317:1181–1184.
1993;162:524–532.
36. Moher D, Pham B, Jones A, et al. Does quality of reports of
56. Garety PA, Kuipers L, Fowler D, Chamberlain F, Dunn G.
randomised trials affect estimates of intervention efficacy
Cognitive behavioural therapy for drug resistant psychosis.
reported in meta-analyses? Lancet. 1998;352:609–613.
Br J Psychiatry. 1994;67:259–271.
37. Schulz KF, Chalmers I, Hayes RJ, Altman DG. Empirical ev-
57. Bentall RP, Haddock G, Slade PD. Cognitive behaviour ther-
idence of bias: dimensions of methodological quality associ-
apy for persistent auditory hallucinations: from theory to
ated with estimates of treatment effect in controlled trials.
therapy. Behav Ther. 1994;25:51–66.
JAMA. 1995;280:178–180.
58. Drury V, Birchwood M, Cochrane R, Manmillan F. Cogni-
38. Marshall M, Lockwood A, Bradley C, Adams C, Joy C, tive therapy and recovery from acute psychosis: a controlled
Fenton M. Unpublished rating scales; a major source of trial. I. Impact on psychotic symptoms. Br J Psychiatry.
bias in randomised controlled trials of schizophrenia. Br J 1996;169:593–601.
Psychiatry. 2000;176:249–252.
59. Kuipers E, Garety P, Fowler D, et al. London-East Anglia
39. Moher D, Jadad AR, Nichol G, Penman M, Tugwell P, randomised controlled trial of cognitive-behavioural therapy
Walsh S. Assessing the quality of randomised controlled for psychosis. I: effects of the treatment phase. Br J Psychia-
trials: an annotated bibliography of scales and checklists. try. 1997;171:319–327.
Control Clin Trials. 1995;16:62–73.
60. Tarrier N, Yusupoff L, Kinney C, et al. A randomised con-
40. Barrowclough C, Haddock G, Lobban F, et al. Group trolled trial of intensive cognitive behaviour therapy for
cognitive-behavioural therapy for schizophrenia: randomised chronic schizophrenia. BMJ. 1998;317:303–307.
controlled trial. Br J Psychiatry. 2006;189:1–7.
61. Daniels L. A group cognitive-behavioral and process-oriented
41. Kemp R, Kirov G, Everitt B, Hayward P, David A. Rando- approach to treating the social impairment and negative
mised controlled trial of compliance therapy. 18-month symptoms associated with chronic mental illness. J Psy-
follow-up. Br J Psychiatry. 1998;17:413–419. chother Pract Res. 1998;7:167–176.
42. Chalmers TC, Smith H, Blackburn B, et al. A method for 62. Levine J, Barak Y, Granek I. Cognitive group therapy for
assessing the quality of a randomized control trial. Control paranoid schizophrenics: applying cognitive dissonance.
Clin Trials. 1981;2:31–49. J Cogn Psychother. 1998;12:3–12.
43. Jadad AR, Moore RA, Carroll D, et al. Assessing the quality 63. Pinto A, La Pia S, Mennella R, Giorgio D, De Simone L.
of reports of randomized clinical trials: is blinding necessary? Cognitive-behavioural therapy and clozapine for clients
Control Clin Trials. 1996;17:1–12. with treatment-refractory schizophrenia. Psychiatr Serv.
44. Juni P, Witschi A, Bloch R, Eggar M. The hazards of scoring 1999;50:901–904.
the quality of clinical trials for meta-analysis. JAMA. 64. Haddock G, Tarrier M, Morrison AP, Hopkins R, Drake R,
1999;282:1054–1060. Lewis S. A pilot study evaluating the effectiveness of individ-
45. Juni P, Altman G, Eggar M. Assessing the quality of con- ual inpatient cognitive behavioural therapy in early psychosis.
trolled clinical trials. BMJ. 2001;323:42–46. Soc Psychiatry Psychiatr Epidemiol. 1999;34:254–258.
46. Kazdin AE, Bass P. Power to detect differences between alter- 65. Halperin S, Nathan P, Drummond P, Castle D. A cognitive-
native treatments in comparative psychotherapy outcome re- behavioural, group-based intervention for social anxiety in
search. J Consult Clin Psychol. 1989;57:138–147. schizophrenia. Aust N Z J Psychiatry. 2000;34:809–813.

536
 Assessing Quality of CBTp Trials

66. Sensky T, Turkington D, Kingdon D, et al. A randomised 80. Startup M, Jackson M, Bendix S. North Wales randomized
controlled trial of cognitive-behavioural therapy for persis- controlled trial of cognitive behaviour therapy for acute
tent symptoms in schizophrenia resistant to medication. schizophrenia spectrum disorders: outcomes at 6 and 12
Arch Gen Psychiatry. 2000;57:165–172. months. Psychol Med. 2004;34:413–422.
67. Bailer J, Takats I, Westermeier C. Efficacy of individualized 81. Cather C, Penn D, Otto M, Yovel I, Mueser K, Goff D. A
cognitive-behavioral therapy for schizophrenic patients with pilot study of functional Cognitive Behavioral Therapy
negative symptoms and social disabilities: a controlled trial. (fCBT) for schizophrenia. Schizophr Res. 2005;74:201–209.
Z Klin Psychol Psychother. 2001;30:268–278. 82. Granholm E, McQuaid JR, McClure FS, et al. A random-
68. Lewis SW, Tarrier N, Haddock G, et al. Randomised con- ized, controlled trial of cognitive behavioral social skills
trolled trial of cognitive-behaviour therapy in early schizo- training for middle-aged and older outpatients with chronic
phrenia: acute phase outcomes. Br J Psychiatry. 2002;181: schizophrenia. Am J Psychiatry. 2005;162:520–529.
suppl 4391–97. 83. Wykes T, Hayward P, Thomas N, et al. What are the effects
69. Turkington D, Kingdom D, Turner T. The Insight Group.Ef- of group cognitive behaviour therapy for voices? A rando-

Downloaded from http://schizophreniabulletin.oxfordjournals.org/ at Universitaetsbibliothek Muenchen on July 4, 2014

fectiveness of a brief cognitive behavioural therapy interven- mised control trial. Schizophr Res. 2005;77:201–210.
tion in the treatment of schizophrenia. Br J Psychiatry. 84. Gaudiano B, Herbert J. Acute treatment of inpatients with
2002;180:523–528. psycholtic symptoms using acceptance and commitment ther-
70. Durham RC, Guthrie M, Morton RV, et al. Tayside-Fife apy: pilot results. Behav Res Ther. 2006;44:415–437.
clinical trial of cognitive-behavioural therapy for medication- 85. Fleiss JL. Measures of effect size for categorical data. In:
resistant psychotic symptoms: results to 3-month follow-up. Cooper H, Hedges LV, eds. The Handbook of Research Synthe-
Br J Psychiatry. 2003;182:303–311. sis. New York, NY: Russell Sage Foundation; 1993:245–281.
71. Hall PL, Tarrier M. The cognitive-behavioural treatment of 86. Everitt BS. Modern Medical Statistics: A Practical Guide.
low self-esteem in psychotic patients: a pilot study. Behav London, UK: Arnold; 2002.
Res Ther. 2003;41:317–332. 87. Tarrier N, Taylor K, Gooding P. Cognitive behavioural inter-
72. Valmaggia L, van der Gaag M, Tarrier N, Pijnenborg GHM, ventions to reduce suicidal behaviour: a systematic review and
Sloof CJ. A randomized controlled trial of cognitive behavior meta-analysis. Behav Modif. In press.
therapy with treatment refractory positive symptoms of 88. Gregg L, Tarrier N. Virtual reality in mental health: a review
schizophrenia. Br J Psychiatry. 2005;186:324–330. of the literature. Soc Psychiatry Psychiatr Epidemiol.
73. Granholm E, McQuaid JR, McClure FS, Pedrelli P, Jeste 2007;42:343–354.
DV. A randomized controlled pilot study of cognitive behav- 89. Moja L, Telaro E, D’Amico R, Moschetti I, Coe L, Liberati
ioural social skills training for older patients with schizophre- A. Assessment of methodological quality of primary studies
nia. Schizophr Res. 2002;53:167–169. by systematic reviews: results of the metaquality cross sec-
74. Gumley A, O’Grady M, McNay L, Reilly J, Power KG, tional study. BMJ. 2005;330:1053.
Norrie J. Early intervention for relapse in schizophrenia: 90. Balk E, Bonis P, Moskowitz H, et al. Correlation of
results of a 12-month randomised controlled trial of cognitive quality measures with estimates of treatment effect in
behavioural therapy. Psychol Med. 2003;33:419–431. meta-analyses of randomized controlled trials. JAMA. 2002;
75. Rector N, Seeman MV, Segal ZV. Cognitive therapy for 287:2973–2982.
schizophrenia: a preliminary randomised controlled trial. 91. Detsky AS, Naylor CD, O’Rourke K, McGeer AJ, L’Abbe
Schizophr Res. 2003;63:1–11. KA. Incorporating variations in the trial quality of individual
76. Jolley S, Garety P, Craig T, Dunn G, White J, Aitken M. randomised trial into meta-analysis. J Clin Epidemiol.
Cognitive therapy in early psychosis: a pilot randomized con- 1992;45:255–265.
trolled trial. Behav Cogn Psychother. 2003;31:473–478. 92. Gotzsche PC. Methodology and overt and hidden bias
77. Kingsep P, Nathan P, Castle D. Cognitive behavioural group in reports of 196 double-blind trials of nonsteroidal anti-
treatment for social anxiety in schizophrenia. Schizophr Res. inflammatory drugs in rheumatoid arthritis. Control Clin
2003;63:121–129. Trials. 1989;10:31–56.
78. Wiersma D, Jenner J, Nienhuis F, van de Willige G. Hallucina- 93. Pildal J, Chan A-W, Hróbjartsson A, Forfang E, Altman D,
tion focused integrative treatment improves quality of life in Gøtzsche P. Comparison of descriptions of allocation con-
schizophrenia patients. Acta Psychiatr Scand. 2004;109:194–201. cealment in trial protocols and the published reports: cohort
79. Bechdolf A, Knost B, Kuntermann C, et al. A randomized study. BMJ. 2005;330:1049.
comparison of group cognitive-behavioural therapy and 94. Tarrier N, Barrowclough C, Haddock G, McGovern J. The
group psychoeducation in patients with schizophrenia. Psy- dissemination of innovative cognitive-behavioural treatments
chiatr Scand. 2004;110:21–28. for schizophrenia. J Ment Health. 1999;8:569–582.

537