Critical Appraisal – D.K. Hunt, MD; Dept of Medicine.
version 07/2008
Online version: http://medicine.uthscsa.edu/journalclub/ PDA version: http://clinical.uthscsa.edu/pda/
Question 1: What is study design / level of evidence (see back of this card)
Question 2: Are the results of this study valid? - Adapted from the Users' Guides to Evidence-Based Practice - http://www.cche.net/
Structured Reading of Articles (PICO)
Therapy Diagnosis/ Causation/ Syst. Review Practice Decision analysis
Screening Harm/Prognosis (meta-analysis) Guideline
Patients/ Controlled? Was a broad Was a well-defined Was article search Was a (recent) Was article search
problem/ If so, were pts spectrum of cohort or case thorough? systematic thorough?
population randomized patients studied? series used? review of
successfully? (see Explicit inclusion literature Explicit inclusion
table 1) If a controlled criteria? done? criteria?
study, were patient
groups similar?
Intervention Double-blinded? Was the new test Was study design 1. Similar results Were all Were all important
Treatment, test, done strong? across studies (e.g. important clinical strategies and
etiologic, or Were groups independently of (RCT>cohort>case homogeneity)? options and outcomes considered?
prognostic factor treated equally the reference control>case 2. Strong study outcomes
aside from exp. standard? series> case designs? considered? Are the probabilities
treatment? report) (RCT>cohort>case credible?
control>case
series> case Are the utilities
report) credible?
3. Large studies
(less publication
bias)?
4. Did the studies
use individual
patient data?
Comparison True control True control True control group? True control True control arm?
group? group? groups?
Outcome 1. Was there 80% What was the Were objective Were outcomes Are practical, Did one choice lead to
follow up? reference outcome criteria consistent across clinically important gains?
2. Was an standard? applied? studies (e.g. important,
Intention-to-treat homogeneity rec.s made? Was this result stable
analysis Was the reference Was there 80% reported (P>0.2)? over varying
performed (how standard applied follow-up? How strong probabilities and
were study to all patients? are the rec.s? utilities (sensitivity
dropouts Association makes analysis)?
handled)? Were outcomes biological sense? Has the guide
3. Were blindly assessed? undergone
outcomes blindly Did exposure peer review &
assessed? precede outcome? testing?
Dose-response Are the rec.s
effect? applicable to
your patients?
Question 2: Is this valid study important? (http://medicine.uthscsa.edu/calc may help with numbers)
Therapy Diagnosis/ Causation/ Overview Harm Decision analysis to
Screening Prognosis
RRR=
CER – EER How likely are out- See Therapy See Therapy and Did 1 choice lead
CER Sens=a/(a+c) comes? Diagnosis important gains?
Spec=d/(b+d)
ARR= PPV=a/(a+b) How precise are the Was same choice
CER - EER NPV=d/(c+d) estimates? preferred despite
LR+=sens/(1-spec) changes in
NNT= OR=ad/bc probabilities
1/ARR or utilities
OR=ad/bc
DEFINITIONS
Disease
If study is negative- RRR= relative risk reduction or risk ratio LR+=positive likelihood ratio
T a b
What was the power? E CER=control event rate EER=experimental (or PPV=positive predictive value
intervention)
S c d NPV=negative predictive value
T ARR=absolute risk reduction or risk difference
Sens=sensitivity
NNT=number needed to treat
Spec=specificity
OR=odds ratio
Levels of Evidence:
Level I A: Homogeneous systematic review of randomized controlled trials (RCTs) Journal Club
B: Single RCT
C. ‘All or none’ study (if biologically plausible and dramatic results) Please provide for each session: the
clinical question, the search strategy used,
A: Homogeneous systematic review of cohort studies a four sentence summary of the article you
Level II
B: Single cohort study found. The four sentences are:
A: Homogeneous systematic review of case-control studies 1) why was the study done
Level III
B: Single case-control study
2) what was the type of study
Level IV
Case series
3) the central result
Level V
Case report
4) the author’s conclusion or clinical
Teaching point: Be careful of changing your practice based on a low level study unless its results are recommendation.
1) dramatic (e.g. the patients lived yet should have died) and 2) biologically plausible.
For more detailed review, add:
Question 3: Should these results change your practice?
1) the level of evidence of the article
Are the results generalizable to your patient? You should also consider your patient’s individual risk,
preferences and concerns. What alternative treatments are available, but not compared in this 2) the structured assessment with PICO
study? (other side of this page)
