THERAPY WORKSHEET: page 1 of 2

Citation:

Are the results of this single preventive or therapeutic trial valid?

Was the assignment of patients to
treatments randomised?
-and was the randomisation list concealed?

Were all patients who entered the trial

accounted for at its conclusion? -and were
they analysed in the groups to which they
were randomised?

Were patients and clinicians kept “blind”

to which treatment was being received?

Aside from the experimental treatment,

were the groups treated equally?

Were the groups similar at the start of the

trial?

Are the valid results of this randomised trial important?

SAMPLE CALCULATIONS:
Occurrence of diabetic Relative Risk Absolute Risk Number Needed
neuropathy Reduction Reduction to Treat
RRR ARR NNT
Usual Insulin Intensive CER - EER CER - EER 1/ARR
Control Event Insulin CER
Rate Experimental
CER Event Rate
EER
9.6% 2.8% 9.6% - 2.8% = 71% 9.6% - 2.8% = 6.8% 1/6.8% = 15 pts,
9.6% (4.3% to 9.3%) (11 to 23)

95% Confidence Interval (CI) on an NNT = 1 / (limits on the CI of its ARR) =

YOUR CALCULATIONS:
Relative Risk Absolute Risk Number Needed
Reduction Reduction to Treat
RRR ARR NNT
CER EER CER - EER CER - EER 1/ARR
CER
THERAPY WORKSHEET: page 2 of 2
Can you apply this valid, important evidence about a treatment in caring for
your patient?
Do these results apply to your patient?
Is your patient so different from
those in the trial that its results
can’t help you?

How great would the potential

benefit of therapy actually be for
your individual patient?

Method I: f Risk of the outcome in your patient,

relative to patients in the trial. expressed
as a decimal: ______

NNT/F = _____/_____ = _________

(NNT for patients like yours)
Method II: 1 / (PEER x RRR) Your patient’s expected event rate if they
received the control treatment:
PEER:______

1 / (PEER x RRR) = 1/________ =

_______
(NNT for patients like yours)
Are your patient’s values and preferences satisfied by the regimen and its
consequences?
Do your patient and you have a
clear assessment of their values
and preferences?

Are they met by this regimen and

its consequences?

Additional Notes:
SYSTEMATIC REVIEW (of Therapy) WORKSHEET: page 1 of 2

Citation:

Are the results of this systematic review of therapy valid?

Is it a systematic review of randomised
trials of the treatment you’re interested
in?

Does it include a methods section that

describes:
finding and including all the
relevant trials?

assessing their individual validity?

Were the results consistent from study to

study?

Are the valid results of this systematic review important?

Translating odds ratios to NNTs. The numbers in the body of the table are the NNTs
for the corresponding odds ratios at that particular patient’s expected event rate
(PEER).

Odds Ratios (OR)

0.9 0.85 0.8 0.75 0.7 0.65 0.6 0.55 0.5
* †
.05 209 139 104 83 69 59 52 46 41
.10 110 73 54 43 36 31 27 24 21
Control .20 61 40 30 24 20 17 14 13 11
Event .30 46 30 22 18 14 12 10 9 8
Rate .40 40 26 19 15 12 10 9 8 7
‡
(CER) .50 38 25 18 14 11 9 8 7 6
.70 44 28 20 16 13 10 9 7 6
§ **
.90 101 64 46 34 27 22 18 15 12

*
The relative risk reduction (RRR) here is 10%.
†
The RRR here is 49%
‡
For any OR, NNT is lowest when PEER = .50
§
The RRR here is 1%
**
The RRR here is 9%
SYSTEMATIC REVIEW(of Therapy) WORKSHEET: page 2 of 2
Can you apply this valid, important evidence from a systematic review in
caring for your patient?
Do these results apply to your patient?
Is your patient so different from
those in the overview that its
results can’t help you?

How great would the potential

benefit of therapy actually be for
your individual patient?

Method I: In the table on page 1, find

the intersection of the closest odds
ratio from the overview and the CER
that is closest to your patient’s
expected event rate if they received
the control treatment (PEER):
Method II: To calculate the NNT
for any OR and PEER:

___1 - {PEER x (1 -
OR)}____
NNT = (1 - PEER) x PEER x (1 -
OR)
Are your patient’s values and preferences satisfied by the regimen and its
consequences?
Do your patient and you have a
clear assessment of their values
and preferences?

Are they met by this regimen and

its consequences?

Should you believe apparent qualitative differences in the efficacy of therapy in some
subgroups of patients? Only if you can say “yes” to all of the following:
1. Do they really make biologic and clinical sense?
2. Is the qualitative difference both clinically (beneficial for some but useless or harmful for
others) and statistically significant?
3. Was this difference hypothesised before the study began (rather than the product of
dredging the data), and has it been confirmed in other, independent studies?
4. Was this one of just a few subgroup analyses carried out in this study?

Additional Notes:
DIAGNOSIS WORKSHEET: page 1 of 2

Citation:

Are the results of this diagnostic study valid?

1. Was there an independent, blind
comparison with a reference (“gold”)
standard of diagnosis?

2. Was the diagnostic test evaluated in

an appropriate spectrum of patients
(like those in whom it would be used
in practice)?

3. Was the reference standard applied

regardless of the diagnostic test
result?

Are the valid results of this diagnostic study important?

SAMPLE CALCULATIONS:
Target Disorder Totals
(iron deficiency anaemia)
Present Absent
Diagnostic Positive 731 b 270 a+b
Test Result (<65 mmol/L) a 1001
(serum ferritin) Negative 78 d 1500 c+d 1578
(>65 mmol/L) c
Totals 809 b+d 1770 a+b+c+d
a+c 2579
Sensitivity = a/(a+c) = 731/809 = 90% Specificity = d/(b+d) = 1500/1770 = 85%
Likelihood Ratio for a positive test result = LR+=sens/(1-spec)=90%/15%=6
Likelihood Ratio for a negative test result=LR-=(1-sens)/spec=10%/85%=0.12
Positive Predictive Value = a/(a+b) = 731/1001 = 73%
Negative Predictive Value = d/(c+d) = 1500/1578 = 95%
Pre-test Probability (prevalence) = (a+c)/(a+b+c+d) = 809/2579 = 32%
Pre-test-odds = prevalence/(1-prevalence) = 31%/69% = 0.45
Post-test odds = Pre-test odds x Likelihood Ratio
Post-test Probability = Post-test odds/(Post-test odds + 1)

YOUR CALCULATIONS:
Target Disorder Totals

Present Absent
Diagnostic Positive a b a+b
Test Result
Negative c d c+d

Totals a+c b+d 1770 a+b+c+d

Sensitivity = a/(a+c) = Specificity = d/(b+d) =

Likelihood Ratio for a positive test result = LR+=sens/(1-spec)=
Likelihood Ratio for a negative test result=LR-=(1-sens)/spec=
Positive Predictive Value = a/(a+b) = Negative Predictive Value = d/(c+d) =
Pre-test Probability (prevalence) = (a+c)/(a+b+c+d) =
Pre-test-odds = prevalence/(1-prevalence) =
Post-test odds = Pre-test odds x Likelihood Ratio =
Post-test Probability = Post-test odds/(Post-test odds + 1) =
DIAGNOSIS WORKSHEET: page 2 of 2

Can you apply this valid, important evidence about a diagnostic test in caring for your
patient?
Is the diagnostic test available,
affordable, accurate, and precise in your
setting?

Can you generate a clinically sensible

estimate of your patient’s pre-test
probability (from practice data, from
personal experience, from the report
itself, or from clinical speculation)

Will the resulting post-test probabilities

affect your management and help your
patient? (Could it move you across a
test-treatment threshold?; Would your
patient be a willing partner in carrying it
out?)

Would the consequences of the test help

your patient?

Additional Notes:
PROGNOSIS WORKSHEET: Page 1 of 2 Sackett June 1996

Citation:

Are the results of this prognosis study valid?

1. Was a defined, representative sample
of patients assembled at a common
(usually early) point in the course of
their disease?

2. Was patient follow-up sufficiently long

and complete?

3. Were objective outcome criteria

applied in a “blind” fashion?

4. If subgroups with different prognoses

are identified, was there adjustment
for important prognostic factors?

5. Was there validation in an

independent group (“test-set”) of
patients?
PROGNOSIS WORKSHEET: Page 2 of 2
Are the valid results of this prognosis study important?

1. How likely are the outcomes over

time?

2. How precise are the prognostic

estimates?

If you want to calculate a Confidence Interval around the measure of

Prognosis:
Clinical Measure Standard Error (SE) Typical calculation of CI
Proportion (as in the rate of some √ {p x (1-p) / n} If p = 24/60 = 0.4 (or 40%) &
prognostic event, etc) where: where p is proportion and n is n=60
number of patients
the number of patients = n SE=√{0.4 x (1-0.4) / 60} = 0.063
(or 6.3%)
the proportion of these patients
who experience the event = p 95% CI is 40% +/- 1.96 x 6.3% or
27.6% to 52.4%

√ {p x (1-p) / n} Your calculation:

n from your evidence: ________ where p is proportion and n is
number of patients SE: ____________

p from your evidence: ________ 95% CI:

Can you apply this valid, important evidence about prognosis in caring for
your patient?
1. Were the study patients similar to your
own?

2. Will this evidence make a clinically

important impact on your conclusions
about what to offer or tell your
patient?

Additional Notes:
HARM/AETIOLOGY WORKSHEET: Page 1 of 2

Citation:

Are the results of this harm study valid?

1. Were there clearly defined groups of
patients, similar in all important ways
other than exposure to the treatment or
other cause?

2. Were treatment exposures and clinical

outcomes measured the same ways in
both groups (e.g., was the assessment
of outcomes either objective (e.g.,
death) or blinded to exposure)?

3. Was the follow-up of study patients

complete and long enough?

Do the results satisfy some “diagnostic

tests for causation”?
• Is it clear that the exposure
preceded the onset of the
outcome?
• Is there a dose-response
gradient?

• Is there positive evidence from

a “dechallenge-rechallenge”
study?

• Is the association consistent

from study to study?

• Does the association make

biological sense?

Are the valid results from this harm study important?

Adverse Outcome Totals
Present Absent
(Case) (Control)
Exposed to the Yes
(Cohort) a b a+b
Treatment No c d c+d
(Cohort)
Totals a+c b+d a+b+c+d
In a randomised trial or cohort study: Relative Risk = RR = [a/(a+b)]/[c/(c+d)]
In a case-control study: Odds Ratio (or Relative Odds) = OR = ad/bc

In this study:
HARM/AETIOLOGY WORKSHEET: Page 2 of 2

Should these valid, potentially important results of a critical appraisal about a

harmful treatment change the treatment of your patient?
1. Can the study results be extrapolated
to your patient?

2. What are your patient’s risks of the

adverse outcome?
††
To calculate the NNH for any Odds Ratio
(OR) and your Patient’s Expected Event
Rate for this adverse event if they were
NOT exposed to this treatment (PEER):

NNH = PEER (OR - 1) + 1 .

PEER (OR - 1) x (1 - PEER)

3. What are your patient’s preferences,

concerns and expectations from this
treatment?

4. What alternative treatments are

available?

Additional Notes:

††
The Number of Patients you Need to treat to Harm one of them.
ECONOMIC ANALYSIS WORKSHEET: Page 1 of 2

Citation:

Are the results of this economic analysis valid?

1. Is this report really asking an
economic question:

• comparing well-defined alternative

courses of action?

• with a specified point-of-view (a

hospital, a ministry of health, or
preferably society as a whole) from
which the costs and effects are being
viewed?
• With clinically useful expressions of
the costs and consequences of the
alternative courses of clinical action?
• Effects equal, and a simple
comparison of costs: “cost-
minimisation” analysis.
• Effects unequal but measured
in the same common unit of
health: “cost-effectiveness
analysis.”
• Effects both unequal and
measured in more than one
kind of unit of health.
• Converted into
monetary units: “cost-
benefit analysis.”
• Converted into
personal preferences
or utilities (QALYs):
“cost-utility analysis.”
2. Does it cite good evidence (that would
meet the Therapy, Diagnosis, or
Overview Guides) on the
efficacy/accuracy of the alternatives?

3. Does it identify all the costs and

effects you think it should, and did it
select credible measures for them?

Are the valid results from this economic analysis important?

1. Are the resulting costs or costs/unit
of health gained impressive?

2. Are the conclusions unlikely to

change with sensible changes in
costs and outcomes?
ECONOMIC ANALYSIS WORKSHEET: Page 2 of 2:
A “league table” of costs to gain one additional quality adjusted life year (QALY):
Cost/QALY
Treatment (£ Aug. 1990)
Cholesterol testing and diet therapy (all adults aged 40-69) 220
Neurosurgical intervention for head injury 240
Advice to stop smoking from general practitioner 270
Neurosurgical intervention for subarachnoid haemorrhage 490
Antihypertensive treatment to prevent stroke (ages 45-64) 940
Pacemaker implantation 1100
Hip replacement 1180
Valve replacement for aortic stenosis 1140
Coronary artery bypass graft (left main vessel disease, severe angina) 2090
Kidney transplant 4710
Breast cancer screening 5780
Heart transplantation 7840
Cholesterol testing and treatment (incrementally) of all adults aged 25-39 14,150
Home haemodialysis 17,260
Coronary artery bypass graft (one vessel disease, moderate angina) 18,830
Continuous ambulatory periotoneal dialysis 19,870
Hospital haemodialysis 21,970
Erythropoietin treatment for anaemia in dialysis patients (assuming
10% reduction in mortality) 54,380
Neurosurgical intervention for malignant intracranial tumours 107,780
Erythropoietin treatment for anaemia in dialysis patients (assuming
no increase in survival) 126,290
adapted from: Mason J, Drummond M, Torrance G: Some guidelines on the use of cost-
effectiveness league tables. BMJ 1993;306:570-2.

Should this economic analysis be applied in your practice?

1. Do the costs in it apply in your own
setting?

2. Are the treatments likely to be as

effective in your setting?

3. Is it worth it?
• If a cost-minimisation analysis, is the
difference in costs big enough to
warrant switching over to the cheaper
one?

• If a cost-effectiveness analysis, is the

difference in effectiveness great
enough for you to want to spend the
difference?

• If a cost-utility analysis, where does it

lie in your local, current league table?

Additional Comments:
DECISION ANALYSIS WORKSHEET: Page 1 of 1

Citation:
Are the results of this clinical decision analysis valid?

1. Were all the important clinical

strategies and outcomes included?

2. Are the probabilities credible? (Was

an explicit and sensible process used
to identify, select, and combine the
best external evidence into
probabilities?)

3. Are the utilities credible? (Were the

utilities obtained in an explicit and
sensible way from credible sources?)

4. Was the robustness of the conclusion

tested? (Was the impact of clinically
sensible differences in probabilities
and utilities determined?)

Are the valid results from this decision analysis important?

1. Did one course of action lead to
clinically important gains in life-
expectancy or other utility measure?

2. Was the same course of action

preferred despite clinically sensible
changes in probabilities and utilities?

Should this decision analysis be applied in your practice?

Do the probabilities apply to your
patient?

If not, can you adjust them

appropriately?

Can your patient state their utilities in a

usable and stable form?

Additional Notes:
GUIDELINES WORKSHEET: Page 1 of 2

Citation:

Are the recommendations in this guideline valid?

1. Were all important decision options
and outcomes clearly specified?

2. Was the evidence relevant to each

decision option identified, validated,
and combined in a sensible and
explicit way?

3. Are the relative preferences that key

stakeholders attach to the outcomes
of decisions (including benefits, risks
and costs) identified and explicitly
considered?

4. Is the guideline resistant to clinically

sensible variations in practice?

Is this valid guideline or strategy potentially useful?

1. Does this guideline offer an
opportunity for significant
improvement in the quality of health
care practice?

• Is there a large variation in current

practice?

• Does the guideline contain new

evidence (or old evidence not yet
acted upon) that could have an
important impact on management?

• Would the guideline affect the

management of so many people, or
concern individuals at such high risk, or
involve such high costs that even small
changes in practice could have major
impacts on health outcomes or resources
(including opportunity costs)?
GUIDELINES WORKSHEET: Page 2 of 2:

Should this guideline or strategy be applied in your practice?

1. What barriers exist to its
implementation?

Can they be overcome?

2. Can you enlist the collaboration of key

colleagues?

3. Can you meet the educational,

administrative, and economic
conditions that are likely to determine
the success or failure of implementing
the strategy?
• credible synthesis of the evidence
by a respected body

• respected, influential local

exemplars already implementing
the strategy

• consistent information from all

relevant sources

• opportunity for individual

discussions about the strategy
with an authority
• user-friendly format for guidelines

• implementable within target group

of clinicians (without the need for
extensive outside collaboration)
• freedom from conflict with
economic incentives,
administrative incentives, patient
expectations, and community
expectations.

Additional Comments: