www.cebm.
net
Study design and Levels of Evidence
Dr Rafael Perera
Director of Research Methods
Research questions
www.cebm.net
determine problem
(case reports/case-series, ecological/cross-sectional studies)
implement activities find potential causes
(case-control/cohort studies)
(surveillance studies, meta-analyses)
design interventions
(randomized controlled trials)
Taxonomy of trial design
www.cebm.net
Hierarchy of Evidence www.cebm.net
Levels of Evidence
www.cebm.net
Level Therapy Prognosis Diagnosis Aetiology
I Systematic Systematic Systematic Systematic
Review Review Review Review
II RCT Inception cohort Cross-sectional Prospective
(consecutive) cohort
III Non-Randomised Untreated control Cross-sectional Retrospective
experimental patients in RCT (non-consecutive) cohort
Comparative with Retrospective Diagnostic case- Case-control /
concurrent cohort control Ecological
control group
IV Case series Cohort patients Case series Cross-sectional
different disease
Case Report Series
www.cebm.net
Describe patients’ characteristics, and may generate ideas for future studies
Randomized Controlled Trial
www.cebm.net
Population
Meet
Inclusion
Criteria?
Sample
Baseline Follow-
Follow-up
Assessment assessments
•Typical RCT randomises two (or more) groups of patients to different treatments
Observational Studies
www.cebm.net
• Ecological
• Cross-sectional
• Case-control
• Cohort or “follow-up” studies
Ecological study
www.cebm.net
• Focuses on the characteristics of population groups
rather than their individual members.
• The group could be defined by
• time (calendar period, birth cohort),
• geography (country, city),
• socio-demographic characteristics (ethnicity, religion).
• Used to examine the differential distribution of diseases
among people with different risk profiles.
• The kinds of comparisons usually take advantage of routinely collected
data and are therefore inexpensive
Cross-sectional study
www.cebm.net
Population
Compute prevalence
Group of interest
of stroke in smokers
(e.g. smokers)
Compare
sample groups
Comparison Group
of interest Compute prevalence
(e.g. non-smokers) of stroke in non-smokers
Starting point
Present
•A cross-sectional study is a single “snapshot” in time
•We can only study current risk factors and diseases (prevalence)
Case-control study
www.cebm.net
Population of diseased
individuals
Sample of diseased
individuals
Sample of non-
non-diseased
individuals
Population of non-
non-diseased
individuals
Past Exposure
History Starting point
•Case-control studies examine the association of disease with past exposure (s)
Cohort study
Population of disease
www.cebm.net
free individuals
Sample
Starting point Future
• Selected group of disease-free people who are classified according to a specific
exposure.
•Observed over time to see who develops the disease or outcomes (s) of interest.
•Can measure incidence (new cases of disease) and thus risk
Principles of Evidence Grading
www.cebm.net
1. Different research evidence for different types of clinical
questions
2. Top level of evidence – Systematic Reviews (Irrespective of type
of clinical question)
3. When SR not available, efficient strategies required to identifying
relevant evidence
4. Study “Level” alone should not be used to grade evidence
5. Balanced assessments use different types of research
Bias in RCTs www.cebm.net
Effect of study features on effect size in 229 trials
Odds Ratio P-value
Allocation 0.70 <0.001
concealment
Random # 0.95 0.58
Inappropriate 1.07 0.32
Exclusions
Double-blind 0.83 0.01
Bias in Diagnostic Studies
www.cebm.net
case-control 3.0 (2.0-4.5)
different reference tests 2.2 (1.5-3.3)
partial verification 1.0 (0.8-1.3)
not blinded 1.3 (1.0-1.9)
non-consecutive 0.9 (0.7-1.1)
retrospective 1.0 (0.7-1.4)
no description test 1.7 (1.1-2.5)
no description population 1.4 (1.1-1.7)
no description reference 0.7 (0.6-0.9)
0 1 2 3 4
Lijmer JG et al. JAMA 1999;282:1062-1067 Relative Diagnostic Odds Ratio
