Lecture 1:
What is critical appraisal?
“Critical appraisal is the process of systematically examining research
evidence to assess its validity, results, and relevance before using it to
inform a decision”
Why critically appraise research evidence?
It is crucial to critically evaluate research evidence to facilitate evidence-
based practice, that is, the use of the best available evidence to guide
decision-making and program design.
The term “best evidence” emphasizes that it is the quality and not the
quantity of evidence that is of primary importance.
Critical appraisal distinguishes the best available evidence from a large
body of research.
Using a best-evidence approach allows you to:
• Retrieve reliable and up-to-date information on interventions that work or
do not for a particular public health topic;
• Control the amount of literature you will need to analyze; And
• Be confident that public health decision-making is based on the “best of
the best” information available on a particular topic.
Is this article relevant to my issue and setting?
1 Read the abstract
Use the information found in the abstract to answer the questions below:
• Are your issues discussed there?
• What are the main findings of the research?
• Do you want to know more after reading the abstract?
• Was the research done in a similar setting to yours?
• Does it address a related question? (Even research that covers your
issue indirectly can be useful.)
• Are there reasons to doubt the findings without reading the whole
article?
You may conclude that the study is not reliable merely by reading the
abstract. If this is the case, move on to another article!
2 Read the Introduction and Discussion sections
To further assess the relevance of a study, you will look to those
sections of the research article which describe in more detail the
objectives and context of that study.
The Introduction and Discussion sections will help to identify the key
concepts, goals, subjects, and themes of the research.
3 Consult the Methods section
The Methods section will give a step-by-step description of exactly
how the study was carried out. In this section you should take note of:
• Where the study was done (on site, at home, administrative setting,
etc.);
• From whom the data was collected (primary from staff, patients,
families; or secondary from databases);
• How the data was collected (interviews, focus groups,
questionnaires, surveys, observations, etc.).
If you have found this article to be useful and wish to continue
evaluating it, move on to the next step. Otherwise, find another one!
What are the author’s conclusions?
1. Compare the abstract to the Discussion section
2. Compare the raw data contained in tables with the results analyzed in
the results analyzed in the Discussion and Conclusions sections
3. How well are the results related to other research on the same topic?
How can the results be applied to public health practice?
The following questions will guide your assessment of the applicability of a
study’s results:
• How can the results be interpreted and applied to public health? Can I
apply it to my program/policy?
• Were all important public health outcomes considered?
• Are the benefits worth the costs and potential risks?
Key Questions for Critical Appraisal of Public Health Research
1) Is the article relevant to your topic?
Are the issues discussed in the abstract of interest to you?
Was the research done in a similar setting to yours?
2) What are the results?
Were the results similar from study to study?
What are the overall results of the study?
How precise are the results?
Can a causal association be inferred from the available data?
3) Are the results valid?
Did the review explicitly address the public health question? Was the
search for relevant studies detailed and exhaustive? Were the primary
studies of high methodological quality?
4) Can the results be applied to public health practice?
Can I apply the results to my program/policy?
Are all important public health outcomes considered? Are the
benefits worth the costs and potential risks?
Lecture 2: types of study designs
- There are various types of scientific studies. The choice of study
depends on the research question
The higher in the triangle the more accurate on evidence bassed
Hierarchy of study designs:
1. Meta-analyses and systematic reviews (review of completed
studies)
2. Randomized controlled trials (experimental with randomization)
3. Quasi-experimental (experimental without randomization)
4. Cohort studies (observational)
5. Case–control studies (observational)
6. Case series (observational) 7. Case reports (observational) 8.
Ideas, editorials, opinions
9. Animal research
10. In vitro (test tube) research
Systematic review:
→ summary of the clinical literature
includes a description of the findings of the collection of research studies.
The systematic review may also include a quantitative pooling of data,
called a meta-analysis.
Meta-Analysis
A way of combining data from many different research studies.
A meta-analysis is a statistical process that combines the findings from
individual studies.
Randomized Controlled Trial
A controlled clinical trial that randomly (by chance) assigns participants to
two or more groups.
One (the experimental group) receiving the intervention that is being tested,
and the other (the comparison group or control) receiving an alternative
(conventional) treatment.
Cohort Study (Observational Study)
A clinical research study in which people who have a certain condition or
receive a particular treatment are followed over time and compared with
another group of people who are not affected by the condition.
a) Prospective cohort studies: studies carried out from the present time
into the future, can be tailored to collect specific exposure data.
b) Retrospective or historical cohort studies: look at medical events
from some time-point in the past up to the present time.
Case-control Study
Case-control studies begin with the outcomes and do not follow people
over time.
Studies which start with the identification of persons with a disease of
interest and a control (comparison, referent) group without the disease.
Cross-sectional study
The observation of a defined population at a single point in time or time
interval.
Case Reports and Series
A report on a series of patients with an outcome of interest. No control
group is involved.
Lecture 3: evidence-based decision making
Evidence:
Evidence is considered the synthesis of all valid research that answers a
specific question and that, in most cases, distinguishes it from a single
research study.
Evidence-based medicine:
The integration of the best research evidence with our clinical expertise and
our patient’s unique values and circumstances.
Evidence-based dentistry:
An approach to oral health care that requires the judicious integration of
systematic assessments of clinically relevant scientific evidence, relating
to the patient’s oral and medical condition and history, with the dentist’s
clinical expertise and the patient’s treatment needs and preferences.
EBDM vs traditional decision making
EBDM is a structured process that incorporates a formal set of rules for
interpreting the results of clinical research and places a lower value on
authority or custom.
In contrast to EBDM, traditional decision making relies more on intuition
(anticipation), unsystematic clinical experience, and pathophysiologic
rationale
EBDM:
PICO:
The first step in evidence-based decision making is asking the right
question.
Dissecting the question you want to ask into its components:
problem or population (P),
intervention (I),
comparison group (C) and
outcomes (O)
The formality of using PICO to frame the question serves two key purposes,
as follows:
1. forces the clinician to focus on what he and the patient believe to be the
most important single issue and outcome.
2. facilitates the next step in the process, the computerized search, by
identifying key terms that will be used in the search.
Skills and Abilities Needed to Apply an Evidence-Based Decision-
Making Process:
To translate the EBDM process into action is based on the abilities and
skills, we need to:
1. Convert information needs and problems into clinical questions so
that they can be answered.
2. Conduct a computerized search with maximum efficiency for
finding the best external evidence with which to answer the question.
3. Critically appraise the evidence for its validity and usefulness
(clinical applicability).
4. Apply the results of the appraisal, or evidence, in clinical practice.
5. Evaluate the process and your performance.
Appraising the Evidence
After identifying the evidence gathered to answer a question, it is important
to have the skills to understand the evidence found.
International evidence-based groups have made this easier by developing
appraisal forms and checklists that guide the user through a structured
series of “YES/NO” questions to determine the validity of the individual
study or SR.
WEBSITES: CONSORT, PRISMA, CASP
Applying the Evidence: Evidence-Based Dentistry in Action
In the treatment itself!
Patient case scenario..
Evaluating the outcomes
The final steps in the EBDM process are to evaluate the effectiveness of the
intervention and clinical outcomes and to determine how effectively the
EBDM process was applied.
For example, one question to ask in evaluating the effectiveness of the
intervention is, “Did the selected intervention or treatment achieve the
desired results?
Lecture 4: Critical appraisal of article contents
The Critical Appraisal starts by evaluating the main sections.
Peer-review process:
Peer review is the system used to assess the quality of a manuscript before
it is published.
How the peer-review process work?
When a manuscript is submitted to a journal, it is assessed to see if it
meets the criteria for submission.
If it does, the editorial team will select potential peer reviewers within the
field of research to peer-review the manuscript and make
recommendations.
Types of peer-review:
Single-blind: the reviewers know the names of the authors, but the authors
do not know who reviewed their manuscript unless the reviewer chooses to
sign their report.
Double-blind: the reviewers do not know the names of the authors, and the
authors do not know who reviewed their manuscript.
Open peer: authors know who the reviewers are, and the reviewers know
who the authors are. If the manuscript is accepted, the named reviewer
reports are published alongside the article and the authors’ response to the
reviewer.
Transparent peer: the reviewers know the names of the authors, but the
authors do not know who reviewed their manuscript unless the reviewer
chooses to sign their report. If the manuscript is accepted, the anonymous
reviewer reports are published alongside the article and the authors’
response to the reviewer
I. Overview of the paper:
a. The publishing journal and the year
b. The article title: Does it state key trial objectives?
c. The author (s) and their institution (s)
The presence of a peer-review process in journal acceptance protocols also
adds academic validity to the assessment criteria for research papers and
hence would indicate a reduced chance of publication of poor quality
research.
II The abstract
Reading the abstract is a quick way of getting to know the article and its
purpose, major procedures and methods, main findings, and conclusions.
III. Introduction section
Introductions generally include background information about the topic
being studied, and describe other relevant research.
IV. The methods section
Describes how the researchers carried out the research.
A good methods section gives the reader enough detail that they could
repeat the study themselves in a way that is identical to the original in every
relevant way, to see if they produce similar results.
V. The results section
reports the results of the study.
RUBRIC
is typically an evaluation tool or set of guidelines used to promote the
consistent application of learning expectations, learning objectives, or
learning standards in the classroom, or to measure their attainment against
a consistent set of criteria.
Rubric for research manuscript
Rubric for research summaries
Rubric for journal articles
Rubric for review articles
Lecture 6: critical appraisal of article contents III
To determine if a study’s findings are trustworthy, you will review the
Methods section.
There are five factors that influence the reliability of research results:
1. Completeness of the methods analyzed
2. Quality and relevance of the measures
3. Quality of the data
4. Ability to control for differences between groups being compared
5. Appropriateness of the statistical methods given the nature of the
data generated
Discussion section
Authors discuss meaning of the results
Comparison between Abstract and Discussion sections:
The discussion section is more detailed and precise than the abstract, and
will explain the limitations of the research and possible implications which
are not mentioned in the abstract.
In the abstract, a short and precise conclusion is only mentioned.
Lecture 7: Journal impact factor
Impact factor
measure applied to a journal and is a measure of the frequency with which
the 'average article' has been cited in a particular year or period.
Impact factor can be calculated after completing the minimum of 3 years of
publication; for that reason journal IF cannot be calculated for new
journals.
The journal with the highest IF is the one that published the most commonly
cited articles over a 2-year period
When assessing the usefulness of a journal, the Journal Impact Factor
alone is not the only criteria to consider. The Journal Impact Factor should
be used with informed peer review.
Care in the use of impact factors
Scientific journals generally rank higher than clinical journals: Due to the
fact that scientific papers tend to cite only scientific and not clinical
articles, whereas clinical papers tend to cite both scientific and clinical
articles.
Self-citation is also possible, increasing the IF: So authors happily cite
previous papers in the same journal and editors may cite editorials. Some
journals have been known to try to manipulate the IF by writing to authors
asking them to add references to articles published in that journal.
Errors, misprints and inconsistencies in citations can distort the IF
IFs are biased toward journals that mainly publish review articles
Greater availability tends to raise the IF
Controversial or poor papers may increase the IF
Predatory journals
If a journal charges publication fees, actively solicits research manuscripts
and does not provide trustworthy peer review or quality editorial services.
Fake impact factor
An impact factor is a rating of how important a journal is, and is used by
researchers to choose the right journal for their manuscript. However, there
are other metrics that are deemed fake if they are designed to mislead the
researcher and the reader.
Lecture 8: misconduct in research
Fees
Most submission and publication fees can be grouped into two categories:
- pre-publication and post-acceptance fees
What are publishing charges?
These fees can vary widely depending on the journal, publisher, article, and
access type. For example, some journals charge a submission or
membership fee before reviewing the manuscript, while others charge an
APC (Article Processing Fees) or a page fee after accepting it.
The most common publishing charge is the APC, which is usually requested
to make an article open access (OA).
Benefits of OA:
Increasing the visibility, impact, and citation of the publication, and
facilitating collaboration and knowledge exchange.
FEES in research publications...
• Pre-publication fees
=> Submission fees.
=> Membership fees.
• Post-acceptance fees
=> Article processing charges (APCs).
=> Page/colour printing charges.
Article processing charges (APCs)
These fees are also known as author publishing charges or publication fees
and are most commonly charged by open access journals.
Types of misconducts
1. Data Fabrication/Data Falsification
Data Fabrication → the intentional misrepresentation of research results.
Data Falsification→ the manipulation of research resources, equipment, or
processes, including omitting and presenting facts, with the objective of
giving a false impression.
Image manipulation
2. Duplicate Submission/Publication and Redundant Publication
Duplicate submission/publication → Submission of a single article/study in
two different journals or publication of an almost similar study in two
journals refers to duplicate submission.
Redundant publication → publication of the same data more than once,
resulting in a rejection or a request to merge the submitted manuscript with
another one and the rectification of published manuscripts.
3. Duplication of Text and/or Figures (Plagiarism)
refers to presenting (or copying) someone’s work, including words, ideas, or
information, as own without proper citation or acknowledgment.
4. Authorship Issues
is important in assigning credibility to work submitted or published.
5. Undeclared Conflict of Interest
situation in which there is a potential for financial or other personal
considerations from authors or reviewers to compromise or present bias in
professional judgment and objectivity.
6. Suspected Manipulation of Peer Review/Bias of Peer Review
Journals are considering carefulness to avoid any conflict of interest
between the reviewer and the authors
7. Manipulation of Citations
Self-Citation: practice of including one’s own work in the cited references of
an article to increase the number of citations attributed to the same author.
=> When the manipulation in citation is considered as a real problem?
Citation manipulation is a problem when references do not contribute to
the scholarly content of the article and are included solely to increase
citations.
8. Violation of Research Ethics
=> Research ethics can include a number of issues, such as:
• Patient consent (medical research)
• Animal experimentation (life sciences)
• Recipient consent (market research)
• User consent (social/online research)
• Studies involving humans or animals must include informed consent on
human subjects, and standard protocols must be followed for the
experiments on animals.
Lecture 9: Misconduct 2
Recommended actions to several types of research misconducts:
Before publication vs after publication
1.Data Fabrication/Data Falsification
the publisher will communicate the complaint to the corresponding
author and ask him/her to provide justifications.
2. Duplicate Submission/Publication and Redundant Publication
By simple comparison
3. Duplication of Text and/or Figures
If the complaint is related to plagiarism, the publications team will
compare the referred text to that of the manuscript on a word-to-word
basis.
4. Authorship Issues
Authors must provide a final list of authors at the time of submission,
ensuring the correct sequence of the names of authors, which
considered for any addition, deletion, or rearrangement after the final
submission of the manuscript.
If a change is essentially required, it can only be done on an Editor’s
approval, for which the Editor-in-chief must receive the following from the
corresponding author:
- The reason
- Confirmation from all the co-authors
5. Undeclared Conflict of Interest
The complainant is made aware that CoI matters cannot be investigated
unless the journal Editor-in-chief informs the corresponding (or
complained-about) author and possibly the institution or company where
the research took place.
6. Suspected Manipulation of Peer Review/Bias of Peer Review
In the beginning, reviewers need to declare and mention any conflict of
interest that could affect the standard of the peer review and could result in
a biased decision.
7. Manipulation of Citations
Some journals have centralized system for identifying excessive self-
citations when such a case arises.
8. Violation of Research Ethics
In the communication to the author in question, the journal’s Editor-in-chief
is requested to indicate whether the matter is likely to be referred to the
institution or company where the research took place,
Lecture 10:
Retraction types..
1. Data Fabrication/Data Falsification
2. Duplicate Submission/Publication and Redundant Publication 3.
Duplication of Text and/or Figures (Plagiarism)
4. Authorship Issues
5. Undeclared Conflict of Interest
6. Suspected Manipulation of Peer Review/Bias of Peer Review
7. Manipulation of Citations
8. Violation of Research Ethics
Recommendation to avoid misconduct in research publications:
=> Training of researchers/authors.
=> Careful use of citations.
=> Contribution of authors
=> avoid plagiarism by early use of detection tools
To avoid retraction
Author have the opportunity of scientific corrections
