I - T E C H T E C H N I C A L I M P L E M E N T A T I O N G U I D E # 2

Guidelines for
Pre- and Post-Testing
I-TECH’s Technical Implementation Guides are a series of practical and instructional papers designed
to support staff and partners in their efforts to create and maintain quality programs worldwide.

Pre- and post-tests are used to measure knowl- skills, e.g., a training on effective counseling skills,
edge gained from participating in a training course. may be difficult to create and score. An alternate
The pre-test is a set of questions given to partici- method of evaluating learning, such as the obser-
pants before the training begins in order to deter- vation of skills demonstrated in a role play, might
mine their knowledge level of the course content. be more appropriate in this situation.
After the completion of the course, participants are
given a post-test to answer the same set of ques- When deciding whether or not to take the time to
tions, or a set of questions of comparable difficulty. do both a pre- and a post-test, consider first what
Comparing participants’ post-test scores to their you most want to learn about your training. If you
pre-test scores enables you to see whether the want to understand exactly what knowledge can
training was successful in increasing participant be credited to the training itself, using a pre- and
knowledge of the training content. post-test methodology is important. If, instead, you
only need to know whether participants can dem-
This guide will provide you with information on onstrate content knowledge or skills by the end of
when and how to use pre- and post-tests, tips for the training, a pre-test is not necessary. By admin-
developing good questions, instructions for how to istering a test only at the end of training, you will
validate and administer pre- and post-tests, and a not be able to show knowledge gains, but you will
description of how to analyze results. be able to assess the extent to which participants
understand the course material.

Using Pre- and Post-Tests Remember, one of the limitations of any test of
knowledge administered immediately after training
Pre- and post-test scores provide information on is that it will not tell you what people will remember
whether or not participants have learned from the one week or one year after the training, nor wheth-
training. In addition, a well-designed pre- and post- er they will apply what they learned in their work.
test can help trainers understand which concepts
or competencies were well taught during the train-
ing and which ones need additional time, or need Developing a Pre- and Post-Test
to be covered using alternative methods.
Tests are instruments or tools used to measure
Pre- and post-tests may not be the best tools to change. If the instrument itself is faulty, it cannot
use for every training. For example, if a course is accurately measure changes in knowledge. A valid
highly interactive with ample opportunity to as- and reliable pre- and post-test must be made up of
sess participants’ knowledge and comfort with the well-written and clear questions. The following are
training content, then pre- and post-tests may not some tips for creating good questions:
be worth the time to administer and analyze. Very
short courses, lasting a day or so, may not merit the Create questions that focus on the primary
time requirements of administering and analyzing course objectives. Try to develop at least one
pre- and post-test data. In addition, pre- and post- question for each course objective. This will en-
tests for trainings focused on relationship-building sure that you are asking participants to demon-
GUIDELINES FOR PRE- AND POST-TESTING: A TECHNICAL IMPLEMENTATION GUIDE PAGE 2

strate their knowledge of what course developers ended, true/false, multiple-choice, and case-based
determined are the most important concepts to — that can be used in your test. Having a variety of
learn across the entire course. You can go one step knowledge questions can help make a test more
further by asking yourself, “What are the 10 most rigorous and interesting. Open-ended questions re-
important things in this course that a physician quire participants to use their own words to respond
— or other health professional — needs to know to a question or comment on a particular scenario.
about HIV care?” Then create your questions from Multiple-choice questions ask respondents to select
this list of 10 concepts, facts, or skills. For a work- among several possible answers. True/false ques-
shop or training on a highly technical subject, such tions ask respondents to consider only two possible
as antiretrovirals (ARVs) or opportunistic infections choices. You can ask respondents to demonstrate
(OIs), a content expert generally needs to develop more specific, detail-oriented learning with multiple-
the test in order to ensure that incorrect options choice questions than with true/false questions.
are plausible, that the right content is covered, etc.
If you are designing a test for people who do not
Do not create questions that demand the memori- speak English as a first language, avoid changing
zation of extraneous (i.e., picky) detail. Participants your question type too many times throughout the
should not be tested on whether they remember a course of the test. Participants may get confused
particular word or phrase, or whether they remem- trying to interpret new directions, which distracts
ber if prevalence rates were 13% or 15%, but rather from the purpose of the test.
on whether they have learned important concepts Creating Open-Ended Questions
and related facts. Create questions that are specific enough so
that participants clearly understand the scope
Only include questions to which there were of the question. Example: “Explain how HIV-in-
clear answers provided during the course. Do fected children are likely to suffer.” This question is
not test participants on concepts or knowledge overly broad. Participants might wonder whether
that were not sufficiently covered in the course. If “suffering” refers to abandonment, physical weak-
there are important concepts you think should be ness, or other complications, or a multitude of other
covered in the course but weren’t, integrate this in- types of suffering. “List 3 reasons why HIV- infect-
formation into your own evaluation of the training ed children are more likely to suffer from failure to
and recommend it be included the next time the thrive.” This question specifically asks participants
course is taught. to address “failure to thrive,” a clinical condition
well-covered in the training session.
Develop a test that will take between 10-25 min-
utes to complete. The amount of time spent on Creating True/False Questions
pre- and post-tests should vary depending on the Construct questions that are simply worded, to
length of the overall training course and the type of the point, and unambiguous. Simple sentences
questions asked. It is reasonable that a test cover- are straightforward and have fewer words than
ing a week-long training would be longer than a test more complex, multiphrase sentences. Vocabulary
covering a two-day training. Open-ended essay or that can be interpreted in different ways makes it
short-answer questions will tend to take longer much more difficult for respondents to answer.
than multiple-choice tests with the same number
of questions. Example: “There are many ways a person can be-
come infected with HIV” uses a word (many) that
can be interpreted in perhaps ten different ways.
Tips for Creating Questions A better question would be one that focused on a
single mode of transmission: “An individual can be-
There are a variety of question types — open-
come infected with HIV through a needlestick.”
GUIDELINES FOR PRE- AND POST-TESTING: A TECHNICAL IMPLEMENTATION GUIDE PAGE 3

Stay away from conjunctions such as “and,” ous for most English speakers. Among those who
“but,” “except,” and “or.” These words imply a speak English as a second or third language, it may
second idea or concept and can be confusing when be merely a spelling or vocabulary test.
respondents are answering True/False questions.
Make the multiple-choice question text longer
Example: The true/false question stem, “HIV can than the text of the answers. The majority of infor-
be transmitted during intercourse but only if the in- mation should be in the question, not the answers.
dividuals are not using a condom,” is problematic. Participants should not be overwhelmed with words
Although the question appears to be true, HIV can when attempting to answer the question correctly.
be transmitted even if individuals are using a con-
dom during intercourse. The “but” provides too Review your questions and answers for usability.
much potential for ambiguity and room for confu- Cover up the answers and look at the question. Some-
sion in the respondent. one knowledgeable of the course content should be
able to answer the question without looking at the
Creating Multiple-Choice Questions answers. If possible, ask a second colleague knowl-
Develop responses that are substantively dis- edgeable of the course content to take the test and
tinct from one another. Answers in a multiple- see how he/she answers the questions. If they have
choice question that are too similar do not provide problems answering, then chances are that your ques-
a respondent with a clear choice. Such questions tions are not specific enough.
can end up testing their ability to make distinctions
in spelling or definition instead of making important Combining Methods:
discerning choices among crucial concepts in HIV. Creating Case-Based Questions
A case-based question presents a short case study
Example: Which of the following is the name for and then poses questions based on the information
the ARV drug abbreviated as ABC? available in the case study. Case-based questions
Abacab require participants to apply discrete facts or bits of
Abacavan knowledge to a situation they will likely encounter
Abacavir in their workplace.
Abracadabra
Case-based questions can be true/false, multiple
Although it might be important for participants in choice, or short answer. Short-answer questions will
an ARV course to know that Abacavir is the name provide you with a better sense of the participants’
for ABC, these responses are more about how well knowledge and ability to apply concepts learned in
they can distinguish slight variations in spelling. A the training course. The drawback to short-answer
better selection of responses would be: questions is that they take more time to grade.
Abacavir
Amprenavir
Acyclovir
Amphotericin B

Develop “incorrect” responses that are poten-

tially plausible but clearly wrong. Even your most
knowledgeable learners should not find the correct
answer extremely obvious; respondents should be
presented with a selection of answers that they must
consider carefully. In the example above, the correct
response in the first list of responses was too obvi-
Pre- and post-tests should take between 10-25 minutes to complete.
GUIDELINES FOR PRE- AND POST-TESTING: A TECHNICAL IMPLEMENTATION GUIDE PAGE 4

Example of a Short Answer Case-Based Question

Mrs. S. is a 34 year-old Ethiopian woman who was referred from the VCT center after testing HIV positive
three weeks ago. Her husband recently died from cryptococcal meningitis; during his hospitalization he
tested positive for HIV, causing her to pursue testing.

Four months ago, Mrs. S. was diagnosed with tuberculosis. Upon her diagnosis, she began a four-drug
treatment for TB, and is currently taking isoniazid and ethambutol. HIV testing was recommended and
refused at the time of TB diagnosis. Other than TB treatment, she is taking no medications, and until
her TB diagnosis her medical history had been normal. She is allergic to trimethoprim/sulfamethoxazole
(Bactrim) which causes a diffuse erythematous maculopapular rash.

On physical exam, she is a thin, well-developed woman in no acute distress. A head, eye, ear, nose, and
throat (HEENT) exam reveals candida on the tongue and buccal mucosa. Heart, lung, and abdomen exam
are normal. Laboratory test data show:

Hemoglobin 11.7 ALT 39

White blood cell 4.8 AST 47
Neutrophils 74%
Lymphocytes 20%
Monocytes 6%
Platelets 167,000 HIV Reactive

1. For this patient, what evidence (history, physical examination, and/or laboratory data) supports begin-
ning antiretroviral therapy (ART)?

2. Do you identify any barriers to good ARV adherence? If so, what would you recommend to address
the barrier(s)?
3. What antiretroviral medication(s) would you initiate?

4. What toxicities could be associated with the antiretroviral medication(s) you chose?

Case questions have a case stem, a lead-in ques-

tion, and possible responses (or a space for a short
answer reply). The case should not give away the
answer, nor should the answer be obvious among
the possible options.

Pre- and post-tests may not be the best tools to use for every training, such
as a one-day short course, or a training on skills that need to be observed.
GUIDELINES FOR PRE- AND POST-TESTING: A TECHNICAL IMPLEMENTATION GUIDE PAGE 5

Examples of Case-Based Multiple-Choice Test Questions

Example 1: A 60-year-old man presents with progressive weakness of arms and legs. He reports
difficulty climbing stairs or combing his hair. He also has difficulty swallowing, but he has no vi-
sual complaints. On physical examination, you note a maculopapular eruption on the eyelids, nose,
cheeks, and knuckles. Joint examination is normal. What is the most likely diagnosis?1
Dermatomyositis
Myasthenia gravis
Polymyalgia rheumatica
Rheumatoid arthritis
Systemic lupus erythematosus

Example 2: In this example, a case is presented and is followed by several multiple-choice questions.
Case Study: John
John is a recently diagnosed HIV-positive male who came to clinic for his initial visit. His initial labo-
ratory studies show the following: Hgb of 9, CD4 count of 199. On exam you notice white plaque
on his tongue and the roof of his mouth. He has a history of tuberculosis 5 years ago, which was
effectively treated. Otherwise, he seems in good health.

The following questions have only ONE right answer. Circle the letter of each correct answer.

1. At what WHO stage of HIV disease is John?

a. Stage 4
b. Stage 3
c. Stage 2
d. Stage 1

2. Is John eligible for antiretroviral medication?

a. Yes
b. No

3. What is the likely diagnosis of the white plaque on John’s tongue?

a. Oral hairy leukoplakia
b. Kaposi’s sarcoma
c. Oral candidiasis (thrush)
d. Lichen sclerosis

Validating Pre- and Post-Tests

All pre- and post-tests must be validated before it was intended because the question was poorly
they are considered a reliable data collection tool. written and had more than one correct answer, or
If participants get a question wrong, it should be because the question addressed content that was
because of lack of knowledge, not because a par- not taught in the course. When a participant gets
ticipant interpreted the question differently than a question correct, it should be a result of knowl-
GUIDELINES FOR PRE- AND POST-TESTING: A TECHNICAL IMPLEMENTATION GUIDE PAGE 6

edge in that subject area, not because the incorrect tests, use numbers instead of names as identifiers
answers were so implausible that it was easy to on the pre- and post-tests. Below are instructions
guess the correct answer. for administering anonymous pre- and post-tests.

As the first step in the validation process, ask four lo- Assign random numbers to each participant.
cal staff to take the test. Ask them to mark any ques- Write numbers from 1 to n (where n equals the
tions that were unclear to them when they were tak- number of participants in the training) on pieces of
ing the test. Have staff discuss with you their answer paper. Ask participants to choose one piece of pa-
to the questions, ensuring that their understanding per and to copy the number they drew on the back
of the test questions was the same as what was of the first page of their participant packet handed
intended. Although staff may not be representative out at the training.
of the participants who will take the test, this is a
good first step for clarifying questions and responses Administer the pre-test before beginning the
before you give the test to a group of training par- training. Hand out pre-tests to participants and ask
ticipants. Make adjustments to the questions based them to note their numbers on the part of the test
on the results of this “internal pre-testing” with staff marked “code number.” Explain to participants that
before administering to training participants. their responses to the test are anonymous — we
don’t know what number they drew. Explain that the
The most important step of the validation process purpose of the pre/post test is to evaluate our train-
takes place with the participants themselves. After ing and to help us answer the question, “Did people
administering the post-test to training participants, learn what we wanted them to from the training?”
review the answers as a group. Ask participants
to explain their answers to the questions to better Ask participants to answer the pre-test questions.
understand how they were interpreting the ques- They can ask for clarification for any of the ques-
tions.It should be clear from the discussion which tions that they do not understand, for example, if
questions were confusing to participants and which there is something in the wording of the question
ones were clearly written. For questions answered that does not make sense to them or is ambiguous.
incorrectly, the discussion should help to deter- A designated trainer should note all such questions
mine whether the question was confusing and or clarifications on a master copy so the test can
participants actually understand the concept being be improved for the next round of training. When
tested, or whether the participants did not acquire participants have finished, collect the pre-tests and
the intended knowledge for some reason. Rewrite give them to the training team lead.
the test based on feedback and administer the test
to another set of participants to make sure adjust- Administer the post-test at the end of the train-
ments clarified any confusing questions. ing. Hand out post-tests and ask participants to write
their code numbers on the top. Remind participants
that their codes are unknown to the trainers, but it
Implementing a Pre- and Post-Test is important they use their same unique code on
both the pre- and post-tests for trainers to assess
Once you have validated your test, you are ready the strengths/weaknesses of the training. Ask par-
to begin using it to collect data about your train- ticipants to answer the post-test questions. Again,
ing course. The most effective way to use pre- and people should feel free to ask for clarification on lan-
post-tests is to match each participant’s pre-test and guage or interpretation. The designated trainer should
post-test answers so you can look at an individual’s take note of all such questions on a master copy.
change in knowledge as well as the knowledge
change of the group as a whole. Since participants When participants are finished, collect post-tests and
may be uncomfortable putting their name on their give them to the team lead. Discuss the questions
GUIDELINES FOR PRE- AND POST-TESTING: A TECHNICAL IMPLEMENTATION GUIDE PAGE 7

and answers to the test with the participants. For tidisciplinary training, did nurses’ scores increase
both teaching and evaluation purposes, it is impor- more than doctors’? Did each individual’s score
tant to go over the correct answers to the post-test increase? Did the overall range of scores change
with participants while you are in class. Reviewing for the group between pre- and post-testing? Think
the answers as a group provides immediate feed- about the knowledge level of the audience your
back to participants on concepts they did not under- training is targeting. If the training is aimed too high,
stand and reinforces the learning of correct informa- those that scored high on the pre-test will show the
tion. It also helps to clear up misconceptions before most increase on the post-test while those who
those misconceptions are institutionalized through scored low may show very little increase. Alterna-
practice. Lastly, reviewing the answers in class pro- tively, if lower scores climb and higher scores are
vides the trainer with important feedback on areas stable, the training may be aimed too low. Depend-
that are still not clear to participants, and helps iden- ing on the purpose of the evaluation information,
tify areas of the curriculum needing adjustment. you may want to see if any knowledge increase
was statistically significant.
Make sure to include time for pre- and post-tests
and the discussion of results clearly in your agenda Next, look at changes in knowledge by question to un-
for the training course. Participants should see the cover which parts of the training were most effective,
discussion of the post-test results as a part of the that is, resulted in the most increase in knowledge. Re-
training, not an optional activity that happens after member that a lack of change in knowledge could indi-
the training course is complete. cate either a poorly designed test question or a weak-
ness in the curriculum. If pre-test scores are high, then
there will be little room for knowledge gain as measured
Analyzing Pre- and Post-Test Results on the post-test. If there are questions that multiple par-
ticipants are missing in both the pre- and post-test, con-
The final step is to analyze the results of the pre- sider adjustments to the curriculum to strengthen weak
and post-tests both by participant and by ques- or unclear content areas.
tion. Looking at the data in both of these ways will
help you learn about both the type of participant Review all the data to make sure you haven’t missed
that learned the most from the training (e.g., those any clues and document any additional interesting
with high or low pre-existing knowledge) and the findings. Use the results to make any necessary
areas of the training that were most effective for adjustments to the training.
the whole group. Using Excel, SPSS, or another
statistical software package to analyze the results 1. Based on Developing Multiple Choice Questions for the Royal College of
Physicians and Surgeons of Canada Certification Examinations, T. Wood &
is not required, but it will greatly facilitate the analy- G. Cole, Educational Research and Development, June 2004. p. 4. Can be
sis process. found online at: www.ranzcog.edu.au/fellows/pdfs/diploma-mcqs/devel-
oping-mcqs-for-RCPSC.pdf.
Create a spreadsheet where each row is a single
participant (identified by ID number) and where
each question has two columns — one column
indicating if the participant correctly answered the
question on the pre-test and one indicating if the
participant correctly answered the question on the
post-test.

Look at changes in correct responses by individual.

Did each individual’s score increase? Are there any
discernable patterns indicating which participant Remember to use the results to make any necessary adjustments to the
scores increased the most? For example, in a mul- training.
GUIDELINES FOR PRE- AND POST-TESTING: A TECHNICAL IMPLEMENTATION GUIDE PAGE 8

Acknowledgments

Funding
This document was developed with funding from Cooperative Agreement U69HA00047 from the US Department of Health and
Human Services Health Resources and Services Agency (HRSA); its contents are solely the responsibility of the authors and do not
necessarily represent the views of HRSA.

About I-TECH
The International Training and Education Center for Health (I-TECH) is a collaboration between the University of Washington and the
University of California, San Francisco. Its mission is to support the development of a skilled health care work force and well-orga-
nized national health delivery systems to provide effective prevention, care, and treatment of infectious diseases in resource-limited
settings. Staff work in Africa, Asia, and the Caribbean in partnership with local ministries of health, universities, non-governmental
organizations, and medical facilities.

I-TECH
901 Boren Avenue, Suite 1100
Seattle, Washington 98104 USA
www.go2itech.org

© 2008 University of Washington

Please seek the permission of I-TECH before reproducing, adapting, or excerpting from this guide.

January 2010 ©I-TECH