POLICY EVALUATION
Introduction
Public policy making does not end with the passage of
legislation and the implementation of programs
authorized by the new law. The next questions are:
Whether the initiative achieved its objectives,
What the effects were and
Whether any policy changes are needed.
Policy evaluation answers these and related questions.
DEFINITION OF POLICY EVALUATION
Policy evaluation is a systematic (orderly) process for
assessing the design, implementation and outcomes of
policies.
Policy evaluation is conducted to assess the extent to
which objective of the policy have been achieved,
identify needed improvements, and /or make
decisions about future programming
Policy evaluation is conducted to check necessity,
efficiency, validity etc.
Evaluation uses social science research methods,
including qualitative and quantitative techniques, to
determine how successful implementation and outcome
have been.
1
Public policy evaluation provides feedback on the
efficiency, effectiveness and performance of public
policies.
Why policy evaluation?
Policy evaluation enables all participants in the policy
process, including legislators, executives, agency officials
and others to:
a) Measure the degree to which a policy has achieved its
goals, i.e it tries to answer question “what have we
done?”
b)Assess the effects- to see how well the process is
meeting objectives and how the community is
benefiting from the policy
c) Identify any needed changes to a policy. i.e is there
any need to make improvement?
Who evaluate?
i) Elites such as Government Agencies, Public
Universities etc
ii)Interest Groups: e.g Business groups
Question: Who evaluates government/public policies?
Answer: often the government evaluates itself
2
When should policy evaluation be conducted?
Evaluation can take place at any stage in the life of a
policy. That is, evaluation can be done prior to policy
implementation and/or after implementation
TYPES/FORMS OF POLICY EVALUATION
Basically there are two main types of policy evaluation.
a) Formative evaluation and
b)Summative evaluation.
a)Formative evaluation
This is done in early stages of policy development and it
examines the operations of the policy, usually for the
purpose of improving the policy and assessing its
implementation.
It assess the needs that a new program should fulfill,
it examine the early stages of a program's
development, or testing a program on a small scale
before broad dissemination
b)Summative evaluation
This is conducted after a program has been fully
implemented. It looks at whether the program is meeting
its objectives, and why or why not. it tend to asks
Whether the policy achieved its intended
goals/objectives
3
Did the program/ policy make a difference
What change occurred in the population
participating/ affected by the policy
What kind of outcomes should we focus on?
i) Outcomes which can clearly be attributed to the
policy
ii)Outcomes which are sensitive to change
iii) Outcomes which are realistic ie they can be
achieved in the time framework of evaluation
STANDARDS FOR EFFECTIVE POLICY
EVALUATION
Policy evaluation needs to be effective. ie policy
evaluation should achieve the intended effect.
The term effectiveness should not be confused with
efficiency
Actually efficiency focuses on whether the policy is
operating at required/optimal level in terms of time,
money, human resources etc.
So effectiveness and efficiency are two separate concepts.
For effective policy evaluation there four (4) standards or
attributes:
a)Utility:
Policy evaluation should serve the information needs
of the intended users. i.e
What kind of information is needed?
4
By whom e.g government, donor, business
group etc
b)Feasibility:
Planned activities in evaluation process should be
realistic given the resources available e.g time,
money, expertise at hand.
How much money, time, and effort can we put
in order to obtain the information in (a) above?
Try to be realistic, practical, diplomatic
c) Propriety:
Policy evaluation should be conducted legally,
ethically, with due regards for the welfare of those
involved i.e policy evaluation should protect rights of
individuals and welfare of those involved.
What are the steps that need to be followed for
the evaluation to be ethical?
d)Accuracy:
Policy evaluation should reveal and convey
technically adequate information for the intended
user.
Will the evaluation produce findings that are
valid and reliable, given the needs of those who
will use the results?
5
POLICY EVALUATION FRAMEWORK
Policy evaluation structure/framework involve six steps
a) Determine purpose and uses:
Because policy evaluation is a cost full in terms of
money, time, one must define the primary purpose of
evaluation first.
Who need this evaluation and why
Who is going to use the information obtained
b)Develop objectives
Objectives are specific statement which describe
what you plan to do with your proposed policy within
a given time period. Ie what do I want to do in this
policy within this period?
The objective to be developed must be SMART
S = simple and specific (no ambiguity)
M = measurable (target rate or the amount of change
eg 80%)
A = achievable (To the targeted population, within
the time framework, in the area)
R = Realistic
T = timely (eg by the end of this year)
c) Develop evaluation Questions
Evaluation question are developed based on purpose,
objectives, resources and time framework of
evaluation. Evaluation questions help to focus the
evaluation
6
d)Collect/gather information
Identify types of information needed (on
variables such as sex, race, age)
Determine sources for information (eg people,
documents, observation)
Select method to collect information (document
review, interviews, observations, focus group
discussion, case studies)
Select procedures for collecting information
(when, where, who will collect information)
e) Analyze and develop conclusion
Here the evaluator needs to categorize order,
manipulate and summarize data in order to obtain
answers to evaluation questions.
f) Report/ communicate findings
Report should include
Policy/program being evaluated
Purpose of evaluation
Methods of data collection
Results of data analysis
Discussion on strengths, weaknesses and
implications of the results
7
Note that
Effective communication should ensure that evaluation
consider what to communicate, to whom, by whom and
how.
BARRIERS FOR EFFECTIVE POLICY
EVALUATION
Policy evaluations require
i) Funding/capital
ii)Time and
iii) Technical skills
