EDUCATIONAL EVALUATIO

6.1 Educational
Evaluation

In the United States, there is a Joint Committee on

Standards for Educational Evaluation which has developed
standards for educational programmes, personnel, and
student evaluation. The Joint Committee standards are
broken into four sections: Utility, Feasibility, Propriety, and
Accuracy. In the Philippines, there is also a society which
looks into educational evaluation, the Philippine Society
for Educational Research and Evaluation (PSERE) but
mainly, educational evaluation standards are set by the
Department of Education. Various European institutions
have also prepared their own standards, more or less
related to those produced by the Joint Committee in the
United States. They provide guidelines about basing value
judgments on systematic inquiry, evaluator competence
and integrity, respect for people, and regard for the
general and public welfare.

In

order to systematize the

evaluation process, The American
Evaluation Association has created a
set of Guiding Principles for
evaluators which can equally apply
in the Philippine context.

These guiding principles are hereunder stated:

Systematic

Inquiry: Evaluators conduct

systematic, data-based inquiries about whatever
is being evaluated. Inquiry cannot be based on
pure hearsay or perceptions but must be based
on concrete evidence and data to support the
inquiry process.
Competence: Evaluators provide competent
performance to stakeholders. The evaluators
must be people or persons of known
competence and generally acknowledged in the
educational field.

 Integrity

/ Honesty: Evaluators ensure the honesty

and integrity of the entire evaluation process. As
such, the integrity of authorities who conduct the
evaluation process must be beyond reproach.

Respect

for People: Evaluators respect the security,

dignity, and self-worth of the respondents, program
participants, clients, and other stakeholders with
whom they interact. They cannot act as is they know
everything but must listen patiently to the accounts of
those whom they are evaluating.

Responsibilities

for General Public Welfare:

Evaluators articulate and take into account the
diversity of interests and values that may be related
to the general and public welfare.

6.2 Evaluation approaches

Evaluation

approaches are the

various conceptual arrangements
made for designing and actually
conducting the evaluation process.

Major classification of
evaluation
House (1980)
Consider all major evaluation approach to
be base on a common ideology which is that is of
liberal democracy,
It's believed that an individual haw freedom
of choices,
He is unique and that the evaluation process
is guided by empirical inquiry truly based on
objective standard.
All evaluation is based on subjectivist ethics in
which the individual's subjective experiences
figure prominently.

Corresponding Way of
Ethical Position
Objectivist epistemology is associated with
the utilitarian ethics.
Knowledge is acquired which is
capable of external verification and evidence
(intersubjective agreement) through methods
and techniques universally accepted and
through presentation of data.
Subjectivist epistemology is associated
with the intuitionist/pluralist ethic.

Two Main Political Perspective of the

House Approach

An approach can be elitist in which idea is to

focus on the perspective of managers and top
echelon people and professionals.
The approach can be also mass-based in
which the focus is on the consumer and
approaches are participatory.

Stufflebeam and
Webster (1980)
Approaches into one of three groups
according to their orientation toward
the role of values, an ethical
consideration.
Values orientations include
approaches primarily intended to
determine the value of some object.

Attribute
Approach

Politically
controlled

Public relations

Experimental
research

Organizer

Threats

Propaganda
needs

Casual
relationships

Purpose
Get, keep
increase
influence,
power
money.

Key Strengths

Key
Weaknesses
or Secure
Violates
the
evidence
principle of full
advantageous
&
frank
or to the client in disclosure.
a conflict.

Create positive Secure

public image.
evidence most
likely to bolster
public support.

Violates
the
principles
of
balanced
reporting,
justified
conclusions,
&objectivity.

Determine
casual
relationships
between
variables.

Requires
controlled
setting,
limits
range
of
evidence,
focuses
primarily
on
results.

Strongest
paradigmfor
determining
casual
relationships.

Management
information
system

Testing
program

Scientific
efficiency

Individual
differences

Continuously
supply
evidence
needed
to
fund, direct,
&
control
program.

Gives
managers
detailed
evidence
about
complex
programs.

Human
service
variables are
rarely
amenable to
the
narrow,
quantitative
definitions
needed.

Compare test
scores
of
individuals &
groups
to
selected
norms.

Produces
valid
&
reliable
evidence
in
many
performance
areas.
Very
familiar
to
public.

Data usually
only
on
tested
performance,
overemphasiz
es test-taking
skills, can be
what
is
taught
or
expected.

Pseudo-evaluation
Politically

controlled and public relations

studies are based on an objectivist
epistemology from an elite perspective.
Although both of these approaches seek to
misrepresent value interpretations about
some object, they go about it a bit
differently. Information obtained through
politically controlled studies is released or
withheld to meet the special interests of
the holder.

Public

relations information is used

to paint a positive image of an object
regardless of the actual situation.
Neither of these approaches is
acceptable evaluation practice,
although the seasoned reader can
surely think of a few examples where
they have been used.

Objectivist, elite, quasi-evaluation

As

a group, these five approaches

represent a highly respected
collection of discipline inquiry
approaches. They are considered
quasi-evaluation approaches
because particular studies
legitimately can focus only on
questions of knowledge without
addressing any question of value.


Experimental
research
Best

approach for determining causal

relationship between variables. The
potential problem with using this as an
evaluation approach is that its highly
controlled and stylized methodology may
not be sufficiently responsive to the
dynamically changing needs of most
human service program.

Management information system (MISs)

Gives

detailed information about

the dynamic operations of
complex programs. However, this
information is restricted to readily
quantifiable data usually
available at regular intervals.

Testing programs
Familiar

to anyone who has attended

school, served in the military, or worked
for a large company. These programs are
good at comparing individuals or groups to
selected norms in a number of subject
areas or to a set of standards of
performance. They only focus on testee
performance and they might not
adequately sample what is taught or
expected.

Objectives-based
approaches
Relate

outcomes to prespecified
objectives allowing judgments to be
made about their level of
attainment. Unfortunately, the
objectives are often not proven to
be important or they focus on
outcomes too narrow to provide the
basis for determining the value of
an object.

Content analysis
A

quasi-evaluation approach
because content analysis
judgment need to be based on
value statements. Instead they
can be used on knowledge.

Objectivists, mass, quasi-evaluation

Accountability

is popular with
constituents because it is intended to
provide an accurate accounting of
results that can improve the quality of
products and services. However, this
approach quickly can turn
practitioners ad consumers into
adversaries when implemented in a
heavy-handed fashion.

Objectivists, elite, true evaluation

Decision-oriented

studies are
designed to provide a knowledge
base for making and defending
decisions. This approach usually
requires the close collaboration
between an evaluator and decisionmaker, allowing it to be susceptible
to corruption and bias.

Policy studies
Provides

general guidance and

direction on broad issues by
identifying and assessing potential
costs and benefits of competing
policies. The drawback of is these
studies can be corrupted or
subverted by the politically
motivated actions of the participants.

Objectivists, mass, true

evaluation
Consumers-oriented

studies are used

to judge the relative merits of goods and
services based on generalized needs and
values, along with a comprehensive
range of effects. This approach does not
necessarily help practitioners improve
their work, and it requires a very good
and credible evaluator to do it well.

Subjectivities, elite, true evaluation

Accreditation

/ certification programs are

based on self-study and peer review of
organizations, programs, and personnel. They
draw on the insights, experience, and expertise
of qualified individuals who use established
guidelines to determine if the applicant should
be approved to perform specified functions.
However, unless performance based standards
are used, attributes of applicants and the
processes they perform often are
overemphasized in relation to measures of
outcomes or effects.

Connoisseur studies use the

highly refined skills of individuals
intimately familiar with the
subject of the evaluation to
critically characterize and
appraise it. This approach can
help others see programs in a
new light, but it is difficult to find
a qualified and unbiased
connoisseur.

Subjectivist, mass, true

evaluation
The

adversary approach focuses on

drawing out the pros and cons of
controversial issues through quasi-legal
proceedings. This helps ensure a balanced
presentation of different perspectives on
the issues, but it is also likely to discourage
later cooperation and heighten animosities
between contesting parties if winners
and losers emerge.

Client-centered

studies address
specific concerns and issue of
practitioners and other clients of the
study in a particular setting. These
studies help people understand the
activities and values involved from a
variety of perspectives. However, this
responsive approach can lead to low
external credibility and a favorable
bias toward those who participated in
the study.

6.3 Evaluation methods and techniques

Evaluation is methodologically diverse

using both qualitative methods and
quantitative methods, including case
studies, survey research, statistical
analysis, and model building among
others. A more detailed list of methods,
techniques and approaches for
conducting evaluations would include
the following.

Accelerated
aging
Action
research
Advance
product
quality
planning
Alternative
assessment
Appreciative
inquiry
Assessment
Axiomatic
design
Benchmarking
Case study
Change
management
Clinical trial

Course
evaluation
Data mining
Delphi
technique
Discourse
analysis
Electronic
portfolio
Environmental
scanning
Ethnography
Experimental
Experimental
techniques
Factor
analysis
Factorial
experimental
Feasibility

Interview
Marketing
research
Meta-analysis
Metrics
Most
significant
change
Multivariate
statistics
change
Naturalistic
observation
Observational
techniques
Opinion
polling
Organizationa
l

learning
Participant

 Competitor
analysis
Consensus
decision
making
Consensusseeking
decisionmaking
Content
analysis
Conversation
analysis
Cost-benefit
analysis

Field
experiment
Fixtureless InCircuit test
Focus group
Force speed
analysis
Game theory
Grading
Historical
method inquiry

Participatory
impact
pathways
analysis
Policy analysis
Process
improvement
Project
management
Qualitative
research
Quality audit
Quality circle
Quality control
Quality
management
Quality Mgt
Quantitative
research

 Questionnaire
Questionnaire
construction
Root cause
analysis
Rubrics
Sampling
School
accreditation
Self-assessment
Six-sigma
Standardized
testing
Statistical
process control
Statistical survey
Statistics
Strategic
planning
Structured
interviewing
Systems theory
Student testing
Total quality
management

6.3 Evaluation Methods and Techniques

Evaluation

is methodologically diverse
using both qualitative methods and
quantitative methods, including case
studies, survey research, statistical
analysis, and model building among
others. A more detailed list of methods,
techniques and approaches for
conducting evaluation would include the
following:

Accelerated
aging
Action
research
Advance
Product
Quality
Planning
Alternative
assessment
Appreciative
inquiry
Assessment
Axiomatic
design
Benchmarki
ng
Case study
Change
managemen
t
Clinical trial

Course
evaluation
Data mining
Delphi
technique
Discourse
analysis
Electronic
portfolio
Environmental
scanning
Ethnography
Experiment
Experimental
techniques
Factor
analysis
Factorial
experiment
Feasibility
study
Field

Interview
Marketing
research
Metaanalysis
Metrics
Most
significant
change
Multivariate
statistics
Naturalistic
observation
Observatio
nal
techniques
Opinion
polling
Organizatio
nal learning
Participant
observation

Quantitativ
e research
Questionna
re
Questionna
re
constructio
n
Root cause
analysis
Rubrics
Sampling
Schoolaccreditatio
n
Selfassessment
Sis sigma
Standardize
d testing
Statistical
process

 Cohort study
Competitor
analysis
Consensus
decisionmaking
Consensusseeking
decisionmaking
Content
analysis
Conversatio
n analysis
Cost-benefit
analysis

Fixtureless
In-Circuit
Test
Focus group
Force field
analysis
Game
theory
Grading
Historical
method
Inquiry

Participatory
Impact
Pathways
Analysis
Policy
analysis
Process
improvemen
t
Project
management
Qualitative
research
Quality audit
Quality circle
Quality
control
Quality
management
Quality Mgt

Statistical
survey
Statistics
Strategic
planning
Structured
interviewi
ng
Systems
theory
Student
testing
Total
Quality
Manageme
nt
Triangulati
on

6.4 The CIPP evaluation model

Stufflebeam

(1983) developed a very useful

approach in educational evaluation known as the
CIPP or Context, Input, Process, Product approach
(although this model has since then been
expanded to CIPPOI (where the last two stand for
Outcome and Impact respectively). The approach
essentially systematizes the way we evaluate the
different dimensions and aspects of curriculum
development and the sum/total of student
experiences in the educative process. The model
requires that stakeholders be involved in the
evaluation process.

 The

approach is illustrated below:

The CIPP model evaluation

INPUTS
PRODUCT

Product
Is there one final exam at the end or several during the course?
Is there any informal assessment?
What is the quality of assessment (i.e. what levels of KSA are
assessed?)
What are the students KSA levels after the course?
Is the evaluation carried out for the whole [* | In-line. WMF*]
process?
How do students use what they have learned?
How was the overall experience for the teachers and for
students?
What are the main lessons learned?
Is there an official report?
Has the teachers reputation improved or been ruined as a
result?

These guide questions are not answered by the

teacher only or by a single individual. Instead,
there are many ways in which they can be
answered. Some of the more common methods
are listed below:
Discussion with class
Informal conversation or observation
Individual student interviews
Evaluation forms
Observation in class/session of teacher/trainer
by colleagues
Video-tape of own teaching (micro-teaching)

Organizational

documents
Participant contract
Performance test
Questionnaire
Self-assessment
Written test


6.4 Summary of Keywords and Phrases:
Assessment

is the process of gathering and

analyzing specific information as a part of an
evaluation.
Competency evaluation is a means for
teachers to determine the ability of their students
in their ways besides the standardized test.
Course evaluation is the process of evaluating
the instruction of a given course.
Educational evaluation is evaluation that is
conducted specifically in an educational setting.

 Immanent

evaluation opposed by Gilles

Deleuze to value judgement.
Performance evaluation is a term from
the field of language testing it stands in
contrast to competence evaluation.
Program evaluation is essentially a set of
philosophies and techniques to determine if a
program works.