Unit#1

Introduction to measurement assessment

and evaluation

COURSE CODE:8602 (EDUCATIONAL ASSESSMENT AND MEASUREMENT)

RESOURCE PERSON : EHTESHAM ABBAS

For Daily Updates Regarding Allama Iqbal Open University And

Different Educational Institue, Save This Contact 03127644396.
After Saving this Contact Reply With "Yes Saved"
And Get Daily Alerts. (In Sha Allah)
Todays session will cover following
aspects

 Concept of measurement, assessment and evaluation

 Classroom Assessment: Why, What, How and When
 Types of Classroom Assessment
 Principles of Classroom Assessment
 Role of class room assessment

For Daily Updates Regarding Allama Iqbal Open University And

Different Educational Institue, Save This Contact 03127644396.
After Saving this Contact Reply With "Yes Saved"
And Get Daily Alerts. (In Sha Allah)
Measurement, assessment and evaluation

 Measurement:
 In educational perspective measurement refers to the process of obtaining
numerical values of students progress.
 It is the process of assigning numerical values.
 This process provides the information regarding how much a student has
learnt.
 Measurement provides quantitative description of the students’
performance
 For example: paper marks in numbers,
 for example Rafaih solved 23 arithmetic problems out of 40. But it does not
include the qualitative aspect for example, Rafaih’s work was neat.
For Daily Updates Regarding Allama Iqbal Open University And
Different Educational Institue, Save This Contact 03127644396.
After Saving this Contact Reply With "Yes Saved"
And Get Daily Alerts. (In Sha Allah)
Measurement, assessment and
evaluation
 Assessment
 It is a process to collect, analyze, interpret and communicate the
information about student learning.
 Educational assessment is the process of documenting usually in
measureable terms, knowledge, skills and attitudes
 The process of gathering information to make informed decisions
 It might be qualitative or quantitative.
 Assessment is a broad term that includes testing. For example, a teacher
may assess the knowledge of English language through a test and assesses
the language proficiency of the students through any other instrument for
example oral quiz or presentation. Based upon this view, we can say that
every test is assessment but every assessment is not the test.
Measurement, assessment and
evaluation

 Testing
 Test is a tool used to collect information about students learning.
 Set of questions in organized way.
 Is a systematic procedure to measure a students learning.
 Is quantitative descriptions.
 Other tools for students information
 Observation, interview etc

For Daily Updates Regarding Allama Iqbal Open University And

Different Educational Institue, Save This Contact 03127644396.
After Saving this Contact Reply With "Yes Saved"
And Get Daily Alerts. (In Sha Allah)
Measurement, Assessment and
Evaluation
For Daily Updates Regarding Allama Iqbal Open University And
 Evaluation Different Educational Institue, Save This Contact 03127644396.
After Saving this Contact Reply With "Yes Saved"
 Is a process of value judgment. And Get Daily Alerts. (In Sha Allah)
 Evaluation is much more comprehensive term than measurement and
assessment.
 It includes both quantitative and qualitative descriptions of students’
performance.
 It always provides a value judgment regarding the desirability of the
performance for example, Very good, good etc.
 The central idea in evaluation is "value." When we evaluate a variable, we
are basically judging its worthiness, appropriateness and goodness.
Measurement, assessment and
evaluation
Classroom Assessment: Why, What,
How and When

 It is an integral part of teaching learning process.

 It is widely accepted that effectiveness of teaching learning process is
directly influenced by assessment.
 Hamidi (2010) developed a framework to answer the Why; What, How and
When to assess. This is helpful in understanding the true nature of this
concept.
 Why to Assess: Teachers have clear goals for instruction and they assess to
ensure that these goals have been or are being met.
 What to Assess: In classroom assessment, teachers are supposed to assess
students' current abilities in a given skill or task. The teacher can assess
students’ knowledge, skills or behaviour related to a particular field.
For Daily Updates Regarding Allama Iqbal Open University And
Different Educational Institue, Save This Contact 03127644396.
After Saving this Contact Reply With "Yes Saved"
And Get Daily Alerts. (In Sha Allah)
Classro om Assessment: Why, What,
How and When

 Who to Assess: Teachers should treat students as 'real learners', not as

course or unit coverers. Teachers should understand the level and ability of
their learners (e.g. slow learner, active)
 How to Assess: Teachers employ different instruments, formal or informal, to
assess their students. They can adjust the assessment types to what they
are going to assess.
 When to Assess: There is a strong agreement of educationists that
assessment is interwoven into instruction. Teachers continue to assess the
students learning throughout the process of teaching. (E.g. Formative and
summative assessment)

For Daily Updates Regarding Allama Iqbal Open University And

Different Educational Institue, Save This Contact 03127644396.
After Saving this Contact Reply With "Yes Saved"
And Get Daily Alerts. (In Sha Allah)
Types of Classroom Assessment

 Assessment is a purposeful activity aiming to facilitate students’ learning

and to improve the quality of instruction.
 Assessment is systematic way to collect data of students learning
interpetate and report .
 Generally divided in two types.
 Formative assessment (Assessment for Learning)
 Summative Assessment( Assessment of Learning)

For Daily Updates Regarding Allama Iqbal Open University And

Different Educational Institue, Save This Contact 03127644396.
After Saving this Contact Reply With "Yes Saved"
And Get Daily Alerts. (In Sha Allah)
Types of classroom assessment

 Formative assessment (Assessment for Learning)

 Day to day (e.g. class room test daily, interview to diagnose the students,
question during lecture to check students activeness)
 Gather qualitative and quantitative both
 Ongoing and continuous process
 The purpose of this assessment is to improve teaching learning process
 Use for immediate decisions in students learning
 Help teachers to determine next step during the learning process.
For Daily Updates Regarding Allama Iqbal Open University And
Different Educational Institue, Save This Contact 03127644396.
After Saving this Contact Reply With "Yes Saved"
And Get Daily Alerts. (In Sha Allah)
Types of classroom assessmentin

 Instructional strategies of formative assessment

 Observation
 Questioning strategies
 Self and peer assessment
 Students record keeping

For Daily Updates Regarding Allama Iqbal Open University And

Different Educational Institue, Save This Contact 03127644396.
After Saving this Contact Reply With "Yes Saved"
And Get Daily Alerts. (In Sha Allah)
Types of classroom assessment

 Summative assessment (Assessment of Learning)

 Evaluate the students learning at the end of the semester or year.(E.g end
test, final paper)
 The purpose of this assessment is to help the teacher, students and parents
know how well student has completed the learning task.
 is used to assign a grade to a student which indicates his/her level of
achievement in the course or program.

For Daily Updates Regarding Allama Iqbal Open University And

Different Educational Institue, Save This Contact 03127644396.
After Saving this Contact Reply With "Yes Saved"
And Get Daily Alerts. (In Sha Allah)
Types of classroom assessment

 Types of summative assessment

 Final Examination (a truly summative examination)
 Term Examination(submitted at the end of session)
 Projects (submitted at the end of session)

For Daily Updates Regarding Allama Iqbal Open University And

Different Educational Institue, Save This Contact 03127644396.
After Saving this Contact Reply With "Yes Saved"
And Get Daily Alerts. (In Sha Allah)
Principles of Effective Classroom
Assessment
 Main principles of classroom assessment when you are going
to assess your class. Following are some the some main
principal to assess the classroom
 It should be
 Be formative
 Be Clearly objectives
 Be Specific
 Be Diagnostics (leaner strengths and weakness)
 Be Well-planned and organized
 It should serves the teacher, the student and institution.
 Be Flexible not rigid For Daily Updates Regarding Allama Iqbal Open University And
 Be Learner centered Different Educational Institue, Save This Contact 03127644396.
After Saving this Contact Reply With "Yes Saved"
And Get Daily Alerts. (In Sha Allah)
Role of classroom assessment

 To motivate students for learning (e.g. Low grade students motivated for
improvement
 To diagnose the problems, learning difficulties etc.(e.g.mathamtics
questions)
 For grading purpose (e.g. scoring, pass fail concept)
 Remedies of teaching to improve teaching
 Performance indicator for teacher, students and institution

For Daily Updates Regarding Allama Iqbal Open University And

Different Educational Institue, Save This Contact 03127644396.
After Saving this Contact Reply With "Yes Saved"
And Get Daily Alerts. (In Sha Allah)
Thank You for Listening!
Course Code: 8602 (Educational Assessment and
Evaluation)
Unit 2
OBJECTIVES AND ASSESSMENT
 Learning Objectives
Today’s session will cover the following
aspects

 What Are Tests ?

 Purpose and Principles Of Tests
 Types Of Tests
 Taxonomy of Educational Objectives
 Bloom Taxonomy
 Solo Taxonomy
 Table of Specification and its Preparation
For Daily Updates Regarding Allama Iqbal Open University And
Different Educational Institue, Save This Contact 03127644396.
After Saving this Contact Reply With "Yes Saved"
And Get Daily Alerts. (In Sha Allah)
Why Test is necessary for Assessment?
(Your previous knowledge)
 What is a Test

 It is a tool with organized questions to collect

information about students learning.
 Test is set of questions asked in specific order. It
is a tool to collect information about students’
learning in quantitative form.

For Daily Updates Regarding Allama Iqbal Open University And

Different Educational Institue, Save This Contact 03127644396.
After Saving this Contact Reply With "Yes Saved"
And Get Daily Alerts. (In Sha Allah)
 Purpose Of Tests

• Monitoring Students’ Progress

• Diagnosing Learning Problems
• Assigning Grades
• Classification and Selection of
Students
• Evaluating Instruction
 Principles Of Tests

• Clear purpose
For Daily Updates Regarding Allama Iqbal Open University And
• Relevancy Different Educational Institue, Save This Contact 03127644396.
After Saving this Contact Reply With "Yes Saved"
And Get Daily Alerts. (In Sha Allah)

• Clear Language

• Validity and Reliability

• Table Of Specification
 Test and its Types

Types of test

 Subjective type tests

 Objective type tests
 Multiple choice tests
 Close ended tests
 Matching items tests
For Daily Updates Regarding Allama Iqbal Open University And
Different Educational Institue, Save This Contact 03127644396.
After Saving this Contact Reply With "Yes Saved"
And Get Daily Alerts. (In Sha Allah)
Taxonomy and Classification of
Educational Objectives

Taxonomy of educational objectives

(three domains )

( I ) The Cognitive (Knowledge)

( ii ) The Affective (Attitude, Values)

( iii ) The Psychomotor (Skills)

For Daily Updates Regarding Allama Iqbal Open University And

Different Educational Institue, Save This Contact 03127644396.
After Saving this Contact Reply With "Yes Saved"
And Get Daily Alerts. (In Sha Allah)
Writing Cognitive Domain Objectives
 Bloom’s Revised
Taxonomy (2001)

For Daily Updates Regarding Allama Iqbal Open University And

Different Educational Institue, Save This Contact 03127644396.
After Saving this Contact Reply With "Yes Saved"
And Get Daily Alerts. (In Sha Allah)
 Preparation of Table of
Specification
• Table of specifications is a tool used to ensure that a test measures
the content and thinking skills that the test intends to measure.

K C A Total
Content Strand

No of % No of % No of % No of %
Sr. no.

Items weight Items weighta Items weight Items weigh

age in ge age tage

Test in Test in Test in Test

1 Geo 4 12.5 4 12.5 3 9.31 11 34.3

metr

y
 How to Calculate Test Items in
Cells?

 For example Knowledge has 31% weightage in curriculum. There

are 11 items for Geometry in the test (i.e. 34% in curriculum). Then
items of geometry to measure knowledge level of students can
be calculated as

 (Number of test items for content strand or Unit)x(knowledge wise

weightage in %)

 = 11x(31/100)

For Daily Updates Regarding Allama Iqbal Open University And

Different Educational Institue, Save This Contact 03127644396.
After Saving this Contact Reply With "Yes Saved"
And Get Daily Alerts. (In Sha Allah)
 SOLO Taxonomy
For Daily Updates Regarding Allama Iqbal Open University And
Different Educational Institue, Save This Contact 03127644396.
After Saving this Contact Reply With "Yes Saved"
And Get Daily Alerts. (In Sha Allah)
• SOLO: Structure of Observed Learning Outcomes

• Developed by Biggs and Collis (1982)

Thank You for Listening!
 Students have to select correct
option/answer from a given choices

Multiple Choice
Questions
True/False
Matching items
Which is the capital
city of Pakistan? The capital city of
Pakistan is:

A. Paris A. Paris.
B. Lisbon B. Lisbon.
C. Islamabad C. Islamabad.
D. Rome D. Rome.

For Daily Updates Regarding Allama Iqbal Open University And

Different Educational Institue, Save This Contact 03127644396.
After Saving this Contact Reply With "Yes Saved"
And Get Daily Alerts. (In Sha Allah)
 Important Facts
 Use Simple Language
 Make the Questions Brief and Clear
 Avoid Asking Negative Questions
 Offer an Appropriate Numbers of Distracters

For Daily Updates Regarding Allama Iqbal Open University And

Different Educational Institue, Save This Contact 03127644396.
After Saving this Contact Reply With "Yes Saved"
And Get Daily Alerts. (In Sha Allah)
 Effective  Difficulty of Construction
 Versatile
 Reduced Guessing
 Can cover broad
range of content

For Daily Updates Regarding Allama Iqbal Open University And

Different Educational Institue, Save This Contact 03127644396.
After Saving this Contact Reply With "Yes Saved"
And Get Daily Alerts. (In Sha Allah)
 A True-False test item requires the student to
determine whether a statement is true or false.

 Also known as a “binary-choice” item.

For Daily Updates Regarding Allama Iqbal Open University And

Different Educational Institue, Save This Contact 03127644396.
After Saving this Contact Reply With "Yes Saved"
And Get Daily Alerts. (In Sha Allah)
 Easy to construct  Assess lower order
 Easy to score thinking skills
 Helpful for poor  Poor representative of
students students learning
achievement

For Daily Updates Regarding Allama Iqbal Open University And

Different Educational Institue, Save This Contact 03127644396.
After Saving this Contact Reply With "Yes Saved"
And Get Daily Alerts. (In Sha Allah)
 Consist of :
(i) Column of premises
(ii) Column of responses

For Daily Updates Regarding Allama Iqbal Open University And

Different Educational Institue, Save This Contact 03127644396.
After Saving this Contact Reply With "Yes Saved"
And Get Daily Alerts. (In Sha Allah)
 Terms with definitions
 Phrases with other phrases
 Causes with effects
 Parts with larger units
 Problems with solutions

For Daily Updates Regarding Allama Iqbal Open University And

Different Educational Institue, Save This Contact 03127644396.
After Saving this Contact Reply With "Yes Saved"
And Get Daily Alerts. (In Sha Allah)
 Factual information  Time consuming for
can be tested students
 Valuable in content  Not good for higher
areas that have a lot levels of learning
of facts

For Daily Updates Regarding Allama Iqbal Open University And

Different Educational Institue, Save This Contact 03127644396.
After Saving this Contact Reply With "Yes Saved"
And Get Daily Alerts. (In Sha Allah)
 Students have to provide correct
answer

Essay Type
Completion
items

Short answers
 Answer is needed to complete the
sentence.
 These are also known as “Gap-Fillers.

For Daily Updates Regarding Allama Iqbal Open University And

Different Educational Institue, Save This Contact 03127644396.
After Saving this Contact Reply With "Yes Saved"
And Get Daily Alerts. (In Sha Allah)
 Student supplies a response to a question
 Most effective for assessing knowledge and
comprehension learning outcomes
 Easy to construct  May overemphasize
 Minimizes guessing memorization of
facts
 Scoring is laborious

For Daily Updates Regarding Allama Iqbal Open University And

Different Educational Institue, Save This Contact 03127644396.
After Saving this Contact Reply With "Yes Saved"
And Get Daily Alerts. (In Sha Allah)
 Measure the students' higher order thinking
skills, such as applying, organizing, synthesizing,
integrating, evaluating, or projecting while at
the same time providing a measure of writing
skills.

For Daily Updates Regarding Allama Iqbal Open University And

Different Educational Institue, Save This Contact 03127644396.
After Saving this Contact Reply With "Yes Saved"
And Get Daily Alerts. (In Sha Allah)
For Daily Updates Regarding Allama Iqbal Open University And
Different Educational Institue, Save This Contact 03127644396.
After Saving this Contact Reply With "Yes Saved"
And Get Daily Alerts. (In Sha Allah)
 Reduce guessing  Time consuming to
 Easy to construct score.
 Allows students to
demonstrate ability
to organize
knowledge, express
opinions, show
originality.

For Daily Updates Regarding Allama Iqbal Open University And

Different Educational Institue, Save This Contact 03127644396.
After Saving this Contact Reply With "Yes Saved"
And Get Daily Alerts. (In Sha Allah)
For Daily Updates Regarding Allama Iqbal Open University And
Different Educational Institue, Save This Contact 03127644396.
After Saving this Contact Reply With "Yes Saved"
And Get Daily Alerts. (In Sha Allah)
Unit 5 # RELIABILITY OF THE
ASSESSMENT TOOLS
CC 8602
PREPARED BY: Ehtesham Abbas
 OBJECTIVES
After studying this unit, prospective teachers will be able
to:
• define reliability in their own words.
• apply the different methods of assuring reliability on the
tests.
• identify the factors affecting reliability.
• construct a test and check how much reliable it is.
• identify measures for reducing the problems in
conducting the tests.
 Reliability
Reliability means Trustworthy. A test score is called reliable
when we have reasons for believing the test score to be stable
and objective.
For example if the same test is given to two classes and is
marked by different teachers even then it produced the
similar results, it may be considered as reliable.
Stability and trustworthiness depends upon the degree to
which score is free of chance error.
We must first build a conceptual bridge between the question
asked by the individual (i.e. are my scores reliable?) and how
reliability is measured scientifically.
When a person thinks of reliability,
many things may come into his mind – my friend is very
reliable, my car is very reliable, my internet bill-paying
process is very reliable, my client’s performance is very
reliable, and so on.
The characteristics being addressed are the concepts such
as consistency, dependability, predictability, variability etc.
Note that implicit, reliability statements, is the behaviour,
machine performance, data processes, and work
performance may sometimes not reliable.
According to Merriam Webster Dictionary:
 “Reliability is the extent to which an experiment, test, or measuring
procedure yields the same results on repeated trials.”
According to Hopkins & Antes (2000):
 “Reliability is the consistency of observations yielded over repeated
recordings either for one subject or a set of subjects.”
Joppe (2000) defines reliability as:
 “…The extent to which results are consistent over time and an accurate representation of
the total population under study is referred to as reliability and if the results of a study can
be reproduced under a similar methodology, then the research instrument is considered
to be reliable.” (p. 1)
For Daily Updates Regarding Allama Iqbal Open University And
Different Educational Institue, Save This Contact 03127644396.
After Saving this Contact Reply With "Yes Saved"
And Get Daily Alerts. (In Sha Allah)
 Types of Reliability
 It has to do with the consistency, or reproducibility, of an examinee's
performance in the test. It's not possible to calculate reliability exactly.
 There are six general classes of reliability estimates, each of which estimates
reliability in a different way.
i) Inter-Rater or Inter-Observer Reliability
 To assess the degree to which different raters/observers give consistent
estimates of the same phenomenon.
 That is if two teachers mark same test and the results are similar, so it
indicates the inter-rater or inter-observer reliability.
Test-Retest Reliability:
 To assess the consistency of a measure from one time to another,
when a same test is administered twice and the results of both
administrations are similar, this constitutes the test-retest reliability.
 (iii) Parallel-Form Reliability:
To assess the consistency of the results of two tests constructed
in the same way from the same content domain.
Here the test designer tries to develop two tests of the similar
kinds and after administration the results are similar then it will
indicate the parallel form reliability.
iv) Internal Consistency Reliability:
 To assess the consistency of results across items within a test, it is
correlation of the individual items score with the entire test.
v) Split half Reliability:
 To assess the consistency of results comparing two halves of single
test, these halves may be even odd items on the single test.
vi) Kuder-Richardson Reliability:
To assess the consistency of the results using all the possible
split halves of a test.
Let's discuss each of these in turn.

For Daily Updates Regarding Allama Iqbal Open University And

Different Educational Institue, Save This Contact 03127644396.
After Saving this Contact Reply With "Yes Saved"
And Get Daily Alerts. (In Sha Allah)
 Inter-Rater or Inter-Observer Reliability
 Whenever we observe or activities of humans, we have to think about the
procedure for reliable and consistent results.
 For this two or more than two observers are assigned to observe the
students or teachers. So how do we determine whether two observers are
being consistent in their observations?
 We probably should establish inter-rater reliability by considering the
similarity of the scores awarded by the two observers.
There are two major ways to actually estimate inter-rater
reliability:
 If your measurement consists of categories -- the raters are checking off which
category each observation falls in -- you can calculate the percent of
agreement between the raters,
For instance, let's say you had 100 observations that were being rated
by two raters.
 For each observation, the rater could check one of three categories.
Imagine that on 86 of the 100 observations, the raters checked the same
category.
 In this case, the percent of agreement would be 86%. OK, it's a crude
measure, but it does give an idea of how much agreement exists,
 and it works no matter how many categories are used for each
observation.

For Daily Updates Regarding Allama Iqbal Open University And

Different Educational Institue, Save This Contact 03127644396.
After Saving this Contact Reply With "Yes Saved"
And Get Daily Alerts. (In Sha Allah)
 The other major way to estimate inter-rater reliability is appropriate when
the measure is a continuous one.
 There, all you need to do is calculate the correlation between the ratings
of the two observers.
 For instance, they might be rating the overall level of activity in a
classroom on a 1-to-7 scale.
 You could have them give their rating at regular time intervals (e.g., every
30 seconds).
 The correlation between these ratings would give you an estimate of the
reliability or consistency between the raters.
 One might think of this type of reliability as "calibrating" the observers.
There are other things one could do to encourage reliability between
observers, even without estimating it.
 Test-Retest Reliability
 Test-retest is a statistical method used to determine a test's reliability. The
test is performed twice;
 in the case of a questionnaire, this would mean giving a group of
participants the same questionnaire on two different occasions.
 This form of reliability is used to judge the consistency of results across
items on the same test.
 Essentially, you are comparing test items that measure the same
construct to determine the tests internal consistency.
We estimate test-retest reliability when we administer the
same test to the same sample on two different occasions.
 This approach assumes that there is no substantial change in the construct
being measured between the two occasions.
 The amount of time allowed between measures is critical.
 We know that if we measure the same thing twice that the correlation
between the two observations will depend in part by how much time
elapses between the two measurement occasions.
 The shorter the time gap, the higher the correlation; the longer the time
gap, the lower the correlation.
 This is because the two observations are related over time -- the closer in
time we get the more similar the factors that contribute to error. Since this
correlation is the test-retest estimate of reliability, you can obtain
considerably different estimates depending on the interval.
 Split-Half Reliability
 Suppose you have to develop a test of 30 items and you want to know that
how reliable the test is? What you have to do is to administer the test, mark
it and divide it in to two parts, in such a way that place all the even
numbered items (2,4,6…………) in one half and the odd numbered items
(1,3,5…………..) in the second. Calculate the reliability by using the
Spearman-Brown prophecy formula given below.
 Actually in split-half reliability we randomly divide all items that claim to
measure the same contents into two sets.
 We administer the entire instrument to a sample of students
 and calculate the total score for each randomly divided half.
 The split-half reliability estimate is simply the correlation between these two
total scores.
 Normally a single test is used to make two shorter alternate forms. This
method has the advantage that only one test administration is required,
and therefore memory and the practice and maturation effects are not
involved.
 Furthermore, it does not require two tests. So it has many advantages over
parallel form and test-retest methods, therefore it is the most frequently
used method of finding internal consistency of the classroom tests.
 The formula used for the reliability of the full test is Spearman-Brown
prophecy formula as given below.
 Reliability of the Full Test = 2(reliability of the half test)
__________________________
1+ (reliability of the half test)
 Parallel-Form Reliability
 In parallel form reliability we have to create two different tests from the
same contents to measure the same learning outcomes. The easiest
way to accomplish this is to write a large set of questions that address
the same contents and then randomly divide the questions into two sets.
 Now it’s time to administer both instruments to the same students at the
same time. The correlation between the two parallel forms is the
estimate of reliability. One major problem with this approach is that you
have to be able to write lots of items that reflect the same contents.
 This is often no easy to do job. Furthermore, this approach makes
the assumption that the randomly divided halves are parallel or
equivalent. Even by chance, this will sometimes not be the case.
The parallel forms approach is very similar to the split-half reliability
described earlier.
For Daily Updates Regarding Allama Iqbal Open University And
Different Educational Institue, Save This Contact 03127644396.
After Saving this Contact Reply With "Yes Saved"
And Get Daily Alerts. (In Sha Allah)
 The major difference is that parallel forms are constructed so that the two
forms can be used independent of each other and considered equivalent
measures. For instance, we might be concerned about a testing threat to
internal validity. If we use Form A for the pretest and Form B for the
posttest, we minimize that problem.
 It would even be better if we randomly assign individuals to receive Form
A or B on the pretest and then switch them on the posttest.
 With split-half reliability we have an instrument that we wish to use as a
single measurement instrument and only develop randomly split halves for
purposes of estimating reliability.

For Daily Updates Regarding Allama Iqbal Open University And

Different Educational Institue, Save This Contact 03127644396.
After Saving this Contact Reply With "Yes Saved"
And Get Daily Alerts. (In Sha Allah)
 Internal Consistency Reliability
 In internal consistency reliability estimation, we use our single test. The test is administered
to a group of students on one occasion to estimate reliability. In effect we judge the
reliability of the instrument by estimating how well the items that reflect the same content
give similar results.
 We are looking at how consistent the results are for different items for the same construct
within the measure. There are a wide variety of internal consistency measures that can be
used.
 Kuder Richardson Reliability
 The estimates of internal consistency of the test are commonly calculated by using Kuder-
Richardson methods.
 These measures to extent to which items within one form of the test have as much in
common with one another as do the items in that one form with corresponding items in an
equivalent form.
For Daily Updates Regarding Allama Iqbal Open University And
Different Educational Institue, Save This Contact 03127644396.
After Saving this Contact Reply With "Yes Saved"
And Get Daily Alerts. (In Sha Allah)
B.Ed Course 8602
EDUCATIONAL ASSESSMENT
AND EVALUATION

Ehtesham Abbas

For Daily Updates Regarding Allama Iqbal Open University And

Different Educational Institue, Save This Contact 03127644396.
After Saving this Contact Reply With "Yes Saved"
And Get Daily Alerts. (In Sha Allah)
UNIT # 06

VALIDITY OF THE
ASSESSMENT TOOLS

For Daily Updates Regarding Allama Iqbal Open University And

Different Educational Institue, Save This Contact 03127644396.
After Saving this Contact Reply With "Yes Saved"
And Get Daily Alerts. (In Sha Allah)
 Validity
 The validity of an assessment tool is the degree to which it
measures for what it is designed to measure.

 The concept refers to the appropriateness, meaningfulness,

and usefulness of the specific inferences made from test
scores.
 Example
 You are intended to measure the intelligence and if math and
vocabulary truly represent intelligence then a math and
vocabulary test might be said to have high validity when used as
a measure of intelligence.

 Give your example

For Daily Updates Regarding Allama Iqbal Open University And

Different Educational Institue, Save This Contact 03127644396.
After Saving this Contact Reply With "Yes Saved"
And Get Daily Alerts. (In Sha Allah)
 Need of Test Validity
 Test validity, or the validation of a test, explicitly means validating the use of a
test in a specific context

 To make sure that a test measures the skill, trait, or

attribute it is supposed to measure.

 To yield reasonable consistent results for individual

 To measure with reasonable degree of Accuracy.

Validity versus Reliability
Methods of Measuring Validity
 1. Content Validity
 Content validity evidence involves the degree to which the content of the test
matches a content domain associated with the construct.

 Items in a test appear to cover whole domain.

 Types of Content Validity

Face validity Curricular

The extent to which
It is an estimate of
the content of the test
whether a test appears
matches the
to measure a certain
objectives of a
criterion. It is
specific curriculum
appearance of test.
as it is formally
described.
 2. Construct Validity
 Construct is the concept or the characteristic that a test is designed to measure.
 According to Howell (1992) Construct validity is a test’s ability to measure
factors which are relevant to the field of study.
 Types of Construct Validity

Convergent
Discriminant

Convergent validity Discriminant validity

refers to the degree to occurs where
which a measure is constructs does not
correlated with other correlate with other
measures. measures.
 3. Criterion Validity
 Criterion validity evidence involves the correlation between the test and a
criterion variable (or variables) taken as representative of the construct.
 It compares the test with other measures or outcomes (the criteria) already
held to be valid.
 3.1. Concurrent Validity
 Concurrent validity refers to the degree to which the scores taken at one point
correlates with other measures (test, observation or interview) of the same
construct that is measured at the same time.
 3.2 Predictive Validity
 Predictive validity assures how well the test predicts some future behaviour of
the examinee.
 If higher scores on the Boards Exams are positively correlated with higher
G.P.A.’s in the Universities and vice versa, then the Board exams is said to
have predictive validity.
 Factors Affecting Validity
 Instructions to Take a Test
 Difficult Language Structure
 Inappropriate Level of Difficulty
 Poorly Constructed Test Items
 Ambiguity in Items Statements
 Length of the Test
 Improper Arrangement of Items
 Identifiable Pattern of Answers
For Daily Updates Regarding Allama Iqbal Open University And
Different Educational Institue, Save This Contact 03127644396.
After Saving this Contact Reply With "Yes Saved"
And Get Daily Alerts. (In Sha Allah)
 Relationship between
Validity and Reliability
 Reliability is a necessary requirement for validity
 Establishing good reliability is only the first part of establishing validity
 Reliability is necessary but not sufficient for validity.
 Unit 7

PLANNING AND ADMINISTERING

CLASSROOM TESTs

For Daily Updates Regarding Allama Iqbal Open University And

Different Educational Institue, Save This Contact 03127644396.
After Saving this Contact Reply With "Yes Saved"
And Get Daily Alerts. (In Sha Allah)

Course Code: 8602 (Educational Assessment and Evaluation)

PLANNING AND ADMINISTERING CLASSROOM TEST
Today’s session will cover the following aspects

• Planning a Test

• Writing Supply Type and Select Type Test Items

• Administering a Test

• Scoring a Test
Why Test Planning and Administration is important?
(Your Previous Knowledge)
 Planning a Test
i. Pre-testing
• Instructions at beginning
• Readiness, Motivation
• To know about achieved objectives

ii. During the Instruction Testing

• Provides bases for formative assessment
• Monitor learning progress
• Detect learning errors
• Provide feedback for students and teachers

iii. End of Instruction Testing

• Measure intended learning outcomes
• Used for formative assessment
• Provides bases for grades, promotion etc.
Table of Specification
 (Matching Items)
• Consist of Two Parallel Columns.
• Left termed (premises) on the right, (responses)
Matching Items Good for: Knowledge and Comprehension

Tips for writing True/False Questions

• Keep both columns Short
• Surety of responses for all Premises
• Left Column longer, Right Shorter (phrases, words, and symbols)
• Description in numbered and options identified by letter
• More options of responses than descriptions
• Put all items on a single page
• Logical order in responses
(chronological, alphabetical, etc.)
 Multiple Choice Questions (MCQs)
Three Parts The capital city of Pakistan is
1. The Question/Stem (body of the question) A. Paris.
2. The Correct Answer (the key of the question) B. Lisbon.
3. Incorrect Alternatives (distracters) C. Islamabad.
D. Rome.
Multiple Choice Questions Good for: Application, Synthesis, Analysis, and Evaluation levels

RULES FOR WRITING MULTIPLE-CHOICE QUESTIONS

• Use Reasonable Distracters
• Use a Question Format
• Emphasize Higher-Level Thinking
• Keep Option Lengths Similar
• Be Grammatically Correct
• Avoid Clues to the Correct Answer
• Avoid Negative Questions
• Use Only One Correct Option
• Give Clear Instructions
• Use Only a Single, Clearly-Defined Problem
• Avoid “All the Above” Option or “None of the Above”
• Don’t Use MCQ When Other Item Types Are More Appropriate
 Supply Type Items (Completion)
Student fills in one or more blanks in a statement.
Example: The capital city of Pakistan is -----------------
Completion Items Good for
Knowledge, Comprehension and higher level Outcomes

Tips for writing Completion Items

• Use a Single-word answer
• State Problem to be examined
• Use of Precise and accurate language
• Don’t use Senseless statement
• Indicate Units of numerical answers
 Supply Type Items (Essay)
Restricted Response:
Describe Similarities and differences in the lives of people living in Islamabad and Faisalabad.
Extended Response:
Identify as many different ways to generate electricity in Pakistan as you can?

Tips for Writing Good Essay Items

• Questions of essential knowledge
• Ask sense able questions
• Preference for specific questions.
• Use relevant items for each domain.
• Avoid a choice among optional questions.
• Test the question by writing an ideal answer to it.
• Specify the time allotment for each item
• Divide a question into separate components when there are obvious multiple
questions or pieces to the intended responses.
 Administering a Test
I. Test Assembly
• Group together all item of Similar Format
• Arrange test items from Easy to Hard
• Space the items for Easy Reading
• Keep items and their options on the Same Page
• Position illustrations, tables, charts, pictures diagrams or maps near
descriptions
• Answer keys must be Checked Carefully
• Provide adequate and proper Space for Name and Date
• Test directions must be Precised and Clear
• Test must be Proofread to make it error free
• Make all the item Unbiased (gender, culture, ethnic, racial etc.)
 Administering a Test
II. Reproduction of the Test

Practical Steps for Printing

• Manage Printing (if test takers are large in number)
• Manage Photocopy from a proper/new machine
• Use Good Quality of the Paper and Printing
• Retain Original Test in your own Custody
• Be careful while Making Sets of the test
• Manage Confidentiality of the test
 Administering a Test
III. Administration of the Test
• Maintain a Positive attitude for achievement
• Maximize achievement Motivation
• Equalize Advantages to all the students
• Provide Easy, Comfortable and Proper Seats
• Provide Proper system of Light, temperature, air and water.
• Clarify all the Rules and regulations
• Remind the students to Check their Copies
• Monitor students continuously
• Minimize Distractions
• Give Time Warnings properly
• Collect Test Uniformly
• Count the Answer Sheets, seal it in a bag and hand it over
 Administering a Test
IV. Test Taking Strategies
• Students need to Follow Directions carefully
• Students need to understand how to Budget their Time
• Students need to Check their Work
• Read the Entire Test Item and possible Answers very Carefully.
• Answer the Easier Questions First
• Students need to make Educated Guesses
• Use Test Item Formats for practice
• Review the Practice Items and answer choices with students.
• Practice using answer sheets.
 Administering a Test
V. Steps to Prevent Cheating

• Take Special Precautions during preparation, storage and

administration of test.
• Provision of Sufficient Space on their desks to work easily
• If Scratch Paper is used have it turned in with the test.
• Testing Hours must be watched carefully
• Two Forms of the Tests can also be used
• Use Special Seating Arrangements while placing the students
• Create and maintain a Positive Attitude for the value of tests
 Scoring the Test (Essay)
• Prepare a Scoring Key in advance
• Outline of the acceptable answer
• Main Features of the Answer and the Weights assigned to each.
• (Content, 4 points, Comprehensiveness: 2, Logical organization: 2, Irrelevant
material: Deduct upto 2, Misspelling of technical terms: Deduct 1/2 to 2, Major
grammatical mistakes: 1 to 2 points, and ignore Poor handwriting, misspelling
of non-technical terms and minor grammatical errors.
• Use an appropriate scoring method (point or rating)
• Read a sampling of the papers to get a „feel‟ of the quality of the answers.
• Score One Question through All of the papers before going on to the next
• Adopt a Definite Policy regarding factors which may not be relevant to learning
outcomes being measured
• Score the papers Anonymously
 Scoring with Example
Test Item
Name and describe five of the most important factors of unemployment in Pakistan. (10 points)

Rubric/Scoring Criteria:

• 1 Point for Each of the Factors Named, to a maximum of 5 points

• One point for each Appropriate Description of the factors named, to

a maximum of 5 points

• No Penalty for spelling, punctuation, or grammatical error

• No Extra Credit for more than five factors named or described.

• Unnecessary Information will be ignored.

How a test can be made effective through
Planning and Administering it?
Thank You for Listening!
INTERPRETING TEST SCORE

For Daily Updates Regarding Allama Iqbal Open University And

Different Educational Institue, Save This Contact 03127644396.
After Saving this Contact Reply With "Yes Saved"
And Get Daily Alerts. (In Sha Allah)
COURSE CODE : 8602 UNIT : 8
 MEASURES OF CENTRAL TENDENCY

▶ A measure indicating the value to be expected of a typical

or middle data point.
 MEAN OR AVERAGE

▶Arithmetic Mean ▶It is defined as the sum of all the

observations divided by
the number of observations. It is denoted by X.
 EXAMPLE OF MEAN

▶Example: ▶5, 10, 12, 16, 8, 42, 25, 15, 10, 7

▶Solution: 5+10+12+16+8+42+25+15+10+7=150/10

▶ Mean = 15
For Daily Updates Regarding Allama Iqbal Open University And
Different Educational Institue, Save This Contact 03127644396.
After Saving this Contact Reply With "Yes Saved"
And Get Daily Alerts. (In Sha Allah)
 MEDIAN

Median is the middle most value of a set of data.

If the number of observations is in odd form, then median is

the mid value.

If the number of observations is even form, then median is

the average of two middle values.
CONT….
 MODE

▶The most frequent value that occurs in the set of data is

called mode.
MEASURES OF DISPERSION
 RANGE
▶It is defined as difference between largest and smallest
observations in a set of data. Range = R = Xm - X0 ▶Where Xm
the largest observation X0 = the smallest observation.
 STANDARD DEVIATION
▶ It determines whether the scores are generally near or far from mean.
 VARIANCE
▶Variance (σ2) in statistics is a measurement of the spread
between numbers in a data set. That is, it measures how far
each number in the set is from the mean and therefore from
every other number in the set.
 NORMAL CURVE
▶One way of presenting out how data are distributed is to plot
them in a graph. ▶Normal or bell curved is distribution of data
may naturally occur in several possible ways, with
a number of possibilities for standard deviation
 SKEWNESS
 Skewness tells us about the amount and direction of the
variation of the data set.
It is a measure of symmetry.
METHODS OF DATA PRESENTATIONS
LINE GRAPHS
 SCATTER PLOTS

A graph of data that is a set of points.

Thanks 4 Attention
Allah Hafiz
REPORTING TEST SCORES
EDUC2114 Classroom Assessment

8602
Unit 09
 Reporting
Objectives
 1. Understand the purpose of reporting test scores
 2. Explain the functions of test scores
 3. Describe the essential features of progress report
 4. Enlist the different types of grading and reporting
systems
 5. Calculate CGPA
 6. Conduct parent teacher conferences
Functions of and reporting and grading

 Instructional uses.
 Feedback to students.
 Administrative and guidance uses.
 Informing parents about their children’s
performance.
 Reporting ?

 A document containing information organized in a

narrative, graphic, or tabular form.
 Purpose of Reporting

 Define the scope of testing

 Present the results of testing
 Draw conclusions and make recommendations
 Evaluate and decision making
 TYPES OF SCORES

Raw Scores Standard Scores

The standard scores indicate

a student’s relative position
A Raw Score is simply the
in a group. It expresses
number of questions a
test performance in terms
student answers correctly
of standard deviation units
for a test
from the mean
 Raw Scores

A raw score provides an indication of the

variability in performance among students in a
classroom
 T Scores:

 Any set of normally distributed standard scores

that has a mean of 50 and SD of 10.

T = (Z x 10) + 50
 Z Score =
(Raw Score –Mean)/Standard deviation

Note: Z score is always minus when the raw score is

smaller than the mean.
 Z Score =
(Raw Score –Mean)/Standard deviation

Note: Z score is always minus when the raw

score is smaller than the mean.
 GRADING

 Grading refers to the process of using symbols, such as

letter to indicate various types of students progress.
Common Methods of Grading :

 Letter
grades
 Number/Percentage grade
 %age Strengths:

 Easy to use
 Easy to interpret theoretically
 Provide a concise summary
 May be combined with letter grades
 More continuous than letter grades
 Do’s of parent-teacher conferences

 Be friendly and honest.

 Be honest
 Be willing to listen and explain
 Be willing to accept parents’ feelings
 Be professional and maintain a positive attitude
 Develop a packet of conference including student’s goals,
samples of work, and reports or notes from other staff.
 Don’ts of the parent teacher conference

 Don’t argue
 Don’t get angry
 Don’t ask embarrassing questions
 Don’t reject parents’ suggestions
 REPORTING TEST SCORE
Unit 9 8602

Prepared by:
Ehtesham Abbas
 OBJECTIVES

After studying the Unit, the students will be able to:

1. understand the purpose of reporting test scores
2. explain the functions of test scores
3. describe the essential features of progress report
4. enlist the different types of grading and reporting systems
5. calculate CGPA
6. conduct parent teacher conferences
 INTRODUCTION
“Reporting Test Scores” is about measuring the performance of students
by providing a profile of their progress and reporting the scores of tests in
different ways in context to the different purposes. There is a long
tradition that students’ skills are measured by some of testing procedures.
Invariably, the product of testing is a score, a ‘yardstick’ by which an
individual student is compared with others and/or by which progress is
documented. Teachers and other educators use tests, and subsequently
test scores in a variety of ways.
The first major topic of the unit deals with the functions of test scores and
progress reports of students after taking any test. As there are different
functions of grading and reporting systems with respect to its uses like
instructional uses, providing feedback to students for administrative use
and guidance and informing parents about their children’s performance.
 The second key topic in the unit discussed is the “Types of
Test Scores and Progress Reports”. Here two types of
reporting test scores are discussed. First is Norm-referenced
tests which include raw scores, grade norms, percentiles,
stanines, and standard scores. Second is Criterion-
referenced test which include system of pass-fail and the
other types of the practices that are used to report the
progress of students.
 The third major theme is “Calculating CGPA and Assigning
Letter Grades” It includes the method of calculating CGPA
and different steps which are concerned with assigning letter
grades in reporting test scores such as combining the data,
selecting the proper frame of reference for grading and
determining the distribution of grades etc.
 Reporting:
 A document containing information organized in a narrative,
graphic, or tabular form, prepared on ad hoc, periodic,
recurring, regular, or as required basis.
Functions of Test Scores and Progress Reports
The different functions of grading and reporting systems are
given as under:

1. Instructional uses

2. Feedback to students

3. Administrative and guidance uses

4. Informing parents about their children’s

performance
 TYPES OF SCORE

1.Raw Scores :
A Raw Score is simply the number of questions a
student answers correctly for a test. For example, if
a student responds to 65 items correctly on an
objective test in which each correct item counts one
point, the raw score will be 65.
Uses:

 A raw score provides an

indication of the variability in
performance among students
in a classroom.
Limitations:
 A rawscore by itself has no
meaning. It can be interpreted
only by comparing it with some
standard such as total number of
items for a test or with raw scores
earned by a comparison group.
Stanine (Standard nine)
 Standard nine) : Stanine scores
express test results in equal steps
that range from 1 (lowest) to 9
(highest). The average is a score of
5. In general, stanine scores 1,2
& 3 are below average, 4,5 & 6 are
average and 7, 8 & 9 are above
average
Standard Scores
 The standard scores indicate a
student’s relative position in a
group. It expresses test
performance in terms of standard
deviation units from the mean
 The mean is the arithmetical
average. The standard
deviation is a measure of the
spread of scores in a group.
Types of standard Scores

Z – Score
If a mean and standard deviation
can be calculated for a given set
of raw scores, each raw scores
can be expressed in terms of its
distance from the mean in
standard deviation units or z –
scores.
 Z – Score =
Raw Score –Mean/Standard deviation

Note: Z – score is always minus

when the raw score is smaller than
the mean.
T Scores:
 any set of normally distributed
standard scores that has a
mean of 50 and SD of 10.
Multiplying the z – score by 10
and adding the product to 50 can
obtain T Scores.
Advantages
 only positive integers are provided

 Interpretation is relatively
simple once the concept of T –
Score is grasped.
GRADING
 Grading refers to the process of
using symbols, such as letter to
indicate various types of
students progress (Nitko 2001).
Common Methods of
Grading :
 Lettergrades :
There is a great flexibility in
the number of grades that can
be adopted i.e. 3 – 11.
Limitations:

 Meaning of grades may vary

widely

 Do not describe
strengths/weakness of
students
Strengths:

 Easy to use

 Easyto interpret
theoretically

 Provide a concise summary

 Number/Percentage grades

(5, 3, 2, 1, 0) or (98%, 80%, 60%

etc.)
It is same as letter grades. Only
difference is that instead of letters
numbers of percentage is used.
Strengths:

 Easy to use
 Easy to interpret theoretically
 Provide a concise summary
 May be combined with letter
grades
 More continuous than letter
grades
Limitations:

 Meaning of grades may vary

widely
 Do not describe
strengths/weaknesses of students
 Meaning may need to be
explained or interpreted.
Two category grades
 Itis good for courses that
require mastery of learning.
Strengths:
Less emotional for students.

Limitations :
Less reliable
Does not contain enough
information about student’s
achievement
Provides no indication of the level of
learning.
CHECK LIST AND RATING SCALE

 They are more detailed and

since they are too detailed it is
cumbersome for teachers to
prepare
Strengths
 Present
detailed lists of students’
achievements

 Canbe combined with letter

grades

 Good for clinical evaluation

Limitations:

May become too detailed to

easily comprehend

Difficult for record keeping.

Advantages of Grades

 Grades are divided in to 5 – 7

divisions to which student’s
performance is assigned as
compared to 101 (0 – 100)
divisions of conventional marking.
 It is a convenient method.
 Chances of errors are minimized
Disadvantages of Grades:

The assigned grades varies

from teacher to teacher
2. Do not indicate students
strengths or weaknesses
3. Foster unfair competition
among students.
REPORTING
Conducting Parent-Teacher
Conferences
The first conference is usually
arranged in the beginning of the
school year to allow parents and
teachers to get acquaintance and
preparing plan for the coming
months. Teachers usually receive
some training to plan and conduct
such conferences.
1. Prepare for the conference

Review the goals and objectives

Organize the information to present
If portfolios are to discuss, these are well-
arranged
Start and keep positive focus
Announce the final date and time as per
convenience of the parents and children
Consider socio-cultural barriers of students
/ parents
 Cont..

Check with other staff who

works your advisee
Develop a packet of conference
including student’s goals, samples
of work, and reports or notes from
other staff.
 2. Rehearse the conference
with students by role-playing
Students present their goals,
learning activities, samples of work
Students ask for comments and
suggestions from parents
 Do’s of parent-teacher
conferences
Be friendly
Be honest
Be positive in approach
Be willing to listen and explain
Be willing to accept parents’
feelings
Be careful about giving advice
Be professional and maintain a
positive attitude
 Cont..

Begin with student’s strengths

Review student’s cumulative
record prior to conference
Assemble samples of student’s
work
List questions to ask parents and
anticipate parents’ questions
Conclude the conference with an
overall summary
 Don’ts of the parent teacher
conference
Don’t argue
Don’t get angry
Don’t ask embarrassing
questions
Don’t reject parents’ suggestions
Don’t blame parents
Don’t talk too much; be a good
listener
Thank You for being
good listener