Assessment

Assessment 1
 What is assessment?
 What is the function of assessment in teaching and
learning?
 What are the various methods of assessment?
 How does an instructor use assessment?
 How does an instructor create effective assessment
questions?
Assessment
 the ability to continuously analyze, appraise, and judge
a student’s performance
 process of gathering measurable information to meet
evaluation needs
 Most important instructor skill
 Grading
 Older term for assessment
 Measure of learning progress needed by both student
and instructor
Assessment 2
 Categories of assessment
 Traditional
 Written testing
 Rote and understanding
 Authentic
 Application and correlation
 Demonstrate skill or ability to solve real world problem
 Use of criteria or standards
 Rubric

 Formal
 Written or documented
 Informal
 Oral
Assessment 3
 Diagnostic assessments
 assess student knowledge or skills prior to a course of
instruction.
 Formative assessments
 wrap-up of the lesson and to set the stage for the next
lesson
 Summative assessments
 used periodically throughout the training to measure
how well learning has progressed to that point.
 chapter quiz, end-of-course test
Characteristics of Effective
Assessment
 Objective
 Flexible
 Acceptable
 Comprehensive
 Constructive
 Organized
 Thoughtful
 Specific
Objective
 The effective assessment is objective, and focused on
student performance. It should not reflect the
personal opinions, likes, dislikes, or biases of the
instructor. Instructors must not permit judgment of
student performance to be influenced by their
personal views of the student, favorable or
unfavorable.
Flexible
 Sometimes a good student turns in a poor
performance, and a poor student turns in a good one.
A friendly student may suddenly become hostile, or a
hostile student may suddenly become friendly and
cooperative. The instructor must fit the tone,
technique, and content of the assessment to the
occasion, as well as to the student.
 The ongoing challenge for the instructor is deciding
what to say, what to omit, what to stress, and what to
minimize at the proper moment.
Acceptable
 The student must accept the instructor in order to
accept his or her assessment willingly. Students must
have confidence in the instructor’s qualifications,
teaching ability, sincerity, competence, and authority.
Usually, instructors have the opportunity to establish
themselves with students before the formal
assessment arises. If not, however, the instructor’s
manner, attitude, and familiarity with the subject at
hand must serve this purpose.
 Instructors must not rely on position(title) to make
assessment acceptable to student
Comprehensive
 The instructor must decide whether the greater
benefit comes from a discussion of a few major points
or a number of minor points. The instructor might
assess what most needs improvement, or only what the
student can reasonably be expected to improve. An
effective assessment covers strengths as well as
weaknesses. The instructor’s task is to determine how
to balance the two.
Constructive
 An assessment is pointless unless the student benefits
from it. Praise for its own sake is of no value, but praise
can be very effective in reinforcing and capitalizing on
things that are done well, in order to inspire the
student to improve in areas of lesser accomplishment.
When identifying a mistake or weakness, the
instructor must give positive guidance for correction.
Organized
 An assessment must be organized. Almost any pattern
is acceptable, as long as it is logical and makes sense to
the student.
Thoughtful
 The instructor must not minimize the inherent dignity
and importance of the individual. Ridicule, anger, or
fun at the expense of the student never has a place in
assessment. While being straightforward and honest,
the instructor should always respect the student’s
personal feelings. For example, the instructor should
try to deliver criticism in private.
Specific
 The instructor’s comments and recommendations
should be specific. Students cannot act on
recommendations unless they know specifically what
the recommendations are. A statement such as, “Your
second weld wasn’t as good as your first,” has little
constructive value. Instead, the instructor should say
why it was not as good, and offer suggestions on how
to improve the weld. If the instructor has a clear, well-
founded, and supportable idea in mind, it should be
expressed with firmness and authority, and in terms
that cannot be misunderstood.
Traditional Assessment
 As defined earlier, traditional assessment generally refers to
written testing, such as multiple choice, matching,
true/false, fill in the blank, etc. Written assessments must
typically be completed within a specific amount of time.
There is a single, correct response for each item. The
assessment, or test, assumes that all students should learn
the same thing, and relies on rote memorization of facts.
Responses are often machine scored, and offer little
opportunity for a demonstration of the thought processes
characteristic of critical thinking skills.
 Useful in assessing student factual knowledge.
Characteristics of Good Test
 Reliability
 Validity
 Usability
 Objectivity
 Comprehensiveness
 Discrimination
Reliability
 Reliability is the degree to which test results are
consistent with repeated measurements. If identical
measurements are obtained every time a certain
instrument is applied to a certain dimension, the
instrument is considered reliable. The reliability of a
written test is judged by whether it gives consistent
measurement to a particular individual or group.
Validity
 Validity is the extent to which a test measures what it
is supposed to measure, and it is the most important
consideration in test evaluation.
 To estimate validity, several instructors read the test
critically and consider its content relative to the stated
objectives of the instruction. Items that do not pertain
directly to the objectives of the course should be
modified or eliminated.
Usability
 Usability refers to the functionality of tests. A usable
written test is easy to give if it is printed in a type size
large enough for students to read easily.
Objectivity
 Objectivity describes singleness of scoring of a test.
Essay questions provide an example of this principle. It
is nearly impossible to prevent an instructor’s own
knowledge and experience in the subject area, writing
style, or grammar from affecting the grade awarded.
Selection-type test items, such as true/false or multiple
choice, are much easier to grade objectively.
Comprehensiveness
 Comprehensiveness is the degree to which a test
measures the overall objectives.
 The instructor has to make certain the evaluation
includes a representative and comprehensive sampling
of the objectives of the course.
Discrimination
 Discrimination is the degree to which a test
distinguishes the difference between students. In
classroom evaluation, a test must measure small
differences in achievement in relation to the objectives
of the course.
 Should have a wide range of scores and all levels of
difficulty.
Authentic Assessment
 Authentic assessment is a type of assessment in which
the student is asked to perform real-world tasks, and
demonstrate a meaningful application of skills and
competencies. Authentic assessment lies at the heart
of training today’s aviation student to use critical
thinking skills. Rather than selecting from
predetermined responses, students must generate
responses from skills and concepts they have learned.
Collaborative Assessment
Four Step Series:
 Replay
 Reconstruct
 Reflect
 Redirect
Replay
 ask the student to verbally replay the flight or
procedure. Listen for areas in which the instructor’s
perceptions differ from the student’s perceptions, and
discuss why they do not match.
Reconstruct
 the reconstruction stage encourages the student to
learn by identifying the key things that he or she
would have, could have, or should have done
differently during the flight or procedure.
Reflect
 Insights from questions:
 What was the most important thing you learned today?
 What part of the session was easiest for you? What part
was hardest?
 Did anything make you uncomfortable? If so, when did
it occur?
 How would you assess your performance and your
decisions?
 Did you perform in accordance with the PTS?
Redirect
 Correlation with other learning:
 How does this experience relate to previous lessons?
 What might be done to mitigate a similar risk in a future
situation?
 Which aspects of this experience might apply to future
situations, and how?
 What personal minimums should be established, and
what additional proficiency flying and/or training might
be useful?
Self-assessment
 The purpose of the self-assessment is to stimulate growth
in the student’s thought processes and, in turn, behaviors.
The self-assessment is followed by an in-depth discussion
between the instructor and the student, which compares
the instructor’s assessment to the student’s self-
assessment.
 Uses a rubric
 guide for scoring performance assessments in a reliable, fair,
and valid manner
 Rubrics in collaborative assessment
 Maneuver or Procedure “Grades”
 Single-Pilot Resource Management (SRM)
Maneuver or Procedure Grades
 Describe
 Explain
 Practice
 Perform
 Not observed
Describe
 At the completion of the scenario, the student is able
to describe the physical characteristics and cognitive
elements of the scenario activities, but needs
assistance to execute the maneuver or procedure
successfully.
Explain
 At the completion of the scenario, the student is able
to describe the scenario activity and understand the
underlying concepts, principles, and procedures that
comprise the activity, but needs assistance to execute
the maneuver or procedure successfully.
Practice
 At the completion of the scenario, the student is able
to plan and execute the scenario. Coaching,
instruction, and/or assistance will correct deviations
and errors identified by the instructor.
Perform
 At the completion of the scenario, the student is able
to perform the activity without instructor assistance.
The student will identify and correct errors and
deviations in an expeditious manner. At no time will
the successful completion of the activity be in doubt.
Not Described
 Any event not accomplished or required.
Single-Pilot Resource
Management (SRM) Grades
 Explain
 Practice
 Manage-Decide
Explain
 student can verbally identify, describe, and understand
the risks inherent in the flight scenario,
 but needs to be prompted to identify risks and make
decisions.
Practice
 student is able to identify, understand, and apply SRM
principles to the actual flight situation.
 Coaching, instruction, and/or assistance quickly
corrects minor deviations and errors identified by the
instructor.
 The student is an active decision maker.
Manage-Decide
 the student can correctly gather the most important
data available both inside and outside the flight deck,
identify possible courses of action, evaluate the risk
inherent in each course of action, and make the
appropriate decision.
 Instructor intervention is not required for the safe
completion of the flight
How to use?
 Post flight brief
 Replay-reconstruct-reflect-redirect
 Check performance against rubric
 Is it harder than traditional assessment?
 Just more structured version of postflight critique
 Danger of fitting traditional grading into rubric
 Goal is progressive learning
 Student may have to be taught collaborative
assessment
 Not as useful in early stages of training
Choosing Effective
Assessment Method
 When deciding how to assess student progress,
aviation instructors can follow a four-step process.
 Steps
 determine level-of-learning objectives
 list indicators of desired behaviors
 establish criterion objectives
 develop criterion-referenced test items
Critique and Oral Assessments
 Used in conjunction with either traditional or
authentic assessment, the critique is an instructor-to-
student assessment. These methods can also be used
either individually, or in a classroom setting.
Ways of Conducting Critique
 Student-led
 Generates interest but may be inefficient
 Small Group
 Instructor should furnish criteria and guidelines
 Individual Student Critique by Another Student
 Instructor must maintain control
 Self-critique
 Written critique
 Instructor can spend more time on it
 Students can keep and refer to it
 Permanent record
Oral Assessment
 The most common means of assessment is direct or
indirect oral questioning of students by the instructor.
Questions may be loosely classified as fact questions
and HOTS questions. The answer to a fact question is
based on memory or recall. This type of question
usually concerns who, what, when, and where. HOTS
questions involve why or how, and require the student
to combine knowledge of facts with an ability to
analyze situations, solve problems, and arrive at
conclusions.
Desirable results of oral
assessment
 Reveals the effectiveness of the instructor’s training
methods
 Checks student retention of what has been learned
 Reviews material already presented to the student
 Can be used to retain student interest and stimulate
thinking
 Emphasizes the important points of training
 Identifies points that need more emphasis
 Checks student comprehension of what has been learned
 Promotes active student participation, which is important
to effective learning
Characteristic of Effective
Questions
 Apply to the subject of instruction.
 Be brief and concise, but also clear and definite.
 Be adapted to the ability, experience, and stage of
training of the students.
 Center on only one idea (limited to who, what, when,
where, how, or why, not a combination).
 Present a challenge to the students
Types of Questions to Avoid
 Puzzle
 Oversize
 Toss-up
 Bewilderment
 Trick questions
 Irrelevant questions
Answering Questions from
Students
 Be sure that you clearly understand the question
before attempting to answer.
 Display interest in the student’s question and frame an
answer that is as direct and accurate as possible.
 After responding, determine whether or not the
student is satisfied with the answer.
 Question is too early at stage of training
 If you don’t know, admit it and promise to find out.