For Factor 6: Analysis of Assessment Procedures TWS Aid TWS Aid
For Factor 6: Analysis of Assessment Procedures
TWS AidTWS Aid
MajorMajor Section Requirements:Section Requirements:
1. Graph comparing pre and post assessments2. Worksheet showing Learning Gain Scores
(LGS) for each student3. Learning Achievement Table depicting % of
students achieving “mastery level” for each objective
4. Text/Graph/Chart analyzing how subgroups performed relative to each objective
Examples of Objectives Classified Low Level of Difficulty
Cognitive: Knowledge & Comprehension (Bloom); Knowledge (Sternberg)Affective: Receiving & Responding toPsychomotor: Reflex movements & Fundamental movements
In a given music example, the student should be able to identify all of the basic elements of music (cognitive)The student will report two likes and two dislikes as a response to the assigned reading (affective)The student performs a tennis serve with correct form (psychomotor)
Examples of ObjectivesClassified Middle Level of Difficulty
Cognitive: Application & Analysis (Bloom); Skills/Performance/Applications (Stenberg)
Affective: Valuing & organizing of valuesPsychomotor: Perceptual abilities & Physical abilities
After watching a video on scientific theories the student will list at least one difference between the use of the term “theory” in science and the use of “theory” in non-science contexts (cognitive)During a debate, the student will defend the right of scientists to conduct research in a three-minute statement (affective)Within the time allowed the student will adjust a microscope so that the image is clear (psychomotor)
Examples of ObjectivesClassified High Level of Difficulty
Cognitive: Synthesis & Evaluation (Bloom); Reasoning Ability (Stenberg)Affective: Internalizing valuesPsychomotor: Skilled movements & Nondiscursive communication
Given a map with six distinct geographical features, students will be able to evaluate the best location for building a new city (cognitive)Student members of jazz band will perform solo improvisations in Count Basie style for One O’Clock Jump (affective)The student accurately performs a tennis serve, including correct spin, speed and placement of the ball in the opposite side of the court (psychomotor)
Student 1 Student 2 Student 3 Student 4 Student 5 Student 6 Student 7
100%
80%
60%
40%
20%
0%
Graph for Pre/Post Assessment Data for Each Graph for Pre/Post Assessment Data for Each StudentStudent
Pre-Assessment Score
Post-Assessment Score
This example graph shows a comparison of the pre & post-assessment data for each student.
Concept of Gain Scores
A gain score is the actual gain divided by the potential gain.
GAIN SCORE = ACTUAL GAIN
POTENTIAL GAIN
.82
82%What does it mean?What does it mean?
Gain Score Worksheet
Pre Post GainStudent # Score Score Score
1 40 90 .832 90 95 .503 40 804 30 855 85 88 .206 75 100 +1.007 60 90 .758 40 95 .929 40 40 .0010 75 50 -1.0011 45 70 .4512 60 80 .5013 40 95 .92
SolveSolve
GAIN SCOREUsing the formula, Using the formula, calculate the calculate the missing gain scores.missing gain scores.
Gain Score Worksheet
Pre Post GainStudent # Score Score Score
1 40 90 .832 90 95 .503 40 80 .67.674 30 85 .79.795 85 88 .206 75 100 +1.007 60 90 .758 40 95 .929 40 40 .0010 75 50 -1.0011 45 70 .4512 60 80 .5013 40 95 .92
Include the average of the gain scores.
“ 50%” Average Learning Gain
GAIN SCORE
Solutions:80 – 40
100 – 404060
= .67=
85 – 30100 – 30
5570
.79==
Group Average .50
Negative Gain Scores
Example:Pre-Assessment = 75 %Post-Assessment = 50 %
Formula: Post-Assess % - Pre-Assess % 100% - Pre-Assess %
What happens when a student scores lower on the post-assessment than on the pre-assessment?
-25 25
50 - 75100 - 75
This student could have gained up to 25 points, but instead lost 25 This student could have gained up to 25 points, but instead lost 25 points, a value equaling 100 % of what could have been gained.points, a value equaling 100 % of what could have been gained.
= = = - 1.00-2525
Gain Scores for Pre-Assessments of Gain Scores for Pre-Assessments of 100%100%
What happens when a student scores 100% on the pre-assessment?
When a student scores 100% on the pre-assessment, that score must be changed to 99%. Otherwise, the potential gain will equal zero and the gain score will be undefined.
Changing the student’s pre-assessment score to 99% will make the potential gain equal 1. Then, using 1 as the denominator (instead of 0) will allow the score to be defined.
Actual Gain
0
Actual Gain Potential Gain
Gain Score == undefined
If the potential gain equals zero, then
Explanation:
Remember:
Other Points About Gain Scores
Gain scores are not a perfect measure
The student teacher/intern should include more than just assessment of knowledge in the assessment plan and gain score calculations
The student teacher/intern will not being evaluated or judged on the amount of gain produced
The Learning Achievement The Learning Achievement TableTablePurpose: To identify the percentage of students who achieved a
minimum mastery level for EACH learning objective.
All assessment data (tests, performances, portfolios, formal and informal questioning, checklists, rubrics, etc.) could be used in this table to calculate percentages.
Required in order to capture some forms of performance assessment that are criterion (given once at the end) assessments.
The student teacher/intern sets the criteria for what is considered mastery.
The student teacher/intern is not going to be evaluated by degree of mastery.
Example of a Learning Achievement Example of a Learning Achievement TableTable
Objective Percent of Students Achieving Mastery
Lists 12 reasons for the occurrence of the Industrial Revolution
88%
Integrates technology throughout presentation
78%
Total Average Mastery 83%
Setting Mastery Levels
Example for: Knowledge Objective & Assessment TypeExample for: Knowledge Objective & Assessment TypeObjectiveObjective: List 12 reasons for the Industrial RevolutionAssessmentAssessment: Fill in the blank – paper and pencil testMastery LevelMastery Level: 9 out of 12 is considered mastery
Example for: a Performance Objective & Assessment TypeExample for: a Performance Objective & Assessment TypeObjectiveObjective: Student integrates technology throughout presentationAssessmentAssessment: Student presentations at the end of the unitMastery LevelMastery Level: Score of a 3 on a 1-3 rubric factor for technology (see
rubric)
Mastery should not be set too high or too low.Mastery should not be set too high or too low. The student teacher/intern needs to be able to examine The student teacher/intern needs to be able to examine
each student’s performance for each objective.each student’s performance for each objective.
Sub-Group ComparisonsSub-Group Comparisons
Purpose: To provide evidence that the student teacher/intern can deal effectively with students with various needs and from various backgrounds.
The student teacher/intern picks the groups of individuals
Could use SES, ESL/Non-ESL, race/ethnicity, disabilities, achievement levels, and gender as potential comparisons
Comparisons can be depicted: in text using a narrative while referring to gain scores and objective mastery % with a chart or graph (example on next slide)
Example Graph for Sub-Group Comparison
100%
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
ESL,84%
Non-ESL,86%
Could also useGain Scores here.
Perc
en
tag
e