Session Overview - ISU Sites

1

Monitoring Student ProgressMonitoring Student Progressin Algebra:in Algebra:

A New Approach to AssessmentA New Approach to Assessment

Anne FoegenAnne FoegenJeannette OlsonJeannette OlsonJanice EdwardsJanice EdwardsApril 28, 2006April 28, 2006

Funded by U.S. Department of Education Award #HC324C030060Funded by U.S. Department of Education Award #HC324C030060

Session Overview Session Overview IntroductionsIntroductions Overview and progress monitoringOverview and progress monitoring Measures and proceduresMeasures and procedures A teacherA teacher’’s perspectivess perspectives Questions and discussionQuestions and discussion

2

Project AAIMSProject AAIMS 3 year project, funded through U.S.3 year project, funded through U.S.

Department of EducationDepartment of Education Focus on algebra instruction andFocus on algebra instruction and

assessment for students with and withoutassessment for students with and withoutdisabilitiesdisabilities

Development of progress monitoringDevelopment of progress monitoringmeasures for Algebra Imeasures for Algebra I

Collaborating with 3 Iowa school districtsCollaborating with 3 Iowa school districts

Project AAIMS WebsiteProject AAIMS Website

3

Project AAIMS WebsiteProject AAIMS Website

www.ci.hs.iastate.edu/aaimswww.ci.hs.iastate.edu/aaims Description of grant activitiesDescription of grant activities Technical reports on research studiesTechnical reports on research studies Sample copies of algebra probesSample copies of algebra probes Resources for algebra teachersResources for algebra teachers Contact information to reach usContact information to reach us

UnderstandingUnderstandingProgress MonitoringProgress Monitoring……

……and how it differs fromand how it differs fromtraditional approaches totraditional approaches to

classroom assessmentclassroom assessment

4

Contrasting Two Approaches toContrasting Two Approaches toClassroom AssessmentClassroom Assessment

Traditional assessment (masteryTraditional assessment (masterymonitoring)monitoring)

Progress monitoringProgress monitoring

Traditional orTraditional orMastery MeasurementMastery Measurement

Did the student learn what was taught in thisDid the student learn what was taught in thischapter/unit/skill sequence?chapter/unit/skill sequence?

Assessment materials mirror instructionalAssessment materials mirror instructionalmaterialsmaterials

Focus is SHORT-term; over a limited set ofFocus is SHORT-term; over a limited set ofproblems, concepts, or skillsproblems, concepts, or skills

This approach is used in both general andThis approach is used in both general andspecial educationspecial education

5

Hypothetical Algebra CurriculumHypothetical Algebra Curriculum1.1. Connections to algebraConnections to algebra2.2. Properties of real numbersProperties of real numbers3.3. Solving linear equationsSolving linear equations4.4. Graphing linear equations and functionsGraphing linear equations and functions5.5. Writing linear equationsWriting linear equations6.6. Solving and graphing linear inequalitiesSolving and graphing linear inequalities7.7. Systems of linear equations and inequalitiesSystems of linear equations and inequalities8.8. Exponents and exponential functionsExponents and exponential functions9.9. Quadratic equations and functionsQuadratic equations and functions10.10. Polynomials and factoringPolynomials and factoring11.11. Rational expressions and equationsRational expressions and equations12.12. Radicals and more connections to geometryRadicals and more connections to geometry


6


MaryMary’’s Chapter Test Scoress Chapter Test Scores

7

Potential Problems with thisPotential Problems with thisApproachApproach

Performance on limited-skill assessmentsPerformance on limited-skill assessmentscan be misleading.can be misleading.

Assessment does not reflect maintenance orAssessment does not reflect maintenance orgeneralization.generalization.

Assessment is designed by teachers or soldAssessment is designed by teachers or soldwith textbooks, with unknown reliability andwith textbooks, with unknown reliability andvalidity.validity.

Progress MonitoringProgress Monitoring Is the student learning the important conceptsIs the student learning the important concepts

and skills in this course/content area?and skills in this course/content area? Assessment materials range across topics/Assessment materials range across topics/

chapters/unitschapters/units Focus is LONG-term; entire yearFocus is LONG-term; entire year’’s curriculums curriculum

included in each assessmentincluded in each assessment Historically, this approach has been used moreHistorically, this approach has been used more

frequently in special educationfrequently in special education

8


Progress MonitoringProgress Monitoring Formative evaluationFormative evaluation Objective data to informObjective data to inform

instructional decision makinginstructional decision making Graphed data to facilitate visualGraphed data to facilitate visual

interpretationinterpretation Repeated measurement using briefRepeated measurement using brief

taskstasks

9

Mary's Math Progress

02468

10121416182022242628303234363840

Sep

t. 7

Sep

t. 9

Sep

t. 13

Sep

t. 20

Sep

t. 27

Oc.t 4

Oct. 11

Oc t. 18

Oct. 25

Nov

. 1

Nov

. 8

Nov

. 15

Nov

. 22

Dec

. 1

Dec

. 8

Dec

. 15

Nu

mb

er o

f P

ro

ble

ms C

orrect

in 3

Min

ute

s

Characteristics ofCharacteristics ofProgress Monitoring MeasuresProgress Monitoring Measures

Brief (2 to 10 minutes); consistent timingBrief (2 to 10 minutes); consistent timing Repeatable; constant difficultyRepeatable; constant difficulty Easy to administer and scoreEasy to administer and score Associated with other indicators of proficiencyAssociated with other indicators of proficiency

(teacher ratings, grades, standardized test(teacher ratings, grades, standardized testscores)scores)

Sensitive to changes in student performanceSensitive to changes in student performanceover timeover time

10

Project AAIMSProject AAIMSProgress Monitoring MeasuresProgress Monitoring Measures

Basic SkillsBasic Skills AA Algebra FoundationsAlgebra Foundations BB Content Analysis-Content Analysis- CC

Constructed Response Constructed Response TranslationsTranslations DD Content Analysis-Content Analysis- EE

Multiple Choice Multiple Choice

Basic Skills (A)Algebra Probe A-31 Page 1

Solve:

9 + a = 15

a =

Solve:

10 – 6 = g

g =

Evaluate:

12 + (– 8) + 3

Simplify:

9 – 4d + 2 + 7d

Simplify:

2x + 4 + 3x + 5

Simplify:

5(b – 3) – b

Solve:

12 – e = 4

e =

Solve:

q • 5 = 30

q =

Simplify:

4(3 + s) – 7

Evaluate:

8 – (– 6) – 4

Simplify:

b + b + 2b

Simplify:

2 + w(w – 5)

Solve:

18

12

6=

b

b =

Solve:

1 foot =12 inches

5 feet = ____ inches

Simplify:

7 – 3(f – 2)

Simplify:

4 – 7b + 5(b – 1)

Evaluate:

– 5 + (– 4) – 1

Simplify:

s + 2s – 4s

Solve:

63 ÷ c = 9

c =

Solve:

x + 4 = 7

x =

Simplify:

2(s – 1) + 4 + 5s

Simplify:

– 5(q + 3) + 9

Simplify:

8m – 9(m + 2)

Evaluate:

9 + (– 3) – 8

11

Form A: Basic Skills (in Algebra)Form A: Basic Skills (in Algebra) 60 items; 5 minutes60 items; 5 minutes Problems include:Problems include:

Solving basic fact equations;Solving basic fact equations; Applying the distributive property; Applying the distributive property; Working with integers; Working with integers; Combining like terms; Combining like terms; Simplifying expressions;Simplifying expressions; Applying proportional reasoningApplying proportional reasoning

Scoring: # of problems correctScoring: # of problems correct

Algebra Foundations (B)Algebra Probe B -1 Page 1

What is the slope?

What is the y

intercept?

Fill in the

empty box:

s 3s

6 18

7 21

8

9 27

Fill in the

empty box:

n 4n+7

1 11

2 15

3

4 23

Fill in the

empty box:

b

5 2

3 0

0 -3

-2 -5

What is the slope?

What is the y intercept?

If y > 9, two

possible values for

y are _____ and

_____

6 • 4 + 1 =

________

Simplify:

7f + (2f + f)

Solve:

n + 3 = 8

n = ________

Evaluate 4 b + 2

when

b = 1 ________

and when

b = 3 ________

Write the

expression for this

phrase:

6 less th an a

number

(-2) • ( -4) =

________

Graph the expression

m > 6

Write a word

phrase for this

expression:

n + 9

8 ÷ 2 + 4 • 3 =

________

23 = ________

Write the expression for

this phrase:

9 times a number

-8 -6 -4 -2 0 2 4 6 8

12

Form B: Algebra FoundationsForm B: Algebra Foundations 42 items (50 points); 5 minutes42 items (50 points); 5 minutes Problems represent five core concepts/skillsProblems represent five core concepts/skills

essential to conceptual understanding inessential to conceptual understanding inalgebraalgebra Writing and evaluating variables and expressionsWriting and evaluating variables and expressions Computing expression (integers, exponents, andComputing expression (integers, exponents, and

order of operations)order of operations) Graphing expressions and linear equationsGraphing expressions and linear equations Solving 1-step equations and simplifyingSolving 1-step equations and simplifying

expressionsexpressions Identifying and extending patterns in data tablesIdentifying and extending patterns in data tables

Scoring: # of problems correctScoring: # of problems correct

Content Analysis - Constructed Response (C)

13

Form C: Content Analysis-Form C: Content Analysis-Constructed ResponseConstructed Response

16 items; 7 minutes16 items; 7 minutes Problems are sampled from 2-3 coreProblems are sampled from 2-3 core

concepts in each chapter, with theconcepts in each chapter, with themiddle chapters represented in highermiddle chapters represented in higherproportion than the early/late chaptersproportion than the early/late chapters

Scoring: Items are worth from 1 to 6Scoring: Items are worth from 1 to 6points, depending on degree ofpoints, depending on degree ofdifficulty. Partial credit is possible.difficulty. Partial credit is possible.

Translations

14

Form D: TranslationsForm D: Translations

43 items; 7 minutes43 items; 7 minutes Problems are modeled on an NSF-fundedProblems are modeled on an NSF-funded

middle school curriculum in which algebra ismiddle school curriculum in which algebra istaught in a manner consistent with the NCTMtaught in a manner consistent with the NCTMstandardsstandards

Task requires students to match varyingTask requires students to match varyingrepresentations for relationships betweenrepresentations for relationships betweentwo variables: data tables, graphs, equations,two variables: data tables, graphs, equations,story scenariosstory scenarios

Scoring: # problems correct - # problemsScoring: # problems correct - # problemsincorrectincorrect

Content Analysis - Multiple Choice (E)

15

Form E: Content Analysis-Form E: Content Analysis-Multiple ChoiceMultiple Choice

16 items; 7 minutes16 items; 7 minutes Problems are sampled from 2-3 coreProblems are sampled from 2-3 core

concepts in each chapters 1-8concepts in each chapters 1-8 Must show work to obtain partial creditMust show work to obtain partial credit Scoring: Up to 3 points per problem, -1Scoring: Up to 3 points per problem, -1

point penalty for circling an incorrectpoint penalty for circling an incorrectanswer without showing any workanswer without showing any work(guessing)(guessing)

Consumer Report Version-Fall 05Consumer Report Version-Fall 05

Content Analysis-Content Analysis-MCMC

TranslationsTranslations

Content Analysis-CRContent Analysis-CR

Algebra FoundationsAlgebra Foundations

Basic SkillsBasic Skills

FeasibilityFeasibilityValidityValidityReliabilityReliabilityMean,Mean,RangeRange

16

Current StudyCurrent Study

2 districts using Basic Skills (A) probes2 districts using Basic Skills (A) probesand Content Analysis-Multiple Choiceand Content Analysis-Multiple Choice(E) probes for progress monitoring(E) probes for progress monitoring

1 district using Foundations (B) probes1 district using Foundations (B) probesand Content Analysis-Multiple Choiceand Content Analysis-Multiple Choice(E) probes for progress monitoring(E) probes for progress monitoring

Current StudyCurrent Study

Administer 2 versions of A or B probes midAdminister 2 versions of A or B probes midmonthmonth

Administer 2 versions of E probes at the endAdminister 2 versions of E probes at the endof the monthof the month

Teachers mail probes to us to be scored.Teachers mail probes to us to be scored. We send scored probes and spreadsheetsWe send scored probes and spreadsheets

back to teachers.back to teachers.

17

Our hopeOur hope……

These measures are designed so These measures are designed sostudents students ““growgrow”” enough in relatively enough in relativelyshort periods of timeshort periods of time

Data from these measures can be usedData from these measures can be usedto support teachersto support teachers’’ instructional instructionaldecision makingdecision making

Teacher ResponsibilitiesTeacher Responsibilities

Distribute & collect student/parentDistribute & collect student/parentconsent formsconsent forms

Administer probesAdminister probes Forward consent forms & probes toForward consent forms & probes to

AAIMS coordinatorAAIMS coordinator Share/Discuss probe scores withShare/Discuss probe scores with

studentsstudents

18

Analyzing StudentAnalyzing Student’’s Scoress Scores

Are students showing growth in their algebraAre students showing growth in their algebraknowledge?knowledge? Why not? Little effort or little increase inWhy not? Little effort or little increase in

algebra knowledge?algebra knowledge?

Compare probe scores to studentCompare probe scores to student’’s algebras algebragradegrade Course grade more reflective of study habits, yetCourse grade more reflective of study habits, yet

student showing growth of knowledge throughstudent showing growth of knowledge throughprobe scoresprobe scores

High Achieving StudentHigh Achieving Student

19

Average Achieving StudentAverage Achieving Student

Low Achieving StudentLow Achieving Student

20

Teacher ConcernsTeacher Concerns

Motivating students to put forth effort onMotivating students to put forth effort onprobesprobes

Self evaluation of Algebra knowledgeSelf evaluation of Algebra knowledge Extra CreditExtra Credit TreatsTreats Continually explaining purpose of probesContinually explaining purpose of probes

Teacher BenefitsTeacher Benefits

Identify individual studentsIdentify individual students’’ strengths strengthsand weaknessesand weaknesses

Identify class strengths andIdentify class strengths andweaknessesweaknesses

Determine retention of skills over timeDetermine retention of skills over time Evaluate special education students andEvaluate special education students and

their math goalstheir math goals

21

Student ConcernsStudent Concerns

Will probes scores effect my courseWill probes scores effect my coursegrade?grade?

““ProgressProgress”” testing testing

Student BenefitsStudent Benefits

Identify strengths and weaknessesIdentify strengths and weaknesses Check for improvement of scoresCheck for improvement of scores

22

Future DirectionsFuture Directionsfor Project AAIMSfor Project AAIMS

2006-07: Research on the influence of2006-07: Research on the influence ofteachersteachers’’ use of progress monitoring use of progress monitoringdata on student performancedata on student performance

To follow our progress, visit theTo follow our progress, visit theProject AAIMS website:Project AAIMS website:

www.ci.hs.iastate.edu/aaimswww.ci.hs.iastate.edu/aaims

Questions and DiscussionQuestions and Discussion

Session Overview - ISU Sites

Documents