Using Student Data as a Basis for Feedback to Teachers Ronnie Detrich Wing Institute Cal-ABA, 2011.

Using Student Data as a Basis for Feedback to Teachers

Ronnie DetrichWing Institute

Cal-ABA, 2011

Everybody’s Talking

• Teacher accountability often offered as solution for education troubles.

• Numerous proposals to pay and evaluate teachers based on student performance.o Proposals usually linked to performance on high stakes

tests. Very weak means of influencing teacher behavior.

Difficulties with High Stakes Tests as Basis for Feedback

• Low frequencyo Once per year.

• Long delay between behavior and feedback.o Test in spring, feedback following fall.

• No direct link between teacher behavior and feedback.o Feedback is about student performance.o Does not specify what teachers do to influence student

scores.

Alternatives to High Stakes Tests

• Formative assessmento Curriculum-based measuremento Direct measures of behavior.

• Characteristicso Frequent measuremento Easy and briefo Reliable and valid.

Why Feedback to Teachers is Important Hattie (2009)

Function of Feedback Systems

• Assure that all students are benefitting from educational services.

• Student performance primary source of feedback to educators.o There are constraints.

What Should be Targets for Feedback

• Logistically difficult to provide feedback on all aspects of teacher behavior.

• Student performance influenced by variables not under teacher control:o Curriculum-usually selected at another level.

If curriculum is ineffective then inappropriate to hold teacher accountable.

Effectiveness of curriculum can be measured by student performance but only implemented with integrity.

• How well educators implement an intervention is primary target for feedback.

Mortenson & Witt, 1998

Importance of Treatment of Integrity

• Students cannot benefit from interventions they do not experience.

• Unless educator knows what they are supposed to be doing they cannot do more of it.

Dimensions of Treatment Integrity(Dane & Schneider,

1998)

• Exposure (Dosage): the extent to which participants are exposed to the intervention as prescribed.o Curricula usually prescribe frequency and duration of

exposure that is necessary for benefit. Ex: 3/week for 30 minutes/session.

o Failing to satisfy either can impact student benefit.Ex: 1/week for 30 minutes.

Dimensions of Treatment Integrity(Dane & Schneider,

1998)

• Adherence: the extent to which the components of an intervention are delivered as prescribed.o Most common measured dimension.o It is necessary but not sufficient to produce benefits.

Adherence with low dosage not likely to produce positive outcomes.

Dimensions of Treatment Integrity(Dane & Schneider,

1998)

• Quality of delivery: qualitative measure of how well the intervention is implemented.o Applied behavior analysts acknowledge importance.

Have not developed good measures.

o Possible measures through social validity methods: EnthusiasmSincerity Variations in inflection and content of speech.

The Role of Student Data?

• Student data provides feedback about progress.• If we know about adequacy of treatment integrity

then can make decisions:o Adequacy of interventiono Adequacy of implementation

If implementation is inadequate then focus should be on improving educator behavior.

If implementation is adequate then focus should be on changing intervention so student can succeed.

Decisions can be made about increasing or decreasing intensity of intervention.

Grade Level Standard

Aim Line

Trend Line

Visual Aids Can Enhance Data Interpretation

Taking Treatment Integrity Measurement to Scale

• Insufficient resources to frequently observe all educators working with students to determine adequacy of implementation.

• One solution: Measure student behavior and analyze at different units of analysis:o Schoolo Classroomo Individual student

• Any level there is gap between obtained and expected performance assess levels of treatment integrity.

Taking Treatment Integrity Measurement to Scale

• Requires that data are routinely collected across all levels.o RtI:

CBM on all students 3/year.

o PBSContinuous measurement of Office Discipline Referrals School-wide Evaluation Tool (SET) measures treatment integrity

at the school level. Schools with scores higher than 80% greater impact on social

behavior than schools that do not exceed 80%.

Implications of Providing Feedback to Educators

• Moves sole responsibility for student outcomes from teacher to broader system.

• If teachers are to effectively implement:o System selects effective interventions for teachers to

implement. o Assure that implementers are actually trained.o System must monitor teachers implementation of

intervention.o Assure that resources are to present to implement

effectively.

Thank you

Copies of all presentations may be downloaded atwinginsitute.org