Principal(les) for Data Collection: Important Ingredients for Successful and Targeted Interventions Richard P. West, Ph.D. Terry Humphreys, M.S. Tim G.

Principal(les) for Data Collection: Important

Ingredients for Successful and Targeted Interventions

Richard P. West, Ph.D.

Terry Humphreys, M.S.

Tim G. Smith, M.S.

Matthew J. Taylor Ph.D.Utah State University and Cache County School District

http://www.csf.usu.edu

If Schools Are To Achieve All They Can,

They Will Need…• Better information about what

works (Best Practices)• Tools for monitoring progress• Tailored assistance in developing

and implementing appropriate policy

• More skillful communication and more public involvement in reform

Education Commission of the States, 1998

• Better schools result from better decisions, and better decisions result from better data

• Sustained improvement in academic achievement requires changes in the school environment

• An ethic of collegiality and cooperation is necessary to bring about meaningful school reform

PR

INC

IPLE

S

Indicators of School QualityMonitoring the School Environment

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

Exte

rnal

Vari

ab

les

Inte

rnal

Vari

ab

les

Web of Causation for Academic Achievement

Instruction

AcademicAchievement

Web of Causation for Social Competence

Punishment

Social Competence

Natural selection ofmetabolic adaptation

to starvation

Socialpressures

Industrialsociety

Hereditaryfactors

Dietary excesses insaturated fat, cholesterol,

calories, saltObesity

Personality &emotional

stress

Cigarettesmoking

Lack ofexercise

Coronaryartery

distribution

Diabetes orcarbohydrateintolerance

Hyperlipidemia Hypertension Increasedcatecholamines

Thrombotictendency

Significantcoronary

atherosclerosis

Myocardialsusceptibility

Deficiency incollateralcirculation

Coronaryocclusion

Myocardialinfarction

The authors note that “Despitethe apparent complexity of thisdiagram, it is undoubtedly anoversimplification and willcertainly be modified by further study.” (p. 5).

Web of Causation for Myocardial Infarction (Heart Attacks)

Taken from Friedman, G. D. (1994).Primer of Epidemiology (5th Ed.).New York: McGraw-Hill, p.4.

The Indicators of School Quality

• Parent Support

• Teacher Excellence

• Instructional Quality

• Administration

• Student Commitment

• Safety

• Resources

Areas of Risk

1. Home Language “Is English the primary language spoken at home?”

2. Mobility “Have you moved more than once in the past three years?”

3. Peer Associations “Do you generally approve of your child’s closest friends?”

4. Family Bonding “Do your neighbors generally monitor their children’s activities?”

5. Community Affiliation “Do you regularly attend community, social, or religious meetings?”

6. Academic Risk “Do you have a high school diploma/GED?”

7. Economic Risk “Do you have Internet access at home?”

ISQ and Academic Achievement

• The variables measured by ISQ account for more than 80% of the variance of academic achievement scores

• Even when “risk” is removed from the equation, the correlations between ISQ variables and achievement are statistically significant

Hierarchy of Risk• Economic Status• Community Affiliation• Family Bonding• Mobility• Academic Status• Home Language• Peer Acceptance

% of them that is also at-risk for…ES CA FB M AS HL PA

Economic Status ----- 60 45 25 8 7 2Community Affiliation 85 ----- 61 35 11 10 3Family Bonding 96 91 ----- 39 17 15 4Mobility 94 90 68 ----- 29 26 7Academic Status 100 100 100 100 ----- 89 22Home Language 100 100 100 100 100 ----- 25

Ofschoolsfoundto be

at-riskfor…

Peer Acceptance 100 100 100 100 100 100 -----

At-Risk Schools

A Tale of Two Schools

"It was the best of times, it was the worst of times.”

Universal

All Students

Targeted

At-Risk Students

Clear Communication of Behavioral Expectations

Bobcat Pride(Rules, Values, Common Language)

Administrative Intervention(Instructions

Individual NegotiationsContracts)

Relationships and Bonding System-wide Advisement

Extra-Curricular ProgramsMentoring

Relationship-building

Teaching Emphasis•Academic Skills•Social Skills•Self-management Skills

Teaching Social SkillsBehavior Modeling

(Expectations, Modeling, Practice, Fluency, Evaluation)

Intensive Teaching(Planned

AndOpportunistic

Teaching)

Recognition for Appropriate Behavior

Bobcat 200(Praise Notes/Boards

Recognition ProgramsGood Behavior Game)

Increase)

Instructive Praise

Bobcat Tracks

Office Referrals

2551

19841824

0

500

1000

1500

2000

2500

3000

2001/2002 2002/2003 2003/2004

Office Referrals OCTOBER

312

267

224

123

0

50

100

150

200

250

300

350

2001 2002 2003 2004

Total Level 1 Violations by Groups

At-Risk students, in this case, are identified as having 5 or more violations MARCH 2004

6%

59%

25%

41%

Level 1 Violations by At-Risk Groups

142

6

236

18

278

43

656

67

0

100

200

300

400

500

600

700

TwentyPlus

Ten Plus Five Plus Total

TotalIndividuals

MARCH 2004

Violations by LocationBus2%

Cafeteria3%

Classroom (excluding

tardies)67%

Commons1%

Hallways19%

Outside8%

Universal

All Students

Targeted

At-Risk Students

Clear Communication of Behavioral Expectations

Bobcat Pride(Rules, Values, Common Language)

Administrative Intervention(Instructions

Individual NegotiationsContracts)

Relationships and Bonding System-wide Advisement

Extra-Curricular ProgramsMentoring

Relationship-building

Teaching Emphasis•Academic Skills•Social Skills•Self-management Skills

Teaching Social SkillsBehavior Modeling

(Expectations, Modeling, Practice, Fluency, Evaluation)

Intensive Teaching(Planned

AndOpportunistic

Teaching)

Recognition for Appropriate Behavior

Bobcat 200(Praise Notes/Boards

Recognition ProgramsGood Behavior Game)

Increase)

Instructive Praise

Bobcat Tracks

The Future of ISQ

• Indicators of PBS

• Domain and PBS Checklists

Elementary Students

• Do your teachers always give clear instructions?

• Are you often confused about how to behave at school?

• Do you like to read?

• Do your teachers tell you when you do well?

Secondary Students

• Do all of your teachers generally give clear instructions?

• Is there an adult at this school who you can approach for help?

• Are you frequently confused about what is expected of you at school?

• Would you know where to get help if you fell behind in your schoolwork?

• Have you been recognized individually in the last school week for behaving well?

Staff

• Do you post clearly stated expectations for behavior in your classroom?

• Do teachers regularly encourage students to come to them for extra help?

• Is there a coordinated effort by all school staff to teach appropriate social skills?

• Do all of your students know where to get help to catch up academically?

• Are you encouraged by the administration to recognize positive student behaviors?

Checklist of Contextual FactorsChecklist of Contextual Factors

1. A well-written set of behavioral standards and expectations exists at this school

2. The set of expectations is short (generally from 5 to 7 items)3. Students were involved in the development, refinement, and

communication of the standards of behavior4. The behavioral expectations are statements of how to behave

well, rather than what not to do5. Behavioral expectations are posted prominently throughout the

school6. Behavioral expectations are emphasized in each classroom (e.g.

explicitly taught, reminded, and encouraged)7. Students are able to remember and repeat statements of

behavioral expectations

Clear Communication of Expectations for Performance

Adapted from G. Roy Mayer (2001) California State University,Los Angeles

Checklist of Contextual FactorsChecklist of Contextual Factors

8. Strong administrative support for staff exists (e.g. good teaching is recognized, faculty requests are acted upon promptly)

9. Strong staff support for one another exists (e.g. staff confer with one another regarding instruction and discipline)

10. Staff greet and help students feel welcome in the classroom11. Staff interact with and show interest in students in various settings12. Staff have many more positive than negative interactions with

students13. Students generally comply willingly with staff requests and

instructions14. Students tend to “hang around” staff, engaging in conversations, etc.15. Staff are really well acquainted with each and every student, and are

familiar with students’ personal characteristics, attributes, and challenges

Relationships and Bonding

Adapted from G. Roy Mayer (2001) California State University,Los Angeles

Checklist of Contextual FactorsChecklist of Contextual Factors

16. The school assumes responsibility for learning of academic skills17. Curriculum in all areas is organized to emphasize active rather

than passive responding, with many tailored opportunities for all students to respond

18. Academic assignments are adjusted to students’ functional levels19. Sufficient additional academic support is provided to struggling

students20. The school assumes responsibility for learning of social skills21. Social skills are identified and taught effectively emphasizing

fluency and generalized performance in natural settings22. Failure to meet high expectations of performance is followed by

individual intensive teaching rather than punishment23. Students receive explicit instruction and support in self-

management

Skill-Building Emphasis: Academic, Social, and Self-Management Skills

Adapted from G. Roy Mayer (2001) California State University,Los Angeles

Checklist of Contextual FactorsChecklist of Contextual Factors

24. Recognition is provided by the administration to students who meet the behavioral expectations

25. Recognition is provided by classroom teachers to students who meet the behavioral expectations

26. All students receive frequent and appropriate recognition for their accomplishments and efforts to meet high standards of good behavior

27. At-Risk students receive more frequent and personalized (tailored) recognition for their efforts to meet high standards and expectations (in both academic and deportment)

28. Evidences exist in this school of efforts to pay more attention to good behavior and success than to problem behavior and mistakes

Recognition of Appropriate Behavior

Adapted from G. Roy Mayer (2001), California State University,Los Angeles

CONTEXTUAL FACTORSCONTEXTUAL FACTORS

“It appears that changing these identified contextual factors not only can help prevent antisocial behavior, but also can help to create an environment more

conducive to learning”G. Roy Mayer (2001)

California State University,Los Angeles

Nine Contextual Factors that Contribute to Punitive School Environments and Promote

Antisocial Behavior

• Low student involvement in school activities• Unclear rules for student deportment• Weak or inconsistent administrative support• Student academic failure• Student deficiency in social & personal management skills• Problems discriminating prosocial & antisocial behavior• Consequences delivered inconsistently• Inadvertent reinforcement of antisocial behavior• Over reliance on punitive methods of control (Mayer, 1995; Similar to home-based contextual

factors noted by Loeber, Stouthammer-Loeber & Green, 1987 and Reid & Patterson, 1991)