POSITIVE CLASSROOMS, POSITIVE CHILDREN Evaluating the Incredible Years Teacher Classroom Management programme in Ireland Dr Sinead McGilloway Principal.

‘POSITIVE CLASSROOMS, POSITIVE CHILDREN’ Evaluating the Incredible Years

Teacher Classroom Management programme in Ireland

Dr Sinead McGillowayPrincipal Investigator

Mental Health and Social Research UnitDepartment of Psychology NUI Maynooth

Overview

The Incredible Years Ireland Study: our role and who we are

Study background:

- EBD in the classroom

- research objectives/questions

- study design

Outline of key findings

Conclusion

The research evaluation: Our role

To undertake a 4-year national community-based evaluation of the Incredible Years programme in Ireland:

- Parent (BASIC) training

- Teacher Classroom Management training**

- Combined parent and small group (Dina) training

To work collaboratively with Archways and other community-based organisations and schools in executing the research

To generate high quality findings that will impact on policy and practice in Ireland (and elsewhere) -> improve the lives of children with EBD and their families

The Research Team

NUI Maynooth Dr Sinead McGilloway

Lynda Hyland

Dr Grainne NiMhaille

Yvonne Leckey

Paul Kelly

Prof Donal O’Neill

Mairead Furlong

Church of Ireland College of Education

Dr Anne Lodge

Bangor University, Wales Dr Tracey Bywater

Trinity College Dublin Prof Catherine Comiskey

Queen’s University Belfast Dr Michael Donnelly

EBD in the classroom

Child conduct problems in school academic under-achievement, poor attendance and early school leaving

poor mental health, juvenile delinquency

Impact on the teacher’s ability to teach the class effectively Negative impact on classroom environment and the

teacher-child relationship

Can affect teacher job satisfaction, stress and likelihood of burnout

DES survey (2006): 32% of trainee teachers had fair or ineffective classroom management skills

The Incredible Years TCM programme

Social and behavioural learning principles Help teachers promote pro-social behaviour Reduce undesirable and aggressive behaviour Establish more positive relationships with pupils and

facilitate peer-to-peer bonding Encourage teachers to collaborate with parents and

promote parent involvement in school (Webster-Stratton, 2005)

The intervention A brief group-based programme Five one-day sessions (each month) Videotape modelling, role plays and discussions

Research questions

1. Does the IY TCM programme improve classroom management competencies and provide skills to better manage challenging behaviour?

2. To what extent does the TCM programme improve child behaviour in the classroom?

3. How do outcomes for teachers and children change over time?

4. What are the experiences of teachers and which factors facilitate or inhibit the effective implementation of the programme?

5. How cost-effective is the programme?

Research Design

Three separate studies:

1. Main study: A clustered Randomised Control Trial (RCT) design (intervention and control group)

2. A smaller qualitative study assessing experiences and views of teachers who took part in the RCT

3. A cost analysis to assess the cost-effectiveness of the programme

Method1. RCT 22 teachers from 11 schools in Limerick area (445 children aged 4-

7yrs from Junior and Senior Infant classes; 53% female) - Child behaviour (teacher & researcher-completed) - Teacher behaviour and self-report Initial screening - low’, ‘medium’ and ‘high’ EBD (n=217) Assessed at baseline (pre-intervention) and 6 months later

2. Qualitative sub-study 1:1 interviews with 11 teachers from 6 schools (baseline and 6-mth

follow-up only) Thematic analysis

3. Cost analysis Detailed cost data on: (1) non-recurrent initial training costs of

facilitators; (2) pre-delivery cost of programme; (3) group training costs; and (4) teacher substitution costs outcome data

RCT Flow Diagram

Schools (n=12) in the greater Limerick area approached by Archways to participate in the research

Names of 2 other schools provided to the research team by Archways. Both consented to participate

Three schools were unable to participate (reluctant to participate [n=2] and time concerns [n=1])

Nine schools provided informed consent

All teachers completed and returned questionnaire measures

T-SDQ child scores computed and index children identified. Parental informed consent obtained

Randomisation process conducted; 11 intervention group teachers and 110 index children; 11 control

group teachers and 107 children

Baseline (T1) observations carried out in all classrooms (n=22) with all teachers (n=22) and index children (217)

Intervention group teachers (n=11) receive TCM training (110 Index children)

Follow-up (T2) assessments conducted immediately after TCM training completed (ie. 6 months after baseline assessment).

Total number of participating schools (n=11), teachers (n=22), children (n=445)

234* index children were selected for observation from the initial sample of 445 on the basis of their total SDQ scores

* 17 children were lost to follow-up.

Mean SDQ ‘Total Difficulties’ scores across the sample range

1.2

6.4

15.9

7.8

1.5

7.3

15.86

8.2

0

2

4

6

8

10

12

14

16

18

Low Middle High Total

Control Treatment

Key findings: 1. RCT

Baseline: 26% of children showed significant social, emotional and behavioural difficulties at baseline (SDQ)

Several statistically significant improvements in pre-post intervention teacher behaviour:

- use of fewer negative classroom management strategies - give children more time to comply with questions/instructions - positive upward trend in positive classrm management strategies

Increase in the intervention group in self-reported frequency of use and perceived usefulness of positive classroom management strategies

All teachers reporting feeling more confident in their ability to manage conduct problems

Teacher ‘no opportunity’ for Intervention and Control Groups

0

2

4

6

8

10

12

14

Baseline Follow-up

No

Op

po

rtu

nit

y

Control

Intervention

Child outcomes

Children in intervention group:

- showed significant reductions in emotional symptoms

- positive trends with respect to overall behaviour

Those most ‘at risk’ showed most significant improvements:

- overall behaviour

- peer problems

- general distress and social impairment

SDQ Total Difficulties score for ‘high risk’ children

0

2

4

6

8

10

12

14

16

18

20

Baseline Follow-up

SD

Q T

ota

l fo

r h

igh

sco

rers

Control

Intervention

2. Qualitative study (Lynda Hyland)

Theme 1: Reality of the classroom environment

All teachers reported significant behavioural challenges in their classrooms; some also reported physical and verbal aggression:

“…he has thrown things behind his head and hit children with them. And he has even been violent towards adults as well too, children and adults… numerous teachers I’d say at this stage have gotten kicked and punched when he lashes out.” (T1)

Teacher stress:

“Some days I went home and just cried, because I was just so exhausted and frustrated with the situation. Because, you know, I could see it in the other children, and I could see it in the other parents as well.” (T4)

Reluctance to seek support

Theme 2: TCM learning experience

Increased awareness: Course helped crystallise theoretical knowledge into conscious practical applications in the classroom

Experiential learning: A very positive aspect of the programme

“ I’m delighted that I went on the training. As I said overall, you’re a way more confident teacher dealing with behaviour. You just have better and more resources and different things to draw upon.” (T11)

Role of the Group leaders: Building relationships and facilitating learning

“… (The group leaders) were very, very good. They were very open and… they’d

ask and they would try their best - even give you ideas to try this or that or try the other.” (T1)

‘I am not alone’: Social support and the learning process

Theme 4: Impact on child behaviour - A ‘therapeutic milieu?’

A number of reported improvements in pupil behaviour: overall conduct staying more ‘on task’ - less distracted fewer attention-seeking behaviours better peer relations

“They would be encouraging each other. They have started to get on better with each other and they’re co-operating amongst themselves even.” (T1)

“…for the children…,you know who were doing well and who might often get overlooked, they were encouraged and you know, they were boosted and their behaviour improved as a result.” (T2)

Theme 5: The ‘positive classroom’

Teachers reported having much more positive classroom experiences, and seeing the classroom as a happy place to work and learn

They reported feeling much less stressed and better equipped to deal with situations when they arise:

“We’ve been having more and more really, really good days where they’ve been enjoying it and I’ve been enjoying it. And you know, they have been saying things like, ‘oh you should get a sticker too teacher’ …you were very good today’ and all this ‘kind of thing’. So I think it helps me personally feel like I can cope with anything now.” (T7)

“ I think it’s good. I think it’s very good…I definitely would recommend people to do it” (T3)

All interviewees would highly recommend programme to another

teaacher/school

3. Cost Analysis (Prof.Donal O’Neill)

Total cost of delivery: €22,012.92 Average cost/teacher: €2012.92 Average cost/child: €100.65 (average class size=20)

Approx 1% of total expenditure per primary school child (Eurostat, 2007)

Combined with outcome (SDQ) data -> €52.97 per 1 point change in SDQ scores (ICER); e.g. it would cost approx €158.91 to bring the average highest scoring child in sample to within ‘normal’ range of behaviour

Much lower than other education-based interventions ($4863-$20,972 per child/youth)

Conclusion Findings support those from elsewhere

Significant improvements in key areas: - Teacher skills/competencies in managing behavs - Child behaviour - Classroom atmosphere/ positive classroom envt

Programme was regarded as being very worthwhile

Other benefits: teacher stress, process of self-reflection, better home-school collaboration

Very modest costs

TCM programme is effective, cost-effective, highly acceptable to teachers, well suited to an Irish context and consistent with policy and practice guidance in Ireland

heooThe good nTeh ews…

Thanks to…

All participating schools, their principals, teacher participants

The staff of Archways

The Atlantic Philanthropies & Dormant Accounts fund

The Expert Advisory Committee and Peer Reviewers

All members of the research team and especially the MHSRU team at NUI Maynooth

www.iyirelandstudy.ie [email protected]