Evaluation of Family SMILES: an NSPCC intervention for children whose parents have a mental illness

Evaluation of Family SMILES: an NSPCC

intervention for children whose parents

have a mental illness

Prakash Fernandes and Rachel Margolis

What is the Family SMILES programme?

Family SMILES ServiceFamily SMILES

Family SMILES is targeted at children and young people aged 8 to 14 whose parents have a major mental illness that has contributed significantly to

the child having been identified as being vulnerable or at risk

FaIn Brief

• The original programme was developed in Australia by Erica Pitman

• 8 Group work sessions• The aim of the group is to help children understand

about mental health and develop some life skills to help them cope

• The parent also receives one to one support to understand their child’s experience of living in a family with mental illness

• The family then completes a support plan to ensure the child is safe and their wishes are considered in the event of the parent becoming ill again

• The NSPCC have been running the Family SMILES service since September 2011.

• Family SMILES is run in the following NSPCC service centres: Belfast, Bristol, Gillingham, Grimsby, Ipswich, Jersey, Manchester, Middlesbrough, Prestatyn and Southampton

• At the end of 2013, 144 parents and 158 children had gone through the programme.

5

In Brief- cont’d

Where is Family SMILES run and how many people have used the service?

Overview of the evaluation

7

To enable children and young people to

process their thoughts and

feelings

To enable children and

young people to feel better about

themselves

To reduce children and young people's emotional and behavioral problems

ObjectivesTo provide

individual work with parents

To provide group support for

children

To provide joint work with children and

parents

Toimprove the

well-being of children and young people

and reduce isolation

Specific aims

Overall aim

OutputsGroup workInformationSignpostingPeer support

To enhance parents protective behaviors

Individual workInformationSignposting

Joint work

Safety plan

8

Evaluation designOutcome Tool Perspective

To enable children and young people to feel better about themselves

Adapted Rosenberg Self Esteem Scale

Child

To reduce children and young people’s emotional and behavioural issues

Goodman’s Strengths and Difficulties Questionnaire (SDQ) Health of the Nation Outcome Scales (HoNOSCA)

Child/ Parent Practitioner

To enable children to process their thoughts and feelings

Children’s evaluation wheel Child

To enhance parents’ protective parenting/ to improve safeguarding of children

Child Abuse Potential Inventory (CAPI) Parents evaluation wheel

Parent

Qualitative interviews with children, parents, practitioners and referrers

Quasi experimental design . Full report in December 15

Key Findings from interim report

To reduce children and young people’s emotional & behavioural problems (children/ parent’s perspective)

SDQs completed at both T1 and T2 (n=72)

• The mean at T1 is 15.3and the mean at T2 is 13.6. The change in score is statistically significant (p=0.01)

The data also indicates a shift from a level of difficulty that is of clinical concern to that of a normal level.( Exact McNemar’s test p=0.01)

T1 T20

2

4

6

8

10

12

14

16

18

T1

T2

Time frame

Average score

T1 T20%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Borderline/Clinical rangeNormal range

Reduction in children’s emotional/behavioural difficulties (2)

• There has been a decrease in mean scores across all subscales of the SDQ. However the change is only statistically significant for conduct problems (0.04) and peer problems subscale.( p= 0.01)

Conduct problemsEmotional symptoms Hyperactivity Peer problems Pro-social0

1

2

3

4

5

6

7

8

9

10

T1

T2

SDQ subscales

Average score

To reduce children and young people’s emotional & behavioural problems (practitioners' perspective)

56 HoNOSCAs have been completed at T1 and T2.

• The mean score at T1 is 8.04 and the mean at T2 is 6.68. The change in mean score between T1 and T2 is statistically significant. ( p= 0.02 )

T1 T20

1

2

3

4

5

6

7

8

9

T1

T2

Time frame

Average score

13

Outcome : To enable children and young people to feel better about themselves72 Adapted Rosenberg Self-esteem questionnaires have been completed at T1 and T2

T1 T20

5

10

15

20

25

T1

T2

Time frame

Mean score

• There has been an increase from a mean of 19.35 at T1 to 21.29 at T2. The change is statistically significant p= 0.01

Outcome: To enhance Protective Parenting

Subscale Mean T1

Mean T2

Statistically significant (p<=0.05)

Distress (cut off=152) 185.7 168.5 Yes

Rigidity (cut off=30) 20.9 18.1 Yes

Unhappiness (cut off=23) 40 31.6 Yes

Problems with child and self (cut off=11)

7.5 6.1 No

Problems with family (cut off=18) 17.7 14.8 No

Problems with others (cut off=20) 17.5 17 No

Total score (cut off 215) 289.2 256.2 Yes

Ego strength scale 9.7 12.5 Yes

Loneliness scale 11.5 11.3 No

Overall reduction in behaviours that contributes to risk in children as shown by the CAPI. CAPIs suitable for evaluation at T1 and T2 (n=22)

Outcome: To enable children and young people to process their thoughts & feelings

• The changes between T1 and T2 are statistically significant for all criteria except for easily making friends and having a fun time.

Criteria T1 T2 Statistically significant

I can talk to someone if I'm worried about my parent's health 3.65 4.07 Yes

I can talk to my parent about how their mental health affects me 3.25 3.7 Yes I can easily make friends 3.82 4.03 No

I am able to have a fun time when I want 3.95 4.04 No

I feel supported by others around me 3.85 4.22 Yes

Children’s evaluation wheels (n=94)

Parents’ perception of changes to their parenting

16

22 parents completed the wheel at both Time 1 and Time 2

Criteria T1 T2 Statistically significant

How much I think that my child is affected by my behaviour 3.64 3.73 No

How supported I feel in taking care of my child 3.41 4.05 No

How confident I feel in asking for help when I need it 3.55 4 No

How much knowledge I have about children's' needs at different stages of their development 3.64 4.14 No

How confident I feel that I am doing the best I can for my child 3.82 4.23 Yes

Key Findings: Qualitative interviews with children and parents

More effective communication within the family about mental health“I never really felt comfortable talking to

my mum about anything. My mum is starting

to tell me stuff, and usually before I would

probably go: ‘I don’t really care’. But I do care now

- Young person

17

“But I was able to talk to him then more about it and I learnt that from SMILES.

It’s OK to discuss things with your child and it’s OK to say ‘I’m not feeling well’,

which before I would have said nothing. Just ‘Oh, mummy hurt her back today’ or

‘mummy tripped and fell’ or something, I’d come up with these reasons for being

in bed or if I’d been crying or whatever. Definitely, definitely made a big

difference.”(Parent, Severe mental health problem)

“I thought I was doing right by letting her deal with her own thing, when actually now she’ll ask me for five minutes and we’ll sit and talk” (Parent)

Key Findings: Facilitators and Barriers

18

• The group work experience: age range ; gender; some awareness of parent’s mental health problems prior to group starting.

“I could talk about things which actually other people had experienced with their own mums. It was really helpful.” (Child)

I used to bottle it up and I used to find it hard to open myself up. So I used to kind of sit alone… But then when I had been to that group for eight weeks somehow I just couldn’t end up holding it in and maybe that was a better way of me showing I’m hurting... Now I’ve learnt that telling someone is the best thing and bottling up is the worst kind of thing” (Child)

• The individual work with parents: readiness to accept impact of their mental health on their child; how they are managing their mental health problems; how they perceive change in the child’s behaviour at home; engagement (or lack of) from the entire family; inability of the programme to address family dynamics

“She (practitioner) asked me a question and it stuck in my head: “How do the boys feel about you?” I’ve never asked that because you take it for granted and that was the turning point” (Parent)

Next Steps for the evaluation

19

• Analysis of the comparison group data

• Analysis of the interview data from referrers and

practitioners

• Analysis of Time 3 data

2020

Summaries of all our evaluation plansEach summary includes:• Details of the evaluation methods we are using for the service• Details of any challenges faced and how we overcame them• A list of the measurement tools we are using • Contact information of the staff member involved in the evaluation

Tools for measuring outcomes for children and Families We've outlined our experiences of using each tool, along

with details of what it measures and how easy it is to use.Each measure includes:

• An explanation of the measure• Details of who to use it with• The time it takes to administer • Any training or expertise required • It’s validity and reliability • Cost• Our experience of using it

And much, much more….

Our Impact and Evidence Hub can be found at www.nspcc.org.uk/evidencehub 

Join the debate at:@nspccpro

Search #evidencehub

Our hub features…

  The NSPCC Impact and Evidence Hub