Studying ‘what works?’ and other questions Liz Newbronner Firefly Research & Evaluation Prof Brian Taylor University of Ulster.

Studying ‘what works?’ and other questions

Liz Newbronner

Firefly Research & Evaluation

Prof Brian Taylor

University of Ulster

2

Knowledge Building Inductive research = exploring ideas

that lead to a model or understanding Builds on anomalies in deductive research

Theoretical reflection on ideas to create & connect concepts & models to be explored & tested

Deductive research = testing ‘does it work’ by an experimental method Builds on theory & inductive research

3

Different Feel

Inductive research is more open-ended, exploratory, venture into the unknown

Deductive reasoning is a more precise and rational process of testing a premise (or hypothesis) against the real world by using questions such that the answers will disprove the hypothesis if it is not true

4

4

Different Kinds of Questions Perspectives: What do older people think of

their experience of hospital discharge arrangements?

Outcomes: What is the effect on likelihood of returning home for at least six months of X hospital discharge arrangement?

Prevalence & incidence: what fraction of older people being discharged from hospital receive care from X type of discharge team?

5

Defining your question(s)

Clarify question(s) before choosing method If you want to prove, measure, compare,

contrast you need a quantitative (numbers) method, at the least a scale with two alternative options

If you want to explore perceptions, to pursue a novel idea, create a new theory, choose a primarily qualitative method

6

Basic Types of Study Design Experiment – does A cause B? Survey – prevalence or correlation

between A and B QuaLitative – how can we best

understand (model) perceptions or behaviours

Evaluation – how good is a service or activity

Audit - how good is a service or activity compared to a standard

“Let the punishment fit the crime?!”

The choice of most appropriate method depends primarily on the research question and feasibility not on the topic

7

NB Design is not the same as data collection tools

Questionnaires (postal/email/web) Interviews Focus Groups Data Extraction from files (paper/computer) Observations Diaries Vignettes (case scenarios) etc

8

Mixed Methods – justify!

QuaLitative then quaNTitative = Preliminary study identifies factors to

measure Helps determine sample size for

significance by indicating range of variableQuaNTitative then quaLitative Quotations bring numbers to life Increasing use in government reports Useful for audit & service evaluation

9

Group work task

In your group:Develop an outline study designHighlight the methodological challenges you think the study is likely to presentConsider any practical challenges that would need to be overcomePrepare three key points/issues for the feedback session

10

Study 1: Respite care services for carers of children with disabilities

Following critical feedback from carers a local authority has re-shaped the respite services it provides and commissions

In-home, day and residential respite care are available but providers and locations have changed

The LA wishes to understand which respite services work best for families caring for children:with different types of disabilityin different age groupsin different family situations

The study needs to make recommendations for any further changes in provision or allocation of resources

11

Study 2: Support needs of carers of people with acquired brain injury

A national charity has secured funding to undertake the research

They are particularly interested in the support needs of spouse carers and those with dependent children

They want to understand more about:the types of information, advice and support carers would valuewhen they want information, advice and support

The results will be used to a) inform a review of the charity’s national advice service and b) help local groups influence NHS and LA services

12

Study 3: Impact of specialist dementia nurses on the health of carers of people with dementia

A national research body wishes to commission research to examine whether regular and ongoing support from a specialist dementia nurse has an impact on:Carers emotional and physical healthTheir capacity to continue caring (if they wish to) Their access to other sources of information, advice and support

The results of the study will be used to inform national NHS commissioning

13

Design questions to consider

Is the study research or evaluation? What are the main research or evaluation

questions? Who will be the main audiences for the findings? How will the results be used? What will be the main ‘outputs’? Likely timeframes and resources?

14

Methodological challenges to consider

Defining interventions and processes Selecting outcome/impact measures Enabling participation - identifying and

contacting carers Sampling and representativeness Context and confounding factors Rigour and results

15

Challenges in studies of what works1. Defining the intervention

(intervention fidelity)

2. Defining outcome measures

3. Balancing external with internal validity

4. Eliminating confounding factors

5. Ethical and legal issuesTaylor BJ (2012) ‘Intervention Research’ (Ch 27) in Gray M, Midgley J

& Webb S (Eds.) Social Work Handbook, New York: Sage16

Design a study on:1. Effectiveness of respite care to help family

carers to cope with a family member with (learning disability or mental health problem or older person or… )

2. Prevalence of ill-health amongst family carers caring for a family member with (brain injury or a mental health problem or… )

3. Family carers perceptions and concept of themselves as a carer for a family member with (dementia or severe learning disability or… )

17

Challenges of ‘what works?’ The Cochrane Collaboration is an

independent, non-profit, volunteer-based, international collaboration of over 30,000 people committed to systematically reviewing the best evidence of the effectiveness of health and social care interventions

There are now over 70 systematic reviews in the Cochrane Collaboration Library of interventions that could be undertaken by Social Workers (www.cochrane.org)

18

A few Cochrane Reviews relating to carers• Support for carers of people with dementia• Information provision for stroke patients and

their caregivers• Crisis intervention for people with severe

mental illness• Inspiratory muscle training for cystic fibrosis• Assistive technology for memory support in

dementia• Shared decision making interventions for

people with mental health conditions

19

A few more Cochrane Reviews relating to carers Respite care for people with dementia &

their family carers Hospital at home early discharge Helping people recognise early signs of

recurrence in depression Subjective barriers to prevent wandering

of cognitively impaired people Non-pharmacological interventions for

wandering of people with dementia at home

20

Need “a fundamental step change in breadth, depth and quality of the UK research base in social work and social care”

Economic & Social Research Council (2008) Strategic Adviser for Social Work and Social Care Research: Commissioning Brief, Swindon: ESRC

21

Process: Knowledge from Research

Need a social work equivalent to medicine development: Theory - why might it work? Laboratory experiments - does it ever work? Animal experiments – does it work on a living

creature where there is no threat to humans? Tests on healthy volunteers - safety Tests on those with the problem – effectiveness Post-approval reporting – side effects in the

general population including co-morbidities

22

Research Process Model

Medical Research Council (2008) Developing and Evaluating Complex Interventions, London: MRC

23

1. Development Evidence & theory Model the process

2. Pilot feasibility Test

procedures Plan evaluation

3. Evaluation Effective? Understand

change process

4. Implement Disseminate Monitor

Studying ‘what works?’ and other questions

THE ENDLiz Newbronner

Firefly Research & Evaluation

Prof Brian Taylor

University of Ulster