If it works in Edmonton, will it work in Edinburgh ? Professor Helen Roberts DPhil FFPH General and Adolescent Paediatrics Unit, UCL Institute of Child.

If it works in Edmonton, will it work in Edinburgh ?

Professor Helen Roberts DPhil FFPHGeneral and Adolescent Paediatrics Unit,

UCL Institute of Child Health, London [email protected]

... might systematic reviewing help?

.. Well up to a point Lord Copper.

SRs can offer• A synthesis of studies in a

particular field which no policy maker or practitioner, however diligent, could hope to read themselves and sometimes

• A challenge to received wisdom

What we say… What non-systematic reviewers are really thinking…

We carried out a comprehensive search of the literature, including Felonbase, Quackline, NERD alerts, etc

They searched the one or two databases where 95% of the literature is published, plus an unnecessarily large number of oddly-named databases that no-one has ever heard of

Mark Petticrew’s Cochrane lecture 2007

What systematic reviewers say What non-systematic reviewers are really thinking…

We found 1,000,000 studies, of which 1 nearly met the inclusion criteria, but not quite

They excluded all the really useful stuff

We concluded that more research is needed

…and then they concluded that more money is needed

Mark Petticrew’s Cochrane lecture 2007

Meaning well and doing good

Nowhere is wisdom more necessary than in the guidance of charitable impulses. Meaning well is only half our duty; thinking right is the other and equally important half

Samuel Gridley Howe, 1866

Tabloid ‘solutions’

• Licences for smokers • Separate queues in the supermarket for

alcohol• Letters to parents of fat kids• Exercise hours for large firms

Traffic management in Athens(of the south)

• Attempts to reduce traffic• Odd and even numbered number plates on different days• Tax breaks for greener cars • The consequences of interventions with strong face validity

aren’t always what the experts think they might be

Mentoring to reduce offending

• Anti-social behaviour in childhood and adolescence is a problem for young people, their families, the police, communities and politicians

• In 2003, the Government announced funding for mentoring schemes in England, arguing that ‘mentors can make a real difference to... some of the most vulnerable people ....There are ... excellent examples of schemes which really work’.

• Mentoring is non-invasive and does not require drug treatment. It is easy to see why it might work, and why it might be attractive to politicians and policy makers.

• Unfortunately it can have the reverse of the desired effect

What might help in working out what works?

• Reversing the lack of a rigorous outcome evaluation culture

• Recognition that quick fixes are rare and that social and educational interventions have as much or even more potential to do good or harm as health interventions

What else might work ?

• Include high quality qualitative and ethnographic studies

• Paying greater attention to the experiences of people on the receiving end of interventions, policies and practices

(c) Helen Roberts

Horses for courses• Effectiveness Does this

work? Does it work better than doing something else, or nothing?

• Effectiveness of service delivery

How does it work ?• Safety

Does it do more good than harm ?

• Cost effectiveness Is it worth buying?

• Acceptability Will children or parents want to (at best) or be willing to (at worst) take it up ?

• Appropriateness Is this the right service for these children ?

• Quality How good is the service?

• Salience Does it matter? Evidence, hierarchies and typologies: Horses for Courses, Petticrew M. and Roberts H. Journal of Epidemiology and Community Health, 57:527-529, 2003

The Road to Wisdom

The road to wisdom? Well, it's plainAnd simple to express:Errand errand err again,but lessand lessand less. Piet Hein