Local Surveys Tools 29th October 2009 The Acute Survey Co-ordination Centre: Jason Boyd.

Local Surveys Tools29th October 2009

The Acute Survey Co-ordination Centre:Jason Boyd

Improving local use of surveys by… Adapting existing acute survey questionnaires

Expand question bank contentRemove or revise existing questions

Developing a “Day Case Surgery” survey Provision of methodology documents

comparable with national surveys Provide advice to trusts on conducting surveys

Questionnaire content: methods

• Analysis of previous survey data• Examining patient free-text comments• Feedback survey of trusts/contractors (Maternity)• Online survey of trusts/contractors to prioritise

question development (Inpatients)• Focus groups (Inpatients, Day case & Outpatients)• Stakeholder consultation

Cognitively tested new/revised questions

Local survey tools available

• Accident & Emergency Department• Adult Inpatient• Day Case Surgery• Maternity Services• Outpatient Department

Tools available for each survey• Guidance manuals– “Guide to conducting a local … survey”– “Guide to mailing questionnaires, data entry and cleaning”– “Guide to analysis, reporting and dissemination of results”

• Online question bank tool• Covering letters and multi-language sheet• Sample construction and data entry

spreadsheets• Model service contracts• Survey development report

Ask the right questions

• To evaluate service improvement initiatives

• Monitoring progress against national targets or trust-wide initiatives

• Comparison of results over time

• Analysis by different groups

Online questionnaire tool

Expanded question bank toolsSurvey ‘Core’ questions

in last national survey (n)

Questions in online bank (n)

Demographic questions in bank

(n)

A&E Department 50 100 12

Maternity 96 (132) 120 15

Day case surgery - 122 15

Outpatients 58 126 14

Inpatients 87 203 12

Questions designed by the trust…Issues uniquely important to a trust may

require questions not in the question bankDevelop new survey questions

• Co-ordination Centre can advise• Survey Contractors can advise• Consider:

Effect on response ratePlacement of questions

Pre-test questions

Ethics and R&D approval

• Local surveys all derived from Research Ethics Committee approved national surveys (except Day Case Surgery)

• Patient information material REC approved format

• Local surveys are “service evaluation”

Ethics not required

Trust R&D notified

Data protection

• Use unique patient reference numbers• Use contracts with any external organisation• Never link patient name and address

information to their responses• Be aware: patient identity can be inferred

from data• Anonymise the data of small groups• Encrypt datasets between use

Sample size

• Consider the organisational level to be surveyed, i.e. trust, department, ward

• Estimate confidence intervalsRecommend minimum 100 respondents per

group for any sub-group analysis

• Adjust for expected Response RateConsult RR from previous surveysUnderstand demographic variations in RR

Maximise response rates• A higher response rate results in more

representative and reliable findings:Publicise the survey to staff and the publicQuestionnaire lengthCohesive questionnaire design and relevant questionsReminder letters for postal surveysAppropriate fieldwork periodComplementary collection methods

Real-time and frequent feedbackTelephone surveysQualitative methods such as interviews, focus groups, forums

Telephone help-line and translation services

Sampling methodology• Sampling methods vary between surveys

– random sampling: outpatients, A&E– consecutive sampling: inpatients, day case

surgery, maternity– different exclusion and inclusion criteria

• Check for sampling errors

• Check for deceased patients

• Consider additional sample information

When to conduct a local survey

• Avoid duplication with the national patient survey programme:

– Survey samples should not overlap (remember many patients will access more than one service in their care pathway)

• Avoid fieldwork during major holiday periods

• Consider staff workload

Using a survey contractor

• 12 CQC approved survey contractors• Issues to consider– Staff time– Cost– Expertise– Data protection– Reporting– Perceptions of objectivity– Service contracts

Using results: access to previous data

• CQC publishes national survey results:– Trust-based data: benchmark reports,

comparative scores workbook (benchmark data)– National data: historical tables, briefing note and

press releases

• Individual trust spreadsheets provided to trusts

• Previous trust data held by Co-ordination Centre – available on request

Standardising data and duplicating benchmark data

• Changes in survey results can be due to differences in the sample composition!

• Results should be “standardised” when compared to the national surveys and over time

• All national programme benchmark tables standardised for age and sex (and for ‘route of admission’ for inpatients)

• Refer to the data cleaning and analysis guides to duplicate national programme results

Standardising data and duplicating benchmark data

• The Co-ordination Centre can supply:

– All previous national benchmark data for the trust (‘core’ questions only)

– Demographic proportion tables for the most recent national survey to allow standardisation of data to these

– Advice on standardisation and data weighting

Getting help from the Co-ordination Centre…

• Telephone and e-mail support

• Advice on:

• Sampling

• Questionnaire design

• Analysis and standardisation

• Updating and adding additional local surveys

Contacting the Co-ordination Centre

• Current staff: Sally Donovan (Manager) Esther Howell (Senior Research Associate) Jason Boyd (Senior Research Associate) Geraldine Cooney (Senior Research Associate) Elisabeth Garratt (Research Associate) Harriet Hay (Research Associate)

• Telephone: 01865 208127

• Email: [email protected]