ENC B 1 Chair: Dr Jonty Heaversedge Chief Officer: Andrew Bland The best possible health outcomes for Southwark people Deep-Dive Review: Evaluating the CCG’s use of Patient Experience Information Omar Al-Ramadhani Planning and Assurance Manager
ENC B
1 Chair: Dr Jonty Heaversedge Chief Officer: Andrew Bland
The best possible health outcomes for Southwark people
Deep-Dive Review:
Evaluating the CCG’s use of
Patient Experience
Information
Omar Al-Ramadhani Planning and Assurance Manager
ENC B
2 Chair: Dr Jonty Heaversedge Chief Officer: Andrew Bland
The best possible health outcomes for Southwark people
1. Aim ................................................................................................................................... 3
2. Introduction ..................................................................................................................... 3
3. Sources of Patient Experience Information and Themes ............................................ 5
3.1 Section Summary: Current Information .................................................................................... 5
3.2 Overview of Data ..................................................................................................................... 5
3.3 Nationally-Generated Patient Experience Data ....................................................................... 7
3.4 Locally-Generated Patient Experience Data .......................................................................... 10
4. How the CCG Currently uses Patient Experience Information and how we can
Improve our Approach ......................................................................................................15
4.1 Section Summary – The Use of Patient Experience Information ............................................ 16
4.2 Strategic Planning ................................................................................................................. 17
4.3 Procurement .......................................................................................................................... 19
4.4 Monitoring and Evaluation ..................................................................................................... 20
5. Recommendations.........................................................................................................24
Appendix A - Friends and Family Test.............................................................................25
Appendix B - Sources of Patient Experience Information .............................................28
Appendix C – HRWD Tracker ............................................................................................31
ENC B
3 Chair: Dr Jonty Heaversedge Chief Officer: Andrew Bland
The best possible health outcomes for Southwark people
1. Aim
This report aims to review how NHS Southwark CCG uses patient experience information to inform
service design and improve the quality and safety of NHS services used by Southwark patients. The
review looks at patient experience information obtained from national and local sources and
additionally considers the gaps in the CCG’s approach. In summary, the review focuses on
answering the following questions:
Is the information available to the CCG fit for purpose and are there any gaps?
How does the CCG collate and analyse patient experience data and to what extent are we
effective in using this information to improve services?
What are the most frequently identified patient experience issues that require improvement?
2. Introduction
Commissioning services that offer a good standard of patient experience has consistently been a
key priority for the CCG. Improving patient experience has been a key objective in the CCG’s
operating, strategic plans and recently published CCG Quality Framework. The CCG also
recognises that good patient experience is a fundamental component of good quality care.
Understanding patients’ views of commissioned services is an essential feature of monitoring the
services we commission.
The CCG’s commitment to improving patient experience is also reflected in national policy. In the
white paper, Equity and Excellence: Liberating the NHS, published in July 20101, the government
made clear its commitment to improving patient experience and creating an NHS that is ‘genuinely
centred on patients and carers’. Patient experience has gained increased recognition as a core
component of good quality care2 with the NHS Outcomes Framework making clear that the
provision of a ‘good experience’ of care for patients is a central goal for both NHS commissioners
and providers.
Patient experience has also been linked to improved patient outcomes. A systematic review of 55
studies in primary care and hospitals found consistent positive associations between patient
experience, patient safety and clinical effectiveness for a wide range of disease areas. This
supports the case for including patient experience as one of the central measurements of quality in
healthcare and underlines its importance3.
In response to heightened national focus on patient experience following the reports into Mid-
Staffordshire; Winterbourne View and Keogh’s review into hospital mortality, the NHS has increased
the production of patient experience information in order to ensure both providers and
commissioners monitor and improve performance. NHS organisations now generate and have
access to a significant volume of data. However, evidence tells us that data is not yet systematically
1 Department of Health. 2010. Equity and Excellence: Liberating the NHS.
2 Department of Health. 2008. High Quality Care for All - NHS Next Stage Review Final Report.
3 The Health Foundation. 2013. Measuring Patient Experience.
ENC B
4 Chair: Dr Jonty Heaversedge Chief Officer: Andrew Bland
The best possible health outcomes for Southwark people
used by commissioners of NHS services. A recent paper published in the British Medical Journal4
highlights that, whilst multiple sources of patient experience information is available to healthcare
organisations, not enough is being done to use that information to monitor and improve services.
The following sections of the review – sections 3 and 4 – look in further detail at some of the
information that is regularly available to the CCG. These sections look at some of the key themes in
these data and also review how the CCG currently analyses and utilises this information over the
course of the commissioning cycle.
Section 5 concludes the review with a number of practical recommendations for the CCG to take
forward.
Note: This review reflects on and incorporates recommendations made in the CCG Quality
Framework.
4 A. Coulter, L. Locock, S. Ziebland and J. Calabrese. 2014. Collecting data on patient experience is not enough: they
must be used to improve care. The British Medical Journal.
ENC B
5 Chair: Dr Jonty Heaversedge Chief Officer: Andrew Bland
The best possible health outcomes for Southwark people
3. Sources of Patient Experience Information and Themes
This section looks at the patient experience information currently available to the CCG and
highlights areas where the CCG could make further use of these data sources. Useful data on
patient experience should have the following characteristics:
a) Provides adequate coverage over its commissioned activity
b) Is broad enough to highlight areas that may require improvement and
c) Granular enough to make it possible to drill down to understand issues in more depth.
3.1 Section Summary: Current Information
a) The CCG regularly focuses on national data sources to provide an overview of patients’ experience of acute, mental health, primary care and social care services.
b) The CCG needs to develop an effective way of collecting informal/soft data to identify trends and recurring issues (refer to recommendations 1 and 2 in Section 5).
c) The CCG should make more use of patient focus groups in certain service areas. Focus groups can uncover unanticipated issues that surveys fail to identify and they can be a good way of engaging people who do not respond to surveys; and the most disadvantaged patients (The Health Foundation, 2013). The CCG should access information being shared at hospital patient groups. This can be done by attending meetings or asking for minutes/notes of meetings (refer to recommendations 1 and 5b in Section 5).
d) For both of the above the CCG should consider an approach to conducting qualitative surveys of patients or the completion of focus groups. This approach could consider the role of local voluntary sector groups and/or Healthwatch Southwark as they have links to patients and the community (refer to recommendation1 and 5 in Section 5).
e) There are some gaps where information is not routinely collected/analysed, these include some out of hospital pathways, small providers and out of hours doctors services (refer to recommendation 4 in Section 5).
f) The CCG does not have access to much data that measures patient experience at the interface between services except for data on discharge from acute care. The data we have is mainly restricted to activity at specific providers rather than across pathways. An effective way of gathering this information could be through patient interviews and focus groups. Continuing care could be a good area to start to gather this information as the team have good links with patients and carers (refer to recommendation 5 in Section 5).
3.2 Overview of Data
This Section provides an overview of the types of patient experience information available to
the CCG and highlights a number of themes.
ENC B
6 Chair: Dr Jonty Heaversedge Chief Officer: Andrew Bland
The best possible health outcomes for Southwark people
England has been a pioneer in measuring patient experience information and was the first country
to introduce a mandatory survey of hospital inpatients in 2002. National survey reporting has
expanded considerably to now include many aspects of care (box 1)5. There is therefore a
significant volume of intelligence and information available to CCGs. Indeed one challenge faced by
commissioners has been to establish systems for the routine collations and interpretation of patient
experience data.
The CCG commissions a wide array of services from multiple providers and so it must be able to
draw upon a varied approach to capturing patient experience information. All data capture methods
have pros and cons, so a combination of styles may be needed to understand issues at a level
which can be used to inform and drive improvements.
5 A. Coulter, L. Locock, S. Ziebland and J. Calabrese. 2014. Collecting data on patient experience is not enough: they
must be used to improve care. The British Medical Journal.
Box 1: Regular national health and social care surveys carried out in England
Care Quality Commission (CQC) national patient experience surveys
General Practice Patient Survey
National Cancer Patient Experience Survey
VOICES survey of bereaved people
Patient reported outcomes of selected surgical procedures (PROMs)
Adult Social Care Survey
Adult Social Care Carers Survey
CQC social care user surveys
ENC B
7 Chair: Dr Jonty Heaversedge Chief Officer: Andrew Bland
The best possible health outcomes for Southwark people
The diagram below presents a spectrum of patient experience information. The means of collecting
data are grouped into how specific or general the information is and how descriptive it is (see figure
below).
Source: Measuring Patient Experience, The Health Foundation, 2013.
3.3 Nationally-Generated Patient Experience Data
There are a number of nationally directed patient experience surveys that generate data to be used by NHS providers, commissioners and regulators of services. These data and the results at local providers are summarised below and reflect the most recently available data:
Maternity Survey – 2013
King’s College Hospital NHS Foundation Trust (KCH) achieved scores that were comparable with the national average on every question except for ‘being involved enough in decisions about their care during labour and birth’ which the Trust scored below the national average. Guy’s & St. Thomas’ NHS Foundation Trust (GSTT) achieved scores that were comparable with the national average on every question except for two questions relating to staff behaviour. The trust performs below the national average for leaving patients alone at a time that worried them and responding quickly to the call button.
Accident and Emergency Survey – 2012
KCH achieved scores that were comparable with the national average on every question. GSTT achieved scores that were comparable with the national average on every question. The Trust performed better than the national average regarding doctors and nurses listening to what patients had to say, however the Trust performed less favourably against the national average for feeling threatened by other patients or visitors.
ENC B
8 Chair: Dr Jonty Heaversedge Chief Officer: Andrew Bland
The best possible health outcomes for Southwark people
Cancer Survey – 2012/13
KCH under-performed in most areas of the survey, achieving scores in the lowest 20%. The Trust performed particularly poorly in explaining what was wrong to patients and taking into account patients views while discussing treatment.
GSTT performed better than KCH but the trust does have a number of areas that require improvement.
Inpatient Survey – 2013
KCH achieved scores that were comparable with the national average on most of the questions, however, the Trust underperformed in explaining how patients would be put to sleep and how to control pain and the time spent on waiting lists. The Trust performed particularly poorly for giving patients enough notice about when they were going to be discharged. GSTT did not achieve any scores below the national average and performed well in a number of areas.
Patient Experience Data Sources Used in the Keogh Review
The review carried out by Sir Bruce Keogh into hospital mortality outliers6 focussed on a range of domains that are indicative of the level of quality and safety of care; one of the domains was patient experience. The review focused on a number of publically available sources of information which when analysed together provide a valuable insight into the general performance of providers highlighting areas that could require further investigation. The review examined the following sources of information:
National data sources
CQC Inpatient survey
NHS England Cancer survey
PEAT: Privacy and dignity
PEAT: Environment
PEAT: Food
Ombudsman’s rating
Friends and Family test
Patient voice comments
PEAT = Patient Environment Action Team
Note: KCH and GSTT were not included in the review
Other National Surveys
The Care Quality Commission (CQC) co-ordinate surveys to collect feedback on the experiences of people using a range of health care services provided by the NHS. One of the surveys carried out is
6 NHS England, 2013. Review into the quality of care and treatment provided by 14 hospital trusts in England: overview
report.
ENC B
9 Chair: Dr Jonty Heaversedge Chief Officer: Andrew Bland
The best possible health outcomes for Southwark people
annual; the adult inpatient survey, however the rest are carried out every 2-3 years. A cancer annual survey is carried out by Quality Health on behalf of NHS England. Other national surveys include:
Outpatient (2003, 2004/5, 2009 and 2011)
A&E (2003, 2004/5, 2008 and 2012)
Maternity (2007, 2010 and 2013)
GP Patient Survey (collected every 6 months, published annually)
Patient Environment Action Team
The Patient Environment Action Team (PEAT) programme has been superseded by the Patient-Led
Assessments of the Care Environment (PLACE) programme. These self-assessments are
undertaken by teams of NHS and private/independent health care providers and include at least 50
per cent members of the public (known as patient assessors). They focus on cleanliness, food,
hydration, and the extent to which the provision of care with privacy and dignity is supported in
clinical and non-clinical services.
Although the assessments are comprehensive, scoring does not differentiate between levels of
performance well enough. The vast majority of hospitals achieve ratings of good/excellent (8 or 9
out 10), which makes it difficult to assess performance and track improvement. As with most
national surveys PLACE surveys are carried out annually.
Friends and Family Test
The Friends and Family Test (FFT) is a single question survey which asks patients whether they would recommend the NHS service they have received to friends and family who need similar treatment or care. The Friends and Family test is discussed in detail in appendix A.
General Practice Patient Survey
Although the CCG does not commission primary care, it has a statutory duty to assist and support NHS England in securing continuous improvement in the quality of primary care. A source of patient experience information is the national GP Patient Survey which assesses patients’ experiences of the access to and quality of care they receive from their local GPs, dentists and out-of-hours doctor services.
Southwark’s overall survey response rates and satisfaction with GP services and out-of-hours services were similar to the London average.
Patient Voice Comments
Patient Voice Comments measure counts of negative comments adjusted by counts of positive comments about services at a trust. For any source of patient feedback, there will be an expected relationship between the number of positive comments and the number of negative comments across all trusts. For example, the number of negatives would be expected to increase at a given rate as the number of positives increase. Using a statistical procedure called Poisson Regression, results from all trusts are taken to work out what that specific relationship is (i.e. the slope of the regression indicates how quickly negative comments increase relative to positive comments). Then, for each trust, this relationship is applied
ENC B
10 Chair: Dr Jonty Heaversedge Chief Officer: Andrew Bland
The best possible health outcomes for Southwark people
to the number of positive comments to predict what the number of negatives would be. The departure of the observed number of negatives from the expected (predicted) number determines how much of an outlier a given trust is. If a given trust’s observed count of negative comments is much higher than expected, it can potentially be flagged as a risk by the CQC (if the disparity is extreme and statistically significant). The table below shows that GSTT’s observed negative comments is below expected but King’s negative comments are slightly above expected. The CQC has not identified King’s score as a risk.
Expected Observed
King’s 4.75 5.0
GSTT 4.9 4.0
Source: CQC Intelligent Monitoring Report, 13 March 2014
The information available from the CCG’s main providers is included in appendix B.
Limitations of National Surveys
1. Frequency. The annual surveys can be an insightful source of information on a number of areas of care; however by the time responses are gathered and results published, issues may have already been addressed by the provider or new issues may have occurred which are not captured by the results. Running more regular and focused local surveys would help to monitor services more closely and highlight emerging trends.
2. Granularity. National survey questions are fairly general and do not drill down into specialty or
ward level. This makes the information useful for identifying general issues and trends which could require further investigation. An example of this occurred after a national survey showed that London hospitals were performing poorly in maternity care which Southwark CCG followed up by developing a Commissioning for Quality and Innovation (CQUIN) payment to improve certain areas of maternity care at King’s.
3.4 Locally-Generated Patient Experience Data
Patient Experience Data Generated by Foundation Trust Providers
GSTT produce an annual patient experience report which appraises the trust’s performance in recent national surveys, performance in the trust’s own near time feedback surveys, performance against patient experience CQUINs and a summary of patient advice and liaison service performance. The report also reviews findings from the trust’s patient engagement activities. There is a good mix of quantitative and qualitative information. The report highlights areas of good performance and also areas that require improvement. The issues are discussed generally, however the report could form a good basis for drilling down into areas of concern with the trust and identifying specific wards or specialties that require improvement. King’s produce monthly ‘how are we doing’ (HRWD) surveys (refer to Appendix C for the HRWD tracker) which ask patients questions on patient experience (including those covered by CQUINs) and certain points of delivery. The report used to only present quantitative information; however since August 2013 the report has published all patient comments made at the Trust as well as
ENC B
11 Chair: Dr Jonty Heaversedge Chief Officer: Andrew Bland
The best possible health outcomes for Southwark people
comments made about the trust at NHS Choices and Patient Opinions. The quantitative data is broken down by specialty which helps provide a good level of detail to the data. The trust also produces a quarterly patient experience report which provides an update on the trust’s performance in national surveys, the HRWD survey, complaints, PALS and a review of the trust’s rating on NHS Choices. GSTT has replaced its annual community services patient experience report with near time surveying. This provides more regular data and is currently reported quarterly for ten services. The trust is trying to increase the accessibility of the survey in a number of ways including; enhancing the domiciliary telephone process; making surveys more user friendly for children and those with learning disabilities and providing support to help capture the views of more vulnerable patients.
GSTT Community services
District Nursing
Foot Health
Health Visiting
Intermediate Care
Occupational Therapy
Paediatrics
Physiotherapy
Reproductive and Sexual Health
Speech and Language Therapy
Specialist Regional Rehabilitation
There is a large variation in the number of questionnaires submitted for each service with the most being around 1,000 for Foot Health and as little as nine for Specialist Regional Rehabilitation. The surveys do not provide any qualitative data, however they could be effective at identifying high level problem areas. So far the information has not resulted in service improvements because the data has not been able to illuminate any areas that require improvement. Options being considered to improve the surveys include; themed questions on specific areas and bespoke questions for each service. Commissioners are currently designing a survey for district nursing with bespoke questions that will attempt to cover the integration of services. SLaM’s patient experience reporting is centred around the trust’s Patient Experience Data Information Centre (PEDIC) reports which present quantitative data for inpatients, outpatients and each clinical academic group (CAG). The questions in the report have been agreed by commissioners with the trust. The last report (Q3 13/14) is detailed enough to monitor performance over time and initiate discussions with the trust on areas that require improvement. A patient experience survey was carried out by the Southwark IAPT service between January 2013 and December 2013 receiving over 1,000 responses. The survey asked five questions and allowed patients to make comments and so provides both quantitative and qualitative information. On the whole, patients rated the service highly.
ENC B
12 Chair: Dr Jonty Heaversedge Chief Officer: Andrew Bland
The best possible health outcomes for Southwark people
Local Provider Themes
The table below presents areas of patient experience which have been highlighted as under performing from the trust reports mentioned above.
GSTT Acute
Inpatients
Receipt and explanation of the ward pack
Quality of the food
Medication side effects
Noise at night
Cleanliness of toilets and bathrooms
Outpatients
Choice of appointment times
Being kept informed of appointment wait in clinic
Addressing worries and fears
Knowing who to contact
Delays in appointment starting time
King’s
All areas
Being told how long you would wait in outpatients
Complaints response times
Noise at night from staff
SLaM
Inpatients
Receiving a copy of care/recovery plan and jointly producing the plan
Explanation of possible medication side effects
Offered a meeting with the ward pharmacist
Restraint and injected medications used as a last resort with minimum use of force?
GSTT Community Services
NA
Receiving a copy of care/recovery plan and jointly producing the plan
Knowing how to make a complaint
Offered a crisis plan for emergency mental health situations
Being given a full assessment after accessing crisis support at SLaM
ENC B
13 Chair: Dr Jonty Heaversedge Chief Officer: Andrew Bland
The best possible health outcomes for Southwark people
CCG Engagement
NHS Southwark CCG regularly captures patient experience information from the Engagement and
Patient Experience Committee (EPEC), a sub-Committee of the Governing Body and local Patient
Participation Groups (PPGs). The CCG also captures information from large ad hoc patient
engagement events and through a number of service user groups (Discussed further in Section 4).
EPEC and PPG Feedback
Concerns raised at EPEC and locality PPGs are recorded in meeting minutes and are also
communicated to lead officers and the CCG Governing Body through the CCG’s quarterly quality
report. Issues are also raised directly by the Head of Membership and Engagement with
commissioners, however this is currently done informally.
Issues that have been captured include:
Waiting times for inpatient treatment
Outpatient clinic waiting times
Cancelled outpatient and inpatient appointments
Discharge delays from hospital
Referrals being lost
Quality of follow up and after care particularly after cancer treatment.
Timeliness of discharge information
Communication between secondary and primary care.
The CCG can potentially collect large amounts of informative soft data through engagement events
and patient forums, however soft data is not always formally published and so the CCG’s access
and use of it can be limited. Feedback collected from engagement events or patient forums are not
always written up and analysed, which risks information being lost.
Out of Hospital Pathways
Most out of hospital pathway services are contracted to carry out annual patient experience
surveys. The dermatology service returns an annual survey and King’s and GSTT are contracted to
survey patients in the Musculoskeletal Clinical Assessment and Treatment Service (MCATS).
Some out of hospital pathway services (for example the diabetes service) return monthly patient
surveys, however commissioners have highlighted that they don’t have the capacity to review the
data on a regular basis but will analyse the data during a service review.
Feedback indicates that beneficial data has come from conversations with patients during visits (for
example the diabetes quality visits) and by engaging patient groups and forums. The Southwark
and Lambeth diabetes forum has been a good source of information as it represents a diverse and
fairly representative group of patients. Interviewing patients and running focus groups can be
ENC B
14 Chair: Dr Jonty Heaversedge Chief Officer: Andrew Bland
The best possible health outcomes for Southwark people
resource intensive so it has been suggested that collaborating with voluntary services who
already have links with patients may be an effective way of gathering a rich perspective on patient
experience.
Patient experience information across pathways
The CCG lacks access to patient experience information that tracks patient experience across
pathways and the transfer between health and care services. This gap in information has been
recognised nationally and prompted five leading healthcare organisations to initiate an 18 month
project aimed at capturing older people’s experiences of care coordination. The project will develop
a survey tool to measure how older people and service users experience care when it is delivered
by multiple organisations. It is hoped that the final tool set will be available for use in the summer of
20157.
Sources of Information the CCG does not routinely utilise
There are a number of sources of information that the CCG does not routinely use, these are listed
below:
Hospital patient groups – There are many patient groups at King’s that meet to share feedback on
services; these include
King’s In Conversation
King’s In Your Shoes
King’s Maternity Services Liaison Committee
King’s Cancer Voices
Online chat forums and social media – These can be a useful source of patient comments about
services in Southwark. The CCG has used the SE1 online forum to ask Southwark people for their
experiences of using the urgent care centre in June 2013. The discussion generated 3 pages of
comments and allowed the CCG to engage with members of the forum to ask follow up questions.
The CCG has also used the Myhealth London chat forum facility as part of the Call to Action
campaign but the topic did not generate any discussion.
The CCG has been using Twitter to publicise its engagement events and activities. Tweets have
mainly focused on drawing peoples’ attention to the CCG’s website and not for direct engagement.
The Twitter account has 3,766 followers.
7 The Nuffield Trust: New project launched to explore how well health and social care services are coordinated.
http://www.nuffieldtrust.org.uk/media-centre/press-releases/new-project-launched-explore-how-well-health-and-social-care-services. Accessed 14 August 2014.
ENC B
15 Chair: Dr Jonty Heaversedge Chief Officer: Andrew Bland
The best possible health outcomes for Southwark people
4. How the CCG Currently uses Patient Experience Information
and how we can Improve our Approach
This section will review how patient experience information has been used by the CCG to
commission services and how the use of information could be improved.
The CCG’s work can be summarised by the commissioning cycle below:
Involving patients in the commissioning cycle and ensuring services are commissioned to improve
patient experience can lead to more efficient and cost effective pathways and ones that lead to
better outcomes.
ENC B
16 Chair: Dr Jonty Heaversedge Chief Officer: Andrew Bland
The best possible health outcomes for Southwark people
4.1 Section Summary – The Use of Patient Experience Information
a) The CCG has been effective at conducting large scale engagement events like those that inform the CCG’s commissioning intentions and the recent Call to Action event. However, we are aware that some feedback indicates that engagement events are not held early enough in the planning process to ensure that decisions are genuinely informed by patient views. There is a feeling that some engagement events and forums have been used to endorse decisions that have already been made.
b) When carried out in good time, engagement events have been an effective way of gathering
information to inform strategic decisions. It is essential that Commissioners work with the Communications and Engagement team to feedback to patients and participants in our engagement to demonstrate where their involvement has helped to shape our plans.
c) The CCG should undertake a different approach to focussed engagement. Our experience is that engagement in small forums and discussion groups work best when it includes expert patients or current and ex-users. Wherever possible the CCG should focus engagement on issues that patients are able to offer valuable insights and opinions (refer to recommendation 1 in Section 5).
d) The CCG has been effective in driving service improvements through the use of CQUINs.
However without the use of CQUINs (and quality accounts at King’s) it is difficult to engage providers to measure additional areas of patient experience and implement improvement plans. Clinical Quality Review Groups (CQRGs) are used as a way of exerting influence to encourage providers to improve on certain areas of patient experience (refer to recommendation 3 in Section 5).
e) There are good examples of how triangulation of national and local data (King’s maternity care)
and the triangulation of patient experience information and complaints information (mental health) has been effective in driving improvements. This approach should be enhanced as part of the CCG’s assurance of commissioned providers (refer to recommendation 2 in Section 5).
ENC B
17 Chair: Dr Jonty Heaversedge Chief Officer: Andrew Bland
The best possible health outcomes for Southwark people
4.2 Strategic Planning
Examples of Good Practice
The CCG has been effective at carrying out large engagement events to gather patient experience
information. The recent Call to Action event for example was attended by around 70 people and
produced a wealth of information that was captured in a well written report.
Recently, the CCG used a Survey Monkey to gather information on the experience of users of the
talking therapy services to inform the development of the new service. The survey returned over 60
responses and asked a number of questions across a range of themes. The survey also provided
free text boxes to allow for more detailed feedback. Widely accessible electronic surveys like this
can provide valuable benefits particularly if they are coupled with survey methods aimed at hard to
reach and the most vulnerable patient groups.
Services like Survey Monkey allow for:
1) Wide coverage and easy access.
2) Eliminates manual input of survey responses.
3) Real time data charts and analysis.
The findings from the survey confirmed that patients wanted access to the service after 6 pm, they
wanted the choice of a range of languages and wanted a more even number of male and female
counsellors.
There are a number of additional examples of when patients have made a valuable contribution to more focussed pieces of work, like in the review of muskuloskeletal physiotherapy services. Engagement here involved a workshop of current users to inform potential changes to the service (box 2).
Box 2: Provision of physiotherapy and osteopathy services.
In early 2013 the provision of physiotherapy and osteopathy in Southwark was reviewed.
The review found that as well as the provision of physiotherapy at hospitals, several GP
surgeries also provided physiotherapy or osteopathy services.
A consultation engagement event was held with patients on Wednesday 25 September.
14 patients attended, most of who had used a non-core funded Osteopathy Service at
their local practice and/or Physiotherapy Services at various sites. Discussions at the
event focused on osteopathy in particular as a number of patients were concerned about
changes to their current practice based osteopathy services. Although the CCG had
decided that a change to the service was necessary to provide a more equitable and
evidence based service; patients were engaged on a number of options on how the
future service may look.
ENC B
18 Chair: Dr Jonty Heaversedge Chief Officer: Andrew Bland
The best possible health outcomes for Southwark people
The Continuing Care team has worked to support patient representatives during the development of adult personal health budgets (Box 3).
Areas for Improvement
The CCG has generally been less effective at smaller scale engagement of patients at the planning stage of the commissioning cycle. CCG leads have highlighted that on occasion small scale engagement events are not organised at the very beginning of the planning cycle and are carried out to justify strategic decisions that have already been made. It can be difficult for lay people to make valuable contributions at this stage because:
They are sometimes not well informed about the case for change.
Strategic topics are not presented in a way that are easily understood.
The scope of some service changes are sometimes limited by centrally mandated specifications and the need to achieve national targets which limits the scope of engagement.
A number of CCG leads have suggested that asking patients to focus on one or two areas that really benefit from a patient perspective, like the production of information leaflets, may be more beneficial than engaging people on high-level technical issues.
Box 3: Adult Personal Health Budgets The continuing care team tried to overcome these issues when inviting patients to sit on a
steering group to oversee implementation of adult personal health budgets (PHBs). They
chose one patient who was representative of wider groups and one patient who had lived
experience of holding a PHB. To ensure the lay members made useful contributions they
were kept well informed of relevant issues outside of steering group meetings and
additional effort was made to ensure communication was always in a clear format.
ENC B
19 Chair: Dr Jonty Heaversedge Chief Officer: Andrew Bland
The best possible health outcomes for Southwark people
4.3 Procurement
Examples of Good Practice
There are examples of where the CCG has used patient experience information and engagement to
inform procurement decisions.
The Talking Therapies procurement will involve two service users on its steering group and a
representative from Healthwatch Southwark. The process will require a significant time commitment
from those involved as there are monthly meetings (potentially moving to fortnightly) and the
interview process has been planned to take place over a week. The service users are being
supported to take part by receiving technical briefings and receiving compensation.
For mental health services, patients’ involvement in procuring services has been limited. There have
been a couple of examples of children being involved at this stage. Children fed back their views as
part of the 1000 voices initiative and were also involved in helping select early health providers. The
Southwark Youth Council (SYC) and the looked after children user group have been involved in
designing and procuring services (Box 4).
Areas for Improvement
For out of hospital pathways, the general approach is to engage patients through workshops to
discuss procurement options. However, for some services a patient representative from EPEC has
sat on procurement panels even though they are not a current or ex-user of the service (Box 5).
Box 4: Southwark Youth Council Southwark Youth Council (SYC) is a voice for young people run by young people. Young people are redesigning public services and influencing how neighbourhoods are improved. The SYC has been proactive in various decision making activities across the borough. The main members of SYC are young people living, attending school, and working in Southwark. They hold events and workshops as a way for young people to express their views and for SYC to take those views to the decision makers.
Box 5: Procurement of Out of Hospital Pathways Commissioners tried to encourage service users of the ENT and dermatology services to
be involved in the procurement of the new services. However, it was difficult to generate
interest from patients because involvement could require a level of commitment that was
beyond most patients particularly as meetings take place during working hours.
In February 2013, the procurement processes of the ENT and dermatology services were
presented at EPEC. It was agreed at the meeting that there should be a patient
representative present on both procurement panels and both of these representatives
were members of EPEC.
ENC B
20 Chair: Dr Jonty Heaversedge Chief Officer: Andrew Bland
The best possible health outcomes for Southwark people
4.4 Monitoring and Evaluation
Examples of Good Practice
There are a number of good examples of where patient experience information is being used to
monitor and evaluate services.
At King’s, commissioners monitor patient experience performance through monthly HRWD reports
and quarterly Quality and Governance reports and discuss issues further at monthly CQRG
meetings. Working jointly with King’s, commissioners have used patient experience data to initiate
service improvements through the development of CQUINs.
Commissioners have been able to identify areas for service improvement at King’s through patient
experience data and the trust has worked jointly with commissioners to develop CQUINs to improve
performance. CQUINs can be an effective way of improving performance temporarily, however if the
improvements are primarily driven by incentives then there is a risk that performance could
Box 6: King’s Maternity service
In 2011/12, a study of maternity services in England showed that London had one of the
highest maternity death rates; was double the national average and that women in
London were found to be the least satisfied with the care that they received during labour
and birth. King’s HRWD surveys also showed that patient experience was below target
for maternity services. This prompted commissioners to work with the trust to develop a
CQUIN for 2012/13 to improve performance, which did result in an improvement in
performance and the achievement of most of the CQUIN targets.
Box 7: King’s Worst Wards
In 2011/12 commissioners and King’s used the data that informs the HRWD surveys to
identify the worst performing wards. A CQUIN was introduced in 2012/13 to incentivise
performance improvement in those wards and resulted in improvement in patient
experience and achievement of all CQUIN targets.
Box 8: King’s Outpatients
In 2012/13, a CQUIN was developed with the trust to measure and improve patient
experience in specific areas of outpatients that had been identified as requiring
improvement from local HRWD survey results. The areas were identified from survey
results in 2011/12 and a number of indicators were developed to track and improve
performance in areas identified as needing the most improvement.
ENC B
21 Chair: Dr Jonty Heaversedge Chief Officer: Andrew Bland
The best possible health outcomes for Southwark people
deteriorate following removal of the incentive. Inserting penalties into contracts is another potential
approach; however it has proven difficult to insert penalties for indicators that are not nationally
mandated making it difficult to embed quality improvements contractually.
The CSU works with King’s to improve areas of patient experience by discussing patient experience
issues covered in quarterly quality reports at CQRG meetings. It is difficult to quantify the
effectiveness of this approach on improving patient experience.
Quality Accounts
King’s annual Quality Accounts summarise their performance and improvements against quality
priorities and objectives set by themselves for 2013-2014. It also outlines those they have agreed
for the coming year 2014-2015. The main domains of the report cover patient safety, clinical
effectiveness and patient experience. The two main areas of development under patient experience
for 2013/14 were improving outpatient experience and improving patient experience of discharge.
1. Improving outpatient experience in Suite 1
The aims of 13/14 were to agree which 5 questions are most in need of improvement within
Suite 1, develop action plans and agree improvement target percentage for Q2, 3 and 4. The
trust also aimed to achieve a decrease in patient complaints relating to the outpatient
department.
2. Improving patient experience of discharge
The aims of 13/14 were to improve trust-wide patient satisfaction in two key areas; providing
information about medication after discharge and providing patients with information on
what to do and who to contact if they have a concern after discharge. These are questions in
the HRWD local surveys and so improvement could be tracked on a monthly basis.
Mental Health - Examples of effective use of patient experience information
The CCG Mental Health Commissioning Team regularly engage with patient forums and use the
information to feedback to services for improvement. Some of the common issues that have been
raised include day centre provision, inpatient ward food, inpatient ward cleanliness and staff
attitude.
a) Day centre provision has been a particularly common topic of discussion over the last 12
months coupled with the introduction of personal budgets. Council members have voiced
frustration that they have not been kept up-to-date with latest information which led to project
leads organising a news letter for service users.
b) Comments on the quality and quantity of food on wards are fed directly back to the SLaM ward
manager who in turn picks up the issue with the hospitality contract holders.
ENC B
22 Chair: Dr Jonty Heaversedge Chief Officer: Andrew Bland
The best possible health outcomes for Southwark people
Any improvements are fed back to the group at the following meeting.
The CCG Mental Health Commissioning Team monitor performance at SLaM through its PEDIC
reports. Areas of concern highlighted by PEDIC survey results are raised with the trust through
commissioner responses and are discussed at quality meetings with the trust.
The CCG Mental Health Commissioning Team also tries to triangulate patient experience
information with complaints reports and again feeds back concerns in commissioner responses.
Commissioners have also conducted ward inspections at SLaM and have picked up a number of
issues that can affect patient experience from ward staff. These have been raised at quality
meetings with the trust.
Areas for Improvement
The triangulation of patient experience information with other quality and safety information has
been improving. The quarterly CCG Quality Report has included some information on patient
experience and information from the CCG’s engagement structure and public engagement events;
this is a good start, however future reports should include more patient experience information to
allow for more consistent and wider triangulation.
Box 10: SLaM patient experience CQUIN
In 2013/14, a patient experience CQUIN was included in the SLaM contract. The CQUIN
involved developing patient focus groups to feedback on areas that require improvement.
The issues highlighted were staff attitude, feeling respected by staff and the environment.
SLaM then developed action plans to improve these areas and this was monitored by the
CCG mental health team. In Q4, a patient focus group met to review progress against
improvement plans and acknowledged that there had been improvement in all the areas
highlighted at the start of the year.
In 2014/15, commissioners requested that SLaM become more pro-active in asking for
patient feedback. It was agreed that the trust be more proactive in asking patients if they
would like to feedback on their experiences at discharge on an ipad. The ipad also gave
patients the opportunity to fill in PEDIC surveys and write comments on their experiences.
Box 9: Southwark Mind
NHS Southwark CCG commission Lambeth and Southwark Mind to run the Southwark
User Council. All members are current or previous service users and represent
commissioned services and visit their service on a regular basis asking for feedback from
current service users. Feedback is then discussed at the Council monthly meetings
(closed for the first 2 hours) and then open with Commissioners and Service providers in
attendance for the next 1.5 hours. Two members of the council sit on the Mental Health
Programme Management Group.
ENC B
23 Chair: Dr Jonty Heaversedge Chief Officer: Andrew Bland
The best possible health outcomes for Southwark people
CQRGs review patient experience information and this provides an ideal opportunity to triangulate
patient experience information with other quality and safety information. CQRG leads report back to
the Integrated Governance and Performance Committee (IGP) and Governing Body on the main
quality and safety concerns at providers, however there isn’t always a specific focus on patient
experience information. If more patient experience information is reported back, commissioners
could then add their insights to the data and enhance the performance monitoring process.
Organising and Managing the Data
Commissioners have fed back that there is a lack of capacity and a system to collect, analyse and
triangulate data effectively.
Quality Analysis Database
Birmingham CCG has been developing a system that they hope will revolutionise the way they use
patient experience information. They are developing a Quality Analysis Database (QUAND) which
will eventually form a substantial central repository for all the CCG’s patient experience data. They
are designing the system to automatically extract information from national databases like NHS
Choices and Patient Opinion and from local databases like the primary care patient experience
module, Datix. They are also trying to triangulate this data with quality and safety indicators like falls
and pressure ulcers and so the system will be able to automatically extract data from the Strategic
Executive Information System (STEIS).
The system will also allow users to input information manually making it capable of capturing soft
data. Data will be broken down to a granular level allowing users to search at provider, specialty
and even ward level to try and identify trends.
Insights Dashboard
London CCGs are considering procuring a London based patient experience dashboard that will
bring together a wide array of publically available information covering hospitals and GP practices.
There are three different subscription levels available for the CCG to choose from - subscription
levels offer varying degrees of detail and bespoke information. Southwark CCG managers will be
attending the London CCG Engagement Leads meeting on 29 September to review subscription
options.
ENC B
24 Chair: Dr Jonty Heaversedge Chief Officer: Andrew Bland
The best possible health outcomes for Southwark people
5. Recommendations
1) The CCG could further benefit from the use of patient experience information by triangulating national and local information and by triangulating patient experience information with complaints and quality and safety information. This could provide a more comprehensive level of intelligence which could help commissioners identify trends. Commissioners should consider broadening the source of information to include more frequent use of online forums; and reports and intelligence from organisations that work closely with patients like charities and Healthwatch Southwark.
To allow for effective triangulation, the CCG needs to capture more qualitative data to add greater depth to the quantitative data we currently collate. This can be through issues identified in the CCG Quality Report, in the form of written feedback, local ‘soft’ intelligence captured from engagement with patients. The CCG should consider the resource implications of this development.
2) CCG officers should develop quarterly briefings to CQRG leads, with intelligence gained from a triangulated approach to quality and safety. This will allow CQRG leads to monitor performance with providers more effectively and support them to improve services.
3) Commissioners should further encourage providers (particularly large providers) to design and implement improvement plans to enhance patient experience in specific areas of underperformance. Providers are not keen to engage on this in the absence of financial incentives so a collaborative approach overseen by CQRG meetings could be developed.
4) The CCG should work with services that do not currently provide patient experience information to develop appropriate means of capturing information. The regular capture of patient experience and the implementation of improvement plans should be included in all provider contracts.
5) Commissioners should consider ways of understanding in greater detail patients, carers and their families’ experience of care across pathways and not just at provider sites.
a. The CCG should consider commissioning broad-based surveys of patients to understand from a wider sample more about their experience of the care they have recently received. This would need to look at specific issues that we know are important (e.g. the discharge process; compassion of nursing care etc.).
b. A project should be initiated to understand in greater depth some of the issues facing patients more likely to use health services. Patients with Continuing Healthcare packages of care may be a useful cohort to work with in the first instance. The CCG’s continuing care team works closely with individual patients and their carers and could conduct in-depth interviews to understand in more detail patients’ experiences.
6) The CCG should ensure that patient representatives are involved in all clinical procurements, including procurement panels. Patient representatives can help the CCG identify innovation and good practice and identify providers who meet the needs of patients. For this to happen; they should be experienced patients or carers in the specific area and be offered support by the CCG to fulfil this role.
ENC B
25 Chair: Dr Jonty Heaversedge Chief Officer: Andrew Bland
The best possible health outcomes for Southwark people
Appendix A - Friends and Family Test
The Friends and Family Test (FFT) is a single question survey which asks patients whether they
would recommend the NHS service they have received to friends and family who need similar
treatment or care. The FFT was rolled-out across the NHS following recommendations made by the
Nursing and Care Quality Forum (NCQF) to the Prime Minister.
Initially, the test had two main aims as defined by the Department of Health:
To provide a comparable performance measure across providers
An improvement tool to enhance understanding about where and how improvements could be made.
A recent review of the FFT by NHS England endorsed the tests potential to facilitate service
improvement; however it did recognise its limitation in providing a comparable performance
measure across providers.
FFT was initially rolled out for providers of NHS funded acute services for inpatients (including
independent sector organisations that provide acute NHS services) and patients discharged from
A&E (type 1 & 2) from April 2013. As of 1 October 2013 the survey was extended to include all
women of any age who use NHS funded maternity services and from 1 December 2014 it will be
available in GP practices. The test will be rolled out even further next year; from January 2015 it
will be rolled out in mental health and community services and from 1 April 2015, it will be
expanded to NHS dental practices, ambulance services, patient transport services, acute
hospitals outpatients and day cases.
The 2014/15 National CQUIN
FFT is part of the 2014/15 national CQUIN which incentivises providers to improve performance.
The 2014/15 CQUIN will incentivise the following:
1. Early implementation of the test in outpatient and day case departments by 1 October 2014.
2. Increasing or maintaining response rates in acute inpatient services.
i. A baseline response rate in Q1 of at least 25 per cent and by Q4 a response rate that is
both (a) higher than the response rate for Q1 and (b) 30 per cent or over; or
ii. Maintaining a response rate that is over 30 per cent.
3. Increasing or maintaining response rates in A&E.
i. A baseline response rate of at least 15 per cent and by Q4 a response rate that is both (a)
higher than the response rate for Q1 and (b) 20 per cent or over; or
ii. Maintaining a response rate that is over 20 per cent.
ENC B
26 Chair: Dr Jonty Heaversedge Chief Officer: Andrew Bland
The best possible health outcomes for Southwark people
The Question
In its current form, the test asks the following standardised question: “How likely are you to
recommend our ward/A&E department to friends and family if they needed similar care or
treatment?” Patients will use a descriptive six-point response scale to answer the questions with the
following response categories:
1 Extremely likely
2 Likely
3 Neither likely nor unlikely
4 Unlikely
5 Extremely unlikely
6 Don’t know
Friends and Family Test results are calculated using underlying “Net Promoter Score” methodology.
All patients within the target groups will be given the opportunity to respond to the Friends and
Family Test. Patients must be surveyed at, or within 48 hours of, discharge.
Scoring Methodology
The following example explains how the FFT score is calculated. Assume that a trust records the
following results.
Total number of responses
Extremely likely 114
Likely 54
Neither/nor 24
Unlikely 2
Extremely unlikely 6
Total 200
ENC B
27 Chair: Dr Jonty Heaversedge Chief Officer: Andrew Bland
The best possible health outcomes for Southwark people
The FFT score is calculated by the following formula:
Minus
Friends and Family Test Score: 57 – 16 = 41
A common query regarding the scoring methodology is, why those that respond "likely" are
excluded from the calculation?
FFT is meant to drive excellence within patient experience so, if a patient responds with ‘likely’ then
they will not have had the best experience. However, they are not excluded from the calculation,
because they are included in the denominator. If a patient chooses to respond “likely” instead of a
less positive category, that will lead to a higher score. If they choose to respond “likely” instead of
“extremely likely” that will lead to a lower score. Likely responses therefore have an important
influence on the resulting score.
Proportion of respondents who
would be extremely likely to
recommend (response: “extremely
likely”)
114 ÷ 200 (x 100) = 57
Proportion of respondents who would
not recommend (response: “neither
likely nor unlikely”, “unlikely” &
“extremely unlikely”)
24+2+6) = 32 ÷ 200 (x 100) = 16
28 Chair: Dr Jonty Heaversedge Chief Officer: Andrew Bland
The best possible health outcomes for Southwark people
Appendix B - Sources of Patient Experience Information
Source/Report Detail Source
National Voices National coalition of health and social care charities in England. Focus on patient centred care
http://www.nationalvoices.org.uk/
King’s How Are We Doing (HWRD)
Opportunity for patients to fill in surveys on many services areas: Outpatients, Emergency Department, Emergency Dental, Early Pregnancy Unit, Maternity Services, Dental Outpatient Services, Inpatients and Day surgery patients. Carers, relatives, friends or visitors can fill in How are we doing? Comments and Suggestions cards. King’s produce monthly reports detailing results.
Jessica Bush – King’s Head of Engagement and Patient Experience. [email protected]
King’s In Conversation
Listening events with patients regarding the following questions: Are patients always our first priority? Would you recommend our hospitals as places to receive care? What do you think we could do better?
http://www.kch.nhs.uk/about/get-involved/groups-events/kings-in-conversation
King’s In Your Shoes Opportunity for King’s staff to listen to patient views. http://www.kch.nhs.uk/about/get-involved/groups-events/in-your-shoes
King’s Maternity Services Liaison Committee
The MSLC is a group that includes mums as well as midwives, doctors and other health professionals. The group meets about every two months.
http://www.kch.nhs.uk/about/get-involved/groups-events/maternity-services-liaison-committee
King’s Cancer Voices A group made up of patients, relatives and carers affected by cancer and health professionals.
http://www.kch.nhs.uk/about/get-involved/groups-events/cancer-voices
King’s Have Your Say Repository of patient comments http://www.kch.nhs.uk/about/haveyoursay
29 Chair: Dr Jonty Heaversedge Chief Officer: Andrew Bland
The best possible health outcomes for Southwark people
Source/Report Detail Source
GSTT Feedback Opportunity for patients to fill in surveys on Friends and Family test, general surveys, service-specific surveys and Community services.
http://www.guysandstthomas.nhs.uk/patients-and-visitors/feedback/your-feedback.aspx
GSTT 2012-13 annual Patient Experience and Engagement Report
Provides an overview of the work that has taken place during 2012-13
GSTT Board papers http://www.guysandstthomas.nhs.uk/about-us/publications/board-papers/2013-board-papers.aspx#na
The GP Patient Survey GP survey results available at CCG and individual practice level. http://practicetool.gp-patient.co.uk/Ccg/Search?id2=southwark%20ccg&index=0
CQC National Maternity Survey 2013
Key findings from the 2013 national survey of women’s experiences of maternity care.
http://www.cqc.org.uk/public/publications/surveys/maternity-services-survey-2013
CQC National A&E Survey 2012
Asks people about their experiences of A&E departments in 2012. http://www.cqc.org.uk/public/reports-surveys-and-reviews/surveys/accident-and-emergency-2012
CQC National Inpatient Survey 2012
Asks patients who were admitted to hospital services about their experiences in 2012.
http://www.cqc.org.uk/public/reports-surveys-and-reviews/surveys/inpatient-survey-2012
CQC National Outpatient Survey 2011
Asks patients about their views on their most recent visit to an outpatient department.
http://www.cqc.org.uk/public/reports-surveys-and-reviews/surveys/outpatient-survey-2011
30 Chair: Dr Jonty Heaversedge Chief Officer: Andrew Bland
The best possible health outcomes for Southwark people
Source/Report Detail Source
CQC Community Mental Health Survey 2013
Asks patients about their experiences of receiving community mental health services.
http://www.cqc.org.uk/public/publications/surveys/community-mental-health-survey-2013
NHS England National Cancer Patient Experience Programme
Asks patients about their experiences of receiving cancer care. http://www.quality-health.co.uk/surveys/national-cancer-patient-experience-survey
Local quarterly Community surveys
Includes; Children occupational therapy, children physiotherapy, children speech and language therapy, District nursing, foot health, health visiting, paediatrics and reproductive and sexual health
Jean Young, Meridian Desktop
Friends and Family Test
The Friends and Family Test is a single question survey which asks patients whether they would recommend the NHS service they have received to friends and family who need similar treatment or care. Currently the test covers;
i) A&E ii) Inpatients iii) Maternity
http://www.england.nhs.uk/statistics/statistical-work-areas/friends-and-family-test/friends-and-family-test-data/
SLaM PEDIC surveys and reports
There are a number of patient experience surveys carried out by SLaM which cover; outpatients, inpatients and trustwide.
Southwark CCG Mental Health team
Cancer Patient Experience Survey: Insight Report and League Table 2012–13
The 2012–13 National Cancer Patient Experience Survey (CPES) captures a picture of progress among hospital trusts. London trusts are amongst the worst performing in the country
http://www.macmillan.org.uk/Documents/AboutUs/Research/Keystats/2013CPESInsightBriefingFINAL.pdf
31 Chair: Dr Jonty Heaversedge Chief Officer: Andrew Bland
The best possible health outcomes for Southwark people
Appendix C – HRWD Tracker