Faculty showcase 2013 - Opening up clinical performance

Opening up clinical performancevariation and financial incentives in Primary Care quality of care

Evan (Evangelos) Kontopantelis1

1Institute of Population Health

Faculty Research Series, 25 September 2013

Kontopantelis (IPH) Variation and financial incentives 25 September 2013

Outline

1 People

2 The scheme...

3 The research journey

4 Summary


Collaborative work!

Martin RolandTim DoranDavid ReevesStephen CampbellBonnie SibbaldMatt SuttonHugh GravelleJose Valderas...and others...


Improving quality of carea (very) juicy carrot...

A pay-for-performance (p4p) program kicked off in April 2004 withthe introduction of a new GP contract

General practices are rewarded for achieving a set of quality targetsfor patients with chronic conditionsThe aim was to increase overall quality of care and to reducevariation in quality between practices

The incentive scheme for payment of GPs was named the Qualityand Outcomes Framework (QOF)Initial investment estimated at £1.8 bn for 3 years (increasing GPincome by up to 25%)QOF is reviewed at least every two years


Quality and Outcomes Frameworkdetails for years 1 (2004/5) and 7 (2010/11)

Domains and indicators in year 1 (year 7):Clinical care for 10 (19) chronic diseases, with 76 (80) indicatorsOrganisation of care, with 56 (36) indicatorsAdditional services, with 10 (8) indicatorsPatient experience, with 4 (5) indicators

Implemented simultaneously in all practices (a control group wasout of the question)Practices are allowed to exclude patients from the indicators andthe payment calculationsInto the 10th year now (01Mar13/31Apr14); cost for the first 9years was well above the estimate at £9 bn approximately


Quality and Outcomes Frameworkwhat does it mean?

If you haveAtrial fibrillation, asthma, HT, cancer, CHD,HF, CKD, COPD, dementia, depression,diabetes, epilepsy, SMI, osteoporosis, PAD,stroke, hypothyroidismbut also covers LD, obesity, palliative care,sexual health, smoking

e.g. for diabetesmeasure-control BP, chol, glucoseimmunise for influenzaphysical exams (retinal screen, foot exam)...and more... 17 indicators in total


Quality and Outcomes Framework2005 reaction

Drugs cut strokes by a third

Doctors get 20%pay rise just for doing their jobs

DOCTORS are getting 20 percent pay rises for doing simpletasks “they should always havedone,” it was claimed last night.

Health economists, includingGovernment advisers, say thatunder their new contracts GPsare being paid huge bonuses tomeet “easy” targets that do littleto improve patients’ lives.

Such is the level of concernover the value for taxpayers thatministers are already meetingdoctors’ leaders in order to settougher goals.

The move comes just weeksafter the Government boastedhow most GPs had met the tar-gets for performance-related pay.

But Alan Maynard, professor ofhealth economics at York Univer-sity, said yesterday: “It’s quiteridiculous. The Government’s

C H O L E S T E R O L - l o w e r i n gdrugs called statins could pre-vent heart attacks and strokes ina third of patients with diseasedarteries, a study shows.

Most doctors consider statintreatment only when blood cho-lesterol is above a certain levelbut research by British andAustralian scientists showsmany more people with lowercholesterol could also benefit.

Statins work by blocking theaction of an enzyme that enablesthe liver to produce cholesterol.

Research Council scientist DrColin Baigent, who coordinated

By Michael Day

By Geoff Marsh

spent all this money and hasgiven GPs 20 per cent rises just totry and get them to do what theyshould always have done.

“Now they’re being paid extrato look out for people with highblood pressure, which causesheart attacks and strokes, and tomonitor and treat them – but theyshould have been doing that allalong. It’s not rocket science.”

There have already been twoWhitehall meetings after the dis-closure last month that, on aver-age, GP practices achieved 91 percent of the available perfor-mance-related bonuses under thenew contract.

After achieving bonuses, aver-age earnings of a practice surgedby £75,000 while the salary of the

average GP partner rose to£100,000 a year. The first meetingto set tougher targets came with-in a week of the announcement.The second was held last week.

One BMA negotiator, Leeds GPRichard Vautrey, said: “We arelooking at making amendments,but at present we’re not in the position to say exactly whatthese are.

“We would expect the revisedframework to be ready by the endof the year.”

Apart from blood pressure,other target areas have been sin-gled out as too modest. ChrisHam, professor of health policyand management at BirminghamUniversity, said: “My view is that,in principle, the new GP contractis a good thing – GPs are beingpaid on the basis of how well they

treat patients and not just accord-ing to how large their list size is.

“But the fact so many of themmet the targets in the first yearsuggests they were too easy.”

A BMA spokeswoman deniedthe targets were too easy. “Theydemonstrate the vast majority ofdoctors are already providinghigh-quality care for theirpatients,” she said.

Simon William, director of pol-icy at the Patients’ Association,said, however: “The BMA wouldtrumpet the GPs’ performance.It’s a trade union, it’s there torepresent doctors not patients.”

A spokesman for the Depart-ment of Health said: “Theseexcellent results show the newcontract is giving GPs a realincentive to improve the qualityof care.”

the research at Oxford Univer-sity, said: “This study showsstatin drugs could be beneficialin a much wider range ofpatients than is currently con-sidered for treatment.

“What matters most is doc-tors identify all patients at riskof a heart attack or stroke –largely ignoring their presentblood cholesterol level – andthen prescribe a statin at a dailydose that reduces their choles-terol substantially.

“Lowering the bad (LDL) cho-

lesterol … with a statin shouldreduce the risk of a heart attackor stroke by at least one third.”

The study, published online inThe Lancet, suggests patientsgiven a statin would also experi-ence greater benefits if doctorsaimed to achieve larger reduc-tions in cholesterol levels.

The analysis also providesinformation about the safety ofstatins. Earlier studies hadraised concerns they could belinked with an increased risk ofcertain cancers or diseases.

British Heart FoundationProfessor Rory Collins, one of

the study authors, said: “Thiswork shows clearly that statinsare very safe. There is no goodevidence that statins cause can-cer and nor do they increase therisk of other diseases. Althoughstatins can cause muscle pain orweakness, our study shows seri-ous cases are extremely rare.”

Study co-author ProfessorJohn Simes, of Sydney Univer-sity, said: “The benefits of statintreatment were seen in all pat-ient groups, including women,the elderly, individuals with dia-betes and those with and with-out prior heart attack or stroke.”

OPINION 12 DIARY 26 LETTERS 38 TV 39 STARS 43 CROSSWORDS 45 CITY 48 SPORT 52

If you have a story:

Ring 0871 520 2982If you have a photo:

Ring 0871 520 7171or send mobile phone pictures

by texting EXPRESS to07843 500 911

or email to:[email protected]

DO YOU HAVE ASTORYOR APHOTO?

We asked in our poll on Saturday: Should Blair

resign over sendingtroops to Iraq?

Yes:97% No: 3%

RESULTS OFPHONE POLL

3 6 5

8 9 3 4

9 4 5 3 6

9 5 7 1 6

1 5

3 9 5 8 7

7 5 1 4 3

5 2 9 8

1 8 6

Fill in all the squares so that each row, each column,and 3x3 square contain all the digits from 1 to 9. SUDOKU

7 8 6 1 9 3 5 2 4 2 1 9 5 7 4 8 6 3 5 3 4 8 2 6 9 7 1 4 9 7 2 1 5 6 3 8 8 2 5 6 3 7 1 4 9 3 6 1 9 4 8 7 5 2 9 7 2 3 5 1 4 8 6 6 4 3 7 8 9 2 1 5 1 5 8 4 6 2 3 9 7

Yesterday’s solution

This is the Quick Sudoku for the solver in a hurry.

For today’s solution call:0905 072 0051For extra clues call: 0905 072 0052(calls cost 75p per minute from a BT landline)

Another Sudoku: PAGE 45

● The Daily Express Sudoku Book is available from October 3 at £5.99 (p&p included for UK). Don’t miss out, pre-order your copy now! Send cheque or postal order to Express Bookshop, PO Box 200, Falmouth, Cornwall TR11 4WJ or call 0871 434 6091, or you can order on-line at www.expressbookshop.com

LOTTO NUMBERS

WEDNESDAY LOTTOSeptember 21

2 23 33 35 40 42 25Estimated jackpot is £2.8m

Extra: 2 13 18 25 28 47 Estimated jackpot is £8.7m

T-Ball: 5 14 16 18 32 2

DAILY PLAYSaturday1 7 8 12 21 23 24

SATURDAY LOTTOSept 242 11 27 29 36 44 15Estimated jackpot is £8.4mExtra: 3 11 16 34 40 45Jackpot is £9m this weekT-Ball: 6 8 20 26 34 5

EUROMILLIONSSeptember 231 26 31 34 47Lucky star numbers: 4 9

Amsterdam . . .c 17 63Athens . . . . . . .f 25 77Berlin . . . . . . . .sh 16 61Cape Town . . . .s 16 61Corfu . . . . . . . .f 25 77Dublin . . . . . . .c 18 64Faro . . . . . . . . .s 24 75Geneva . . . . . .f 20 68Gibraltar . . . . . .f 23 73Guernsey . . . . .s 17 63Los Angeles . . .f 21 70Madrid . . . . . . .s 22 72Malaga . . . . . . .f 24 75Mallorca . . . . . .s 24 75Malta . . . . . . . .s 25 77Miami . . . . . . . .f 31 88Moscow . . . . . .c 14 57New York . . . . .c 22 72Nice . . . . . . . . .s 24 75Paris . . . . . . . . .c 20 68Rome . . . . . . . .s 23 73Stockholm . . . .s 20 68Sydney . . . . . .r 19 66Tel Aviv . . . . . . .s 34 93Tenerife . . . . . .s 28 82

Summary: Showers in places.

World yesterday

Europe today

C F A windy day with rain spreadingeastwards across most areas,

but the south and east of Englandwill have a fine start.

Outlook tomorrow:

North West East AngliaBreezy with scatteredshowers, locally heavy. A fresh south-westerly wind.High 18C (64F).

Early rain will clear to leavesunny spells. A risk of showerslater. Moderate south-westerlywinds. High 20C (68F).

Northern Ireland London/South EastSunny intervals andshowers. Cool and breezywith a strong south-westerlybreeze. High 16C (61F).

After a cloudy start with rain in places, it will soonbecome drier with spells of sunshine. High 21C (70F).

Wales SouthSome sunshine at times, but with scattered showers,frequent in the west. Freshwinds. High 18C (64F).

A cloudy start in places, but turning brighter withsunny spells. Afternoonshowers. High 20C (68F).

Midlands South WestSunny spells, but with a few showers from the west.Moderate south-westerlywinds. High 20C (68F).

Some sunshine at times,but with occasional showers.Moderate westerly winds.High 19C (66F).

North East/Yorks Channel IslesA bright day with sunny spells,but breezy with the chance of a shower A fresh breeze.High 18C (64F).

Early rain will soon clear to leave sunny intervals, but with a risk of a shower.High 18C (64F).

Scotland SeaGusty at times with frequent,heavy showers in the west.Drier and brighter further east. High 18C (64F).

North Sea: moderate.Irish Sea: moderate.Channel: moderate.

Rain 19C/66FAmsterdam

Fair 19C/66FBrussels

Fair 22C/72FFrankfurt

Fair 23C/73FGeneva

Fair 22C/72FParis

Fair 25C/77FRome

Warmest: Chivenor 19C (66F).Coldest: Aboyne 2C (36F).Wettest: S’th Uist Range 0.55in.Sunniest: Beccles 5.0hr.

BRITAIN EXTREMES:(24 hours to 2pm yesterday)

c-cloudy, dr-drizzle, f-fair, fg-fog,g-gales, h-hail, hz-haze, m-mist,r-rain, sh-showers, s-sun, sn-snow, sl-sleet, th-thunder,

Around Britain

HIGH TIDE: London Bridge: 9.03am, 10.00pmLiverpool: 7.14am, 7.53pmGreenock: 8.31am, 7.42pmDover: 7.04am, 8.01pm

Belfast 0.0 0.13 9 17Birmingham 2.0 0.00 9 17Bristol 2.3 0.00 10 19Cardiff 3.0 0.01 11 20Edinburgh 2.0 0.04 7 18Glasgow 0.9 0.94 10 15Leeds ** ** 9 16London 3.4 0.00 12 18Manchester 1.1 0.00 8 17Newcastle ** 0.00 10 17Norwich 3.7 0.04 7 16Oxford 1.5 0.00 7 17

CITIES

Sun Rain Temp(hr) (in) (min)(max)

Lighting-up times YesterdayBelfast 7.10pm-7.20amBirmingham 6.54pm-7.03amBristol 6.57pm-7.06amGlasgow 7.03pm-7.14amLondon 6.47pm-6.56amManchester 6.55pm-7.05amNewcastle 6.52pm-7.03am

MOON rises: –, Sets: 5.14pmSUN rises London: 6.55am, Sets: 6.47pmManchester rises: 7.03am, Sets: 6.55pm

Moon, sun and tides

25

30

30

20

15

13

13

16

18

18

18

19

17

1818

20

20

20

21

Weather today

Miami 31C 88F / Moscow 14C 57F

All forecasts and maps provided by PA WeatherCentre

Daily Express Tuesday September 27 20052


Quality and Outcomes Frameworklater reaction


Key questionssince QOF was parachuted in Primary Care

What were levels of achievement when QOF introduced?Did they change over time?What was happening pre-QOF? Was there an increasing trend?Practice characteristics asssociated with high performance? Size?Gap between practices in affluent and deprived areas?Exception reporting rates and evidence for practice ‘gaming’?What happened to non-incentivised aspects of care?Did the intervention effect vary by population group?What about patient satisfaction?Various computer systems used. Does system choice affect care?


Timeline

1998 2014

1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013

2003QuIP tp #2

2005QuIP tp #3

2009GP Patient

Satisfaction Survey

01/04/2004QOF introduced

1998 - 2007QuIP

1998QuIP time point #1

2005QOF y1*

2008GPRD (CPRD)

access

2007QuIP tp #4

2013QOF y9

2004 - 2014QMAS


Timeline

1998 2014

1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013

2003QuIP tp #2

2005QuIP tp #3

2009GP Patient

Satisfaction Survey

01/04/2004QOF introduced

2005Where's the sun?

1998 - 2007QuIP

1998QuIP time point #1

2005QOF y1*

2008GPRD (CPRD)

access

2007QuIP tp #4

2013QOF y9

2004 - 2014QMAS


Levels of achievement in the first year (2004-5)All 8000+ English practices

Median reported achievementwas 83.4% (IQR: 78.2-87.0%)Very high achievement levelsSmall effects ofsociodemographiccharacteristicsException reporting notextensive but the strongestpredictorA few practices scored high byexcluding large numbers ofpatients

T h e n e w e ng l a nd j o u r na l o f m e dic i n e

n engl j med 355;4 www.nejm.org july 27, 2006 375

special article

Pay-for-Performance Programs in Family Practices in the United Kingdom

Tim Doran, M.P.H., Catherine Fullwood, Ph.D., Hugh Gravelle, Ph.D., David Reeves, Ph.D., Evangelos Kontopantelis, Ph.D., Urara Hiroeh, Ph.D.,

and Martin Roland, D.M.

From the National Primary Care Research and Development Centre, University of Manchester, Manchester, United King-dom. Address reprint requests to Dr. Doran at the National Primary Care Re-search and Development Centre, William-son Bldg., University of Manchester, Man-chester M13 9PL, United Kingdom, or at [email protected].

N Engl J Med 2006;355:375-84.Copyright © 2006 Massachusetts Medical Society.

A bs tr ac t

Background

In 2004, after a series of national initiatives associated with marked improvements in the quality of care, the National Health Service of the United Kingdom introduced a pay-for-performance contract for family practitioners. This contract increases exist-ing income according to performance with respect to 146 quality indicators cover-ing clinical care for 10 chronic diseases, organization of care, and patient experience.

Methods

We analyzed data extracted automatically from clinical computing systems for 8105 family practices in England in the first year of the pay-for-performance program (April 2004 through March 2005), data from the U.K. Census, and data on charac-teristics of individual family practices. We examined the proportion of patients deemed eligible for a clinical quality indicator for whom the indicator was met (reported achievement) and the proportion of the total number of patients with a medical con-dition for whom a quality indicator was met (population achievement), and we used multiple regression analysis to determine the extent to which practices achieved high scores by classifying patients as ineligible for quality indicators (exception reporting).

Results

The median reported achievement in the first year of the new contract was 83.4 per-cent (interquartile range, 78.2 to 87.0 percent). Sociodemographic characteristics of the patients (age and socioeconomic features) and practices (size of practice, num-ber of patients per practitioner, age of practitioner, and whether the practi tioner was medically educated in the United Kingdom) had moderate but significant effects on performance. Exception reporting by practices was not extensive (median rate, 6 per-cent), but it was the strongest predictor of achievement: a 1 percent increase in the rate of exception reporting was associated with a 0.31 percent increase in reported achievement. Exception reporting was high in a small number of practices: 1 percent of practices excluded more than 15 percent of patients.

Conclusions

English family practices attained high levels of achievement in the first year of the new pay-for-performance contract. A small number of practices appear to have achieved high scores by excluding large numbers of patients by exception reporting. More research is needed to determine whether these practices are excluding pa-tients for sound clinical reasons or in order to increase income.

The New England Journal of Medicine Downloaded from nejm.org at UNIV OF MANCHESTER JOHN RYLANDS LIB on September 12, 2013. For personal use only. No other uses without permission.

Copyright © 2006 Massachusetts Medical Society. All rights reserved.


Trends of prior achievement and 2004-5 expectationsRepresentative sample of 42 practices

Quality of care for CHD,asthma and diabetes improvedbetween 2003 and 2005,continuing the earlier trendIncrease in rate ofimprovement significant onlyfor asthma and diabetesBetter recording or care?Practices were preparing in2003, in anticipation of thescheme launch?Gains from the schemebecome questionable, from acost-effectiveness perspective


n engl j med 357;2 www.nejm.org july 12, 2007182

cians and the closing of other practices. The 42 practices for which data were available for the longitudinal analysis were still nationally repre-sentative in terms of socioeconomic status, but solo practitioners were underrepresented.4 How-ever, these 42 practices have values close to the national averages for socioeconomic status, pop-ulation density, and type of housing of the pa-tient population, and their performance was also typical of English family practices during the first year of the pay-for-performance program.2 The analysis was restricted to the 42 practices for which data were available for all three time points (1998, 2003, and 2005). The research protocol was approved by the ethics committee of the multi-center Manchester National Health Service.

Data Collection

Trained research staff extracted the data to assess the quality of clinical care for the categories of coronary heart disease (15 clinical indicators), asthma (12 clinical indicators), and type 2 dia-betes (21 clinical indicators). Data were collected from both computerized and handwritten medi-

cal records with the use of evidence-based review criteria10,11 developed with the RAND–UCLA ap-propriateness method.12 Patients with these three conditions were randomly selected from lists of those receiving the relevant drugs (see the Sup-plementary Appendix, available with the full text of this article at www.nejm.org) according to re-peat prescriptions within the previous 6 months, and separate samples of patients treated in 1998, 2003, and 2005 were selected. In 1998, for two practices, there were no eligible patients who had coronary heart disease because of the young age of the patient population, so for that time point, the results for this condition are based on only 40 practices. Data were collected for up to 20 pa-tients for each of the three conditions in each practice in 1998 (some small practices did not have 20 patients for each of the conditions) and for up to 12 patients for each condition in each practice in 2003 and 2005. Data were collected for a total of 2300 patients in 1998, for 1495 pa-tients in 2003, and for 1482 patients in 2005. These data are presented as a pooled analysis across practices.

Although the study did not include conditions that were not rewarded with financial incentives in the pay-for-performance program, there were clinical indicators for coronary heart disease, asthma, and type 2 diabetes for which financial incentives were not provided in 2004. We com-pared 30 indicators for which financial incentives were provided with 17 indicators for which finan-cial incentives were not provided. In this analysis, we excluded three clinical indicators for which this distinction was unclear — that is, it was not clear whether there were financial incentives pro-vided for the indicator at all three time points.

Statistical Analysis

An overall score for the quality of care was com-puted for each patient included for 1998, 2003, and 2005. For each patient with asthma, coronary heart disease, or type 2 diabetes, the score was computed as a ratio: the number of clinical indi-cators for which appropriate care was provided, divided by the number of indicators relevant to that patient. Expressed as a percentage, this score represents the percentage of “necessary care” 10 provided to each patient, within a range from 0 to 100. We adopted this measure for consistency with our previous investigation of this sample.4 Scores for the quality of care at the practice level were

90

Mea

n Pr

actic

e-Q

ualit

y Sc

ore

(%)

80

85

70

60

55

75

65

01998 1999 2000 2001 2002 2003 2004 2005

CHD

Asthma

Diabetes

Year

AUTHOR:

FIGURE:

JOB:

4-CH/T

RETAKE

SIZE

ICM

CASE

EMail LineH/TCombo

Revised

AUTHOR, PLEASE NOTE: Figure has been redrawn and type has been reset.

Please check carefully.

REG F

Enon

1st2nd

3rd

Campbell

1 of 1

07-12-07

ARTIST: ts

35702 ISSUE:

22p3

Figure 1. Mean Scores for Clinical Quality at the Practice Level for Coronary Heart Disease, Asthma, and Type 2 Diabetes, 1998 to 2005.

The quality of care for coronary heart disease (CHD), asthma, and type 2 diabetes was improving between 1998 and 2003, before the introduction of pay for performance. The rate of improvement in quality of care increased significantly for diabetes and asthma between 2003 and 2005, after the in-troduction of pay for performance; the rate for coronary heart disease, which was increasing most rapidly before pay for performance, continued at the same rate after pay for performance was introduced.



n engl j med 357;2 www.nejm.org july 12, 2007 181

s p e c i a l r e p o r t


Quality of Primary Care in England with the Introduction of Pay for Performance

Stephen Campbell, Ph.D., David Reeves, Ph.D., Evangelos Kontopantelis, Ph.D., Elizabeth Middleton, M.Sc., Bonnie Sibbald, Ph.D., and Martin Roland, D.M.

In 2004, the United Kingdom committed £1.8 bil-lion ($3.2 billion) to a new pay-for-performance contract for family practitioners.1 During the first year, the levels of achievement exceeded those an-ticipated by the government, with an average of 83.4% of the available incentive payments claimed.2 However, the quality of care in English family practices had already begun to improve in response to a wide range of initiatives,3-6 including nation-al standards for the treatment of major chronic diseases and a national system of inspection (Ta-ble 1). Family practitioners already had some ex-perience with financial incentives from the lim-ited use of incentive programs that were initiated in 1990.7,8 It is therefore unclear whether the high levels of quality attained after the pay-for-perfor-mance contract was introduced in 2004 reflect improvements that were already under way or whether existing trends toward improvement were accelerated. The effect of the incentive program must be understood in the context of the com-prehensive quality-improvement strategy within which the contract was introduced.

This report presents data from a longitudinal cohort study that measured the quality of care in a representative sample of primary care practices in England at two time points (1998 and 2003)

before the pay-for-performance program was in-troduced and at one time point (2005) after its introduction. A validated set of criteria was used to assess quality in the management of three chronic conditions: asthma, coronary heart dis-ease, and type 2 diabetes. Because some clinical indicators — the measures of the quality of clini-cal care — were not rewarded with financial pay-ments in the 2004 pay-for-performance program, the study design also permitted a comparison of trends in the quality of care for indicators for which financial incentives were provided and for those for which they were not provided in the management of these three conditions.

Methods

In 1998, we measured the quality of care in a strat-ified, random sample of 60 primary care practices in six geographic areas of England. These practices were nationally representative in terms of size, whether the practice was approved for residency training, and the sociodemographic characteris-tics of their populations.9 We followed up 42 of these practices in 2003 and 2005. The reduction in the number of practices was due partly to attri-tion and partly to the retirement of solo physi-

Table 1. Examples of Key Initiatives in the Broad National Quality-Improvement Strategy.

National standards for the treatment of major chronic diseases, such as the National Service Frameworks for coronary heart disease (1999) and diabetes (2003)

Contractual requirement for practitioners to undertake a clinical audit (initially a requirement in the 1990 contract)

Financial incentives for cervical cytologic testing and immunization (early 1990s)

Widespread use of audit and feedback by the Primary Care Trusts

Release of comparative data for quality of care to practitioners (common) and the public (rare) by the Primary Care Trusts

Annual appraisal of all primary care physicians (by the Primary Care Trusts and including discussion of some audit data)

National system of inspection and monitoring of performance (by the Healthcare Commission)




Performance over time and its variationAll 8000+ English practices

Median overallachievement was 85.1%,89.3% and 90.8% in first 3yMedian achievementincreased by 4.4% forquintile of most affluent andby 7.6% for most deprivedGap in medianachievement narrowed from4.0% to 0.8%Financial incentiveschemes can contribute tothe reduction of inequalities

Articles

www.thelancet.com Vol 372 August 30, 2008 731

Information about practice and patient characteristics was taken from the 2006 general medical statistics database, which is maintained by the Department of Health. Practices were grouped into quintiles of equal size on the basis of the level of area deprivation in the census super-output area (a standard, stable unit of geography used in the UK for statistical analysis; average population 7200) where they were located, with data from the Index of Deprivation 2004.20 We calculated the odds of practices from each quintile being in the top and bottom performing 5% of practices with respect to achievement and rates of exclusion by logistic regression. We estimated the associations of practice-level characteristics with practice achievement, exclusion of patients, and changes in these outcomes with multiple linear regressions. These analyses controlled for missing indicators, heterogeneity of variance, and clustering of practices, and we made checks on the robustness of the results to model specifi cations (webappendix). All variables were divided by their standard deviations, thus regression coeffi cients show the increase in standard deviations of the outcome for one standard deviation increase in predictor variables. All statistical analyses were done with Stata software (version 9).

Achievement data for 2004–05, 2005–06, and 2006–07 were available for 8277 general practices in England. Practices were excluded from the study if they had fewer than 1000 patients in any 1 year (49 practices), one or more disease registers were missing (47 practices), the practice relocated to a more or less affl uent area during the period

(164 practices), complete exclusion data were not available (172 practices), or if the practice population changed in size by 25% or more (258 practices). Our main results are drawn from 7637 practices, providing care for more than 49 million patients. We undertook subanalyses for excluded practices (webappendix).

Role of the funding sourceThere was no funding source for this study. The corresponding author had full access to all the data in the study and had fi nal responsibility for the decision to submit for publication.

ResultsThe median overall reported achievement—the propor-tion of patients who were deemed eligible by the prac-tices for whom the targets were achieved—was 85·1% (IQR 79·0–89·1) in year 1, 89·3% (86·0–91·5) in year 2, and 90·8% (88·5–92·6) in year 3. Increases in achievement between years were signifi cant (p<0·0001 in all cases). Although average levels of achievement in-creased over time, variation in achieve ment diminished.

In year 1, progressively lower levels of achievement were associated with increased levels of area deprivation (fi gure 1). Median achievement ranged from 86·8% (IQR 82·2–89·6) for quintile 1 (least deprived) to 82·8% (75·2–87·8) for quintile 5 (most deprived), with variation in achievement between practices increasing with deprivation (fi gure 1).

Ove

rall

repo

rted

ach

ieve

men

t (%

)

Year 1 (2004–05)0

20

40

60

80

100

Year 2 (2005–06) Year 3 (2006–07)Quality and outcome framework year

Quintile 1Quintile 2Quintile 3Quintile 4Quintile 5

Figure 1: Distribution of scores for overall reported achievement by deprivation quintile for year 1 (2004–05) to year 3 (2006–07)Central line shows median achievement and box shows interquartile range; whiskers represent range of achievement scores. Circles represent statistical outliers—ie, individual practices with achievement scores outside the range: fi rst quartile–(1·5×IQR) to third quartile+(1·5×IQR).

most deprived

most affluent

Articles

728 www.thelancet.com Vol 372 August 30, 2008

Eff ect of fi nancial incentives on inequalities in the delivery of primary clinical care in England: analysis of clinical activity indicators for the quality and outcomes framework Tim Doran, Catherine Fullwood, Evangelos Kontopantelis, David Reeves

SummaryBackground The quality and outcomes framework is a fi nancial incentive scheme that remunerates general practices in the UK for their performance against a set of quality indicators. Incentive schemes can increase inequalities in the delivery of care if practices in affl uent areas are more able to respond to the incentives than are those in deprived areas. We examined the relation between socioeconomic inequalities and delivered quality of clinical care in the fi rst 3 years of this scheme.

Methods We analysed data extracted automatically from clinical computing systems for 7637 general practices in England, data from the UK census, and data for characteristics of practices and patients from the 2006 general medical statistics database. Practices were grouped into equal-sized quintiles on the basis of area deprivation in their locality. We calculated overall levels of achievement, defi ned as the proportion of patients who were deemed eligible by the practices for whom the targets were achieved, for 48 clinical activity indicators during the fi rst 3 years of the incentive scheme (from 2004–05 to 2006–07).

Findings Median overall reported achievement was 85·1% (IQR 79·0–89·1) in year 1, 89·3% (86·0–91·5) in year 2, and 90·8% (88·5–92·6) in year 3. In year 1, area deprivation was associated with lower levels of achievement, with median achievement ranging from 86·8% (82·2–89·6) for quintile 1 (least deprived) to 82·8% (75·2–87·8) for quintile 5 (most deprived). Between years 1 and 3, median achievement increased by 4·4% for quintile 1 and by 7·6% for quintile 5, and the gap in median achievement narrowed from 4·0% to 0·8% during this period. Increase in achievement during this time was inversely associated with practice performance in previous years (p<0·0001), but was not associated with area deprivation (p=0·062).

Interpretation Our results suggest that fi nancial incentive schemes have the potential to make a substantial contribution to the reduction of inequalities in the delivery of clinical care related to area deprivation.

Funding None.

IntroductionIn 1997, the UK Government made an explicit commitment to tackle health inequalities, and has since pursued several social and health policies to this end.1–3 The National Health Service (NHS) was targeted with an ambitious programme of initiatives to improve quality that aimed to eliminate unacceptable variations in the standard of health care.3,4 The UK Government reiterated its original commitment by making health inequalities a health-service priority for 2008–09, with the specifi c objective of improving life expectancy in areas with the worst health and deprivation. Primary-care services are intended to have a central role in achieving this objective.5

The most substantial UK Government intervention in primary care was the re-negotiation of the national contract with general practitioners in 2003. At the core of the new contract was the quality and outcomes framework, which links fi nancial incentives to the quality of care that is provided by practices.6 Quality is measured against a set of clinical activity indicators relating to aspects of care for several common chronic diseases, with practices rewarded according to the proportion of eligible patients

for whom each target is achieved. Further payments are awarded for aspects of practice organisation and for administering patient surveys. To protect patients from inappropriate care, the scheme allows practices to exclude patients who they deem inappropriate from specifi c indicators, for reasons such as extreme frailty, intolerance of a particular drug, and declining treatment.7

In the fi rst full year of the scheme (2004–05), practices generally reported high levels of achievement for the clinical indicators,8 and levels of achievement have, on average, increased every year for most indicators.9 However, there are concerns that practices serving deprived populations have achieved lower levels of performance,10 have received less generous fi nancial rewards,11 and might have excluded more patients than have those serving more affl uent populations.12 If this tenet is true, the incentive scheme could be driving an overall increase in quality of care at the cost of widening existing health inequalities, and hence undermining Government policy.

The early years of public-health interventions are often damaging in terms of health equity.13–16 Victora and colleagues’ inverse equity hypothesis17 proposes that

Lancet 2008; 372: 728–36

Published OnlineAugust 12, 2008

DOI:10.1016/S0140-6736(08)61123-X

See Comment page 692

National Primary Care Research and Development Centre, University of Manchester,

Manchester, UK (T Doran MD, C Fullwood PhD,

E Kontopantelis PhD, D Reeves PhD)

Correspondence to:Dr Tim Doran, National Primary

Care Research and Development Centre, Williamson Building,

University of Manchester, Manchester M13 9PL, UK

[email protected]


Trends of prior achievement and 2006-7 expectationsRepresentative sample of 42 practices

By 2007 the rate ofimprovement had slowed forall three conditionsQuality of care fornon-incentivised aspectsdeclined for asthma and CHDNo significant changes onaccess to care or oninterpersonal aspects of careContinuity of care reducedimmediately after theintroduction of the schemeQOF accelerated short-termimprovements

T h e n e w e ngl a nd j o u r na l o f m e dic i n e

n engl j med 361;4 nejm.org july 23, 2009372

incentives as compared with those that were not. For heart disease, the scores for aspects of care that were linked to incentives showed a bigger immediate increase when the pay-for-performance system was introduced (P = 0.05), although this trend was not significant as calculated in the lin-ear model (P = 0.46). The long-term trends (scores

in the post-introduction period vs. scores in the pre-introduction period) did not differ significant-ly (P = 0.06). However, the difference was signifi-cant when calculated with the use of the boot-strapping method (P = 0.05) or the linear model (P = 0.03), and in absolute terms, the mean qual-ity score for aspects of care for heart disease that were not linked to incentives declined after 2005, whereas the quality score for care that was linked to incentives increased. For asthma, the immedi-ate effect of pay for performance did not differ between care that was and care that was not linked with incentives (P = 1.00), but the trends subsequently diverged (post-introduction period vs. pre-introduction period, P = 0.006; post-intro-duction period vs. introduction period, P = 0.05), with the mean score for care that was not linked to incentives declining after 2005, and the mean score for care that was linked to incentives in-creasing. Trends in diabetes care did not differ at any time according to whether the care was linked to incentives.

Communication, Waiting Times, and Continuity of Care

The percentages of patients able to see a physician within 48 hours, as well as the mean scores on the physician-communication scale, showed no significant changes in trend. Continuity of care declined significantly after the introduction of pay for performance (P<0.001) and remained at this lower level (Tables 1 and 2 and Fig. 2).

Estimated Overall Effect of Pay for Performance

For outcomes in which there was evidence that pay for performance altered the trend in quality improvement, we used coefficients from the inter-rupted time-series analysis to compute estimates of the increase in scores beyond that expected from the trend in the pre-introduction period (back-transforming the results from the logit analysis, with estimated 95% confidence limits). As compared with the expected level of improve-ment based on the pre-introduction trend, the pay-for-performance scheme was associated with an improvement in the quality of care for diabetes of 7.5 percentage points in 2005 (95% CI, 4.7 to 10.4) and 6.9 percentage points in 2007 (95% CI, 3.8 to 10.0). For asthma, the increase in quality poten-tially attributable to pay for performance was 9.4 percentage points in 2005 (95% CI, 3.9 to 15.0)

22p3

90

Scor

e

85

80

70

65

55

75

60

01998 1999 2000 2001 2002 2003 2007200620052004

Coronaryheart disease

Asthma

Diabetes

Year

B

A

AUTHOR:

FIGURE:

JOB:

4-CH/T

RETAKE

SIZE

ICM

CASE

EMail LineH/TCombo

Revised

AUTHOR, PLEASE NOTE: Figure has been redrawn and type has been reset.

Please check carefully.

REG F

Enon

1st

2nd3rd

Campbell

1 of 2

xx-xx-09

ARTIST: ts

360xx ISSUE:

100

Scor

e

90

80

60

50

20

70

40

30

01998 1999 2000 2001 2002 2003 2007200620052004

Communicationwith physicians

Continuity of care

Able to get an appointmentwithin 48 hr (particular doctor)

Able to get an appointmentwithin 48 hr (any doctor)

Year

Figure 1. Mean Scores for the Quality of Care at the Practice Level, 1998–2007.

Panel A shows scores for the quality of care provided for coronary heart disease, asthma, and diabetes. Quality scores range from 0% (no quality indicator was met for any patient) to 100% (all quality indicators were met for all patients). Panel B shows scores for patients’ perceptions of commu-nication with physicians, access to care, and continuity of care. Communi-cation was assessed by asking seven questions, with the answers scored on a six-point scale ranging from “very poor” to “excellent”; continuity of care was assessed with the use of the same six-point scale and a single question: “How often do you see your usual doctor?” Access to care was scored as the percentage of patients who reported that they were able to get an ap-pointment within 48 hours. All scores were rescaled to range from 0 to 100.



special article

T h e n e w e ngl a nd j o u r na l o f m e dic i n e

n engl j med 361;4 nejm.org july 23, 2009368

Effects of Pay for Performance on the Quality of Primary Care in England

Stephen M. Campbell, Ph.D., David Reeves, Ph.D., Evangelos Kontopantelis, Ph.D., Bonnie Sibbald, Ph.D., and Martin Roland, D.M.

From the National Primary Care Research and Development Centre, University of Manchester, Manchester (S.M.C., D.R., E.K., B.S., M.R.); and the University of Cambridge General Practice and Primary Care Research Unit, Institute of Public Health, Cambridge (M.R.) — both in the United Kingdom. Address reprint requests to Dr. Campbell at the National Primary Care Research and Development Centre, University of Manchester, Oxford Rd., Manchester M13 9PL, United Kingdom, or at [email protected].

N Engl J Med 2009;361:368-78.Copyright © 2009 Massachusetts Medical Society.

A bs tr ac t

Background

A pay-for-performance scheme based on meeting targets for the quality of clinical care was introduced to family practice in England in 2004.

Methods

We conducted an interrupted time-series analysis of the quality of care in 42 repre-sentative family practices, with data collected at two time points before implemen-tation of the scheme (1998 and 2003) and at two time points after implementation (2005 and 2007). At each time point, data on the care of patients with asthma, diabetes, or coronary heart disease were extracted from medical records; data on patients’ perceptions of access to care, continuity of care, and interpersonal aspects of care were collected from questionnaires. The analysis included aspects of care that were and those that were not associated with incentives.

Results

Between 2003 and 2005, the rate of improvement in the quality of care increased for asthma and diabetes (P<0.001) but not for heart disease. By 2007, the rate of im-provement had slowed for all three conditions (P<0.001), and the quality of those aspects of care that were not associated with an incentive had declined for patients with asthma or heart disease. As compared with the period before the pay-for-performance scheme was introduced, the improvement rate after 2005 was unchanged for asthma or diabetes and was reduced for heart disease (P = 0.02). No significant changes were seen in patients’ reports on access to care or on interpersonal aspects of care. The level of the continuity of care, which had been constant, showed a re-duction immediately after the introduction of the pay-for-performance scheme (P<0.001) and then continued at that reduced level.

Conclusions

Against a background of increases in the quality of care before the pay-for-perfor-mance scheme was introduced, the scheme accelerated improvements in quality for two of three chronic conditions in the short term. However, once targets were reached, the improvement in the quality of care for patients with these conditions slowed, and the quality of care declined for two conditions that had not been linked to incentives. Continuity of care was reduced after the introduction of the scheme.




Does the size of the practice matter?All 8000+ English practices

In y1 smallest practices hadthe lowest median reportedachievement ratesPerformance improved overtime; smallest practicesimproved at the fastest rateCaught up by y3 but displayedmore variation in performanceSmall practices representedamong best and worstQOF reduced variation inperformance and differencesbetween large and smallpractices

British Journal of General Practice, September 2010

T Doran, S Campbell, C Fullwood, et al

e339

was over two times higher for group 8. In year 1, theproportion of practices among the worst performing5% ranged from 13.4% for group 1 to 0.7% for group8. There was little absolute change in theseproportions over time.

Reported achievementThe median overall reported achievement, theproportion of patients deemed appropriate by thepractice for whom the targets were achieved, was85.2% in year 1, 89.3% in year 2, and 90.9% inyear 3. Increases in achievement between yearswere statistically significant (P<0.005 in all cases).Median reported achievement in year 1 varied with

number of patients, from 83.8% for group 1 to 85.9%for group 5 (Figure 2). As was the case for pointsscored, variation in achievement decreased withincreasing number of patients; hence, theinterquartile range was greatest for group 1(1000–1999 patients, 16.2%) and smallest for group8 (≥12 000 patients, 6.6%). However, in contrast tothe findings for points scored, both the highest andthe lowest achievement rates were attained bysmaller practices: 13.2% of practices from group 1were among the highest achieving 5% in year 1, and12.1% were among the lowest achieving 5%. Incontrast, the corresponding proportions for group 8were 1.5% and 0.9% (Table 3).By year 3 there was little difference in average

achievement rates between practices of differentsize; a spread of 1.1% covered the medianachievement of all groups. However, by this time,group 1, which had the lowest median achievementin year 1, had the highest median achievement(91.5%), and group 8 the lowest (90.4%). Variation inachievement between practices decreased for allgroups in year 2 and again in year 3, with the greatestreduction for group 1; to 9.3% in year 2 and 7.2% inyear 3. These patterns were consistent across all 48individual indicators. Despite the changes over time,smaller practices remained more likely to be bothvery high or very low achievers in year 3. Practiceswith fewer than 3000 patients (groups 1 and 2)represented 20.1% of all practices, but 46.7% of thehighest-achieving and 45.1% of the lowest-achieving practices.

Exception reporting and populationachievementPractices’ reported achievement rates depend, inpart, on the number of patients they exclude.Practices excluded a median of 6.6% of patients inyear 2 and 7.4% in year 3. In both years there was atrend for higher exception reporting rates in practiceswith more patients, ranging from 6.3% for group 1(1000–1999 patients) to 6.8% for group 8 (≥12 000patients) in year 2, rising to 6.5% and 7.7%respectively in year 3 (Figure 3). As with achievementrates, there was greater variation in exceptionreporting rates in smaller practices, with a higherproportion of both the highest and lowest exceptionreporters in group 1 (Table 3).The median overall population achievement, the

proportion of all patients for whom the targets wereachieved, including those exception reported by thepractice, was 83.0% in year 2 and 83.8% in year 3(Figure 4). The distributions were similar to those forreported achievement, with group 1 having thehighest median population achievement in year 3(84.6%) but also the greatest variation (interquartile

100

80

60

40

20

0

2004/2005 2005/2006 2006/2007

QOF year

list size 1000–19992000–29993000–39994000–59996000–79998000–999910000–11 99912000 or more

Perc

enta

ge

ofa

llav

aila

ble

QO

Fp

oin

ts

Figure 1. Distribution ofpractice scores forpercentage of total QOFpoints scored by number ofpatients, year 1 (2004–2005)to year 3 (2006–2007).

100

80

60

40

20

0

2004/2005 2005/2006 2006/2007

QOF year

list size 1000–19992000–29993000–39994000–59996000–79998000–999910000–11 99912000 or more

Ove

rall

rep

ort

edac

hie

vem

ent

Figure 2. Distribution ofpractice scores for overallreported achievement bynumber of patients, year 1(2004–2005) to year 3(2006–2007).

Central white line shows median scores and box shows interquartile range (IQR); whiskersrepresent range of scores. Circles represent statistical outliers — that is, individual practiceswith points scores outside the range: first quartile — (1.5 × IQR) + (1.5 × IQR).

Central white line shows median scores and box shows interquartile range (IQR); whiskersrepresent range of scores. Circles represent statistical outliers — that is, individual practiceswith points scores outside the range: first quartile — (1.5 × IQR) + (1.5 × IQR).

ABSTRACTBackgroundSmall general practices are often perceived to provideworse care than larger practices.

AimTo describe the comparative performance of smallpractices on the UK’s pay-for-performance scheme,the Quality and Outcomes Framework.

Design of studyLongitudinal analysis (2004–2005 to 2006–2007) ofquality scores for 48 clinical activities.

SettingFamily practices in England (n = 7502).

MethodComparison of performance of practices by list size, interms of points scored in the pay-for-performancescheme, reported achievement rates, and populationachievement rates (which allow for patients excludedfrom the scheme).

ResultsIn the first year of the pay-for-performance scheme,the smallest practices (those with fewer than 2000patients) had the lowest median reported achievementrates, achieving the clinical targets for 83.8% of eligiblepatients. Performance generally improved for practicesof all sizes over time, but the smallest practicesimproved at the fastest rate, and by year 3 had thehighest median reported achievement rates (91.5%).This improvement was not achieved by additionalexception reporting. There was more variation inperformance among small practices than larger ones:practices with fewer than 3000 patients (20.1% of allpractices in year 3), represented 46.7% of the highest-achieving 5% of practices and 45.1% of the lowest-achieving 5% of practices.

ConclusionSmall practices were represented among both the bestand the worst practices in terms of achievement ofclinical quality targets. The effect of the pay-for-performance scheme appears to have been to reducevariation in performance, and to reduce the differencebetween large and small practices.

Keywordsincentives; quality; primary care.

INTRODUCTIONSmall general practices in the UK, particularly thosethat are single handed, are often accused ofproviding poor-quality care. The 2000 NHS Plan citeda need to ‘confirm that single-handed (solo)practices are offering high standards’.1 The ShipmanInquiry identified advantages and disadvantagesassociated with single-handed practice, anddescribed an implicit unwritten policy to reduce thenumbers of solo practices in the UK; noting thatconcern about single-handed practitioners was notrecent.2 In 2002, the Audit Commission concludedthat there were good arguments for preserving adiversity of practice sizes and types: ‘One challengeis to ensure that the trend towards larger practicesdoes not mean that patients lose out on some of theadvantages that smaller practices currently offer’.3

However, in 2008, the NHS Next Stage Reviewcontinued the pressure on small practices bysuggesting that they be congregated as franchisedpractices, and also advocated groupings of largernumbers of doctors in new GP-led health centres.4

A new general medical services contract wasintroduced in the UK in 2004 incorporating a pay-for-

T Doran, MD, clinical research fellow; S Campbell, PhD,senior research fellow; C Fullwood, PhD; E Kontopantelis,PhD, research associate, National Primary Care Research and

Development Centre, University of Manchester, Manchester.

M Roland, DM, professor of health services research, Instituteof Public Health, University of Cambridge, Cambridge.

Address for correspondenceDr Tim Doran, National Primary Care Research and

Development Centre, Williamson Building, University of

Manchester, Manchester M13 9PL.

E-mail: [email protected]

Submitted: 20 July 2009; Editor’s response: 12 January

2010; final acceptance: 14 April 2010.

©British Journal of General Practice

This is the full-length article of an abridged version

published in print. Cite this article as: Br J Gen Pract 2010;

DOI: 10.3399/bjgp10X515340.

British Journal of General Practice, September 2010

T Doran, S Campbell, C Fullwood, et al

e335

Performance of small generalpractices under the UK’s Quality

and Outcomes FrameworkTim Doran, Stephen Campbell, Catherine Fullwood,

Evangelos Kontopantelis and Martin RolandBJGP 2010


Changes in patient experiences42 practices, random samples of chronic condition and all patients

No changes in 2003-7quality of care forcommunication, nursingcare, coordination &overall satisfactionAspects of accessimproved for chronicdisease patients onlyBoth samples seeing theirusual GP less & lesssatisfied in continuityRelated to incentives toprovide rapidappointments

ANNALS OF FAMILY MEDICINE ✦ WWW.ANNFAMMED.ORG ✦ VOL. 8, NO. 6 ✦ NOVEMBER/DECEMBER 2010

502

PATIENT EXPERIENCES DURING REFORMS

RESULTSResponse rates for the patient survey are shown in

Table 2. Mean practice level GPAQ scale and item

scores are shown in Table 3 for both the chronic ill-

ness samples and the random samples of patients in

2003, 2005, and 2007 respectively. Table 4 displays the

results from the regression analyses.

Patients in the Chronic Illness SamplesMean ratings of quality by patients with chronic illness

were similar in all 3 time periods for the GPAQ scale

scores for communication, coordination, nursing, and

overall satisfaction (Table 3). This fi nding was borne

out by the regression analysis results (Table 4), which

indicated no signifi cant change between 2003 and

2007 on any of these measures (P >.05). There was

also no signifi cant change on any of the GPAQ speed-

of-access items (P >.05), including ability to see a

particular physician or any physician within 48 hours.

There were, however, negative changes in mean scores

from 2003 to 2007 with regard to the 2 continuity-

of-care items: how often patients reported being able

to see their usual physician (decrease of 6.9%; 95%

confi dence interval [CI], –8.4 to –4.4) and satisfaction

with this experience (decrease of 4.8%; 95% CI, –6.2

to –3.4). Signifi cant differences between 2003 and

2007 scores were observed for 3 of the 5 GPAQ access

items relating to urgent appointments: ability to book

an urgent appointment with any doctor (improvement

with odds ratio [OR] = 1.24; 95% CI, 1.06 to 1.45), sat-

isfaction with this experience, and satisfaction with the

ability to book an urgent appointment with a particular

doctor (increase of 2.6%; 95% CI, 0.9 to 4.2). Differ-

ences in the speed-of-access items are not observable

in Table 3, however, as they only emerge after control-

ling for patient-level characteristics.

Patients Randomly Sampled From Registered ListsResults for patients randomly sampled from practice

lists matched those for patients with chronic illness in

all aspects except the speed-of-access items: no signifi -

cant changes over time were observed in any of these

items. The only signifi cant changes from 2003 to 2007

were in regard to continuity of care, with a reduction

in how often patients reported being able to see their

Table 2. Response Rates for Patient Surveys

Patient Group2003% (n)

2005% (n)

2007% (n)

Chronic illness samples 55 (1,092) 52 (1,040) 50 (922)

Random samples of registered patients

47 (3,873) 45 (3,601) 37 (3,104)

Table 3. Summary of Practice Mean GPAQ Scale and Individual Item Scores 2003, 2005, and 2007, for Cross-Sectional Samples of Patients With Chronic Illness and Random Samples of Adult Patients

GPAQ Scale and Item

Samples of Patients With Chronic Illness

Mean (SD)a

Samples of Randomly Selected Patients

Mean (SD)a

2003 2005 2007 2003 2005 2007

Communication scale 74.5 (10.1) 74.5 (9.3) 76.0 (8.5) 69.5 (10.0) 68.4 (9.4) 69.9 (8.9)

Coordination scale 72.1 (7.4) 71.0 (5.8) 73.0 (5.9) 67.0 (5.8) 68.6 (5.1) 68.3 (6.1)

Nursing care scale 76.0 (6.1) 76.1 (6.0) 75.3 (8.1) 73.9 (6.2) 73.4 (5.3) 72.9 (7.1)

Overall satisfaction scale 81.6 (8.0) 80.6 (8.6) 81.3 (7.7) 75.2 (9.2) 74.7 (10.6) 75.6 (7.9)

Item: In general, how often do you see your usual doctor (continuity of care)?

77.2 (8.0) 74.9 (9.4) 70.4 (9.6) 68.2 (13.1) 62.8 (13.5) 62.5 (11.6)

Item: Rating of how often patients get to see their usual doctor (rating of continuity of care)

71.5 (9.8) 69.1 (9.9) 67.4 (11.5) 64.6 (13.1) 61.0 (12.2) 61.2 (10.4)

Item: Do you get an appointment with a particu-lar doctor within 48 hours?

36.9 (29.5) 38.7 (26.3) 37.0 (22.6) 33.4 (26.2) 35.9 (25.1) 32.3 (21.0)

Item: Rating of how quickly an appointment can be made with a particular doctor in the practice

57.3 (17.7) 58.0 (14.1) 56.6 (15.6) 50.0 (18.6) 52.2 (16.0) 53.0 (13.6)

Item: Do you get an appointment with any doc-tor within 48 hours?

64.5 (23.9) 68.0 (22.6) 67.2 (19.9) 61.5 (24.2) 63.4 (21.0) 63.8 (22.4)

Item: Rating of how quickly an appointment can be made with any doctor in the practice

65.2 (15.2) 64.5 (13.8) 65.5 (14.5) 59.2 (17.2) 61.2 (14.2) 62.4 (13.1)

Item: If you need an urgent appointment can you get one on the same day?

81.8 (17.7) 81.2 (15.5) 82.2 (16.8) 79.6 (17.1) 78.2 (17.5) 79.4 (18.8)

GPAQ = General Practice Assessment Questionnaire.

Note: See the Appendix for a description of how the scales were scored.

a Figures relate to raw practice-level scores (mean and standard deviation of practice means).

13/09/2013 Changes in Patient Experiences of Primary Care During Health Service Reforms in England Between 2003 and 2007

www.annfammed.org/content/8/6/499.long 1/18

+

The Annals of Family Medicinewww.annfammed.org

doi: 10.1370/afm.1145Ann Fam Med November 1, 2010 vol. 8 no. 6 499-506

Changes in Patient Experiences of

Primary Care During Health Service

Reforms in England Between 2003 and

2007

Stephen M. Campbell, PhD1, Evangelos Kontopantelis, PhD1,

David Reeves, PhD1, Jose M. Valderas, PhD1, Ella Gaehl, MPhil1,

Nicola Small, MPhil1 and Martin O. Roland, DM2

Author Affiliations

CORRESPONDING AUTHOR: Stephen Campbell, PhD, National Primary CareResearch and Development Centre, University of Manchester, Williamson Bldg,Oxford Road, Manchester, M13 9PL, UK, [email protected]

Abstract

PURPOSE Major primary care reforms have been introduced in recent years in

the United Kingdom, including financial incentives to improve clinical quality

and provide more rapid access to care. Little is known about the impact of

these changes on patient experience. We examine patient reports of quality of

care between 2003 and 2007, including random samples of patients on

practice lists and patients with long-term conditions.

METHODS We conducted a cross-sectional design study of family practices in

which questionnaires were sent to serial samples of patients in 42

representative general practices in England. Questionnaires sent to samples of

patients with chronic disease (asthma, angina, and diabetes) and random

samples of adult patients (excluding patients who reported any long-term


Patient satisfaction2007-8 GP Access Survey, all English practices

Young, Asian, working FT, withlong commuting times: lowestlevels of satisfaction andexperience of accessAbility to take time off to visitGP eliminated thedisadvantage in accessPatients in small practicesmore positive for all aspects ofaccess; except opening hoursPositive access to careassociated with higher QOFscores and slightly lower ratesof emergency admission

0

20

40

60

80

100

% o

f pos

itive

res

pons

es

<1 1−2

2−3

3−4

4−6

6−88−

10

10−1

2>=

12

Phone access

0

20

40

60

80

100

% o

f pos

itive

res

pons

es

<1 1−2

2−3

3−4

4−6

6−88−

10

10−1

2>=

12

Appointment within 2 days

0

20

40

60

80

100

% o

f pos

itive

res

pons

es

<1 1−2

2−3

3−4

4−6

6−88−

10

10−1

2>=

12

Advance appointment (>2 days)

0

20

40

60

80

100

% o

f pos

itive

res

pons

es

<1 1−2

2−3

3−4

4−6

6−88−

10

10−1

2>=

12

Appointment with a particular GP

Practice size in 1,000s of patients

by practice list sizeSatisfaction & positive experience

RESEARCH ARTICLE Open Access

Patient experience of access to primary care:identification of predictors in a national patientsurveyEvangelos Kontopantelis1*, Martin Roland2, David Reeves1

Abstract

Background: The 2007/8 GP Access Survey in England measured experience with five dimensions of access:getting through on the phone to a practice, getting an early appointment, getting an advance appointment,making an appointment with a particular doctor, and surgery opening hours. Our aim was to identify predictors ofpatient satisfaction and experience with access to English primary care.

Methods: 8,307 English general practices were included in the survey (of 8,403 identified). 4,922,080 patients wererandomly selected and contacted by post and 1,999,523 usable questionnaires were returned, a response rate of40.6%. We used multi-level logistic regressions to identify patient, practice and regional predictors of patientsatisfaction and experience.

Results: After controlling for all other factors, younger people, and people of Asian ethnicity, working full time, orwith long commuting times to work, reported the lowest levels of satisfaction and experience of access. Forpeople in work, the ability to take time off work to visit the GP effectively eliminated the disadvantage in access.The ethnic mix of the local area had an impact on a patient’s reported satisfaction and experience over and abovethe patient’s own ethnic identity. However, area deprivation had only low associations with patient ratings.Responses from patients in small practices were more positive for all aspects of access with the exception ofsatisfaction with practice opening hours. Positive reports of access to care were associated with higher scores onthe Quality and Outcomes Framework and with slightly lower rates of emergency admission. Respondents inLondon were the least satisfied and had the worst experiences on almost all dimensions of access.

Conclusions: This study identifies a number of patient groups with lower satisfaction, and poorer experience, ofgaining access to primary care. The finding that access is better in small practices is important given the increasingtendency for small practices to combine into larger units. Consideration needs to be given to ways of retainingthese and other benefits of small practice size when primary care services are reconfigured. Differences betweenpopulation groups (e.g. younger people, ethnic minorities) may be due to differences in actual care received ordifferent response tendencies of different groups. Further analysis is needed to determine whether case-mixadjustment is required when comparing practices serving different populations.

BackgroundAccess to health services is a prerequisite for any highquality health care system. Conceptually, access can beclassified as a dimension of care on its own, separatedfrom dimensions of quality [1,2] though it has moreoften been seen as one of the essential elements of

quality [3,4]. For the National Health Service (NHS),access is a high policy priority. The NHS Plan [5] in2002 stated that patients should be able to see a healthprofessional within 24 hours and a general practitionerwithin 48 hours, and in 2004, GPs were given a financialincentive to achieve this target Many GPs responded tothe incentive by using a model of ‘Advanced Access’which attempts to match demand and capacity on aday-to-day basis [6]. In the US, this model has been suc-cessful in both accelerating entry into the system and

* Correspondence: [email protected] Primary Care Research and Development Centre, University ofManchester, Manchester, M13 9PL, UKFull list of author information is available at the end of the article

Kontopantelis et al. BMC Family Practice 2010, 11:61http://www.biomedcentral.com/1471-2296/11/61

© 2010 Kontopantelis et al; licensee BioMed Central Ltd. This is an Open Access article distributed under the terms of the CreativeCommons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, andreproduction in any medium, provided the original work is properly cited.


Advising NICE on removing indicatorsAll English practices, 2004/5 to 2009/10

Indicators should be removed onstatistical criteria and the economicsof incentivesReplaced with other indicators todrive improvement in other areasIt is unknown what will happen tothe withdrawn indicators

BMJ | 24 APRIL 2010 | VOLUME 340 899

ANALYSIS

Increasing numbers of countries are using indi-cators to evaluate the quality of clinical care, with some linking payment to achievement.1 For performance frameworks to remain effective the indicators need to be regularly reviewed. The frameworks cannot cover all clinical areas, and achievement on chosen indicators will even-tually reach a ceiling beyond which further improvement is not feasible.2 3 However, there has been little work on how to select indictors for replacement. The Department of Health decided in 2008 that it would regularly replace indicators in the national primary care pay for performance scheme, the Quality and Outcomes Framework,4 making a rigorous approach to removal a prior-ity. We draw on our previous work on pay for performance5 6 and our current work advising the National Institute for Health and Clinical Excellence (NICE) on the Quality and Outcomes Framework to suggest what should be consid-ered when planning to remove indicators from a clinical performance framework.

First UK decisions The Quality and Outcomes Framework cur-rently includes 134 indicators for which gen-eral practices can earn up to a total of 1000 points. Negotiations between the Department of Health and the BMA’s General Practitioners Committee last autumn led to an agreement to remove eight clinical indicators worth 28 points in April 2011 (table 1). The eight indicators are

all process measures and reward actions such as taking blood pressure or taking blood to measure cholesterol, glucose, or creatinine concentra-tions for people with relevant chronic diseases. The framework rewards the action itself rather than a clinically informed response to results or intermediate outcomes such as better control of blood pressure or cholesterol levels. It is there-fore not surprising that achievement of these process indicators is high (median >95% and interquartile range <4.5%) with little change in

rates or variation across practices since 2005-6, the second year of the Quality and Outcomes Framework.

In many schemes, including the Quality and Outcomes Framework, providers can “except” certain patients from inclusion in the denomi-nator figures for an indicator on grounds such as extreme frailty or contraindications to a specified drug. Exception reporting rates are also low for these eight indicators (median <5% and inter-quartile range <3%).

How to identify when a performance indicator has run its course In April 2011 eight clinical indicators will be removed from the UK Quality and Outcomes Framework. David Reeves and colleagues explain why they were chosen and suggest a rationale for future decisions

National achievement and exception rates for indicators that are to be removed from the Quality and Outcomes Framework in 20117

Indicator (measurement of ) ConditionMedian (interquartile range) achievement (%) Median (interquartile range) rate of exceptions (%) Paired

indicator*2005-6 2006-7 2007-8 2005-6 2006-7 2007-8Blood pressure Coronary heart disease 98.2 (96.7-99.3) 98.5 (97.2-99.5) 98.4(97.1-99.4) 0.8 (0-1.8) 0.8 (0-1.7) 0.7 (0-1.6) Yes Haemoglobin A1c Diabetes 97.4 (95.1-98.8) 97.8 (95.9-99.0) 97.7 (96.0-98.9) 2.7 (1.4-4.4) 2.5 (1.3-4.2) 2.4 (1.2-3.9) Yes Blood pressure Diabetes 98.8 (97.6-99.7) 99.0 (98.0-100) 98.9 (97.9-99.6) 1.1 (0.3-2.2) 1.0 (0.3-2.1) 1.0 (0.3-2.1) Yes Serum creatinine Diabetes 96.7 (94.1-98.3) 97.4 (95.3-98.7) 97.4 (95.6-98.7) 1.9 (0.9-3.4) 1.7 (0.8-3.2) 1.6 (0.7-3.1) NoTotal cholesterol Diabetes 96.4 (93.8-98.1) 96.9 (94.8-98.3) 96.8 (94.8-98.2) 2.2 (1.1-3.9) 2.1 (1.0-3.8) 2.0 (0.9-3.6) Yes Serum creatinine and thyroid stimulating hormone

Mental health (lithium) 100 (100-100) 100 (100-100) 100 (100-100) 0 (0-0) 0 (0-0) 0 (0-0) No

Blood pressure Stroke 97.4 (95.1-100) 97.7 (95.8-100) 97.6 (95.8-99.5) 1.3 (0-3.3) 1.2 (0-2.9) 1.1 (0-2.7) YesThyroid function Hypothyroidism 96.8 (94.5-98.7) 96.7 (94.5-98.5) 96.5 (94.3-98.3) 0 (0-0.9) 0 (0-0.9) 0 (0-0.8) No

*Paired indicators relate to control of the relevant measure—for example, the indicator that focuses on recording blood pressure in patients with coronary heart disease, is paired with another indicator that rewards on the basis of the proportion of patients whose last blood pressure reading was ≤150/90 mm Hg.

BMJ | 24 april 2010 | VoluMe 340 899

ANALYSIS

Increasing numbers of countries are using indi-cators to evaluate the quality of clinical care, with some linking payment to achievement.1 For performance frameworks to remain effective the indicators need to be regularly reviewed. The frameworks cannot cover all clinical areas, and achievement on chosen indicators will even-tually reach a ceiling beyond which further improvement is not feasible.2 3 However, there has been little work on how to select indictors for replacement. The Department of Health decided in 2008 that it would regularly replace indicators in the national primary care pay for performance scheme, the Quality and Outcomes Framework,4 making a rigorous approach to removal a prior-ity. We draw on our previous work on pay for performance5 6 and our current work advising the National Institute for Health and Clinical Excellence (NICE) on the Quality and Outcomes Framework to suggest what should be consid-ered when planning to remove indicators from a clinical performance framework.

First UK decisions The Quality and Outcomes Framework cur-rently includes 134 indicators for which gen-eral practices can earn up to a total of 1000 points. Negotiations between the Department of Health and the BMA’s General Practitioners Committee last autumn led to an agreement to remove eight clinical indicators worth 28 points in April 2011 (table 1). The eight indicators are

all process measures and reward actions such as taking blood pressure or taking blood to measure cholesterol, glucose, or creatinine concentra-tions for people with relevant chronic diseases. The framework rewards the action itself rather than a clinically informed response to results or intermediate outcomes such as better control of blood pressure or cholesterol levels. It is there-fore not surprising that achievement of these process indicators is high (median >95% and interquartile range <4.5%) with little change in

rates or variation across practices since 2005-6, the second year of the Quality and Outcomes Framework.

In many schemes, including the Quality and Outcomes Framework, providers can “except” certain patients from inclusion in the denomi-nator figures for an indicator on grounds such as extreme frailty or contraindications to a specified drug. Exception reporting rates are also low for these eight indicators (median <5% and inter-quartile range <3%).

How to identify when a performance indicator has run its course In April 2011 eight clinical indicators will be removed from the UK Quality and Outcomes Framework. David Reeves and colleagues explain why they were chosen and suggest a rationale for future decisions

National achievement and exception rates for indicators that are to be removed from the Quality and Outcomes Framework in 20117

Indicator (measurement of ) ConditionMedian (interquartile range) achievement (%) Median (interquartile range) rate of exceptions (%) Paired

indicator*2005-6 2006-7 2007-8 2005-6 2006-7 2007-8Blood pressure Coronary heart disease 98.2 (96.7-99.3) 98.5 (97.2-99.5) 98.4(97.1-99.4) 0.8 (0-1.8) 0.8 (0-1.7) 0.7 (0-1.6) Yes Haemoglobin A1c Diabetes 97.4 (95.1-98.8) 97.8 (95.9-99.0) 97.7 (96.0-98.9) 2.7 (1.4-4.4) 2.5 (1.3-4.2) 2.4 (1.2-3.9) Yes Blood pressure Diabetes 98.8 (97.6-99.7) 99.0 (98.0-100) 98.9 (97.9-99.6) 1.1 (0.3-2.2) 1.0 (0.3-2.1) 1.0 (0.3-2.1) Yes Serum creatinine Diabetes 96.7 (94.1-98.3) 97.4 (95.3-98.7) 97.4 (95.6-98.7) 1.9 (0.9-3.4) 1.7 (0.8-3.2) 1.6 (0.7-3.1) NoTotal cholesterol Diabetes 96.4 (93.8-98.1) 96.9 (94.8-98.3) 96.8 (94.8-98.2) 2.2 (1.1-3.9) 2.1 (1.0-3.8) 2.0 (0.9-3.6) Yes Serum creatinine and thyroid stimulating hormone

Mental health (lithium) 100 (100-100) 100 (100-100) 100 (100-100) 0 (0-0) 0 (0-0) 0 (0-0) No

Blood pressure Stroke 97.4 (95.1-100) 97.7 (95.8-100) 97.6 (95.8-99.5) 1.3 (0-3.3) 1.2 (0-2.9) 1.1 (0-2.7) YesThyroid function Hypothyroidism 96.8 (94.5-98.7) 96.7 (94.5-98.5) 96.5 (94.3-98.3) 0 (0-0.9) 0 (0-0.9) 0 (0-0.8) No

*Paired indicators relate to control of the relevant measure—for example, the indicator that focuses on recording blood pressure in patients with coronary heart disease, is paired with another indicator that rewards on the basis of the proportion of patients whose last blood pressure reading was ≤150/90 mm Hg.


Non-incentivised aspects of careSample of 148 representative practices from the CPRD

Achievement rates improvedfor most indicators in thepre-incentive periodSignificant initial gains inincentivised indicators but nogains in later yearsNo overall effect on the rate ofimprovement for nonincentivised indicators in2004-5But by 2006-7 achievementrates significantly below thosepredicted by pre-incentivetrends

Figure

Mean achievement rate of 148 general practices for quality of care indicators from 2000-1 to 2006-7. Performance indicatorsgrouped by activity and whether they were incentivised under the QOF scheme, which came into force from 2004-5. (Themean rate is the mean of the adjusted means for the individual indicators within each group)

Reprints: http://journals.bmj.com/cgi/reprintform Subscribe: http://resources.bmj.com/bmj/subscribers/how-to-subscribe

BMJ 2011;342:d3590 doi: 10.1136/bmj.d3590 Page 12 of 12

RESEARCH

Effect of financial incentives on incentivised andnon-incentivised clinical activities: longitudinalanalysis of data from the UK Quality and OutcomesFrameworkTim Doran clinical research fellow1, Evangelos Kontopantelis research associate1, Jose M Valderasclinical lecturer 2, Stephen Campbell senior research fellow 1, Martin Roland professor of healthservices research3, Chris Salisbury professor of primary healthcare4, David Reeves senior researchfellow 1

1National Primary Care Research and Development Centre, University of Manchester, Manchester M13 9PL, UK; 2NIHR School for Primary Care

Research, Department of Primary Health Care, University of Oxford, Oxford OX3 7LF; 3General Practice and Primary Care Research Unit, University

of Cambridge, Cambridge CB2 0SR; 4Academic Unit of Primary Health Care, University of Bristol, Bristol BS8 2AA

AbstractObjective To investigate whether the incentive scheme for UK generalpractitioners led them to neglect activities not included in the scheme.

Design Longitudinal analysis of achievement rates for 42 activities (23included in incentive scheme, 19 not included) selected from 428identified indicators of quality of care.

Setting 148 general practices in England (653 500 patients).

Main outcome measures Achievement rates projected from trends inthe pre-incentive period (2000-1 to 2002-3) and actual rates in the firstthree years of the scheme (2004-5 to 2006-7).

Results Achievement rates improved for most indicators in thepre-incentive period. There were significant increases in the rate ofimprovement in the first year of the incentive scheme (2004-5) for 22 ofthe 23 incentivised indicators. Achievement for these indicators reacheda plateau after 2004-5, but quality of care in 2006-7 remained higherthan that predicted by pre-incentive trends for 14 incentivised indicators.There was no overall effect on the rate of improvement fornon-incentivised indicators in the first year of the scheme, but by 2006-7achievement rates were significantly below those predicted bypre-incentive trends.

Conclusions There were substantial improvements in quality for allindicators between 2001 and 2007. Improvements associated withfinancial incentives seem to have been achieved at the expense of smalldetrimental effects on aspects of care that were not incentivised.

IntroductionOver the past two decades funders and policymakers worldwidehave experimented with initiatives to change physicians’behaviour and improve the quality and efficiency of medicalcare.1 Success has been mixed, and attention has recently turnedto payment mechanism reform, in particular offering directfinancial incentives to providers for delivering high qualitycare.2 In 2004 in the UK the Quality and Outcomes Framework(QOF) was introduced—a mechanism intended to improvequality by linking up to 25% of general practitioners’ incometo achievement of publicly reported quality targets for severalchronic conditions.3

Should these incentives succeed, the potential benefits forpatients with the relevant conditions are considerable.4 Incentivesmight also improve general organisation of care, benefitingprocesses and conditions beyond those covered by theincentives.5 Financial incentives have several potentialunintended consequences, however. For example, they mightresult in diminished provider professionalism, neglect of patientsfor whom quality targets are perceived to be more difficult toachieve, and widening of health inequalities.6 7 Doctors mightalso focus on the conditions linked to incentives and neglectother conditions8 or, where certain activities are incentivisedwithin the management of a particular condition, might neglectother activities for patients with that condition.Practices in England generally performed well on incentivisedactivities in the first year of the UK incentive scheme, andoverall performance improved over the next two years.9-11 It is

Corresponding to: Tim Doran [email protected]

Extra material supplied by the author: Methodological appendix (see http://www.bmj.com/content/342/bmj.d3590/suppl/DC1)

Reprints: http://journals.bmj.com/cgi/reprintform Subscribe: http://resources.bmj.com/bmj/subscribers/how-to-subscribe

BMJ 2011;342:d3590 doi: 10.1136/bmj.d3590 Page 1 of 12

Research

RESEARCH


Reasons for exception reporting patientsAll English practices in 2008-9

Median exception rate was 2.7% (IQR 1.9-3.9%) overall and0.44% (0.14-1.1%) for informed dissentCommon reasons logistical (40.6%), clinical contraindication(18.7%), patient informed dissent (30.1%)Higher rates associated with: larger practices, higher deprivation,failure to secure maximum remuneration in previous yearCost of the provision relatively low at £0.58 per patient (£31m)Relatively few patients excluded for informed dissent, suggestingthat the incentivised activities were broadly acceptable to patientsFigures

Fig 1 Proportion of patients exception reported by indicator and reason, 2008-9. For 37 indicators for which reasons forexception reporting were ascribable (see table 1). Indicators ordered by type of activity (measurement or outcome) and byrate of exception reporting attributable to informed dissent

Fig 2 Total remuneration for all practices attributable to achievement of targets and exception reporting, by clinical indicator(all 62). Total remuneration is based on population achievement rates. Remuneration attributable to achieving targets isbased on reported achievement rates. Remuneration attributable to exception reporting is the difference between totalremuneration and remuneration attributable to achieving targets

No commercial reuse: See rights and reprints http://www.bmj.com/permissions Subscribe: http://www.bmj.com/subscribe

BMJ 2012;344:e2405 doi: 10.1136/bmj.e2405 (Published 17 April 2012) Page 11 of 11

RESEARCH

Exempting dissenting patients from pay forperformance schemes: retrospective analysis ofexception reporting in the UK Quality and OutcomesFramework

OPEN ACCESS

Tim Doran clinical research fellow 1, Evangelos Kontopantelis research fellow 1, Catherine Fullwoodresearch associate 2, Helen Lester professor of primary care 3, Jose M Valderas clinical lecturer 4,Stephen Campbell senior research fellow 1

1Health Sciences Research Group-Primary Care, University of Manchester, Manchester M13 9PL, UK; 2Manchester Academic Health ScienceCentre, Central Manchester University Hospitals NHS Foundation Trust, University of Manchester; 3School of Health and Population Sciences,University of Birmingham, Birmingham, UK; 4NIHR School for Primary Care Research, Health Services and Policy Research Group, Departmentof Primary Care Health Sciences, University of Oxford, Oxford, UK

AbstractObjective To examine the reasons why practices exempt patients fromthe UK Quality and Outcomes Framework pay for performance scheme(exception reporting) and to identify the characteristics of generalpractices associated with informed dissent.

Design Retrospective analysis.

Setting Data for 2008-9 extracted from the clinical computing systemsof general practices in England.

Participants 8229 English family practices.

Main outcome measures Rates of exception reporting for 37 clinicalquality indicators, associations of patient and general practice factorswith exception rates, and financial gain for practices relating to their useof exception reporting.

Results The median rate of exception reporting was 2.7% (interquartilerange 1.9-3.9%) overall and 0.44% (0.14-1.1%) for informed dissent,but variation in rates was wide between practices and across indicators.Common reasons for exception reporting were logistical (40.6% ofexceptions), clinical contraindication (18.7%), and patient informeddissent (30.1%). Higher rates of informed dissent were associated with:higher numbers of registered patients, higher levels of local areadeprivation, and failure of the practice to secure maximum remunerationin the previous year. Exception reporting increased the cost of thescheme by £30 844 500 (€36 877 700; $49 053 200) (£0.58 per patient),with two indicators accounting for a quarter of this additional cost.

Conclusions The provision to exception report enables practices toexempt dissenting patients without being financially penalised. Relativelyfew patients were excluded for informed dissent, however, suggestingthat the incentivised activities were broadly acceptable to patients.

IntroductionSince early 2000 payers across healthcare systems worldwidehave experimented with pay for performance schemes thatexplicitly link doctors’ remuneration to quality of care, withmixed success.1-3 In the United Kingdom, a national scheme forprimary care—the Quality and Outcomes Framework—wasintroduced in 2004, providing financial incentives to familypractices for meeting targets on a range of clinical,organisational, and patient experience indicators.4Most practiceshave performed well under the scheme,5-8 but improved patientoutcomes have not been consistently evident.9-11 Even if theysuccessfully stimulate improved performance, pay forperformance schemes have several potential unintendedconsequences. In particular, given a method of remunerationthat financially rewards doctors for performing procedures,prescribing drugs, and controlling biological variables, patientsmay be coerced or refused care if they are non-compliant.12 13

Understanding these potential risks, the designers of the QualityOutcomes Framework included two mechanisms intended toprotect patients from coercive care. Firstly, upper payment

Correspondence to: T Doran [email protected]

Extra material supplied by the author (see http://www.bmj.com/content/344/bmj.e2405?tab=related#webextra)

Business rules for clinical indicatorsExample of business rules for clinical indicators (fig A1) and exception reporting rates for indicators excluded from main analysis (fig A2)Table showing clinical indicators excluded from main analysis

No commercial reuse: See rights and reprints http://www.bmj.com/permissions Subscribe: http://www.bmj.com/subscribe

BMJ 2012;344:e2405 doi: 10.1136/bmj.e2405 (Published 17 April 2012) Page 1 of 11

Research

RESEARCH


Changes to the incentivesAll English practices, 2004-5 to 2009-10

Four influenza immunisationindicators for CHD, COPD,diabetes and strokeUpper threshold increased by5% only for CHD in 2006-7The increase led to increase inthe % of immunised CHDpatients by 0.41% but also toexception reporting by 0.26%Making quality targets moredemanding can lead toimprovement in quality of carebut can also have otherconsequences

Minimumthreshold

y1/2

Mminimumthreshold

y3/5

Maximumthreshold

y1/2

Maximumthreshold

y3/5

020

040

060

080

010

00N

umbe

r of

pra

ctic

es

0 20 40 60 80 100

Percentage of patients

08/09 07/08 06/07 05/06 04/05

CHD 12

Reported achievement

Family Doctor Responses to Changes inIncentives for Influenza Immunizationunder the U.K. Quality andOutcomesFramework Pay-for-PerformanceSchemeEvangelos Kontopantelis, Tim Doran, Hugh Gravelle, RosalindGoudie, Luigi Siciliani, and Matt Sutton

Objective. To analyze the effect of setting higher targets, in a primary care pay-for-performance scheme, on rates of influenza immunization and exception reporting.Study Setting. The U.K. Quality and Outcomes Framework links financialrewards for family practices to four separate influenza immunization rates forpatients with coronary heart disease (CHD), chronic obstructive pulmonary dis-ease, diabetes, and stroke. There is no additional payment for immunization ratesabove an upper threshold. Patients for whom immunization would be inappropri-ate can be excepted from the practice for the calculation of the practice immuni-zation rate.Data. Practice-level information on immunizations and exceptions extracted fromelectronic records of all practices in England 2004/05 to 2009/10 (n = 8,212–8,403).Study Design. Longitudinal random effect multilevel linear regressions comparingchanges in practice immunization and exception rates for the four chronic conditionsbefore and after the increase in the upper threshold immunization rate for CHDpatients in 2006/07.Principal Findings. The 5 percent increase in the upper payment threshold for CHDwas associated with increases in the proportion of immunized CHD patients (0.41 per-cent, CI: 0.25–0.56 percent), and exception was reported (0.26 percent, CI: 0.12–0.40percent).Conclusions. Making quality targets more demanding can not only lead to improve-ment in quality of care but can also have other consequences.Key Words. Quality and Outcomes Framework (QOF), influenza immunization,pay-for-performance, upper threshold

©Health Research and Educational TrustDOI: 10.1111/j.1475-6773.2011.01362.xRESEARCHARTICLE

1117

Health Services Research

Health Services Research, 2012


Patient level diabetes careSample of 148 representative practices from the CPRD

In 2004-5 quality improvedover-and-above thispre-incentive trend by 14.2%By 2006-7 the improvementabove trend was smaller at 7.3Levels of care variedsignificantly for sex, age, yearsof previous care, number ofco-morbid conditions

Recorded quality of primary care forpatients with diabetes in Englandbefore and after the introductionof a financial incentive scheme:a longitudinal observational study

Evangelos Kontopantelis,1 David Reeves,1 Jose M Valderas,2,3

Stephen Campbell,1 Tim Doran1

▸ An additional data ispublished online only. To viewthis file please visit the journalonline (http://bmjqs.bmj.com)

1Health Sciences Primary CareResearch Group, University ofManchester, Manchester, UK2Health Services and PolicyResearch Group, NIHR School forPrimary Care Research,Department of Primary HealthCare, University of Oxford,Oxford, UK3European Observatory of HealthSystems and Policies, LondonSchool of Economics, London,UK

Correspondence toDr Evangelos Kontopantelis,Health Sciences Primary CareResearch Group, 5th floorWilliamson Building,Manchester M13 9PL, UK;[email protected]

Received 29 March 2012Revised 26 June 2012Accepted 12 July 2012Published Online First22 August 2012

To cite: Kontopantelis E,Reeves D, Valderas JM, et al.BMJ Qual Saf 2013;22:53–64.

ABSTRACTBackground The UK’s Quality and OutcomesFramework (QOF) was introduced in 2004/5,linking remuneration for general practices torecorded quality of care for chronic conditions,including diabetes mellitus. We assessed theeffect of the incentives on recorded quality ofcare for diabetes patients and its variation bypatient and practice characteristics.Methods Using the General Practice ResearchDatabase we selected a stratified sample of 148English general practices in England, contributingdata from 2000/1 to 2006/7, and obtained arandom sample of 653 500 patients in which23 920 diabetes patients identified. Wequantified annually recorded quality of care atthe patient-level, as measured by the 17 QOFdiabetes indicators, in a composite score andanalysed it longitudinally using an InterruptedTime Series design.Results Recorded quality of care improved for allsubgroups in the pre-incentive period. In the firstyear of the incentives, composite qualityimproved over-and-above this pre-incentive trendby 14.2% (13.7–14.6%). By the third year theimprovement above trend was smaller, but stillstatistically significant, at 7.3% (6.7–8.0%). After3 years of the incentives, recorded levels of carevaried significantly for patient gender, age, yearsof previous care, number of co-morbidconditions and practice diabetes prevalence.Conclusions The introduction of financialincentives was associated with improvements inthe recorded quality of diabetes care in the firstyear. These improvements included somemeasures of disease control, but most capturedonly documentation of recommended aspects ofclinical assessment, not patient management oroutcomes of care. Improvements in subsequent

years were more modest. Variation in carebetween population groups diminished underthe incentives, but remained substantial in somecases.

INTRODUCTIONIn the last 15 years the National HealthService in the UK has undergone a series ofreforms aimed at improving the quality ofcare for people with chronic conditions.These include the creation of the NationalInstitute for Health and ClinicalExcellence, and the introduction ofNational Service Frameworks which setminimum standards for the delivery ofhealth services in specified clinical areas,including diabetes mellitus.1 The quality ofprimary care generally, and of diabetes carein particular, improved in the early 2000s,2

partly in response to these quality improve-ment initiatives.3 In 2004 new contractualarrangements for family doctors allowedthem to opt out of out-of-hours care andlinked financial incentives to quality ofcare under the Quality and OutcomesFramework (QOF),4 the largest and mostambitious pay-for-performance schemeever attempted in health care.5 6

The QOF initially included 76 clinicalindicators, covering a range of processes ofcare (eg, measurement of blood pressure)and intermediate outcomes (eg, glycaemiccontrol). A further 70 indicators coveredaspects of practice organisation and patientexperience of care. Eighteen of the clinicalindicators related to care for patients withdiabetes, reflecting in part the nationalimportance of the disease, the recordedprevalence of which had increased by 75%

ORIGINAL RESEARCH

Kontopantelis E, et al. BMJ Qual Saf 2013;22:53–64. doi:10.1136/bmjqs-2012-001033 53

group.bmj.com on September 13, 2013 - Published by qualitysafety.bmj.comDownloaded from

Recorded QOF care did not vary significantly by areadeprivation before or after the introduction of theincentivisation scheme. However, the effect of the inter-vention did vary with area deprivation: patients attend-ing practices from the most deprived quartile appear tohave gained less from the intervention compared withpatients in the most affluent quartile of practices, by4.9% in 2004/5 and 3.8% in 2006/7.There was significant variation in recorded QOF care

by practice diabetes prevalence rates, but the differenceswere small and diminished over time. The interventioneffect also varied with practice diabetes prevalence.Compared with practices in the first quartile (lowest dia-betes prevalence), the QOF effect was larger for practicesin the second and third quartiles—by 1.4% and 2.1%respectively in the short term (2004/5) and by 3.2% and4.8% respectively in the long term (2006/7).

Sensitivity analysesThe sensitivity analysis (based on untransformed data)agreed with the main analysis in all respects except forthe relationship between the intervention and patientgender. In the sensitivity analysis both the short- andlong-term impact of the pay-for-performance schemewas significantly larger, though small in scale, for femalepatients (1.2%, p=0.048 and 2.5%, p=0.01 respectively).

DISCUSSION

The main aim of pay-for-performance schemes is toincentivise physicians to provide high quality care andthereby improve patient outcomes. Research to datesuggests that pay-for-performance schemes have limitedimpact when implemented in isolation, but when sup-ported by other quality improvement initiatives canhave a positive effect on quality of care.3 We foundthat recorded quality of primary care in the UK, as

measured by the QOF diabetes indicators, was alreadyimproving prior to the introduction of the scheme in2004, and that it improved at an accelerated rate inthe first years of its implementation, supporting thefindings of previous studies.11 23 However, this acceler-ated improvement did not seem to benefit all popula-tion groups equally.

Strengths and limitations of the studyThe main strength of our study was in the use of datafor individual patients drawn from a nationally represen-tative sample of practices. However, the study is subjectto certain limitations. First, the QOF was introduced uni-versally and was not implemented as part of a rando-mised experiment. The lack of a practice control groupentails that analyses of its effect in quality of care areonly possible using quasi-experimental methods. Resultsobtained with these methods can be method- andassumptions-sensitive; however, the interrupted time-series design is one of the most effective and powerfulof all quasi-experimental designs and is routinely beingused as the best possible approach when such researchscenarios arise.24 Second, we are reliant on the accurateand consistent recording of data by practices; however,usage of clinical computing systems has changed overtime and practices may have exaggerated their perform-ance in response to the financial incentives. This studyhas investigated the quality of recorded diabetes care andthere may be differences with care actually delivered.On the other hand, improved measurement is a neces-sary prerequisite for improved quality of care and onecould argue that it is improved quality of care. Third,most of the measures refer to documentation levels anddo not necessarily lead to the intended improvements inoutcomes if not properly followed-through or the inter-ventions are not offered in an appropriate manner (eg,advising a patient briefly ‘perhaps you should considerquitting smoking’ in order to ‘tick’ the relevant QOFbox is not really a smoking cessation intervention).Fourth, some quality indicators are dependent onothers, for example, indicator DM12 (blood pressurecontrolled) cannot be met unless indicator DM11(blood pressure measured) has also been met. However,we aimed to quantify and assess overall quality of care asmeasured by the whole diabetes domain in the QOFand to be as inclusive as possible. Fifth, the conditionswe modelled to investigate the effect of co-morbiditywere not an exhaustive list and only the presence orabsence was modelled and not the severity of each con-dition. However, the number of chronic co-morbidities isa well-established marker of the overall clinical complex-ity of a patient.25 Sixth, our findings assume that theobserved trends in indicator achievement prior to QOFwould have continued unchanged had the incentive

Figure 2 Aggregate patient level Quality and Outcomes

Framework care and predictions based on the

pre-incentivisation trend.

BMJ Qual Saf 2012;0:1–12. doi:10.1136/bmjqs-2012-001033 9

Original research

group.bmj.com on September 14, 2012 - Published by qualitysafety.bmj.comDownloaded from

2000/1 2001/2 2002/3 2003/4 2004/5 2005/6 2006/7new diagno 44.7 50.4 56.5 65.3 73.4 74.2 74.31-4 years 48.4 53.9 59.4 71.1 80.9 83 83.25-9 years 46.4 51.9 56.8 69.1 78.7 81.4 81.810+ years 45.4 50 55.1 66.7 77.6 79.3 80.4

2000/1 2001/2 2002/3 2003/4 2004/5 2005/6 2006/7new diagnoses 44.7 50.4 56.5 65.3 73.4 74.2 74.31-4 years 48.4 53.9 59.4 71.1 80.9 83 83.25-9 years 46.4 51.9 56.8 69.1 78.7 81.4 81.810+ years 45.4 50 55.1 66.7 77.6 79.3 80.4

40

45

50

55

60

65

70

75

80

85

90

aggr

egat

e re

cord

ed Q

OF

care

scor

e


Clinical computer systemsAll English practices, 2007-8 to 2010-11

Seven clinical computersystems consistently activecollectively holding ≈ 99% ofthe market shareSystem choice strongestpredictor of performance on62 stable care indicatorsDifferences greatest forintermediate outcomeindicatorsParticular systemcharacteristics facilitate higherquality of care, better datarecording or both?

North East

North West

London

West Midlands

Yorkshire & the Humber

South West

East Midlands

East of England

South Central

South East Coast

(85.4,85.7](85.1,85.4](84.8,85.1](84.5,84.8](84.2,84.5][83.9,84.2] LVVision 3ProdSysOneXPCSSynergyPractice ManagerPremiere

NOTE: Chart size proportional to number of practices in area

Average practice scores by Strategic Health Authority, 2010−11

Overall population achievement (62 indicators)and GP systems products

Relationship between quality of careand choice of clinical computingsystem: retrospective analysis of familypractice performance under the UK’squality and outcomes framework

Evangelos Kontopantelis,1,2 Iain Buchan,3 David Reeves,1,2 Kath Checkland,1

Tim Doran4

To cite: Kontopantelis E,Buchan I, Reeves D, et al.Relationship between qualityof care and choice of clinicalcomputing system:retrospective analysis offamily practice performanceunder the UK’s quality andoutcomes framework. BMJOpen 2013;3:e003190.doi:10.1136/bmjopen-2013-003190

▸ Prepublication history andadditional material for thispaper is available online. Toview these files please visitthe journal online(http://dx.doi.org/10.1136/bmjopen-2013-003190).

Received 9 May 2013Revised 4 July 2013Accepted 5 July 2013

For numbered affiliations seeend of article.

Correspondence toDr Evangelos Kontopantelis;[email protected]

ABSTRACTObjectives: To investigate the relationship betweenperformance on the UK Quality and OutcomesFramework pay-for-performance scheme and choice ofclinical computer system.Design: Retrospective longitudinal study.Setting: Data for 2007–2008 to 2010–2011, extractedfrom the clinical computer systems of general practicesin England.Participants: All English practices participating in thepay-for-performance scheme: average 8257 each year,covering over 99% of the English population registeredwith a general practice.Main outcome measures: Levels of achievement on62 quality-of-care indicators, measured as: reportedachievement (levels of care after excludinginappropriate patients); population achievement (levelsof care for all patients with the relevant condition) andpercentage of available quality points attained.Multilevel mixed effects multiple linear regressionmodels were used to identify population, practice andclinical computing system predictors of achievement.Results: Seven clinical computer systems wereconsistently active in the study period, collectivelyholding approximately 99% of the market share. Of allpopulation and practice characteristics assessed,choice of clinical computing system was the strongestpredictor of performance across all three outcomemeasures. Differences between systems were greatestfor intermediate outcomes indicators (eg, control ofcholesterol levels).Conclusions: Under the UK’s pay-for-performancescheme, differences in practice performance wereassociated with the choice of clinical computingsystem. This raises the question of whether particularsystem characteristics facilitate higher quality of care,better data recording or both. Inconsistencies acrosssystems need to be understood and addressed, andresearchers need to be cautious when generalisingfindings from samples of providers using a singlecomputing system.

ARTICLE SUMMARY

Article focus▪ Practice and patient-level characteristics are

known predictors of quality of care, as measuredby the Quality and Outcomes Framework (QOF)indicators.

▪ Various general practitioner (GP) clinical com-puter systems are used in the UK but their distri-bution over time and location is unknown.

▪ GP clinical computer systems differ in softwarearchitecture, user interface and clinical codinglists but their effect on quality of care has neverbeen examined.

Key messages▪ Seven systems were found to hold 99% of the

market share, with clear geographical variation intheir distribution.

▪ Levels of performance on the QOF differed to asmall extent across clinical computer systems,even after controlling for practice and patientcharacteristics. Quantified differences were smallbut not negligible since they translate to system-atic variation in recorded care for hundreds ofthousands of patients nationwide.

▪ Researchers that utilise primary care databases,which collect data from a single clinical system,need to be cautious when generalising their find-ings to all English practices.

Strengths and limitations of this study▪ This is the first study that investigates the effect

of GP clinical computer system choice on mea-sured quality of care.

▪ We used data for over 99% of all English prac-tices and there is no risk of inductive fallacy.

▪ There are more aspects to quality of care thanwhat is recorded under the QOF; this is an obser-vational study and causality is difficult to establishand it is possible that QOF-oriented practiceshave particular clinical system preferences.

Kontopantelis E, Buchan I, Reeves D, et al. BMJ Open 2013;3:e003190. doi:10.1136/bmjopen-2013-003190 1

Open Access Research

group.bmj.com on September 13, 2013 - Published by bmjopen.bmj.comDownloaded from


Getting there...

Very high levels of initialachievent leading tooverpaymentStill confusion about whetherthe aim is to reward highquality of care or to driveimprovementIt appears that the same levelsof care would have been meteventually; QOF just took usthere quicker

050

010

0015

00N

umbe

r of

pra

ctic

es

0 20 40 60 80 100

Percentage of patients

08/09 07/08 06/07 05/06 04/05

Overall, 48+2 (smoking) indicators

Reported achievement


Getting there...

Reduced inequalities but negatively affected continuity andaspects of non incentivised careQOF led to bigger, better organised practices but patients do notseem to like thatException reporting a cheap provision to ensure no patientdiscriminationSmall changes in the scheme details can have big effects onquality of careThe intervention effect varied by patient groupsStrongest predictor of achievement was clinical system


What’s to be done with it?good on paper but massive cost

Limit incentives to intermediateoutcome indicators only?Remove the upper thresholds forindicators, thereby reducing thecost and driving improvement?Re-negotiate the paymentplatform for the same reason?Bigger range of indicators andonly a random set assessed eachyear?Drop the whole scheme andre-invest in other ventures?


More questions

What happens when the incentive for an indicator is removed?What is the effect of the incentivisation on harder outcomes?

mortalitycomplications (e.g. for diabetes)

What really happens with exceptions and is there any gaming?


Thank you for listening!

Comments and questions: [email protected]


Faculty showcase 2013 - Opening up clinical performance

Science

nancial incentives

overall quality of care

gp practices

set of quality targets

clinical care

clinical performance

palliative care

gp income