Including post-discharge mortality in calculation of hospital standardised mortality ratios: retrospective analysis of hospital episode statistics

Including post-discharge mortality in calculation ofhospital standardised mortality ratios: retrospectiveanalysis of hospital episode statistics

OPEN ACCESS

Maurice E Pouw resident1, L M Peelen post-doctoral researcher2, K GMMoons professor of clinicalepidemiology 2, C J Kalkman professor of anesthesiology 1, H F Lingsma post-doctoral researcher 3

1Department of Anesthesiology, University Medical Center Utrecht, Heidelberglaan 100, PO Box 85500, 3508 GA Utrecht, Netherlands; 2JuliusCenter for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht, Netherlands; 3Department of Public Health, ErasmusMC, PO Box 2040, 3000 CA Rotterdam, Netherlands

AbstractObjectives To assess the consequences of applying different mortalitytimeframes on standardised mortality ratios of individual hospitals and,secondarily, to evaluate the association between in-hospital standardisedmortality ratios and early post-discharge mortality rate, length of hospitalstay, and transfer rate.

Design Retrospective analysis of routinely collected hospital data tocompare observed deaths in 50 diagnostic categories with deathspredicted by a case mix adjustment method.

Setting 60 Dutch hospitals.

Participants 1 228 815 patients discharged in the period 2008 to 2010.

Main outcome measures In-hospital standardised mortality ratio, 30days post-admission standardised mortality ratio, and 30 dayspost-discharge standardised mortality ratio.

ResultsCompared with the in-hospital standardised mortality ratio, 33%of the hospitals were categorised differently with the 30 dayspost-admission standardised mortality ratio and 22% were categoriseddifferently with the 30 days post-discharge standardised mortality ratio.A positive association was found between in-hospital standardisedmortality ratio and length of hospital stay (Pearson correlation coefficient0.33; P=0.01), and an inverse association was found between in-hospitalstandardised mortality ratio and early post-discharge mortality (Pearsoncorrelation coefficient −0.37; P=0.004).

Conclusions Applying different mortality timeframes resulted indifferences in standardised mortality ratios and differences in judgmentregarding the performance of individual hospitals. Furthermore,associations between in-hospital standardised mortality rates, length ofstay, and early post-discharge mortality rates were found. Combiningthese findings suggests that standardised mortality ratios based onin-hospital mortality are subject to so-called “discharge bias.” Hence,early post-discharge mortality should be included in the calculation ofstandardised mortality ratios.

IntroductionIn the past few decades, quality of care in hospitals has beensubject to growing attention from physicians and regulators. Invarious countries, standardised mortality ratios are used in anattempt to judge the quality of hospital care.1-4However, severalauthors have raised concerns that differences in standardisedmortality ratios may not reflect differences in quality of caredelivered.5 6 Reasons that have been put forward include thequality of the data used, the limitations of case mix adjustment,and several methodological issues.7-12 Another limitation ofmortality rate as a quality measure is the current focus onin-hospital mortality—that is, deaths that occur during hospitaladmission. Analyses based only on in-hospital deaths arepotentially biased by differences in hospitals’ dischargepractices. For example, hospitals that transfer high risk patientsto other more specialised hospitals may have lower thanexpectedmortality, because some of their patients die elsewhere.Furthermore, the average length of hospital stay has decreasedsignificantly in the past few decades and may therefore haveshifted mortality away from the hospital to post-dischargedestinations.13 14 A recent study by Yu et al (2011) showed thatfor certain surgical procedures approximately a quarter ofpostoperative deaths occurred after discharge and that 12% tookplace just one day after discharge from hospital.15 Metersky etal (2012) concluded that approximately 50% of older patientswho died from pneumonia within 30 days of admission did notdie in hospital but after discharge.16 We will refer to thisphenomenon as “early post-discharge mortality.”Differences in discharge practices or lengths of stay betweenhospitals may thus affect their in-hospital mortality rates. Suchbiases arising from differences in discharge practices could haveimportant consequences for hospitals in an environment ofpublic reporting and payment by results. When the timeframe

Correspondence to: M E Pouw [email protected]

No commercial reuse: See rights and reprints http://www.bmj.com/permissions Subscribe: http://www.bmj.com/subscribe

BMJ 2013;347:f5913 doi: 10.1136/bmj.f5913 (Published 21 October 2013) Page 1 of 12

Research

RESEARCH

to observe death is fixed or is prolonged to the post-dischargeperiod, these “discharge” biases may be countered. For example,the United Kingdom used to report standardised mortality ratiosbased on in-hospital mortality but recently prolonged thetimeframe to “30 days post-discharge.”17 A commonly usedalternative timeframe is the “30 days post-admission” timeframethat covers the fixed period from admission to 30 dayspost-admission.15 16 18 In this study, we will explore bothtimeframes.The aim of our study was to assess the effect of differentmortality timeframes on standardised mortality ratios ofindividual hospitals and judgment of their performance. Asecondary objective was to investigate the relation betweenin-hospital standardised mortality ratio and early post-dischargemortality, length of hospital stay, and transfer rates. We useddata from more than two million Dutch hospital discharges toexplore the differences between hospital standardised mortalityratios calculated by using either in-hospital mortality, 30 dayspost-admission mortality, or 30 days post-discharge mortality.

MethodsDataDutch Hospital Data, the holder of the national HospitalDischarge Register, gave us permission to use their database todo this study. The Hospital Discharge Register containsdischarge data of general and academic Dutch hospitals andcomprises patients’ characteristics such as age and sex as wellas medical variables such as date of admission, date of discharge,diagnoses, and comorbidities. The register follows theICD-9-CM (international classification of diseases, 9th revision,clinical modification) to register discharge diagnoses.Participation of hospitals in the Hospital Discharge Register isvoluntary. In the period 2007-10, the total number of hospitalsin the Netherlands was 100, of which 84 participated in theregister and contributed to this study.To obtain information on deaths that occurred after dischargefrom hospital, Statistics Netherlands (www.CBS.nl) linkedrecords from the Hospital Discharge Register to the Dutchpopulation register. The population register contains personaldetails such as the date of birth, date of death (if applicable),sex, and address of all residents in the Netherlands. Because theHospital Discharge Register is pseudonymised, only date ofbirth, sex, and truncated postal code (four digits) are availablefor linkage with the population register. Statistics Netherlandsregularly evaluates the linkage of the Dutch national HospitalDischarge Register with the population register and concludesthat it is of good quality and forms an adequate basis forstatistical analyses.19We used the combined dataset to computethe time to death (subtracting the date of admission from thedate of death on the death certificate) and to fit the statisticalmodels.

Risk adjusted mortality modelsStatistics Netherlands calculates the Dutch hospital standardisedmortality ratios each year as follows.20 Only in-patient recordswith a primary diagnosis belonging to one of 50 selecteddiagnostic groups (coded using the Clinical ClassificationSystem21) were selected. These 50 diagnostic groups accountfor approximately 80% of all in-hospital deaths in theNetherlands. For each of the 50 selected diagnostic groups, aprediction model is estimated to calculate the expectedprobability of mortality of an admission. The models are logisticregression models with mortality as the dependent variable andage, sex, socioeconomic status, severity of main diagnosis,

urgency of admission, comorbidities, source of admission, andmonth of admission as predictor variables. Firstly, regressionmodels are estimated using all predictors. Subsequently, reducedmodels are estimated, dropping non-significant variables byusing a backward stepwise elimination procedure.22 Thestandardised mortality ratio of a diagnostic group of a hospitalis the ratio of the observed number of deaths and the expectednumber of deaths as calculated on the basis of the regressionmodel. The sum of the observed mortalities of all 50 diagnosticgroups divided by the sum of all expected mortalities times 100gives the hospital-wide standardised mortality ratio. Astandardised mortality ratio greater than 100 indicates highermortality than expected.To calculate standardised mortality ratios with differenttimeframes, we recalibrated the 50 prediction models byredefining “mortality” according to the timeframe used. In thisrecalibration procedure, we used the same variables asindependent predictors and re-estimated the coefficients in theregression model. We chose not to include additional(post-discharge) variables in the post-discharge predictionmodels, because then determining whether differences instandardised mortality ratios are due to the different timeframeused or to the introduction of a new variable would be difficult.All regression models were estimated in R version 2.15.

Comparison of hospital standardisedmortality ratios based on different mortalitytimeframesWe first examined the extent to which standardised mortalityratios depend on the mortality timeframe definition. We madehistograms and scatterplots to evaluate the magnitude anddirection of change in performance when we substituted anin-hospital standardised mortality ratio for a ratio with anothertimeframe. In addition, we classified hospitals into three groupson the basis of the 95% confidence interval of their standardisedmortality ratios. If the 95% confidence interval of thestandardised mortality ratio included the reference value of 100,we categorised the hospital into the group “as expected.” Weregarded a hospital as “better than expected” or “worse thanexpected” if the confidence interval of the standardisedmortalityratio was respectively below 100 or above 100. We analysedhow many hospitals would be categorised differently when adifferent timeframe was used.

Effect of discharge patterns on in-hospitalstandardised mortality ratioWe examined the following variables for their association within-hospital standardised mortality ratio: “early post-discharge”mortality rate (defined as mortality between discharge and 30days post-admission divided by the number of alive discharges),average length of hospital stay, and average transfer rate to othermedical facilities such as hospitals, nursing homes, and othermedical institutions (excluding care homes). For all of thesevariables, we evaluated the association with in-hospitalstandardised mortality ratio by using the Pearson correlationcoefficient. We used SPSS 16.0.2 for all analyses.

ResultsData and risk adjusted mortality modelsThe dataset contained 2 387 604 discharges, of which 2 149958 (90%) could be uniquely paired with the population register.Table 1⇓ shows patients’ characteristics for the linkable andnon-linkable discharges. In summary, the non-linkable

No commercial reuse: See rights and reprints http://www.bmj.com/permissions Subscribe: http://www.bmj.com/subscribe

BMJ 2013;347:f5913 doi: 10.1136/bmj.f5913 (Published 21 October 2013) Page 2 of 12

RESEARCH

discharges were on average younger, more oftenmale, andmoreoften admitted urgently, and they had a lower in-hospitalmortality rate.We used Hospital Discharge Register data of all patientsdischarged in the period 2007-10 that could be uniquely pairedwith the population register to estimate the coefficients of the50 prediction models for in-hospital mortality, 30 dayspost-admission mortality, and 30 days post-discharge mortality.The estimated models are available on request.The Dutch hospitals are categorised as eight academic hospitals,84 general hospitals, and eight specialised hospitals such as eyehospitals and epilepsy clinics. For all eight academic hospitalsand 52 general hospitals, we analysed the three year standardisedmortality ratios (2008-10). We excluded 32 general hospitalsfrom analysis for one of three reasons: no or insufficientparticipation in the Hospital Discharge Register (for example,start of participation after 2009), inadequate data (for example,no registration of comorbidities), or no permission to publish.We excluded the specialised hospitals from analysis because oftheir unique patient profiles.20 We used the three yearstandardised mortality ratio because this is common practice inthe Netherlands.20

In 2008-10 the 60 included hospitals discharged 1 228 815patients. Across these hospitals, the mean in-hospital mortalityrate for the 50 diagnostic groups was 4.9% (SD 0.7%). Averagelength of hospital stay in 2008-10 was 7.2 (SD 0.7) days. Forthe 60 hospitals, 1 199 889 (97.8%, SD 0.7%) patients, had alength of stay shorter than 30 days. The in-hospital mortalityrate until 30 days post-admission was 4.7% (0.7%). Thein-hospital mortality rate for patients with a length of stay longerthan 30 days was 11.6% (2.4%).The overall mortality rate at 30 days (both in-hospital and outof hospital) was 7.2% (0.8%). Themortality rate from admissionto 30 days post-discharge was 8.4% (0.9%). The earlypost-discharge mortality rate was 2.7% (0.4%). Table 2⇓ givesa full overview of mortality rates and discharge statistics.

Comparison of hospital standardisedmortality ratios based on different mortalitytimeframesFigure 1⇓ shows histograms of in-hospital standardisedmortalityratios, 30 days post-admission standardised mortality ratios,and 30 days post-discharge standardised mortality ratios.Between hospital variability was less with 30 dayspost-admission and 30 days post-discharge ratios. Figure 2⇓shows scatterplots indicating how in-hospital standardisedmortality ratios change when 30 days post-admission ratios and30 days post-discharge ratios are used.Tables 3⇓ and 4⇓ show whether these different standardisedmortality ratios also lead to different judgment of individualhospitals. On the basis of in-hospital standardised mortalityratio, 17 hospitals performed better than expected (95%confidence interval <100) and nine hospitals performed worsethan expected (95% confidence interval >100). Using 30 dayspost-admission standardised mortality ratio, 20 hospitals werejudged differently compared with in-hospital standardisedmortality ratios (table 3⇓). With 30 days post-dischargestandardised mortality ratio, 13 hospitals were categoriseddifferently (table 4⇓).

Effect of discharge patterns on in-hospitalstandardised mortality ratioTable 5⇓ shows the association between in-hospital standardisedmortality ratio and early post-discharge mortality rates, lengthof stay, and transfer rates. The in-hospital standardised mortalityratio had a positive correlation with length of stay (Pearsoncorrelation coefficient 0.33; P=0.01) and a negative correlationwith early post-discharge mortality rates (Pearson correlationcoefficient −0.37; P=0.004). The correlation between length ofstay and early post-discharge mortality rates was negative(Pearson correlation coefficient −0.30; P=0.02).According to the dataset used, 9.4% (SD 3.8%) of dischargeswere transferred to another medical institution. We noted thatfour hospitals had no recorded transfers. The correlation betweenin-hospital standardised mortality ratio and transfer rate wasnot statistically significant (Pearson correlation coefficient−0.06; P=0.661).

DiscussionThis study examined the effect of applyingmortality timeframesthat included the post-discharge period on the standardisedmortality ratios of individual hospitals. Compared withstandardised mortality ratios based on in-hospital mortality, wefound that these timeframes resulted in differences in ratios andeven altered judgments regarding the performance of individualhospitals. Furthermore, we found associations betweenin-hospital standardised mortality ratio, length of stay, and earlypost-discharge mortality. Combining these findings suggeststhat standardised mortality ratios based on in-hospital mortalitymay be subject to so-called “discharge bias.”23

The presence of discharge bias is suggested by severalobservations in our analysis. Firstly, we found an inverse relationbetween in-hospital standardised mortality ratio and earlypost-discharge mortality, implying that lower in-hospitalmortality may actually reflect higher post-discharge mortalityinstead of the assumed higher degree of quality of care.Secondly, a shorter average length of stay was associated withlower in-hospital standardised mortality ratio. To be consideredas “better performing,” hospitals with low length of stay andwith low in-hospital mortality should also have low or averagepost-dischargemortality. However, we found that the correlationbetween average length of stay and early post-dischargemortality was negative, implying that shorter average length ofstay is associated with higher post-discharge mortality. If ahospital decides to reduce length of stay without changing thecare delivered, more patients will die after discharge instead ofduring admission; as a consequence, the in-hospital standardisedmortality ratio will decrease. This phenomenon may beincreasingly important, as the average length of stay hasconsistently declined over the past few decades,13 14 and it willmost likely continue to decline in the future owing to economicpressures on hospital beds.Finally, the histograms in figure 1⇓ show that the betweenhospital variability in standardised mortality ratio decreasedwhen 30 days post-admission and 30 days post-dischargetimeframes were used, suggesting that at least part of thevariation of in-hospital standardised mortality ratio can beexplained by mortality occurring shortly after discharge.Altogether, the influence of discharge bias may be substantialfor individual hospitals, as a large number of hospitals (20/60)were categorised differently when the 30 days post-admissiontimeframe was used.

No commercial reuse: See rights and reprints http://www.bmj.com/permissions Subscribe: http://www.bmj.com/subscribe

BMJ 2013;347:f5913 doi: 10.1136/bmj.f5913 (Published 21 October 2013) Page 3 of 12

RESEARCH

Comparison with other studiesOur results are in accordance with previous work and underlinethe risk of discharge bias when using in-hospital mortalitystatistics. For example, Vasilevskis et al (2009) studied intensivecare unit admissions and concluded that variations in transferrates and discharge timing seem to bias in-hospital standardisedmortality ratio calculations.23 In our study, we found astatistically significant association between early post-dischargemortality and in-hospital standardised mortality ratios but nostatistically significant association between transfer rates andin-hospital standardised mortality ratios. However, the fact thatfour hospitals did not record any transfers to other medicalinstitutions at all suggests that the quality of registration of thisvariable in our database is questionable, at least for somehospitals. Therefore, the lack of a statistically significant effectcould also be due to a poor quality of registration of thisvariable. A recent study by Drye et al (2012) concluded that forpatients admitted with diagnoses of acute myocardial infarction,heart failure, and pneumonia, in-hospital mortality rates favourhospitals with shorter length of stay and higher transfer ratescompared with 30 days post-admission mortality rates.24 Theyreported that a higher length of stay was associated with higherin-hospital mortality. Our results are in line with theseobservations. Rosenthal et al (2000) examined the relationbetween in-hospital mortality and hospital discharge practicesby using data on 13 834 patients with congestive heart failurein the United States.25 They found that the classification ofhospitals as statistically significant outliers on the basis of theirin-hospital standardisedmortality ratios was noticeably differentfrom the classification based on 30 days post-admission ratios.This observation may well suggest the presence of dischargebias. In addition to the previously mentioned studies, our studyis, to our knowledge, the first to include a broad hospitalpopulation taking into account 50 diagnoses with a higher apriori mortality risk. Therefore, our results may be moreapplicable when studying hospital-wide performance—forexample, when using hospital standardised mortality ratios toassess quality of care.

Limitations of studyThe pseudonymised, administrative data used for this studyhave some limitations that need to be considered. Firstly,approximately 10% of the pseudonymised admissions couldnot be linked to the population register and had to be excluded.Unfortunately, we had no other means to retrieve thepost-discharge mortality of these non-linkable admissions, sowe do not knowwhether these discharges might have influencedthemagnitude and direction of the difference between in-hospitalstandardised mortality ratio and post-discharge mortality.However, Statistics Netherlands considers this number oflinkable admissions sufficient to do statistical analysis.19Secondly, we could not determine whether the 50 diagnosticgroups analysed, accounting for 80% of in-hospital mortality,also accounted for a high percentage of post-dischargemortality.This is because admissions that did not belong to the 50 analyseddiagnostic groups were not linked to the population register, sowe could not calculate post-discharge mortality for theseadmissions. Thirdly, because participation of hospitals in theHospital Discharge Register database was on a voluntary basis,not all hospitals participated, potentially reducing the variationin hospitals’ performance (especially if poorly performinghospitals selectively decline to participate). However, theincluded hospitals probably give a fair representation of allDutch general hospitals, because the crude mortality rates ofthe excluded hospitals were similar to those of included

hospitals. Finally, we had no means to determine whether apatient had been admitted to, or was transferred from, anon-participating hospital. Consequently, the death of a patienttransferred from or to a hospital that did not contribute to theHospital Discharge Register database was assigned only to theadmitting or referring hospital that participated in the register.

Implications of findingsIncreasingly, pay for performance programmes and selectivepurchasing are based on outcomes rather than adherence toprocess variables. If hospital mortality—either at the aggregatelevel (hospital standardised mortality ratio) or by specialty,diagnosis, or procedure—is used as a performance measure,guarding against bias and reducing the potential for “gaming”is essential. We found that the between hospital variability ofin-hospital standardised mortality ratios could be partlyexplained by differences in post-discharge mortality and lengthof stay. This skews interpretation of quality of care againsthospitals with longer lengths of stay and lower post-dischargemortality rates. Therefore, we recommend includingpost-discharge deaths in the mortality analyses by usingtimeframes that incorporate the early post-discharge period. Ofcourse, mortality after discharge may also be affected by factorsbeyond the hospital’s control, such as quality of outpatient careor quality of other referring and admitting hospitals. However,this could be beneficial from a societal perspective, as hospitalswill have a stake in organising adequate handover andpost-discharge care. In addition, collection of post-dischargedata is currently not routine and acquiring these data may becostly. Nevertheless, our study suggests that 30 dayspost-admission or 30 days post-discharge standardisedmortalityratios are less vulnerable to discharge bias than are in-hospitalstandardised mortality ratios and may therefore be preferableif standardised mortality ratios are to be used for assessment ofhospitals’ performance.

30 days post-admission mortality versus 30days post-discharge mortalityWhether to use a 30 days post-admission timeframe or a 30days post-discharge timeframe is still a matter for debate. Themajor advantage of a 30 days post-admission timeframe is thefixed window of time in which care is measured. The timeframeis equal for all hospitals, whatever their discharge policy or theiropportunities to reduce the length of stay, such as the nearvicinity of palliative care centres or other more specialisedhospitals. However, patients dying in the hospital after 30 daysof admission will be mistakenly regarded as survivors,introducing a potential gaming element for hospitals (an extremeexample would be the incentive to keep patients with a poorprognosis alive until at least 30 days after admission). With the30 days post-discharge timeframe, these patients are alsoanalysed but the timeframe of measurement is no longer fixed.A more elegant method would be to determine the besttimeframe for each diagnosis or procedure. For example, a 30days post-admission timeframe is commonly used for surgicalprocedures, but for some diagnoses (such as pneumonia) a longertimeframemay be preferable. Also, a combination of timeframesis sometimes used—for example, the 30 days post-admissiontimeframe for patients discharged within 30 days combinedwith the in-hospital timeframe for patients admitted for longerthan 30 days. This has been successfully applied in theEuroSCORE, a tool to predict operative mortality for patientsundergoing cardiac surgery.26 Further research is needed todetermine the optimal window of time for every specificdiagnosis. On the basis of our findings and the literature, the 30

No commercial reuse: See rights and reprints http://www.bmj.com/permissions Subscribe: http://www.bmj.com/subscribe

BMJ 2013;347:f5913 doi: 10.1136/bmj.f5913 (Published 21 October 2013) Page 4 of 12

RESEARCH

days post-admission timeframe in combination with thein-hospital timeframe for patients admitted for longer than 30days may best balance the risk of discharge bias and maintainthe advantage of a fixed timeframe.

ConclusionsSelecting mortality timeframes that include the post-dischargeperiod changes the standardised mortality ratios of individualhospitals and affects judgments about performance. Furthermore,short length of stay was associated with low in-hospital mortalitybut higher post-discharge mortality. These findings suggest thatincorporating early post-discharge mortality in the standardisedmortality ratio will reduce the effect of discharge bias.

Contributors MEP conceived and designed the statistical analysis plan,analysed the data, and drafted and revised the paper. LMP and KGMManalysed the data and revised the paper. CJK and HFL analysed thedata and drafted and revised the paper. All authors contributed to thefinal manuscript. MEP is the guarantor.Funding: This study was part of a study commissioned by the DutchMinistry of Health, Welfare and Sport. The ministry had no role in studydesign; in the collection, analysis, and interpretation of data; in thewriting of the report; or in the decision to submit the article for publication.Competing interests: All authors have completed the ICMJE uniformdisclosure form at www.icmje.org/coi_disclosure.pdf and declare: nosupport from any organisation for the submitted work; no financialrelationships with any organisations that might have an interest in thesubmitted work in the previous three years; no other relationships oractivities that could appear to have influenced the submitted work.Ethical approval: Not needed.Data sharing: Technical appendix and statistical code are available(after permission from Statistics Netherlands and the DHD) from thecorresponding author at [email protected]. The dataset is availableon request via the DHD (www.dutchhospitaldata.nl).

1 Jarman B, Galt S, Alves B, Hider A, Dolan S, Cook A, et al. Explaining differences inEnglish hospital death rates using routinely collected data. BMJ 1999;318:1515-20.

2 Jarman B, Pieter D, van der Veen AA, Kool RB, Aylin P, Bottle A, et al. The hospitalstandardised mortality ratio: a powerful tool for Dutch hospitals to assess their quality ofcare? Qual Saf Health Care 2010;19:9-13.

3 Ben-Tovim D, Woodman R, Harrison JE, Pointer S, Hakendorf P, Henley G. Measuringand reporting mortality in hospital patients. Australian Institute of Health and Welfare,2009. (AIHW Cat. No. HSE 69.)

4 Canadian Institute for Health Information. HSMR technical notes, updated September2012. www.cihi.ca/CIHI-ext-portal/pdf/internet/HSMR_TECH_NOTES_201202_en.pdf.

5 Pitches DW, Mohammed AM, Lilford RJ. What is the empirical evidence that hospitalswith higher-risk adjusted mortality rates provide poorer quality care? A systematic reviewof the literature. BMC Health Serv Res 2007;7:91.

6 Shahian DM, Wolf RE, Iezzoni LI, Kirle, L, Normand SLT. Variability in the measurementof hospital-wide mortality rates. N Engl J Med 2010;363:2530-9.

7 Tu Y-K, Gilthorpe MS. The most dangerous hospital or the most dangerous equation?BMC Health Serv Res 2007;7:185-9.

8 Powell H, Li LL, Heller RF. Accuracy of administrative data to assess comorbidity inpatients with heart disease: an Australian perspective. J Clin Epidemiol 2001;54:687-93.

9 Van Gestel YRBM, Lemmens EPP, Lingsma HF, de Hingh IHJT, Rutten HJT, CoeberghJW. The hospital standardised mortality ratio fallacy: a narrative review. Med Care2012;50:662-7

10 Van den BoschWF, Kelder JC, Wagner C. Predicting hospital mortality among frequentlyreadmitted patients: HSMR biased by readmission. BMC Health Serv Res 2011;11:57.

11 Mohammed MA, Deeks JJ, Girling A, Rudge G, Carmalt M, Stevens JA, et al. Evidenceof methodological bias in hospital standardised mortality ratios: retrospective databasestudy of English hospitals. BMJ 2009;338:b780.

12 Pouw ME, Peelen LM, Lingsma HF, Pieter D, Steyerberg E, Kalkman CJ, et al. Hospitalstandardized mortality ratio: consequences of adjusting hospital mortality with indirectstandardization. PLOS One 2013;8:e59160.

13 Borghans I, Heijink R, Kool T, Lagoe RJ, Westert GP: Benchmarking and reducing lengthof stay in Dutch hospitals. BMC Health Serv Res 2008;8:220.

14 Slobbe LCJ, Onyebuchi AA, de Bruin A, Westert GP. Mortality in Dutch hospitals: trendsin time, place and cause of death after admission for myocardial infarction and stroke: anobservational study. BMC Health Serv Res 2008;4:8:52.

15 Yu P, Chang DC, Osen HB, Talamani MA. NSQIP reveals significant incidence of deathfollowing discharge. J Surg Res 2011;170:e217-24.

16 Metersky ML, Waterer G, Nsa W, Bratzler DW. Predictors of in-hospital vs postdischargemortality in pneumonia. Chest 2012;142:476-81.

17 Campbell MJ, Jacques RM, Fotheringham J, Maheswaran R, Nicholl J. Developing asummary hospital mortality index: retrospective analysis in English hospitals over fiveyears. BMJ 2012;344:e1001.

18 Bratzler DW, Normand SL, Wang Y, O’Donnell WJ, Metersky M, Han LF, et al. Anadministrative claims model for profiling hospital 30-day mortality rates for pneumoniapatients. PLOS One 2011;6:e17401.

19 De Bruin A, Kardaun J, Gast F, de Bruin E, van Sijl M, Verweij G. Record linkage ofhospital discharge register with population register: experiences at Statistics Netherlands.Statistical Journal of the United Nations 2004;ECE 21:23-32.

20 Israels A, van der Laan J, de Bruin A, Ploemacher J, Verweij G. HSMR 2010:methodological report. www.cbs.nl/en-GB/menu/themas/gezondheid-welzijn/publicaties/artikelen/archief/2011/2011-hsmr2010-report.htm.

21 Elixhauser A, Andrews RM, Fox S. Clinical classifications for health policy research:discharge statistics by principal diagnosis and procedure. Provider Studies ResearchNote 17. Agency for Health Care Policy and Research, 1993. (AHCPR Publication No93-0043.)

22 Lawless JF, Singhal K. Efficient screening of nonnormal regression models. Biometrics1978;34:318-27.

23 Vasilevskis EE, Kuzniewicz MW, Dean ML, Clay T, Vittinghoff E, Rennie DJ, et al.Relationship between discharge practices and intensive care unit in-hospital mortalityperformance: evidence of a discharge bias. Med Care 2009;47:803-12.

24 Drye EE, Normand SL, Wang Y, Ross JS, Schreiner GC, Han L, et al. Comparison ofhospital risk-standardized mortality rates calculated by using in-hospital and 30-daymodels: an observational study with implications for hospital profiling. Ann Intern Med2012;156:19-26.

25 Rosenthal GE, Baker DW, Norris DG,Way LE, Harper DL, SnowRJ. Relationship betweenin-hospital and 30-day standardized hospital mortality: implications for profiling hospitals.Health Serv Res 2000;34:7.

26 Roques F, Nashef SAM, Michel P, Gauducheau E, de Vincentiis C, Baudet E, et al. Riskfactors and outcome in European cardiac surgery: analysis of the Euro-SCOREmultinational database of 19030 patients. Eur J Cardiothorac Surg 1999;15:816-23.

Accepted: 02 September 2013

Cite this as: BMJ 2013;347:f5913This is an Open Access article distributed in accordance with the Creative CommonsAttribution Non Commercial (CC BY-NC 3.0) license, which permits others to distribute,remix, adapt, build upon this work non-commercially, and license their derivative workson different terms, provided the original work is properly cited and the use isnon-commercial. See: http://creativecommons.org/licenses/by-nc/3.0/.

No commercial reuse: See rights and reprints http://www.bmj.com/permissions Subscribe: http://www.bmj.com/subscribe

BMJ 2013;347:f5913 doi: 10.1136/bmj.f5913 (Published 21 October 2013) Page 5 of 12

RESEARCH

What is already known on this topic

The in-hospital standardised mortality ratio is used globally in an attempt to measure the quality of hospital careHospitals with a low in-hospital standardised mortality ratio are regarded as having a high degree of quality of careThese hospitals should also have low early post-discharge mortality, as hospitals that perform well should have fewer patients dyingshortly after discharge

What this study adds

Including mortality after discharge in the calculation of standardised mortality ratios not only changed the outcomes but also alteredjudgments regarding the performance of individual hospitalsIn-hospital standardised mortality ratio and early post-discharge mortality were inversely associated, suggesting that low in-hospitalmortality may reflect high post-discharge mortality instead of the assumed high quality of careTherefore, early post-discharge mortality should be included in the calculation of standardised mortality ratios

Tables

Table 1| Baseline characteristics. Values are numbers (percentages) unless stated otherwise

Excluded admissions (n=237 646)Included admissions (n=2 149 958)Characteristics

58.964.1Average age (years)

114 841 (48.3)1 013 519 (47.1)Male sex

142 895 (60.1)1 260 927 (58.7)Urgent admission

8.07.5Average length of stay (days)

8987 (3.8)104 337 (4.9)In-hospital death

Admissions between 2007 and 2010 were not included if no unique link was possible between Hospital Discharge Register database and population register.

No commercial reuse: See rights and reprints http://www.bmj.com/permissions Subscribe: http://www.bmj.com/subscribe

BMJ 2013;347:f5913 doi: 10.1136/bmj.f5913 (Published 21 October 2013) Page 6 of 12

RESEARCH

Table 2| Overview of crude mortality rates, transfer rates, and average length of hospital stay

Mean (SD)Measure

4.9 (0.7)In-hospital mortality rate (%)

7.2 (0.8)Hospital mortality rate until 30 days after admission (%)

8.4 (0.9)Hospital mortality rate until 30 days after discharge (%)

7.2 (0.7)Length of hospital stay (days)

97.8 (0.7)Admissions with length of hospital stay <30 days (%)

4.7 (0.7)In-hospital mortality rate at 30 days after admission (%)

2.7 (0.4)Early post-discharge mortality rate (tdischarge−tadmission+30days) (%)

11.6 (2.4)In-hospital mortality rate for admissions >30 days (%)

9.4 (3.8)Transfer rate (%)

No commercial reuse: See rights and reprints http://www.bmj.com/permissions Subscribe: http://www.bmj.com/subscribe

BMJ 2013;347:f5913 doi: 10.1136/bmj.f5913 (Published 21 October 2013) Page 7 of 12

RESEARCH

Table 3| Classification according to 30 days post-admission standardised mortality ratio (SMR) compared with in-hospital SMR

Classification according to 30 days post-admission SMRClassification according to in-hospitalSMR Better than expectedConforms to expectedWorse than expected

036Worse than expected

6235Conforms to expected

1160Better than expected

On the basis of the SMR and its 95% confidence interval, a hospital can be classified in three categories: better than expected, conforms to expected, and worsethan expected. If the SMR is significantly above 100 or significantly below 100, the hospital is considered to have performed respectively worse or better thanexpected. If the SMR does not significantly differ from 100, the hospital’s performance is considered to have conformed to expected. Twenty out of 60 hospitalswere classified differently with the 30 days post-admission timeframe in comparison with in-hospital mortality.

No commercial reuse: See rights and reprints http://www.bmj.com/permissions Subscribe: http://www.bmj.com/subscribe

BMJ 2013;347:f5913 doi: 10.1136/bmj.f5913 (Published 21 October 2013) Page 8 of 12

RESEARCH

Table 4| Classification according to 30 days post-discharge standardised mortality ratio (SMR) compared with in-hospital SMR

Classification according to 30 days post-discharge SMRClassification according to in-hospitalSMR Better than expectedConforms to expectedWorse than expected

036Worse than expected

3283Conforms to expected

1340Better than expected

On the basis of the SMR and its 95% confidence interval, a hospital can be classified in three categories: better than expected, conforms to expected, and worsethan expected. If the SMR is significantly above 100 or significantly below 100, the hospital is considered to have performed respectively worse or better thanexpected. If the SMR does not significantly differ from 100, the hospital’s performance is considered to have conformed to expected. Thirteen out of 60 hospitalswere classified differently with the 30 days post-discharge timeframe in comparison with in-hospital mortality.

No commercial reuse: See rights and reprints http://www.bmj.com/permissions Subscribe: http://www.bmj.com/subscribe

BMJ 2013;347:f5913 doi: 10.1136/bmj.f5913 (Published 21 October 2013) Page 9 of 12

RESEARCH

Table 5| Relations between in-hospital standardised mortality ratio (SMR) and early post-discharge mortality rate, transfer rate, and lengthof stay, and between length of stay and early post-discharge mortality

P valuePearson correlation coefficientRelation

0.004−0.37In-hospital SMR and early post-discharge mortality

0.66−0.06In-hospital SMR and transfer rate

0.010.33In-hospital SMR and length of stay

0.02−0.30Length of stay and early post-discharge mortality

No commercial reuse: See rights and reprints http://www.bmj.com/permissions Subscribe: http://www.bmj.com/subscribe

BMJ 2013;347:f5913 doi: 10.1136/bmj.f5913 (Published 21 October 2013) Page 10 of 12

RESEARCH

Figures

Fig 1 Distributions of hospitals according to in-hospital standardised mortality ratio (SMR), 30 days post-admission SMR,and 30 days post-discharge SMR

No commercial reuse: See rights and reprints http://www.bmj.com/permissions Subscribe: http://www.bmj.com/subscribe

BMJ 2013;347:f5913 doi: 10.1136/bmj.f5913 (Published 21 October 2013) Page 11 of 12

RESEARCH

Fig 2 Scatterplots showing that for some individual hospitals, standardised mortality ratio (SMR) changes if 30 dayspost-admission or 30 days post-discharge ratios are used. The diagonal indicates the points at which in-hospital SMRequals 30 days post-admission SMR or 30 days post-discharge SMR

No commercial reuse: See rights and reprints http://www.bmj.com/permissions Subscribe: http://www.bmj.com/subscribe

BMJ 2013;347:f5913 doi: 10.1136/bmj.f5913 (Published 21 October 2013) Page 12 of 12

RESEARCH