Estimating the returns to UK publicly funded cancer ...

Estimating the returns to UK publicly fundedcancer-related research in terms of the net valueof improved health outcomesGlover et al

Glover et al BMC Medicine 2014 1299httpwwwbiomedcentralcom1741-70151299

RESEARCH ARTICLE Open Access

Estimating the returns to UK publicly fundedcancer-related research in terms of the net valueof improved health outcomesMatthew Glover1 Martin Buxton1 Susan Guthrie2 Stephen Hanney1 Alexandra Pollitt2 and Jonathan Grant23

Abstract

Background Building on an approach developed to assess the economic returns to cardiovascular research weestimated the economic returns from UK public and charitable funded cancer-related research that arise from thenet value of the improved health outcomes

Methods To assess these economic returns from cancer-related research in the UK we estimated 1) public andcharitable expenditure on cancer-related research in the UK from 1970 to 2009 2) net monetary benefit (NMB) thatis the health benefit measured in quality adjusted life years (QALYs) valued in monetary terms (using a base-casevalue of a QALY of GBpound25000) minus the cost of delivering that benefit for a prioritised list of interventions from1991 to 2010 3) the proportion of NMB attributable to UK research 4) the elapsed time between research fundingand health gain and 5) the internal rate of return (IRR) from cancer-related research investments on health benefitsWe analysed the uncertainties in the IRR estimate using sensitivity analyses to illustrate the effect of some keyparameters

Results In 201112 prices total expenditure on cancer-related research from 1970 to 2009 was pound15 billion TheNMB of the 59 million QALYs gained from the prioritised interventions from 1991 to 2010 was pound124 billionCalculation of the IRR incorporated an estimated elapsed time of 15 years We related 17 of the annual NMBestimated to be attributable to UK research (for each of the 20 years 1991 to 2010) to 20 years of researchinvestment 15 years earlier (that is for 1976 to 1995) This produced a best-estimate IRR of 10 compared with 9previously estimated for cardiovascular disease research The sensitivity analysis demonstrated the importance ofsmoking reduction as a major source of improved cancer-related health outcomes

Conclusions We have demonstrated a substantive IRR from net health gain to public and charitable funding ofcancer-related research in the UK and further validated the approach that we originally used in assessing thereturns from cardiovascular research In doing so we have highlighted a number of weaknesses and keyassumptions that need strengthening in further investigations Nevertheless these cautious estimatesdemonstrate that the returns from past cancer research have been substantial and justify the investmentsmade during the period 1976 to 1995

Keywords Medical research investment QALYs Cancer Medical research charities Value of health Rate of returnTime lags Research payback

Correspondence jonathangrantkclacuk2RAND Europe Westbrook Centre Milton Road Cambridge CB4 1YG UK3Kingrsquos Policy Institute Kingrsquos College London Virginia Woolf Building 22Kingsway London WC2R 2LA UKFull list of author information is available at the end of the article

copy Glover et al licensee BioMed Central Ltd This is an Open Access article distributed under the terms of the CreativeCommons Attribution License (httpcreativecommonsorglicensesby40) which permits unrestricted use distribution andreproduction in any medium provided the original work is properly cited

Glover et al BMC Medicine Page 2 of 202014 1299httpwwwbiomedcentralcom1741-70151299

BackgroundEstimating the returns from biomedical and health researchEstimating the economic returns arising from healthresearch develops our understanding of how researchtranslates from lsquobench to bedsidersquo can be used in advocat-ing the case for future investments in medical researchand demonstrates accountability for public and charitableresearch funding to taxpayers and donors Because re-sources used for publicly and charitably funded medicalresearch including cancer research could potentially beput to other purposes for the benefit of society there is anobligation to demonstrate that such investments representgood value In the medical field it is possible to identifyillustrative examples of specific research breakthroughsthat have contributed to substantial benefit in terms oflife-saving interventions or to major improvements in thequality of life of patients with a chronic disease Howeverit is much more difficult to describe systematically the na-ture and extent of the returns to the investment of a wholebody of medical research some of which may inevitablybe less fruitful Furthermore there are tensions betweenadvocacy where interested parties are arguing for moreresearch funding and more dispassionate analysis whichmight conclude that too much money is being spent onresearch As noted in an editorial in Nature in 2010 lsquoMostof the attempts to count the economic benefits of invest-ment in science have been derived from the efforts oflobbying groups and funding agencies to justify sciencespendingrsquo [1]The literature that assesses the value of the benefits of

medical research forms a relatively small field in termsof methodology and quality [23] There is a lack of clearconsensus about key issues such as the best methods touse to assess the value of the health gains and there isalso variability in the extent to which studies have in-cluded all the important components required for a fullanalysis of the cost-effectiveness of investing in researchAs summarised in Table 1 Mushkin [4] in an early studyused a human capital approach to value health gains fromall US biomedical research in terms of the productivitygains from having a healthy workforce [4] This approachhas various weaknesses which were recognised by Mushkinand others [5] including that it tends to overstate thebenefits when lost labour can be replaced by unemployedpeople or through migration and it undervalues healthgains for groups such as the elderly Funding First [6] ad-vanced the field by building on a different approach basedon estimates of the average willingness of individuals topay for small reductions in the risk of death They usedthis figure to value the increased longevity of the USpopulation In a background paper for this Murphy andTopel [7] calculated the enormous economic value thatwould come from finding a cure for cancer and otherdiseases but to date and using the methods they had

adopted the Funding First report claimed lsquothe largestreturns to investment in medical research have comeprincipally from gains against heart disease and strokersquo([6] page 3)A broadly similar approach was adopted in a series of

Australian studies conducted by Access Economics (20032008 and 2011) [8-10] but expanded to allow for im-provements in quality of life based on disability adjustedlife years (DALYs) In the 2003 version of the report no al-lowance was made for the elapsed time between research(input) and improved health and wellbeing (outcome) Inthe 2008 and 2011 iterations this was addressed by pro-jecting potential health and wellbeing gains 40 years intothe future In the 2011 report the authors focused on esti-mating a return on investment for five specific diseasesincluding cancerTo date only three studies that we are aware of have

examined the economic returns from cancer researchTwo of those focused on the costs and benefits of USPresident Nixonrsquos lsquoWar on Cancerrsquo [11-13] Litchenberg[11] in 2004 examined the contribution of pharmaceuticalinnovation to increases in cancer survival rates by lookingat the number of new drugs that had been approved totreat cancer after 1971 (when the War on Cancer wasdeclared) and modelling the impact on cancer mortalityrates in the US He estimated that the increase in approveddrugs accounted for about 50 to 60 of the increase inage-adjusted cancer survival rates Although Litchenberg[11] did not compute a rate of return he did note that thedrug costs to achieve an additional year of life per persondiagnosed with cancer were well below estimates for thevalue of a statistical life Pertinent to the approach adoptedin the current study he concluded lsquoIdeally we would havemeasured the effects of new cancer drugs on the numberof quality adjusted life years (QALYS) but were unable todo so due to lack of datarsquo In two related papers Sun et al[12] and Lakdawalla et al [13] followed a similar concep-tual approach in quantifying the value of gains in cancersurvival but directly compared this with the costs ofresearch and development (RampD) They estimated thatimprovements in cancer survival in the US between 1988and 2000 created 23 million additional life years equiva-lent to roughly US$19 trillion of additional social valueimplying that the average life year gained was worth US$82000 As with Litchenberg [11] Sun et al did notcalculate a return on investment but noted that lsquoThesecalculations suggest that from the patientrsquos point of viewthe rate of return to RampD investments against cancer hasbeen substantialrsquo The third study to look explicitly atcancer is the Deloitte Access Economics study [10] citedabove In that report the authors looked at the rate ofreturn from current (2000 to 2010) research investment incancer by the Australian National Health and MedicalResearch Council (NHMRC) and compared this with

Table 1 Methods used in various studies to assess the benefits from health research

Studyfeatures Mushkin(1979) [4]

Funding first(2000) [6]

Access economics(2003) [8]

HERG et al(2008) [3]

How health gainswere assessed

Top-down by disease categoryoverall gain in each category notlinked to specific interventionAttributed 20 to 30 of totalgain to RampD Reduced morbiditydifficult to assess because littlereduction in days off workbecause of sickness Adjustedthe raw data for example byapplying historical Army andNavy data as an index to recordthe decline in sickness

Top-down overall gain inmortality not linked tospecific interventionsAttributed roughly one-thirdof the total gain to RampD pluslsquosome fraction of the creditfor the other two-thirdsrsquo

Top-down overall gainin mortality and morbiditynot linked to specificinterventions Attributed50 of the total gain toRampD

Top-down as in the2003 study overallgain in mortality andmorbidity not linkedto specific interventionsAttributed 50 of thetotal gain to RampD

Bottom-up identifiedresearch-basedinterventions thenquantified health impact

Top-down overall gainin mortality and morbidityfor five disease areas notlinked to specificinterventions Attributed50 of the total gain toRampD

How health gainswere valued

Human capital approach that isvalues attached to lives savedbetween one period and thenext based on potential futureearnings plus calculation ofvalue of potential working timeno longer lost due to sickness

Used a comparatively highlsquowillingness-to-payrsquo valuederived from laboureconomics

Used the samecomparatively highlsquowillingness-to-payrsquo valueas Funding First

Used a higherlsquowillingness-to-payrsquoestimate than the2003 study this timederived from ameta-analysis ofinternational studies

Used a comparativelylow but arguably realisticvalue of health gain byadopting the figureimplied by the currentlevel of NHS spendingthat is the opportunitycost of a QALY withinthe current NHS budget

Used a lowerlsquowillingness-to-payrsquoestimate than that usedin the 2008 study in linewith Department of Financeand Deregulation guidance

Proportion of nationalhealth gain allocatedto national research

Not discussed as a major issuewe assumed it to be 100

Not discussed as a majorissue in Funding First weassumed it to be 100

Used proportion of globalresearch conducted inAustralia (25) todetermine the proportionof the total research-basedhealth gain to attribute toAustralian research

Uses bibliometricanalysis-based estimateof Australian share ofglobal researchoutput in clinicalmedicine (304)

An analysis of citationsof UK research on UKclinical guidelinessuggests average bestestimate of 17 linkedto UK research

Uses an updatedbibliometric analysis-basedestimate of Australian shareof global research output inclinical medicine (314)

Costs of healthcare considered

No at least not as a separateitem to net-off against thevalue of the health gains

No in initial headline figuresbut Yes in later analysis lsquothegain in the value of life netwhat was spent to attainthe longer life is just 15percent smallerrsquo

No did not net-off thehealthcare costs requiredto achieve the healthgains

No did not net-offthe healthcare costsrequired to achievethe health gains

Yes did net-off thehealth care costs requiredto achieve the health gains

Did not net-off health caredelivery costs but didconsider avoided healthsystem expenditure due togains in wellbeing

Considered elapsedtime betweenresearch and healthgains

Yes 10 years Acknowledged time lagsbetween research andbenefits but this wasapparently not broughtinto calculations

No compared researchexpenditure and healthbenefits in the same yearThis implies the health gainsfrom research are instant

Yes 40 years with rangeof 20 to 60 years usedfor sensitivity analyses

Yes an analysis ofcitations of UK researchon UK clinical guidelinessuggested average bestestimate of 17 years lag

Yes same assumption of40 years as was used in2008 study No sensitivityanalysis around elapsedtime

How the overall rateof return calculated

IRR of 47 Not brought together toprovide an overall IRR

An overall benefitcostsratio for health researchof 240

IRR of 9 for CVDresearch combined with30 for GDP benefits

Benefit-cost ratios for fivedisease areas 61 (CVD) 27(cancer) 11 (SIDS) 12(asthma) and 07(muscular dystrophy)

Abbreviations CVD cardiovascular disease GDP Gross Domestic Product IRR internal rate of return NHS National Health Service QALY quality adjusted life years RampD research and development SIDS Sudden InfantDeath Syndrome

Glover

etalBM

CMedicine

202014 1299

httpww

edcentralcom1741-70151299

gains in wellbeing using DALYS projected for 2040 to2050 In doing so they estimated the net benefit ofNHMRC RampD between 2000 and 2010 to be AU$196billion with a costbenefit ratio of 27 that is for everyAU$1 million invested in cancer research they wouldanticipate a return worth $17 millionA recurring theme in these studies is the extent to

which health gains can be attributed to research-inspiredmedical advances Funding First and Access Economicsadopted a lsquotop-downrsquo (or macro) approach that took ameasure of the overall national health gain from variousfields of medicine and then assumed that a proportionwas attributable to medical research One way of ad-dressing this problem of attribution is by examining in abottom-up manner the impacts of specific projects orprogrammes of research by tracing forwards from theresearch to the benefits that arise Here considerableprogress has been made using the Payback Framework[14-18] but this has relied on the development of specificresource-intensive case studies Other studies have madeprogress in analysing the value of the health gains associ-ated with a series of clinical trials [19] but the major chal-lenge faced by these types of studies is attribution that ishow to show that the health gains that have arisen can beattributed to specific pieces of researchIn 2008 we published a report funded by the Wellcome

Trust and UK Medical Research Council which aimed tobuild on the advances in previous studies and address theexisting limitations so as to develop an approach thatcould be used to measure the economic benefits accruingfrom publicly and charitably funded medical research [3]We analysed two major elements of economic returns thebroad impact on the UK Gross Domestic Product (GDP)and the specific net monetary benefits (NMB) defined asthe health benefit valued in monetary terms minus thecost of delivering that health benefit which arose from theUK application of relevant UK research Our analysis ofthe existing evidence on the GDP or lsquospilloverrsquo benefitsbased largely on US studies from a number of areas ofresearch and certainly not specific to any particular areaof medical research suggested a best estimate of an in-ternal rate of return (IRR) of around 30 We estimatedthe NMB of the health gain using methods similar tothose used here giving an IRR of 9 for cardiovascularresearch This meant that a GBpound100 investment in publiclycharitably funded CVD research produced a combinedstream of benefits thereafter equivalent in value to earn-ing pound039 per year in perpetuity (We also estimated theNMB from mental health research which produced anIRR of 7 however this was based on a more limited ana-lysis because of data limitation and uncertainties aroundthe effects of interventions in mental health which meantthat we were less confident in the results than we were forthe CVD results)

These estimates of the IRR have been widely used inpolicy circles in the UK and beyond [20-23] and in theabsence of any other estimates of the economic impactof biomedical research the figures have often been usedas proxies of the economic impact of medical researchmore broadly A consortium of funders (Wellcome TrustNational Institute of Health Research Cancer ResearchUK (CRUK) and the Academy of Medical Sciences)commissioned a study to further validate the approachand to explore whether the IRR from the net value ofthe health benefits in another area cancer was similaror not Thus this study aimed to estimate the economicreturns from UK publicly and charitably funded cancerresearch on improved health outcomes in the UK specific-ally As with the previous CVD study we accept that thereare international benefits of UK research but this was notin the scope of the current exercise although as we notethis is an area that warrants further investigation Inaddition and as reported separately [24] we undertookfive exploratory case studies to understand qualitativelythe complexity of how research translates into healthbenefitWe present the methods used for the four main steps

that provided the estimated parameters to enable us tocalculate the economic returns from the NMB of the UKhealth gains that we attributed to past UK publicly andcharitably funded cancer-related research and presentthe results expressed as estimates of the IRR with sensi-tivity analyses to illustrate the effects of some of the keyuncertainties Finally we explored the significance of ourfindings in the context of previous studies and the widerpolicy debate on RampD investments and economic impactwe detailed the limitations of our approach and we devel-oped a research agenda for this fledgling field

MethodsOverall conceptual approachFour key sources of data were needed to estimate theIRR of the NMB of the health gains arising from cancerresearch

a time series of the public and charitable funding ofcancer-related research

a time series of the NMB of cancer health gainsderived from the monetised health benefits andthe healthcare costs for selected interventionsa

an estimate of the elapsed time between theinvestment (research funding) and return(health gain) associated with thoseinterventions and

an estimate of the amount of health gain thatshould be attributed to public and charitableresearch investment in cancer-related researchin the UK

With these four data inputs we then calculated a rateof return on the investment in cancer researchIt should be noted that the costs of private sector

RampD investments are accounted for in our analysis aselements within the cost of delivering health care whichare netted off in the NMB The costs to the healthservice of medical interventions produced by the privatesector include the return to the private sector on itsRampD investments

Estimating public and charitable funding of cancer-relatedresearchThe leading funders of cancer research in the UK wereidentified by examining the National Cancer ResearchInstitute (NCRI) Cancer Research Database Between2002 and 2011 the top 10 funders consistently accountedfor over 95 of cancer research spend by the 21 NCRIpartnersb Estimates of annual cancer-related researchfunding between 1970 and 2009 were assembled for these10 organisations plus an estimated contribution to coverFunding Council support for cancer research (the HigherEducation Funding Council for England and similar bod-ies in Wales Scotland and Northern Ireland provide aperformance-related block grant to UK universities basedon the quality and volume of research) A detailed accountof how we estimated these 11 time series is provided (seeAdditional file 1)As also discussed in detail in Additional file 1 in esti-

mating research spend for the Funding Councils and theDepartment of Health (DH)NHS we had to derive afigure specifically for cancer-related research activity inthe UK We settled on a central estimate of 10 of totalpublicly and charitably funded health and biomedicalresearch activity and we also assumed it to be constantover the time period This estimate was derived from anumber of independent sources as follows

Medical Research Council (MRC) spending oncancer research averaged 98 of their totalinvestment (range 46 to 167) between 19701and 200910

Wellcome Trust cancer funding was more erraticranging between 1c and 38 with an average of145 of expenditure being on cancer research

The proportion of peer-reviewed research papers inoncology as a percentage of all UK biomedicaloutputs averaged 92 (range 85 to 95) between1988 and 1995 [25]

The proportion of peer-reviewed research papers inoncology research (as a percentage of all NHSresearch outputs) was 12 between 1990 and1997 [26]

The proportion of mainstream quality-related (QR)funding allocations by the Higher Education Funding

Council for England for lsquoCancer studiesrsquo (that isUnit of Assessment 02) between 2009 and 2012 wasaround 6 of the total biomedical allocation (that isUnit of Assessments 01 to 15 and 44)d

Given the importance of this estimate of 10 for theproportion of research activity that is related to cancer(for those sources where we had no actual breakdown)we also looked at the effect of lower and higher estimatesof 75 and 15 respectively in the sensitivity analyses

Estimating the NMB from cancer-related researchThis element of the research required estimates of thelifetime QALYs gained and the net lifetime costs to theNHS of delivering those QALYs for research-based inter-ventions provided in each year of the period 1991 to2010 The general methods mirrored those used in the2008 study [3] on the returns on investment in CVDresearch and again built up the aggregate net benefitsfrom the bottom up aggregating the QALYs gained andthe net NHS costs from the use of specific interventionsThis approach required 1) identification of the key rele-vant cancer interventions and their level of usage duringthe relevant period and 2) estimates of the QALY gainsand NHS costs associated with the interventions Fromthis information the NMB was calculated as the healthbenefit valued in monetary terms (determined by thequantity of health benefit and a decision-makerrsquos will-ingness to pay for that additional benefit) minus the costof delivering that health benefitIn the CVD study our starting point was previously

published research identifying the cardiovascular inter-ventions that had contributed most health gain [27] Noequivalent studies for cancer were identified that couldprovide a comparable basis for deciding which interven-tions were quantitatively the most important to includein the analysis Thus the three main steps for quantify-ing the total NMB associated with cancer interventionswere 1) to identify the cancer interventions that were thelikely major sources of benefits 2) to identify appropriateestimates of NMB per patient for that subset of cancerinterventions and 3) to construct a time series (for 1991to 2010) of the number of patients receiving each of thesesubsets of cancer intervention in the UK

Identifying the key cancer interventionsAt the outset of the study we had a number of discussionswith cancer research experts to provide us with a broadunderstanding of the main developments in the field overthe past 20 years Informed by these discussions wequantitatively identified those areas that had resulted inthe largest health gain in the UK since 1990 arisingfrom three main sources 1) key cancers where researchand resultant health policies have led to health gains

through a reduction in incidence 2) key cancers forwhich screening programmes have led to health gainsbecause of early detection and 3) key cancers wherethere have been the most significant health gains fromincreased survivalTo identify areas where a reduction in incidence has

been observed cancer incidence data in the UK wereanalysed using UK incidence rates between 1990 and2008 [28] to calculate a percentage change over theperiod This percentage change was then multiplied bymid-period UK incidence (the average per year for 1999to 2001 [29]) to estimate an absolute change in incidenceFour cancer types have seen significantly larger reductionsin incidence between 1990 and 2008 lung (6500) stom-ach (4400) bladder (4400) and cervical (1400) cancersAdditional file 2 gives full details for the 21 cancers Theliterature was consulted to identify possible causes forthese reductions in incidence Overwhelmingly smokingprevention and cessation was cited as the reason for areduction in lung cancers [30] Falls in rates of stomachcancer are also thought to be linked to smoking alongwith declines in Helicobacter pylori and improvements indiet [2931] The picture is less clear given changes in theways these cancers are coded but bladder cancer has beenshown to be associated with smoking too [32] which mayaccount for the decline in rates The fall in cervical cancercan be largely attributed to the roll-out of cervicalscreening since the 1980s which in addition to detectingcancers is able to pick up pre-cancerous abnormalitiesand so reduce the incidence of cancer This has led to afocus on reduction in smoking and on cervical screeningIn addition to cervical screening (which has been in its

present form since 1988) there are currently two othernational screening programmes in the UK aimed at earlydetection of cancers breast cancer screening (introducedin 1988) and colorectal cancer screening (introduced in2006) There is evidence that all three programmes havereduced mortality [33-35] and should be included inour list of priority interventionsThere have been substantial advances in cancer treat-

ment in recent decades which have led to valuable healthgains Surgical techniques remain a cornerstone of treat-ment aided by ever-refined radiotherapy methods Theadvent of new cytotoxic therapies as well as hormonaland biological therapies has greatly increased the availabletreatment options Given the breadth of these treatments(and backed up by the number of treatments that expertopinion had identified) it was necessary to limit the focusof our estimation to a subset which we expected to in-clude most of the health gains likely to have been ob-served between 1991 and 2010 Data on changes insurvival were used as a proxy for health gains Datawere compiled for cancer types on 1-year and 5-yearsurvival rates from CRUK [36] and the Office for National

Statistics (ONS) [37] (see Additional file 2) Rates werecalculated as percentages for the period 1986 to 1990 andcompared with those in 2005 to 2009 to calculate achange in the proportion of people surviving 1 and5 years after diagnosis This change in rate was thenmultiplied by the lsquomid-pointrsquo incidence in 1999 to 2001to estimate the additional number of people survivingThe same three cancer types (albeit in slightly differentorder) were found to have the highest number of additionalpeople surviving for both 1 and 5 years these were pros-tate colorectal and breast cancer These three accountedfor 73 of the estimated gains in 5-year survival Usingclinical guidelines published by the National Institute forHealth and Care Excellence (NICE) a set of the main inter-ventions for each of these three cancer types was identifiedThese interventions were all treatments because althoughthere have been improvements in diagnostics and serviceconfiguration it was assumed that the benefits derivedfrom these should in principle at least be reflected in thenumber of people accessing treatment and in measures oftreatment effectiveness

Identifying estimates of per-patient NMB for the set ofcancer interventionsAs a result of the approach outlined above estimates ofper patient cost and effects were then obtained frompublished studies for the following prioritised areas

Smoking preventioncessation Screening programmes cervical breast and bowel cancer Treatment of breast colorectal and prostate cancer

Smoking preventioncessationThe area where we adopted a very different approach tothat which we had previously used for CVD was smokingIn that study we restricted the analysis to the costs andbenefits arising from NHS smoking cessation interventionsCancer research has not only unequivocally shown thecausal link between smoking (both active and passive) andboth cancer and the risks of cancer (and other healthproblems) but also the effectiveness of various nationalinterventions in reducing smoking rates This cumulativeevidence has contributed to a slow but steady change insmoking behaviour both through direct effects on indivi-dual behaviours and through the many non-NHS interven-tions in the UK (such as legislation and taxation) whichhave followed from and been made possible by this evi-dence and have encouraged existing smokers to quit anddiscouraged others from taking up smoking as summarisedin Figure 1 Therefore health gains from research includenot only the benefit from getting smokers to quit (aided ornot by the NHS) but also in preventing non-smokersfrom ever starting smoking A recent modelling studyfor the UK DH Policy Research Programme provided

Figure 1 Smoking behaviour in England 1982 to 2010 Source General Lifestyle Survey 2010 The Office for National StatisticsCopyright copy 2012 re-used with the permission of The Office for National Statistics

estimates of lifetime life years gained and cost savingsto the NHS of non-smokers and ex-smokers comparedwith smokers [38] The model accounted for the mor-tality benefits from not smoking associated with lungcancer myocardial infarction stroke and chronic ob-structive pulmonary disease In the absence of age-specific smoking rates we used the estimates for menand women aged 35 years and adjusted these to take ac-count of the proportion of life years gained resultingfrom lung cancer reduction and also the adjusted lifeyears gained by the population mean utility values for therelevant ages in order to estimate QALYs gained [39]

Screening programmesTo estimate the NMB of each of the three screeningprogrammes we identified the most appropriate eco-nomic evaluations that modelled the lifetime costs andeffectiveness of offering the screening programmes asdelivered in the UK For both cervical and bowel cancerscreening we used assessments that had informed rele-vant screening policy decisions [4041] In the case ofcervical screening we adjusted the figures presented aslife years gained by an appropriate agesex populationutility values to give an estimate of QALYs [39] Forbreast cancer we used a recently published economicevaluation that had used a life-table model to assess theoverall cost-effectiveness of the NHS screening programmewhich based its assessment of effectiveness on the findingsof the Independent UK Panel on Breast Cancer Screeningand took account of the uncertainty of associated estimatesof benefits harms and costs [3342] In all three casesthese models used take-up rates that were the same orvery similar to those observed in the relevant screeningprogramme during the period in question

Treatment programmesThe full list of treatment interventions included in the esti-mation of health gains for each cancer site are shown inAdditional file 3 These were determined based on NICE Cli-nical Guidelines (CG131 for colorectal cancer [43] CG80 andCG81 for breast cancer [4445] and CG58 for prostate cancer[46]) and cross-checked to ensure that relevant interventionsidentified by experts were included Patient sub-groups wererecognised where distinction in treatments was made orwhere likely differences in cost and benefits existed In breastcancer for instance this distinction was made for node-positive cancers oestrogen receptor-positive cancers HER-2-expressing cancers and prepost-menopausal incidence ofcancers and between early-stage and late-stage cancer Histo-rical comparators for each intervention identified from thecontemporary guidelines were then identified back to 1991For each of the treatment options considered published

economic evaluations were used to estimate per patientcosts and benefits (measured as QALYs) Searches wereconducted using the NHS Economic Evaluation Databaseand MEDLINE to identify economic evaluations of pros-tate breast and colorectal cancer interventions UK-specific estimates were preferred but international evidencewas used where no appropriate UK estimates were avail-able Where they were available NICE technology ap-praisals and National Institute of Health (NIHR) HealthTechnology Assessments were used as the most relevantsources (see Additional file 3) Where exceptionally non-UK cost-effectiveness data had to be used costs were con-verted using purchasing power parity exchange rates

Constructing a time series (1991 to 2010) of usage ofcancer interventionsTo estimate total NMB for the period per-patient QALYgains and net costs for each intervention were multiplied

by the total number of new patients receiving each inter-vention in each year We used the following methods toestimate the time series of usage for the selectedinterventionsFor smoking reductioncessation we used figures derived

from the data on the proportions of smokers ex-smokersand non-smokers for England for each of the years toestimate the net change per year in QALYs gained andNHS savings achieved and related these to populationdata for the UK as a whole [47]For cervical and breast screening programmes we used

figures for the relevant size of the UK age group in eachyear to whom screening was first offered (age 25 for cer-vical and age 50 for breast) For bowel screening we usedthe numbers first offered screening as the programmebegan to be rolled outTo estimate the numbers of people receiving each

treatment intervention over time two primary sourceswere used For surgical procedures (for example colo-rectal excision liver resection and ablation prostatectomyorchiectomy mastectomy and lumpectomy) HospitalEpisodes Statistics [48] were utilised To estimate thenumbers of people receiving drug interventions data onNet Ingredient Cost (NIC) of drugs to the NHS wereused These data were gathered from Health and SocialCare Information Centre (HSCIC) data publications[49] which give details of the total cost of a particulardrug prescribed in primary care (for the PrescriptionCost Analysis) and secondary care (Hospital PrescriptionsAudit Index) in each year For some drugs this informa-tion was not available for the whole of the time period inwhich case assumptions were made on the basis of launchyear and the most recent available time point If thelaunch year occurred during the period 1991 to 2010 alinear interpolation with launch year at pound0 NIC was per-formed For drugs that were not launched during theperiod a last value carried back approach was adoptedusing the most recent year of historical data From theNIC the cost and length of a typical regimen (as esti-mated by NICE costing templates where possible) wereused to calculate the number of complete treatmentsdelivered and hence the number of people receiving aparticular drug in any given year This was then propor-tioned across the indications of a drug and particularpatient group (for example early and late cancers ormultiple cancers)For some older drug interventions NIC data were not

publicly available for any of the years of interest In theseinstances NICE estimates of the proportion of patientslikely to receive interventions (based on guidance costingtemplates) were combined with data on incidence to esti-mate usage numbersFor radiotherapy there was a paucity of data on usage

Data from the National Clinical Analysis and Specialised

Applications Team (NATCANSAT) were available for200910 giving the number of episodes of radiotherapye

It was estimated that 70 of these episodes would be forprimary treatment of a cancer The number of primaryradiotherapy episodes was estimated as a proportion ofthe incidence of each cancer in 200910 This proportionwas applied historically to incidence in order to estimateradiotherapy treatmentThe component figures of numbers of people receiving

treatment interventions were all derived from data forEngland To produce a UK estimate (needed because theresearch spend data is for the UK) figures were adjustedby a factor reflecting Englandrsquos proportion of the adultUK population The screening was based directly onrelevant UK population data and for smoking behaviourthe time series data were for England but have beenapplied to the UK population All cost estimates wereadjusted to 201112 prices using the Hospital andCommunity Health Services Pay and Prices Index [50]For the calculation of NMB we used for the base case

an opportunity cost value of a QALY as used by NICE inits decision-making [5152] This value reflects an estimateof the opportunity cost in terms of QALYs forgone else-where in the health service within its fixed budget Giventhat public spending on health research can justifiably beseen as a decision to spend on research rather than dir-ectly on current healthcare this opportunity cost valueis appropriate to the public decision regarding researchfunding In this study as previously for CVD we char-acterised NICErsquos threshold range as equivalent to anaverage of pound25000 per QALY but considered a broaderrange of values in the sensitivity analysis including avalue of pound70000 which would be broadly consistentwith the commonly proposed QALY threshold of 3 timesGDP per capita [53]

Analysis of UK clinical guidelines to estimate elapsed timeand rate of attributionIn the 2008 report on CVD research the references citedin a sample of clinical guidelines were analysed to informthe estimate of the elapsed time between research spendand net health gain and the proportion of net health gainthat could be attributed to UK research [3] In the currentstudy on cancer research we replicated this approachIn total 31 national clinical guidelines which provided

a broad representation of cancer practice in the UKwere identified Twelve were published by NICE and afurther twelve by the Scottish Intercollegiate GuidelineNetwork (SIGN) The remaining seven guidelines werepublished by either the Royal Colleges or the NationalCancer Screening Programme The reference sections ofthese guidelines were reviewed five had no reference list(four published by NICE one by the National ScreeningProgramme) while one screening guideline had no

references to peer-review journals (that is it referencedonly policy and practice documents) These six guidelineswere excluded from our sample We then used a bespokecomputer programme to extract references from the elec-tronic PDF version of each guideline in three cases theautomated reference extraction failed (because paperswere not referenced in a recognised format) leaving uswith a sample of 22 national guidelinesOf the 5627 references cited in the 22 guidelines 4416

references (78) were automatically extracted excludingduplicate references within a guideline (see Additionalfile 4 for breakdown by guideline) Nine of these refer-ences had no date information and were excluded fromthe analysis of elapsed time leaving a total of 4407 ref-erences The age of a paper cited in a clinical guidelinehas been termed the lsquoknowledge cycle timersquo [54] whichis the average difference between the publication dateof the clinical guideline and the publication date of thecited papers on the guideline The knowledge cycle timewas calculated for the 22 identified guidelines and used toinform the estimated elapsed timeTo estimate the rate of attribution to the UK the 4416

extracted and de-duplicated references were provided tothe Centre for Science and Technology Studies (CWTS)to be matched to their bibliometric database (which is de-rived from the Web of Science)f Of the 4416 extractedreferences CWTS was able to match 4051 (92) whichformed the dataset to estimate the degree of attributionbased on the address field in the cited papers These ad-dresses were used as a proxy for the location in which theresearch was conducted and so it was possible to estimatethe proportion of the cited research that was conducted inthe UK The non-matched references included non-serialoutputs such as books journals that are not indexed onthe Web of Science and incorrect references

Estimation of the rate of returnUsing these four key sources of data we could thenattribute a proportion of the estimated total annual NMBof the cancer health gain as being due to UK research andrelate an equal number of years of investment to years ofNMB lsquolaggedrsquo by an estimate of the average lag betweenresearch and benefit The return was expressed as an IRRwhich is effectively the discount rate that would yield azero net present value The IRR is convenient in enablinga comparison to be made between non-competing in-vestments of different sizes (as well as providing a dir-ect comparison with our previous study) We recognisethe many and various layers of estimates involved Inother circumstances it might be feasible to express theuncertainty as ranges for each parameter in our overallestimate and undertake a formal probabilistic sensitivityanalysis (PSA) However given the nature of the evidencefrom multiple sources for the numerous parameters and

the necessary judgments involved in drawing together andinterpreting the evidence a comprehensive PSA quantita-tively characterising all the uncertainty was not feasiblehere and indeed would be liable to suggest a spuriousprecision Instead we provide a series of one-way andscenario sensitivity analyses to illustrate the effects ofspecific variables on the IRR

ResultsPublic and charitable funding of UK cancer-relatedresearch 1970 to 2011Additional file 1 provides our estimated expenditure byyear by organisation for the 40-year period 1970 to 2009with a summary of cash expenditure provided in Figure 2Figure 3 illustrates estimated public and charitable ex-penditure on cancer-related research from 1970 to 2009in cash and constant 201112 prices (the latter for ourbest estimate) About pound15 billion (in 201112 pricesg) ofcancer-related research funding was invested during thisperiod The data presented in Figure 3 are derived from anumber of different sources and include various assump-tions and estimations For this reason we also provided alsquohighrsquo and lsquolowrsquo scenario for total cancer-related researchexpenditure with a range of pound14 to pound17 billion In Figure 3we also present total public and charitable spending oncancer-related research in cash terms This emphasisesthat in real terms (in 2011 prices the red line) spendingfell between 1970 and 1979 then stagnated until 1986and thereafter increased threefold from pound250 to pound850 mil-lion by the end of the time series in 2009

Net monetary benefitTable 2 shows the contributions to our total estimates oflifetime QALYs gained from the seven areas we addressedclassified by the year in which the intervention was deliv-ered (or in the case of screening the year in which thosetargeted entered the screening programme) Reduction insmoking accounted for 51 of the QALYs gained from theseven areas we prioritised The other two large sources ofQALYs gained were from cervical screening (21) andbreast cancer treatments (19) The other areas we exam-ined were small contributors by comparisonTable 3 shows the lifetime net costs to the NHS for

each of these areas over the 20-year period The keypoints to note here are the high proportion of total netcosts accounted for by breast cancer and prostate cancertreatments Smoking reduction on the other hand reducesnet NHS costs as does colorectal screening although thelatterrsquos introduction late in the period covered means itsabsolute contribution to reducing overall costs is smallTable 4 summarises the NMB when the QALYs have

been valued at pound25000 and the net costs to the NHS ofthe intervention and its long-term sequelae have beendeducted It shows how the total NMB (when measured

Figure 2 Total estimated public and charitable spend on cancer research by source of funding 1970 to 2009 in current (cash) prices

in constant prices) from the research-based interven-tions that we have assessed has been steadily increasingwith an overall increase of 28 over the 20-year periodOver the whole period smoking reduction (providingfor both QALYs and NHS cost savings) accounted for65 of NMB followed by cervical screening (24) andbreast cancer treatments (10) All seven areas we studiedshowed a positive NMB when QALYs were valued atpound25000 However at a QALY value of pound20000 prostate

Figure 3 Public and charitable funding of cancer research 1970 to 20current (cash) prices

and colorectal treatments and breast cancer screening allshowed a negative NMB (that is their net costs exceededthe valuation of the benefits they provide)

Estimating the elapsed timeThe estimate of the elapsed time used in the study wasbased primarily on the analysis of cited references onclinical guidelines (that is knowledge cycle time) Asillustrated in Figure 4 the mean age of the 4407 cited

09 at constant (2011) prices with low and high estimates and in

Table 2 Contributions of the seven areas to the total estimates of lifetime QALYs gained by year 1991 to 2010

QALYs (thousands)

Year Treatment Screening Smokingreduction

Prostate cancer Breast cancer Colorectal cancer Cervical cancer Bowel cancer Breast cancer

1991 8 46 6 71 ndash 2 144 277

1992 9 48 6 70 ndash 2 144 279

1993 10 46 6 68 ndash 2 145 276

1994 11 48 6 67 ndash 2 145 279

1995 10 45 6 65 ndash 2 145 273

1996 11 46 6 66 ndash 2 146 276

1997 11 46 6 63 ndash 3 146 274

1998 11 50 6 59 ndash 2 147 276

1999 13 53 7 56 ndash 2 147 279

2000 15 53 7 55 ndash 2 148 281

2001 17 55 7 54 ndash 2 149 285

2002 20 56 8 53 ndash 2 150 290

2003 22 59 8 53 ndash 2 151 295

2004 24 61 9 56 ndash 2 152 305

2005 21 62 10 60 ndash 2 154 309

2006 22 62 12 61 2 2 155 316

2007 23 65 13 60 5 2 157 324

2008 27 68 14 60 7 2 158 337

2009 26 71 15 63 9 2 159 345

2010 25 74 15 65 12 3 161 354

Total 339 1112 173 1225 35 43 3003 5930

Abbreviation QALY quality adjusted life year

papers on the 22 guidelines was 8 years ranging from 0 to88 years (the median age was 6 years with an interquartilerange of 3 to 10 years) To produce an estimate of elapsedtime between spending on research and health gain asrequired for this study it was necessary to add on to thisvalue the estimates for the period between the awardingof funding and publication and the period between rec-ommendation and use Using the same approach adoptedin the 2008 report we estimated these two periods to totalapproximately 7 years giving a best estimated elapsed timebetween spending on research and health gain of 15 yearswith 10 and 20 years arbitrarily selected as lower and higherestimates for sensitivity analyses

Estimating the amount of health gains that can beattributed to UK researchThe estimate of the proportion of the health gain thatcan be attributed to UK research used in the study wasbased primarily on the analysis of cited references onclinical guidelines A total of 4051 publications were ana-lysed to estimate the proportion of the research that couldbe attributed to the UK The overall percentage across all

guidelines was 17 but as shown in Additional file 4 thisdiffered between specific guidelines

Estimating the IRR from cancer-related researchOur estimates of the NMB produced by year (summarisedin Table 4) were then related to the estimated public andcharitable spend by year on cancer-related research (sum-marised in Figure 3) and expressed as an IRR Calculationof the IRR incorporates our best estimates of the elapsedtime of 15 years (low and high estimates of 10 and20 years) and of the proportion of the NMB that couldbe attributable to UK research (best estimate 17 lowand high range estimates of 10 and 25) Thus in ourbase-case calculation we related 17 of the annualNMB (for each of the 20 years 1991 to 2010) to 20 yearsof the research investment that had occurred 15 yearsearlier (that is for the years 1976 to 1995 in other wordsa subset of the 1971 to 2009 series collated) This pro-duced a base-case estimate of the IRR of 101As is evident from the methods used there is inevitably

considerable uncertainty around the values of all ourestimates Table 5 presents a series of one-way sensitiv-ity analyses to illustrate the effects of some of the main

Table 3 Contributions of the seven areas to the estimates of lifetime costs to the NHS of services delivered by year1991 to 2010

Costs (GBpound million)

1991 199 665 181 41 ndash 34 -277 844

1992 220 687 190 40 ndash 37 -277 897

1993 241 658 185 39 ndash 40 -278 887

1994 272 684 185 39 ndash 42 -278 944

1995 252 646 175 37 ndash 42 -279 874

1996 278 660 182 38 ndash 43 -280 921

1997 269 684 181 36 ndash 53 -281 943

1998 283 720 187 34 ndash 50 -282 993

1999 336 753 214 32 ndash 47 -283 1098

2000 391 746 211 32 ndash 45 -285 1140

2001 456 755 206 31 ndash 43 -287 1204

2002 519 764 202 30 ndash 43 -282 1276

2003 571 794 178 30 ndash 44 -255 1361

2004 614 817 179 32 ndash 44 -254 1432

2005 545 850 182 34 ndash 44 -252 1403

2006 572 851 188 35 -5 45 -258 1428

2007 596 881 177 35 -10 46 -252 1473

2008 613 919 181 35 -15 48 -242 1538

2009 606 950 185 36 -20 48 -237 1569

2010 569 986 186 38 -25 56 -240 1569

Total 8403 15469 3755 704 -75 894 -5358 23793

Abbreviation NHS National Health Service

areas of uncertainty and all changes have predictableeffects For NMB the greatest uncertainty in our calcula-tions probably relates to the magnitude of the benefitsfrom smoking given the indirect nature of the estimateReducing (or increasing) the NMB from smoking by 25produced an IRR of 87 (or 112) for illustration the(unrealistic) extreme of removing entirely the benefitsfrom smoking from our estimates produced an IRR of24 The IRR increased as our estimates of researchfunding were reduced and the proportion of benefitsattributable to UK research increased It was found tobe particularly sensitive to a reduction in the elapsedtime Although taken individually all of the alternativevalues we have explored in this sensitivity analysis showeda reasonable rate of return in combination they couldof course have produced a wider range of estimates forthe IRR

DiscussionTaking into account the necessary assumptions made inour approach the base-case IRR for the NMB from the

health gain from cancer research of approximately 10is remarkably similar to that derived for CVD researchwhere the IRR derived from the health gain was 9 Thesebenefits alone provide a return considerably greater thanthe UK governmentrsquos minimum threshold of 35 forinvestments thus suggesting that investment in cancerresearch is worthwhile Moreover given that CVD cancerand mental health account for about 45 of the total bur-den of disease in the UK [55] we might with increasingconfidence extrapolate this order of rates of return to thewhole of the public and charitable investments in biomed-ical and health research in the UK The important caveatto that statement is that the two of the clinical areas thatwe have analysed in most detail ndash cancer and CVD ndashhaveboth benefitted significantly from the changes in smokingover the period analysedHowever it should be remembered that in our previous

study the rate of return from NMBs of the health gainwas less than a third of the rate of return (30) that wesuggested might relate to the broader GDP gains If weaccept that estimate of returns from GDP then again the

Table 4 Contribution of the seven areas to the estimates of net monetary benefit by year 1991 to 2010 (QALY valueof pound25000)

Net monetary benefit (pound million)

1991 8 490 -33 1729 ndash 7 3885 6085

1992 8 506 -35 1699 ndash 7 3889 6075

1993 9 485 -33 1659 ndash 8 3890 6018

1994 10 504 -32 1646 ndash 8 3894 6030

1995 9 475 -28 1592 ndash 8 3906 5963

1996 -3 483 -28 1609 ndash 9 3920 5989

1997 -3 458 -28 1546 ndash 11 3933 5916

1998 -3 531 -29 1447 ndash 10 3947 5902

1999 -10 566 -40 1374 ndash 9 3966 5866

2000 -15 579 -36 1350 ndash 9 3989 5876

2001 -25 630 -30 1310 ndash 9 4018 5910

2002 -8 647 -5 1283 ndash 9 4037 5963

2003 -14 677 24 1294 ndash 9 4035 6025

2004 -7 705 49 1376 ndash 9 4064 6195

2005 -15 694 67 1460 ndash 9 4099 6314

2006 -16 697 106 1484 64 9 4138 6481

2007 -23 736 149 1469 128 9 4167 6635

2008 51 786 171 1474 192 10 4192 6876

2009 46 816 178 1528 256 10 4216 7050

2010 66 854 183 1596 320 11 4253 7282

Total 65 12318 566 29927 960 179 80437 124452

Figure 4 Age of papers cited on cancer clinical guidelines

Table 5 IRR one-way sensitivity analyses

Analysis IRR

Base casea 101

Research funding estimate

Low 108

High 87

lsquoValuersquo of a QALY GBpound

20000 80

30000 117

50000 161

70000 189

Elapsed time years

10 146

Attribution to UK research

25 130

Effect of smoking cessation

Decrease NMB by 25 87

Increase NMB by 25 112

Omitting benefit of smoking reduction 24

Abbreviations IRR internal rate of return NMB net monetary benefits QALYquality adjusted life yearaBest estimate for research funding and net monetary benefit QALY value ofpound25000 elapsed time of 15 years and attribution to UK research of 17)

overall returns from cancer research would be in the orderof 40 However as we noted then although this estimatewas based on the best available information it wasgenerated from a small empirical literature much of itUS-centred and only a proportion specific to medicalresearch From the papers reviewed a rate of returnbetween 20 and 67 was identified and we took 30as our lsquobest estimatersquo The current study did not revisitthis aspect of the return to investment As discussedbelow we recommend that future research should aimto update and improve on these estimates

What this paper contributesIn this study our main methodological contribution wasto further validate the bottom-up approach we developedin the original Medical Research Whatrsquos it Worth study[3] This new application strengthens our argument thatthe bottom-up approach represents a significant improve-ment on earlier attempts to estimate economic returnsfrom research as it attempts to directly attribute healthgains (as measured by QALYs) to research-derived inter-ventions The alternative lsquotop-downrsquo approaches face thefundamental problem of starting with changes in mortalityor morbidity over time and attributing an estimatedproportion of these changes to biomedical and healthresearch In addition and in line with our previous

work we have taken into account the costs of deliveringthe health gain and the elapsed time between researchinvestment and health gain which earlier studies hadlargely failed to do

Key assumptions and caveatsDespite validating and further developing the approachthere are still a number of key assumptions and caveatsin our estimate of the economic returns from cancer-related research Given this we would be the first toacknowledge that the bottom-up approach by necessityrelies on these assumptions and that our findings doneed to be treated with appropriate caution We docu-ment these assumptions in the interests of transparencyand to stimulate further research The key assumptionsare as follows

Our base-case value of a QALY is pound25000Obviously and as demonstrated by our sensitivityanalysis the IRR is sensitive to the assumed value ofthe health gain measured as QALYs Our base-caseassumption is consistent with our analysis of thereturns to CVD research and reflects the mid-pointin the range of values (of pound20000 to pound30000) cited asthe normal criteria for acceptance of interventions byNICE [51] More recently NICE has increased thisthreshold up to around pound50000 for certain treatmentsthat provide end-of-life benefits particularly late-stagecancer treatments [52] At the same time it has seemedto re-emphasise that the pound20000 threshold shouldapply unless there are special circumstances Althoughthis leaves uncertainty about the most appropriatevalue here (as reflected in our sensitivity analysis)conceptually the argument remains that thislsquoopportunity costrsquo value of a QALY should apply toan assessment of research in that investing inhealth-related research can be seen as an alternativeto spending the money directly on current healthcare We note however that other studies in the USand Australia have used much higher values reflectingindividual willingness to pay for health gains andwe have illustrated in a sensitivity analysis the effectof using a value of the order of three times GDP percapita [53]

The total NMB for interventions not covered isassumed to be zero Our IRR calculation assumesthat all other cancer treatment developmentsinterventions that we have not specifically includedhave in aggregate no effect on the NMB becausefor these the monetised value of the health benefitis equal to the cost of delivering the benefitIn reality there may be some areas that we have notcovered for which the NMB is negative because ofthe high cost of treatment and low incremental

health gain Conversely there are may be otherareas that generate a significant number of QALYsat a relatively low cost We are not in a position toknow whether the net effect of the interventions wedid not examine is positive negative or zero

The total net flow of knowledge betweendisciplines is zero We have assumed that the flowof knowledge is the same into and out of differentresearch fields and from each research field into thecognate treatment areas However we know thatresearch is unpredictable and diffuse and there maybe research disciplines that contribute more thanthey gain from other areas One could argue thatsome of the reduction in mortality from diseasesother than cancer that arises as a result of thereduction in smoking (eg CVD) which we haveexcluded should in fact be included as having beenachieved as an additional advantage arising from theevidence of the effect of smoking on lung cancer

All health gain from treatments is captured in theestimates of the health gain from specificinterventions We have assumed that in principlethe health gain from improved service configurationand all other supportive service changes (includingdiagnostics and imaging) should be captured in theestimates of the gains from specific interventions Inpractice our estimates of QALY gains are mainlyderived from UK-relevant health technologyassessments that are extrapolated from trial datawhich may provide an imperfect estimate of thegain when the interventions are used in routineNHS practice

The definitions of the cancer-related research usedby the research funders captures basic researchthat may have contributed to developments in thisarea This is clearly the case for the cancer-specificfunders such as CRUK as we included all the researchthey funded For MRC funding we relied on thefunder classification which as discussed in Additionalfile 1 was broad and thus should include basic re-search For the Wellcome Trust which accounts foraround 10 of total cancer funding we had to rely onsearch terms We scanned the list of grant titles se-lected through this search strategy and this list sug-gests that fundamental research is being includedalthough we cannot guarantee that it all is in fact in-cluded For the remaining two funders ndash the FundingCouncils and the DHNHS ndash this would not be anissue as their time series were derived through an es-timate of cancer research activity

The knowledge cycle time and attribution ratewere largely determined through bibliometricanalysis of clinical guidelines As part of this studyand reported separately we undertook a series of

case studies that qualitatively explored how researchtranslates into health benefit [24] This workdemonstrates the complexity of biomedical andhealth innovation especially when trying to measurethe time it takes for research to develop into healthbenefits Although the bibliometric approachprovides us with an empirical estimate of both theelapsed time and the rate by which we can attributeUK research to UK health gain it inevitably is agross simplification of a complex process

We have made various assumptions about thebaseline treatment against which we were lookingin research-based developments For example inestimating the net health gain from breast cancertreatments we did not include benefits fromstandard mastectomy but just estimated the benefitsfrom subsequent developments

There is a risk that we may have double-countedthe NMB for individuals who are treated as a resultof screening Conceptually the benefits of screeninginclude the downstream NMB of treatments thatresult from the screening However a number ofissues minimise the likelihood of our doublecounting First we did not include (in the treatmentcalculation) all the benefits of treating an individualdisease (for example breast cancer) but only theadditional benefits of improved (research-based)treatments so any additional people who get lsquobasictreatmentrsquo as a result of screening were countedonly as an advantage to screening Second thebenefits and the future treatment costs of a womanentering a screening programme (which is when weestimated the future QALYs and present value ofassociated net costs) occur in a future year oftenmany years ahead so in taking a 20 year periodthere is limited scope for counting both If we hadperfect data and were looking at all treatmentbenefits over a much longer period we could inprinciple look only at the benefits of treatments thatwould encapsulate all the QALY benefits ofscreening

In acknowledging these assumptions we should makethe important point that an underlying principle weadopted throughout this study and our previous workon CVD was to err on the side of caution that is tomake assumptions that would lead other things beingequal to a lower rate of return However comparedwith our earlier study of CVD we are less confident thatwe have always managed to adhere to the principle ofconservatism For example as discussed above there isan implicit assumption in ascribing the IRR to the wholeof cancer that everything we have not specifically includedhas in aggregate no effect on the NMB (the value of the

health gain is equal to the costs of delivering it) In realitythe aggregate effect of what we have not considered couldbe positive negative or zero Another issue is that in theCVD study conservatism often came from adopting thelower of two (or more) published estimates for specificparameters but for cancer interventions we rarely had achoice of relevant data estimates as discussed in moredepth belowIn addition to these specific assumptions there are a

number of other broader issues that add to the uncertaintyof our estimates and need to be highlighted

We have evidence of linkage between research andhealth gains but no formal evidence of causalityOur analysis relied on the reasonable assumptionthat these health benefits would not have occurredwithout the evidence from medical research and wehave illustrated the often complex nature of thoselinkages in case studies [24] At one level we haveaddressed this issue of causality by our bottom-upapproach adding together the benefits demonstratedthrough clinical trials of new interventions Forthese causality from worldwide medical research isall but a truism However even for these we had toassume that a proportion of the benefit (based on theUK contribution to publications cited in guidelines)arose from UK research It is possible that some oreven all of these interventions might have come intouse in the UK even if there had been no UK cancerresearch but it is improbable that the same level andtiming of benefits would have arisen Causalitycould be argued to be less direct for the benefits ofthe reduction in smoking which made the largestcontribution to the total NMB It is possible butimplausible that changes in smoking behaviourmight have arisen in the absence of any evidence ofthe health effects Certainly our case studies [24]show that there was an extended lag between theinitial evidence of harms to smokers and changesin behaviour and the UK government probablyneeded the cumulative evidence that has emergedover several decades and in particular the evidenceof the harms of environmental tobacco smoke tomake the legislative changes in the face of veryconsiderable resistance There are also additionaluncertainties around the magnitudes of NMB fromsmoking Of the total pound124 billion total NMB pound80billion (or 65) arose from reductions in smokingand the numbers for the increased proportion ofthe population who were non-smokers or ex-smokersis based on self-reported survey data In the sensitivityanalysis (Table 5) if the NMB from smoking reductionwas decreased or increased by (an arbitrary) 25the IRR would reduce to 87 or increase to 112

respectively Omitting the benefits from smokingreduction entirely reduces the IRR to 24 Howeverit should be stressed that we estimated only themortality effects on lung cancer and excluded effectson other cancers (and other disease areas) fromsmoking all of which would mean we probablyunderestimated the impact of smoking reductionHowever taking a perspective of NHS costs onlywe have not included costs to other parts of theeconomy from the various measures to reducesmoking [56]

Variable quality of data on the effectiveness ofscreening The three national screeningprogrammes are important elements in ourestimates The clinical and cost-effectiveness evi-dence for bowel cancer screening is high-quality andtrial-based but the evidence for cervical screeningand even more so for breast cancer screening is lessrobust Therecent review [33] of the clinical evidence hasprovided some clarity to the contentious issue of thenet benefits of breast screening and underpins therelatively simple economic model that we used asthe basis of our estimate of NMB but there isconsiderable uncertainty around these estimates

There is a lack of robust clinical effectiveness andcost-effectiveness data for some interventionsespecially for longstanding treatments This was ageneral problem with well-established surgicaltechniques (for example total mesorectal excisionfor which no cost-effectiveness evidence could befound) and similarly for some of the hormonaltherapies (for example tamoxifen and goserelin)

There are a large number of areas of cancer thatwe did not consider in our analysis Our analysiswas based on a prioritised list of cancer typesgenerated from both expert opinion andepidemiological data By necessity this meant we didnot look at a number of areas (and as noted aboveassumed the NMB arising from these areas to bezero)

Elapsed time was an important variable indetermining the IRR but one that is conceptuallydifficult to measure [24] We wanted to measurethe time between research investment and healthgain but neither of these events occurs at onedefined point Research investment may occur overa period although in many cases given a typicalpattern of investment starting with pilot trials andbuilding to larger-scale studies and finally randomisedcontrolled trials the bulk of the research investmentmay come late in the overall investment period Thepoint at which the bulk of the health gain occurs iseven more difficult to define and will depend on a

range offactors such as the type of intervention and the wayin which it is implemented The issue of time lags wasidentified in the original 2008 report which suggestedthat further research is needed

Given these various issues and the nature of the exercisewhich relies on data and estimates from a wide variety ofsources it is not possible to characterise in any formal waythe overall uncertainty in our estimates The sensitivityanalysis illustrates the effect on the IRR of alternativevalues for some of the key parameters and shows thatthe broad order of magnitude of the IRR is relatively in-sensitive to fairly substantial degrees of uncertainty onspecific elements of the analysis of what has happenedin the past Moreover even without this uncertainty weneed to interpret our analysis of what has happened inthe past with caution as follows

Past performance is not an indicator of futureperformance The IRR is based on pastperformance and cannot be a guarantee of futurereturns particularly for increased levels of researchspending This means that research advocates needto use the estimates provided in this paper verycautiously if wishing to extrapolate them asindicators of likely future returns from researchexpenditure Given the near doubling in cancer-related research funding since the turn of the cen-tury (Figure 3) there will need to be a similar in-crease in NMB in the coming decade to maintainthe current returns It is worth noting that the NMBof bowel screening is not fully reflected in the IRRbecause this screening is ofrecent introduction so there is additional benefit thatwill be realised in the future Likewise pharmaceuticalinterventions are typically priced to maximise thevalue of the benefit at time of introduction so theNMB is close to zero During the coming decadesome of the expensive drugs will come off patent andmay be available more cheaply thus contributing to anincrease in the NMB however other new and expen-sive lsquoon patentrsquo drugs may well be used in preference

We estimated average returns from cancerresearch not the marginal returns From thisanalysis we are not able to say whether the rate ofreturn would have been different if research spendinghad been higher or lower and whether at the marginthe returns to research investment are increasing ordiminishing

The analysis should not be used to makecomparative assessments about the value of researchinto particular interventionscancers Our approachexamined a portfolio of interventionscancer types and

we would caution that the detailed data may not besufficiently robust to make comparisons betweeninterventions within specific cancers

Future research requirementsBased on the key assumptions uncertainties and caveatsdescribed above further research is needed in the follow-ing areas

A deeper understanding of the international flowsof knowledge In our model we estimated theextent to which UK research influences UKpractice using citations on clinical guidelines andthis figure was used in estimating the IRR Howeverthere is a need for a more nuanced understanding ofthese knowledge flows and their impact oninternational health gains for example UK researchis contributing to health gains beyond the UK As aresult our current figure underestimates the globalvalue of UK RampD A study that aimed to measurethe health gains net of healthcare costs in the restof the world as a result of UK medical researchwould address this At a European level it wouldalso be interesting to explore how the investmentsof different European countries in biomedical andhealth research leads to health gains in otherEuropean countries thereby reinforcing the notionof European solidarity

An improved estimate of spillover effects for UKbiomedical and health research Public andcharitable biomedical and health researchexpenditure not only leads to health gains but alsomakes an important contribution to the nationaleconomy Much of the evidence base for estimatinga spillover effect of 30 comes from studiesundertaken in the 1960s and 1970s andor relatesspecifically to agriculture research More recentanalyses for medical research are largely based onUS data Furthermore in this study we alsoassumed that the spillovers are independent ofdisease area but we have no empirical evidence tosupport whether that assumption is justified or notFuture research should aim to provide empiricalestimates of the effects of biomedical and healthresearch for the UK economy ideally at adisease-specific level

Examine another disease area or time period inwhich smoking reduction is likely to have aminimal impact As illustrated in Table 5 the IRRfor cancer research is very dependent on the effectof smoking reduction It would be valuable toundertake an investigation in another clinical area inwhich smoking is not important to see whethersimilar rates of return are found

ConclusionIt is challenging to move beyond the identification of thebenefits from specific examples of research funding andattempt to meet the increasing demands for accountabilityby systematically measuring returns to the investment of awhole body of medical research In this paper we haveestimated the economic benefit of public and charitablefunding of cancer-related research and further validatedthe methodological approach that we originally used inassessing the returns from CVD research Expressed in201112 prices total expenditure on cancer-related re-search from 1970 to 2009 was pound15 billion Over theperiod 1991 to 2010 the interventions we prioritisedin our study produced 59 million QALYs and a NMBof pound124 billion allowing for the net NHS costs result-ing from them and valuing a QALY at pound25000 The pro-portion of the benefit attributable to UK research was17 The lag between research funding and impact forcancer treatments was 15 years Our best estimate of thehealth-gain IRR from UK cancer-related research was10 very similar to that of 9 for CVD research The re-sults suggest that despite the uncertainties around themethods and estimates the historical returns in terms ofthe NMB of the health gains derived in the UK from pub-lic and charitably funded biomedical and health researchare substantial and could by themselves justify the invest-ment made

EndnotesaWe have used the term lsquointerventionsrsquo broadly throughoutthis paper to include treatments screening programmesand a wide range of policies and information that have ledto changes in smokingbNCRI members must have an annual cancer researchspend in the UK in excess of pound1 million and have an ap-propriate peer-review system for ensuring the scientificquality of the research that they fund [57]cUp until 2010 the Wellcome Trust had a policy not tofund cancer research It changed its policy in recognitionthat the basic research it funded was increasingly havingimplications for our understanding of cancerdData provided by the Higher Education Funding Councilfor England in personal correspondenceeData provided at our request by NATCANSAT producedfrom the national radiotherapy dataset for years 2009 to2013fCWTS maintains a bibliometric database of all scientificpublications (including health and biomedical research)for the period 1981 to 2013 This dataset is based on thejournals and serials processed for the Internet versionsof the Science Citation Index Expanded and associatedcitation indices the Social Sciences Citation Index andthe Arts and Humanities Citation Index This databaseis operated for bibliometric purposes in service contracts

under a License Agreement with Thomson Reuters See[58] for more informationgWe used HMG GDP Deflator [59] to estimate constantprices for 2011 (accessed 9 January 2013) We alsocompared the Biomedical Research and DevelopmentPrice Index published by the National Institutes forHealth Office of Budget ([60] accessed 9 January 2013)and concluded that there was no material difference forthe purpose of the current analysis

Additional files

Funding data

Incidence and survival data

Summary of interventions their comparators andsources and health gain data

Guidelines data

Additional file 1

Additional file 2

Additional file 3

Additional file 4

AbbreviationsCRUK Cancer Research UK CVD cardiovascular disease CWTS Centre forScience and Technology Studies DALY Disability adjusted life yearDH Department of Health GDP Gross Domestic Product HSCIC Health andSocial Care Information Centre IRR the internal rate of return MRC MedicalResearch Council NATCANSAT National Clinical Analysis and SpecialisedApplications Team NCRI National Cancer Research InstituteNHMRC National Health and Medical Research Council NHS National HealthService NIC Net Ingredient Cost NICE National Institute for Health andClinical Excellence NIHR National Institute of Health Research NMB NetMonetary Benefit ONS Office of National Statistics QALY Quality adjustedlife year RampD Research and Development SIGN Scottish IntercollegiateGuideline Network

Competing interestsThe authors declare that there are no competing interests

Authorsrsquo contributionsThe project was conceived by JG and MB and designed and executed by allthe authors JG led on the funding analysis JG AP SG on the guidelinesanalysis and MG and MB on the assessment of health gain and thecalculation of the IRR All authors were involved in synthesising andinterpreting the results All authors contributed drafts for various parts of thepaper critically reviewing various iterations and approving the final draftsubmitted

AcknowledgementsWe are grateful for financial support from the Wellcome Trust CancerResearch UK the National Institute of Health Research and the Academy ofMedical Sciences The funders had no influence on the study design datacollection or analysis decision to publish or preparation of the manuscriptThrough membership of the Steering Group the funders did provideongoing suggestions and advice throughout the study and reviewed themanuscript We thank the members of the Steering Group for their adviceand support particularly Liz Allen for chairing along with David Cox AoifeRegan Emma Greenwood and Rachel Quinn Rodrigo Costas and Thed vanLeeuwen (CWTS Leiden) for providing bibliometric analysis Thomas White atthe NCRI for providing us with funding data and reviewing our analysis andcolleagues in our respective organisations for their help in a variety of waysincluding Steven Wooding for reviewing an earlier draft We especiallyextend our thanks to all the cancer experts that we consulted at variousstages of the project particularly Peter Selby Mike Richards Stan KayeGeorgios Lyratzopoulos David Dearnaley Galina Velikova and Tim MaughanNeedless to say we are responsible for any errors misrepresentations orinaccuracies

Author details1Health Economics Research Group Brunel University Uxbridge UK 2RANDEurope Westbrook Centre Milton Road Cambridge CB4 1YG UK 3KingrsquosPolicy Institute Kingrsquos College London Virginia Woolf Building 22 KingswayLondon WC2R 2LA UK

Received 6 March 2014 Accepted 6 May 2014Published

References1 Editorial Unknown quantities Nature 2010 465665ndash6662 Buxton M Hanney S Jones T Estimating the economic value to societies

of the impact of health research a critical review Bull World Health Organ2004 82733ndash739

3 Health Economics Research Group Office of Health Economics RANDEurope Medical Research Whatrsquos it Worth Estimating the economic benefitsfrom medical research in the UK London UK Evaluation Forum 2008 [httpwwwwellcomeacukstellentgroupscorporatesitesitestudioobjectsdocumentsweb_documentwtx052110pdf Accessed January 2014]

4 Mushkin S Biomedical Research Costs and Benefits Cambridge MABallinger 1979

5 Drummond MF Davies LM Ferris FL Assessing the costs and benefits ofmedical research the diabetic retinopathy study Soc Sci Med 199234973ndash981

6 Funding First Exceptional Returns The Economic Value of AmericarsquosInvestment in Medical Research New York Lasker Foundation 2000 [httpwwwlaskerfoundationorgmediapdfexceptionalpdf AccessedJanuary 2014]

7 Murphy KM Topel R The economic value of medical research New YorkLasker Foundation 1998 [httpwwwlaskerfoundationorgpdfeconomicvaluepdf Accessed January 2104]

8 Access Economics Exceptional Returns The Value of Investing in Health RampDin Australia Canberra Access Economics 2003 [httpwwwasmrorgauExceptpdf Accessed January 2014]

9 Access Economics Exceptional Returns The Value of Investing in Health RampDin Australia II Canberra Access Economics 2008 [httpwwwasmrorgauExceptII08pdf Accessed January 2014]

10 Deloitte Access Economics Returns on NHMRC funded Research andDevelopment Canberra Deloitte Access Economics 2011 [httpwwwasmrorgauNHMRCReturnspdf Accessed January 2014]

11 Litchenberg FR The expanding pharmaceutical arsenal in the war oncancer NBER Working Paper Series 2004 Paper No w10328

12 Sun EC Jena AP Lakdawalla DN Reyes CM Philipson TJ Goldman DP Aneconomic evaluation of the war on cancer NBER Working Paper Series2009 Paper No 15574

13 Lakdawalla DN Sun EC Jena AB Reyes CM Goldman DP Philipson TJ Aneconomic evaluation of the war on cancer J Health Econ 201029333ndash346

14 Buxton M Hanney S How can payback from health services research beassessed J Health Serv Res Policy 1996 135ndash43

15 Hanney S Mugford M Grant J Buxton M Assessing the benefits of healthresearch lessons from research into the use of antenatal corticosteroidsfor the prevention of neonatal respiratory distress syndrome Soc Sci Med2005 60937ndash947

16 Wooding S Hanney S Buxton M Grant J Payback arising from researchfunding evaluation of the Arthritis Research Campaign Rheumatology2005 441145ndash1156

17 Wooding S Hanney S Pollitt A Buxton M Grant J Project RetrosightUnderstanding the Returns from Cardiovascular and Stroke ResearchCambridge RAND Europe 2011 [httpwwwrandorgpubsmonographsMG1079html Accessed January 2014]

18 Wooding S Pollitt A Castle-Clarke S Cochrane G Diepeveen S Guthrie SHortvitz-Lennon M Lariviegravere V Ni Chonaill S OrsquoBrien C Olmsted SSchultz D Winpenny E Pincus HA Grant J Mental Health RetrosightUnderstanding the Returns from Research (lessons from schizophrenia)Policy Report Cambridge RAND Europe 2013 [httpwwwrandorgrandeuroperesearchprojectsmental-health-retrosighthtml AccessedJanuary 2014]

19 Johnston SC Rootenberg JD Katrak S Smith WS Elkins JS Effect of a USNational Institutes of Health programme of clinical trials on publichealth and costs Lancet 2006 3671319ndash1327

16 Jun 2014

20 Cancer Research UK Medical Research and the Spending Review 2010[httpinfocancerresearchukorgprod_consumpgroupscr_commonnrepoldocumentsgeneralcontentcr_048533pdf Accessed 20 May 2012]

21 Association of Medical Research Charities Written Submission to theComprehensive Spending Review 2010 [httpwwwamrcorgukpublicationssubmission-department-business-innovation-and-skills-part-comprehensive-spending-review Accessed January 2014]

22 Academy of Medical Sciences Initial Response to the Spending ReviewConsultation 2010 [httpwwwacmedsciacukdownloadphpfile=imagespublication128393883472pdf Accessed May 2012]

23 European Medical Research Councils White Paper II A Stronger BiomedicalResearch for a Better European Future Strasbourg European ScienceFoundation 2011

24 Guthrie S Grant J Hanney S Pollitt A Investigating time lags and attributionin the translation of cancer research A case study approach CambridgeRAND Europe 2014

25 Dawson G Lucocq B Cottrell R Lewison G Mapping the Landscape NationalBiomedical Research Outputs 1988ndash95 London Wellcome Trust 1998

26 Wellcome Trust Putting NHS Research on the Map An Analysis of ScientificPublications in England 1990ndash97 London Wellcome Trust 2001

27 Capewell S Critchley J Unal B IMPACT a Validated Comprehensive CoronaryHeart Disease Model Overview and Technical Appendices Liverpool Universityof Liverpool 2007

28 CRUK statistics [httpwwwcancerresearchukorg AccessedDecember 2013]

29 ONS statistics MB1 series no 32 [wwwonsgovuk AccessedDecember 2013]

30 Karim-Kos HE de Vries E Soerjomataram I Lemmens V Siesling S CoeberghJWW Recent trends of cancer in Europe a combined approach ofincidence survival and mortality for 17 cancer sites since the 1990s EurJ Cancer 2008 441345ndash1389

31 National Cancer Intelligence Network Incidence Stomach Cancer Engl1998ndash2007 [httpwwwncinorgukpublicationsdata_briefingsincidence_of_stomach_cancer_in_england Accessed February 2014]

32 Bine Kjoslashller B Raaschou-Nielsen O Soslashrensen M Frederiksen K Christensen JTjoslashnneland A Overvad K Chapelon FC Nagel G Chang-Claude J BergmannMM Boeing H Trichopoulos D Trichopoulou A Oikonomou E Berrino FPalli D Tumino R Vineis P Panico S Peeters PHM Bueno-de-Mesquita HBKiemeney L Gram IT Braaten T Lund E Gonzalez CA Berglund G Allen NRoddam A et al Tobacco smoke and bladder cancermdashin the EuropeanProspective Investigation into Cancer and Nutrition Int J Cancer 20061192412ndash2416

33 Independent UK Panel on Breast Cancer Screening The benefits andharms of breast cancer screening an independent review Lancet 20123801778ndash1786

34 Libby G Brewster DH McClements PL Carey FA Black RJ Birrell J Fraser CGSteele RJC The impact of population-based faecal occult blood testscreening on colorectal cancer mortality a matched cohort study Br JCancer 2012 107255ndash259

35 Canfell K Freddy S Beral V Cervical cancer in Australia and the UnitedKingdom comparison of screening policy and uptake and cancerincidence and mortality Med J Aust 2006 185482

36 CancerStats ndash Survival [httppublicationscancerresearchukorgdownloadsProductCS_CS_SURVIVALpdf Accessed December 2013]

37 Cancer survival in England - patients diagnosed 2005ndash2009 and followedup to 2010 [httpwwwonsgovukonsrelcancer-unitcancer-survival2005-2009ndashfollowed-up-to-2010stb-cancer-survivalndash2005-09-and-followed-up-to-2010html Accessed December 2013]

38 Godfrey C Shezad A Parrott S Pickett K Economic model of adult smokingrelated costs and consequences for England October 2010 (Revised April 2011)York Public Health Research Consortium Project Report 2011 [httpphrclshtmacukpapersPHRC_A4-06_Final_Reportpdf]

39 Kind P Hardman G Macran S UK population norms for EQ-5D Discussionpaper 182 York University of York 1999 [httpwwwyorkacukmediachedocumentspapersdiscussionpapersCHE20Discussion20Paper20172pdf Accessed January 2014]

40 Karnon J Peters J Platt J Chilcott J McGoogan E Brewer N Liquid-basedcytology in cervical screening an updated rapid and systematic reviewand economic analysis Health Technol Assess 2004 81ndash78 Table 21

41 Whyte S Chilcott J Cooper K Essat M Stevens J Wong R Kalita NReappraisal of the options for colorectal cancer screening Full report for

the NHS Bowel Cancer Screening Programme Sheffield ScHARRUniversity of Sheffield 2011 Table 421

42 Pharoah PD Sewell B Fitzsimmons D Bennett HS Pashayan N Costeffectiveness of the NHS breast screening programme life table modelBMJ 2013 346f2618

43 National Institute for Health and Clinical Excellence Colorectal Cancer TheDiagnosis and Management of Colorectal Cancer CG 131 2011[httpguidanceniceorgukCG131 Accessed December 2013]

44 National Institute for Health and Clinical Excellence Early and LocallyAdvanced Breast Cancer Diagnosis and Treatment CG 80 2009[httpguidanceniceorgukCG80 Accessed December 2013]

45 National Institute for Health and Clinical Excellence Advanced Breast CancerDiagnosis and Treatment CG 81 2009 [httppublicationsniceorgukadvanced-breast-cancer-cg81 Accessed December 2013]

46 National Institute for Health and Clinical Excellence Prostate cancerDiagnosis and Treatment CG58 2008 [httppublicationsniceorgukprostate-cancer-cg58 Accessed December 2013]

47 Health and Social Care Information Centre England Statistics onSmoking 2012 Table 22 [httpwwwhscicgovukcataloguePUB07019smok-eng-2012-reppdf Accessed February 2014]

48 Hospital Episodes Statistics [httpwwwhscicgovukhes AccessedJanuary 2014]

49 Health and Social Care information Centre [httpwwwhscicgovukAccessed January 2014]

50 PSSRU Unit Costs of Health and Social Care 2012 [httpwwwpssruacukproject-pagesunit-costs2012 Accessed January 2014]

51 Guide to the methods of technology appraisal 2008 [httpwwwniceorgukaboutnicehowweworkdevnicetechGuideToMethodsTechnologyAppraisal2008jsp Accessed February 2014]

52 Guide to the methods of technology appraisal 2013 [httppublicationsniceorgukguide-to-the-methods-of-technology-appraisal-2013-pmg9Accessed January 2014]

53 World Health Organisation Macroeconomics and Health Investing in Healthfor Economic Development Report of the Commission on Macroeconomics forHealth 2001 [httpwhqlibdocwhointpublications2001924154550xpdfAccessed May 2014]

54 Grant J Cottrell R Cluzeau F Fawcett G Evaluating ldquopaybackrdquo onbiomedical research from papers cited in clinical guidelines appliedbibliometric study BMJ 2000 3201107ndash1111

55 Murray CJL Richards MA Newton JN Fenton KA Anderson HR Atkinson CBennett D Bernabeacute E Blencowe H Bourne R Braithwaite T Brayne C BruceNG Brugha TS Burney P Dherani M Dolk H Edmond K Ezzati M FlaxmanAD Fleming TD Freedman G Gunnell D Hay RJ Hutchings SJ Ohno SLLozano R Lyons RA Marcenes W Naghavi M et al UK health performancefindings of the Global Burden of Disease Study 2010 Lancet 2013381997ndash1020

56 Department of Health An Audit of the Impact of the Department of HealthrsquosRegulations upon Businesses Final Report September 2013 [httpswwwgovukgovernmentpublicationsdepartment-of-health-regulations-impact-on-business Accessed January 2014]

57 The National Cancer Research Institute [httpwwwncriorguk]58 The Centre for Science and Technology Studies [httpwww

socialsciencesleideneducwts]59 Her Majesties Treasury GDP data [httpwwwhm-treasurygovuk

data_gdp_fightm]60 The National Institute of Health price index [httpofficeofbudgetodnih

govgbiPriceIndexeshtml]

Cite this article as Glover et al Estimating the returns to UK publiclyfunded cancer-related research in terms of the net value of improvedhealth outcomes BMC Medicine

1011861741-7015-12-99

2014 1299

Submit your next manuscript to BioMed Centraland take full advantage of

bull Convenient online submission

bull Thorough peer review

bull No space constraints or color figure charges

bull Immediate publication on acceptance

bull Inclusion in PubMed CAS Scopus and Google Scholar

bull Research which is freely available for redistribution

Submit your manuscript at wwwbiomedcentralcomsubmit

Abstract

Background
Methods
Results
Conclusions
Background
Estimating the returns from biomedical and health research
Methods
Overall conceptual approach
Estimating public and charitable funding of cancer-related research
Estimating the NMB from cancer-related research
Identifying the key cancer interventions
Identifying estimates of per-patient NMB for the set of cancer interventions
Smoking preventioncessation
Screening programmes
Treatment programmes
Constructing a time series (1991 to 2010) of usage of cancer interventions
Analysis of UK clinical guidelines to estimate elapsed time and rate of attribution
Estimation of the rate of return
Results
Public and charitable funding of UK cancer-related research 1970 to 2011
Net monetary benefit
Estimating the elapsed time
Estimating the amount of health gains that can be attributed to UK research
Estimating the IRR from cancer-related research
Discussion
What this paper contributes
Key assumptions and caveats
Future research requirements
Conclusion
Endnotes
Additional files
Abbreviations
Competing interests
Authorsrsquo contributions
Acknowledgements
Author details
References

Glover et al BMC Medicine 2014 1299httpwwwbiomedcentralcom1741-70151299

RESEARCH ARTICLE Open Access

Estimating the returns to UK publicly fundedcancer-related research in terms of the net valueof improved health outcomesMatthew Glover1 Martin Buxton1 Susan Guthrie2 Stephen Hanney1 Alexandra Pollitt2 and Jonathan Grant23

Abstract

Background Building on an approach developed to assess the economic returns to cardiovascular research weestimated the economic returns from UK public and charitable funded cancer-related research that arise from thenet value of the improved health outcomes

Methods To assess these economic returns from cancer-related research in the UK we estimated 1) public andcharitable expenditure on cancer-related research in the UK from 1970 to 2009 2) net monetary benefit (NMB) thatis the health benefit measured in quality adjusted life years (QALYs) valued in monetary terms (using a base-casevalue of a QALY of GBpound25000) minus the cost of delivering that benefit for a prioritised list of interventions from1991 to 2010 3) the proportion of NMB attributable to UK research 4) the elapsed time between research fundingand health gain and 5) the internal rate of return (IRR) from cancer-related research investments on health benefitsWe analysed the uncertainties in the IRR estimate using sensitivity analyses to illustrate the effect of some keyparameters

Results In 201112 prices total expenditure on cancer-related research from 1970 to 2009 was pound15 billion TheNMB of the 59 million QALYs gained from the prioritised interventions from 1991 to 2010 was pound124 billionCalculation of the IRR incorporated an estimated elapsed time of 15 years We related 17 of the annual NMBestimated to be attributable to UK research (for each of the 20 years 1991 to 2010) to 20 years of researchinvestment 15 years earlier (that is for 1976 to 1995) This produced a best-estimate IRR of 10 compared with 9previously estimated for cardiovascular disease research The sensitivity analysis demonstrated the importance ofsmoking reduction as a major source of improved cancer-related health outcomes

Conclusions We have demonstrated a substantive IRR from net health gain to public and charitable funding ofcancer-related research in the UK and further validated the approach that we originally used in assessing thereturns from cardiovascular research In doing so we have highlighted a number of weaknesses and keyassumptions that need strengthening in further investigations Nevertheless these cautious estimatesdemonstrate that the returns from past cancer research have been substantial and justify the investmentsmade during the period 1976 to 1995

Keywords Medical research investment QALYs Cancer Medical research charities Value of health Rate of returnTime lags Research payback

Correspondence jonathangrantkclacuk2RAND Europe Westbrook Centre Milton Road Cambridge CB4 1YG UK3Kingrsquos Policy Institute Kingrsquos College London Virginia Woolf Building 22Kingsway London WC2R 2LA UKFull list of author information is available at the end of the article

copy Glover et al licensee BioMed Central Ltd This is an Open Access article distributed under the terms of the CreativeCommons Attribution License (httpcreativecommonsorglicensesby40) which permits unrestricted use distribution andreproduction in any medium provided the original work is properly cited

Glover

etalBM

CMedicine

202014 1299

httpww

QALYs (thousands)

1991 8 46 6 71 ndash 2 144 277

1992 9 48 6 70 ndash 2 144 279

1993 10 46 6 68 ndash 2 145 276

1994 11 48 6 67 ndash 2 145 279

1995 10 45 6 65 ndash 2 145 273

1996 11 46 6 66 ndash 2 146 276

1997 11 46 6 63 ndash 3 146 274

1998 11 50 6 59 ndash 2 147 276

1999 13 53 7 56 ndash 2 147 279

2000 15 53 7 55 ndash 2 148 281

2001 17 55 7 54 ndash 2 149 285

2002 20 56 8 53 ndash 2 150 290

2003 22 59 8 53 ndash 2 151 295

2004 24 61 9 56 ndash 2 152 305

2005 21 62 10 60 ndash 2 154 309

2006 22 62 12 61 2 2 155 316

2007 23 65 13 60 5 2 157 324

2008 27 68 14 60 7 2 158 337

2009 26 71 15 63 9 2 159 345

2010 25 74 15 65 12 3 161 354

Total 339 1112 173 1225 35 43 3003 5930

1991 199 665 181 41 ndash 34 -277 844

1992 220 687 190 40 ndash 37 -277 897

1993 241 658 185 39 ndash 40 -278 887

1994 272 684 185 39 ndash 42 -278 944

1995 252 646 175 37 ndash 42 -279 874

1996 278 660 182 38 ndash 43 -280 921

1997 269 684 181 36 ndash 53 -281 943

1998 283 720 187 34 ndash 50 -282 993

1999 336 753 214 32 ndash 47 -283 1098

2000 391 746 211 32 ndash 45 -285 1140

2001 456 755 206 31 ndash 43 -287 1204

2002 519 764 202 30 ndash 43 -282 1276

2003 571 794 178 30 ndash 44 -255 1361

2004 614 817 179 32 ndash 44 -254 1432

2005 545 850 182 34 ndash 44 -252 1403

2006 572 851 188 35 -5 45 -258 1428

2007 596 881 177 35 -10 46 -252 1473

2008 613 919 181 35 -15 48 -242 1538

2009 606 950 185 36 -20 48 -237 1569

2010 569 986 186 38 -25 56 -240 1569

Total 8403 15469 3755 704 -75 894 -5358 23793

1991 8 490 -33 1729 ndash 7 3885 6085

1992 8 506 -35 1699 ndash 7 3889 6075

1993 9 485 -33 1659 ndash 8 3890 6018

1994 10 504 -32 1646 ndash 8 3894 6030

1995 9 475 -28 1592 ndash 8 3906 5963

1996 -3 483 -28 1609 ndash 9 3920 5989

1997 -3 458 -28 1546 ndash 11 3933 5916

1998 -3 531 -29 1447 ndash 10 3947 5902

1999 -10 566 -40 1374 ndash 9 3966 5866

2000 -15 579 -36 1350 ndash 9 3989 5876

2001 -25 630 -30 1310 ndash 9 4018 5910

2002 -8 647 -5 1283 ndash 9 4037 5963

2003 -14 677 24 1294 ndash 9 4035 6025

2004 -7 705 49 1376 ndash 9 4064 6195

2005 -15 694 67 1460 ndash 9 4099 6314

2006 -16 697 106 1484 64 9 4138 6481

2007 -23 736 149 1469 128 9 4167 6635

2008 51 786 171 1474 192 10 4192 6876

2009 46 816 178 1528 256 10 4216 7050

2010 66 854 183 1596 320 11 4253 7282

Total 65 12318 566 29927 960 179 80437 124452

Analysis IRR

Base casea 101

Low 108

High 87

20000 80

30000 117

50000 161

70000 189

Elapsed time years

10 146

25 130

Additional files

Funding data

Guidelines data

Additional file 1

Additional file 2

Additional file 3

Additional file 4

16 Jun 2014

1011861741-7015-12-99

2014 1299

Abstract

Background
Methods
Results
Conclusions
Background
Methods
Results
Discussion
Conclusion
Endnotes
Additional files
Abbreviations
Competing interests
Acknowledgements
Author details
References

Glover

etalBM

CMedicine

202014 1299

httpww

QALYs (thousands)

1991 8 46 6 71 ndash 2 144 277

1992 9 48 6 70 ndash 2 144 279

1993 10 46 6 68 ndash 2 145 276

1994 11 48 6 67 ndash 2 145 279

1995 10 45 6 65 ndash 2 145 273

1996 11 46 6 66 ndash 2 146 276

1997 11 46 6 63 ndash 3 146 274

1998 11 50 6 59 ndash 2 147 276

1999 13 53 7 56 ndash 2 147 279

2000 15 53 7 55 ndash 2 148 281

2001 17 55 7 54 ndash 2 149 285

2002 20 56 8 53 ndash 2 150 290

2003 22 59 8 53 ndash 2 151 295

2004 24 61 9 56 ndash 2 152 305

2005 21 62 10 60 ndash 2 154 309

2006 22 62 12 61 2 2 155 316

2007 23 65 13 60 5 2 157 324

2008 27 68 14 60 7 2 158 337

2009 26 71 15 63 9 2 159 345

2010 25 74 15 65 12 3 161 354

Total 339 1112 173 1225 35 43 3003 5930

1991 199 665 181 41 ndash 34 -277 844

1992 220 687 190 40 ndash 37 -277 897

1993 241 658 185 39 ndash 40 -278 887

1994 272 684 185 39 ndash 42 -278 944

1995 252 646 175 37 ndash 42 -279 874

1996 278 660 182 38 ndash 43 -280 921

1997 269 684 181 36 ndash 53 -281 943

1998 283 720 187 34 ndash 50 -282 993

1999 336 753 214 32 ndash 47 -283 1098

2000 391 746 211 32 ndash 45 -285 1140

2001 456 755 206 31 ndash 43 -287 1204

2002 519 764 202 30 ndash 43 -282 1276

2003 571 794 178 30 ndash 44 -255 1361

2004 614 817 179 32 ndash 44 -254 1432

2005 545 850 182 34 ndash 44 -252 1403

2006 572 851 188 35 -5 45 -258 1428

2007 596 881 177 35 -10 46 -252 1473

2008 613 919 181 35 -15 48 -242 1538

2009 606 950 185 36 -20 48 -237 1569

2010 569 986 186 38 -25 56 -240 1569

Total 8403 15469 3755 704 -75 894 -5358 23793

1991 8 490 -33 1729 ndash 7 3885 6085

1992 8 506 -35 1699 ndash 7 3889 6075

1993 9 485 -33 1659 ndash 8 3890 6018

1994 10 504 -32 1646 ndash 8 3894 6030

1995 9 475 -28 1592 ndash 8 3906 5963

1996 -3 483 -28 1609 ndash 9 3920 5989

1997 -3 458 -28 1546 ndash 11 3933 5916

1998 -3 531 -29 1447 ndash 10 3947 5902

1999 -10 566 -40 1374 ndash 9 3966 5866

2000 -15 579 -36 1350 ndash 9 3989 5876

2001 -25 630 -30 1310 ndash 9 4018 5910

2002 -8 647 -5 1283 ndash 9 4037 5963

2003 -14 677 24 1294 ndash 9 4035 6025

2004 -7 705 49 1376 ndash 9 4064 6195

2005 -15 694 67 1460 ndash 9 4099 6314

2006 -16 697 106 1484 64 9 4138 6481

2007 -23 736 149 1469 128 9 4167 6635

2008 51 786 171 1474 192 10 4192 6876

2009 46 816 178 1528 256 10 4216 7050

2010 66 854 183 1596 320 11 4253 7282

Total 65 12318 566 29927 960 179 80437 124452

Analysis IRR

Base casea 101

Low 108

High 87

20000 80

30000 117

50000 161

70000 189

Elapsed time years

10 146

25 130

Additional files

Funding data

Guidelines data

Additional file 1

Additional file 2

Additional file 3

Additional file 4

16 Jun 2014

1011861741-7015-12-99

2014 1299

Abstract

Background
Methods
Results
Conclusions
Background
Methods
Results
Discussion
Conclusion
Endnotes
Additional files
Abbreviations
Competing interests
Acknowledgements
Author details
References

Glover

etalBM

CMedicine

202014 1299

httpww

QALYs (thousands)

1991 8 46 6 71 ndash 2 144 277

1992 9 48 6 70 ndash 2 144 279

1993 10 46 6 68 ndash 2 145 276

1994 11 48 6 67 ndash 2 145 279

1995 10 45 6 65 ndash 2 145 273

1996 11 46 6 66 ndash 2 146 276

1997 11 46 6 63 ndash 3 146 274

1998 11 50 6 59 ndash 2 147 276

1999 13 53 7 56 ndash 2 147 279

2000 15 53 7 55 ndash 2 148 281

2001 17 55 7 54 ndash 2 149 285

2002 20 56 8 53 ndash 2 150 290

2003 22 59 8 53 ndash 2 151 295

2004 24 61 9 56 ndash 2 152 305

2005 21 62 10 60 ndash 2 154 309

2006 22 62 12 61 2 2 155 316

2007 23 65 13 60 5 2 157 324

2008 27 68 14 60 7 2 158 337

2009 26 71 15 63 9 2 159 345

2010 25 74 15 65 12 3 161 354

Total 339 1112 173 1225 35 43 3003 5930

1991 199 665 181 41 ndash 34 -277 844

1992 220 687 190 40 ndash 37 -277 897

1993 241 658 185 39 ndash 40 -278 887

1994 272 684 185 39 ndash 42 -278 944

1995 252 646 175 37 ndash 42 -279 874

1996 278 660 182 38 ndash 43 -280 921

1997 269 684 181 36 ndash 53 -281 943

1998 283 720 187 34 ndash 50 -282 993

1999 336 753 214 32 ndash 47 -283 1098

2000 391 746 211 32 ndash 45 -285 1140

2001 456 755 206 31 ndash 43 -287 1204

2002 519 764 202 30 ndash 43 -282 1276

2003 571 794 178 30 ndash 44 -255 1361

2004 614 817 179 32 ndash 44 -254 1432

2005 545 850 182 34 ndash 44 -252 1403

2006 572 851 188 35 -5 45 -258 1428

2007 596 881 177 35 -10 46 -252 1473

2008 613 919 181 35 -15 48 -242 1538

2009 606 950 185 36 -20 48 -237 1569

2010 569 986 186 38 -25 56 -240 1569

Total 8403 15469 3755 704 -75 894 -5358 23793

1991 8 490 -33 1729 ndash 7 3885 6085

1992 8 506 -35 1699 ndash 7 3889 6075

1993 9 485 -33 1659 ndash 8 3890 6018

1994 10 504 -32 1646 ndash 8 3894 6030

1995 9 475 -28 1592 ndash 8 3906 5963

1996 -3 483 -28 1609 ndash 9 3920 5989

1997 -3 458 -28 1546 ndash 11 3933 5916

1998 -3 531 -29 1447 ndash 10 3947 5902

1999 -10 566 -40 1374 ndash 9 3966 5866

2000 -15 579 -36 1350 ndash 9 3989 5876

2001 -25 630 -30 1310 ndash 9 4018 5910

2002 -8 647 -5 1283 ndash 9 4037 5963

2003 -14 677 24 1294 ndash 9 4035 6025

2004 -7 705 49 1376 ndash 9 4064 6195

2005 -15 694 67 1460 ndash 9 4099 6314

2006 -16 697 106 1484 64 9 4138 6481

2007 -23 736 149 1469 128 9 4167 6635

2008 51 786 171 1474 192 10 4192 6876

2009 46 816 178 1528 256 10 4216 7050

2010 66 854 183 1596 320 11 4253 7282

Total 65 12318 566 29927 960 179 80437 124452

Analysis IRR

Base casea 101

Low 108

High 87

20000 80

30000 117

50000 161

70000 189

Elapsed time years

10 146

25 130

Additional files

Funding data

Guidelines data

Additional file 1

Additional file 2

Additional file 3

Additional file 4

16 Jun 2014

1011861741-7015-12-99

2014 1299

Abstract

Background
Methods
Results
Conclusions
Background
Methods
Results
Discussion
Conclusion
Endnotes
Additional files
Abbreviations
Competing interests
Acknowledgements
Author details
References

QALYs (thousands)

1991 8 46 6 71 ndash 2 144 277

1992 9 48 6 70 ndash 2 144 279

1993 10 46 6 68 ndash 2 145 276

1994 11 48 6 67 ndash 2 145 279

1995 10 45 6 65 ndash 2 145 273

1996 11 46 6 66 ndash 2 146 276

1997 11 46 6 63 ndash 3 146 274

1998 11 50 6 59 ndash 2 147 276

1999 13 53 7 56 ndash 2 147 279

2000 15 53 7 55 ndash 2 148 281

2001 17 55 7 54 ndash 2 149 285

2002 20 56 8 53 ndash 2 150 290

2003 22 59 8 53 ndash 2 151 295

2004 24 61 9 56 ndash 2 152 305

2005 21 62 10 60 ndash 2 154 309

2006 22 62 12 61 2 2 155 316

2007 23 65 13 60 5 2 157 324

2008 27 68 14 60 7 2 158 337

2009 26 71 15 63 9 2 159 345

2010 25 74 15 65 12 3 161 354

Total 339 1112 173 1225 35 43 3003 5930

1991 199 665 181 41 ndash 34 -277 844

1992 220 687 190 40 ndash 37 -277 897

1993 241 658 185 39 ndash 40 -278 887

1994 272 684 185 39 ndash 42 -278 944

1995 252 646 175 37 ndash 42 -279 874

1996 278 660 182 38 ndash 43 -280 921

1997 269 684 181 36 ndash 53 -281 943

1998 283 720 187 34 ndash 50 -282 993

1999 336 753 214 32 ndash 47 -283 1098

2000 391 746 211 32 ndash 45 -285 1140

2001 456 755 206 31 ndash 43 -287 1204

2002 519 764 202 30 ndash 43 -282 1276

2003 571 794 178 30 ndash 44 -255 1361

2004 614 817 179 32 ndash 44 -254 1432

2005 545 850 182 34 ndash 44 -252 1403

2006 572 851 188 35 -5 45 -258 1428

2007 596 881 177 35 -10 46 -252 1473

2008 613 919 181 35 -15 48 -242 1538

2009 606 950 185 36 -20 48 -237 1569

2010 569 986 186 38 -25 56 -240 1569

Total 8403 15469 3755 704 -75 894 -5358 23793

1991 8 490 -33 1729 ndash 7 3885 6085

1992 8 506 -35 1699 ndash 7 3889 6075

1993 9 485 -33 1659 ndash 8 3890 6018

1994 10 504 -32 1646 ndash 8 3894 6030

1995 9 475 -28 1592 ndash 8 3906 5963

1996 -3 483 -28 1609 ndash 9 3920 5989

1997 -3 458 -28 1546 ndash 11 3933 5916

1998 -3 531 -29 1447 ndash 10 3947 5902

1999 -10 566 -40 1374 ndash 9 3966 5866

2000 -15 579 -36 1350 ndash 9 3989 5876

2001 -25 630 -30 1310 ndash 9 4018 5910

2002 -8 647 -5 1283 ndash 9 4037 5963

2003 -14 677 24 1294 ndash 9 4035 6025

2004 -7 705 49 1376 ndash 9 4064 6195

2005 -15 694 67 1460 ndash 9 4099 6314

2006 -16 697 106 1484 64 9 4138 6481

2007 -23 736 149 1469 128 9 4167 6635

2008 51 786 171 1474 192 10 4192 6876

2009 46 816 178 1528 256 10 4216 7050

2010 66 854 183 1596 320 11 4253 7282

Total 65 12318 566 29927 960 179 80437 124452

Analysis IRR

Base casea 101

Low 108

High 87

20000 80

30000 117

50000 161

70000 189

Elapsed time years

10 146

25 130

Additional files

Funding data

Guidelines data

Additional file 1

Additional file 2

Additional file 3

Additional file 4

16 Jun 2014

1011861741-7015-12-99

2014 1299

Abstract

Background
Methods
Results
Conclusions
Background
Methods
Results
Discussion
Conclusion
Endnotes
Additional files
Abbreviations
Competing interests
Acknowledgements
Author details
References

QALYs (thousands)

1991 8 46 6 71 ndash 2 144 277

1992 9 48 6 70 ndash 2 144 279

1993 10 46 6 68 ndash 2 145 276

1994 11 48 6 67 ndash 2 145 279

1995 10 45 6 65 ndash 2 145 273

1996 11 46 6 66 ndash 2 146 276

1997 11 46 6 63 ndash 3 146 274

1998 11 50 6 59 ndash 2 147 276

1999 13 53 7 56 ndash 2 147 279

2000 15 53 7 55 ndash 2 148 281

2001 17 55 7 54 ndash 2 149 285

2002 20 56 8 53 ndash 2 150 290

2003 22 59 8 53 ndash 2 151 295

2004 24 61 9 56 ndash 2 152 305

2005 21 62 10 60 ndash 2 154 309

2006 22 62 12 61 2 2 155 316

2007 23 65 13 60 5 2 157 324

2008 27 68 14 60 7 2 158 337

2009 26 71 15 63 9 2 159 345

2010 25 74 15 65 12 3 161 354

Total 339 1112 173 1225 35 43 3003 5930

1991 199 665 181 41 ndash 34 -277 844

1992 220 687 190 40 ndash 37 -277 897

1993 241 658 185 39 ndash 40 -278 887

1994 272 684 185 39 ndash 42 -278 944

1995 252 646 175 37 ndash 42 -279 874

1996 278 660 182 38 ndash 43 -280 921

1997 269 684 181 36 ndash 53 -281 943

1998 283 720 187 34 ndash 50 -282 993

1999 336 753 214 32 ndash 47 -283 1098

2000 391 746 211 32 ndash 45 -285 1140

2001 456 755 206 31 ndash 43 -287 1204

2002 519 764 202 30 ndash 43 -282 1276

2003 571 794 178 30 ndash 44 -255 1361

2004 614 817 179 32 ndash 44 -254 1432

2005 545 850 182 34 ndash 44 -252 1403

2006 572 851 188 35 -5 45 -258 1428

2007 596 881 177 35 -10 46 -252 1473

2008 613 919 181 35 -15 48 -242 1538

2009 606 950 185 36 -20 48 -237 1569

2010 569 986 186 38 -25 56 -240 1569

Total 8403 15469 3755 704 -75 894 -5358 23793

1991 8 490 -33 1729 ndash 7 3885 6085

1992 8 506 -35 1699 ndash 7 3889 6075

1993 9 485 -33 1659 ndash 8 3890 6018

1994 10 504 -32 1646 ndash 8 3894 6030

1995 9 475 -28 1592 ndash 8 3906 5963

1996 -3 483 -28 1609 ndash 9 3920 5989

1997 -3 458 -28 1546 ndash 11 3933 5916

1998 -3 531 -29 1447 ndash 10 3947 5902

1999 -10 566 -40 1374 ndash 9 3966 5866

2000 -15 579 -36 1350 ndash 9 3989 5876

2001 -25 630 -30 1310 ndash 9 4018 5910

2002 -8 647 -5 1283 ndash 9 4037 5963

2003 -14 677 24 1294 ndash 9 4035 6025

2004 -7 705 49 1376 ndash 9 4064 6195

2005 -15 694 67 1460 ndash 9 4099 6314

2006 -16 697 106 1484 64 9 4138 6481

2007 -23 736 149 1469 128 9 4167 6635

2008 51 786 171 1474 192 10 4192 6876

2009 46 816 178 1528 256 10 4216 7050

2010 66 854 183 1596 320 11 4253 7282

Total 65 12318 566 29927 960 179 80437 124452

Analysis IRR

Base casea 101

Low 108

High 87

20000 80

30000 117

50000 161

70000 189

Elapsed time years

10 146

25 130

Additional files

Funding data

Guidelines data

Additional file 1

Additional file 2

Additional file 3

Additional file 4

16 Jun 2014

1011861741-7015-12-99

2014 1299

Abstract

Background
Methods
Results
Conclusions
Background
Methods
Results
Discussion
Conclusion
Endnotes
Additional files
Abbreviations
Competing interests
Acknowledgements
Author details
References

QALYs (thousands)

1991 8 46 6 71 ndash 2 144 277

1992 9 48 6 70 ndash 2 144 279

1993 10 46 6 68 ndash 2 145 276

1994 11 48 6 67 ndash 2 145 279

1995 10 45 6 65 ndash 2 145 273

1996 11 46 6 66 ndash 2 146 276

1997 11 46 6 63 ndash 3 146 274

1998 11 50 6 59 ndash 2 147 276

1999 13 53 7 56 ndash 2 147 279

2000 15 53 7 55 ndash 2 148 281

2001 17 55 7 54 ndash 2 149 285

2002 20 56 8 53 ndash 2 150 290

2003 22 59 8 53 ndash 2 151 295

2004 24 61 9 56 ndash 2 152 305

2005 21 62 10 60 ndash 2 154 309

2006 22 62 12 61 2 2 155 316

2007 23 65 13 60 5 2 157 324

2008 27 68 14 60 7 2 158 337

2009 26 71 15 63 9 2 159 345

2010 25 74 15 65 12 3 161 354

Total 339 1112 173 1225 35 43 3003 5930

1991 199 665 181 41 ndash 34 -277 844

1992 220 687 190 40 ndash 37 -277 897

1993 241 658 185 39 ndash 40 -278 887

1994 272 684 185 39 ndash 42 -278 944

1995 252 646 175 37 ndash 42 -279 874

1996 278 660 182 38 ndash 43 -280 921

1997 269 684 181 36 ndash 53 -281 943

1998 283 720 187 34 ndash 50 -282 993

1999 336 753 214 32 ndash 47 -283 1098

2000 391 746 211 32 ndash 45 -285 1140

2001 456 755 206 31 ndash 43 -287 1204

2002 519 764 202 30 ndash 43 -282 1276

2003 571 794 178 30 ndash 44 -255 1361

2004 614 817 179 32 ndash 44 -254 1432

2005 545 850 182 34 ndash 44 -252 1403

2006 572 851 188 35 -5 45 -258 1428

2007 596 881 177 35 -10 46 -252 1473

2008 613 919 181 35 -15 48 -242 1538

2009 606 950 185 36 -20 48 -237 1569

2010 569 986 186 38 -25 56 -240 1569

Total 8403 15469 3755 704 -75 894 -5358 23793

1991 8 490 -33 1729 ndash 7 3885 6085

1992 8 506 -35 1699 ndash 7 3889 6075

1993 9 485 -33 1659 ndash 8 3890 6018

1994 10 504 -32 1646 ndash 8 3894 6030

1995 9 475 -28 1592 ndash 8 3906 5963

1996 -3 483 -28 1609 ndash 9 3920 5989

1997 -3 458 -28 1546 ndash 11 3933 5916

1998 -3 531 -29 1447 ndash 10 3947 5902

1999 -10 566 -40 1374 ndash 9 3966 5866

2000 -15 579 -36 1350 ndash 9 3989 5876

2001 -25 630 -30 1310 ndash 9 4018 5910

2002 -8 647 -5 1283 ndash 9 4037 5963

2003 -14 677 24 1294 ndash 9 4035 6025

2004 -7 705 49 1376 ndash 9 4064 6195

2005 -15 694 67 1460 ndash 9 4099 6314

2006 -16 697 106 1484 64 9 4138 6481

2007 -23 736 149 1469 128 9 4167 6635

2008 51 786 171 1474 192 10 4192 6876

2009 46 816 178 1528 256 10 4216 7050

2010 66 854 183 1596 320 11 4253 7282

Total 65 12318 566 29927 960 179 80437 124452

Analysis IRR

Base casea 101

Low 108

High 87

20000 80

30000 117

50000 161

70000 189

Elapsed time years

10 146

25 130

Additional files

Funding data

Guidelines data

Additional file 1

Additional file 2

Additional file 3

Additional file 4

16 Jun 2014

1011861741-7015-12-99

2014 1299

Abstract

Background
Methods
Results
Conclusions
Background
Methods
Results
Discussion
Conclusion
Endnotes
Additional files
Abbreviations
Competing interests
Acknowledgements
Author details
References

QALYs (thousands)

1991 8 46 6 71 ndash 2 144 277

1992 9 48 6 70 ndash 2 144 279

1993 10 46 6 68 ndash 2 145 276

1994 11 48 6 67 ndash 2 145 279

1995 10 45 6 65 ndash 2 145 273

1996 11 46 6 66 ndash 2 146 276

1997 11 46 6 63 ndash 3 146 274

1998 11 50 6 59 ndash 2 147 276

1999 13 53 7 56 ndash 2 147 279

2000 15 53 7 55 ndash 2 148 281

2001 17 55 7 54 ndash 2 149 285

2002 20 56 8 53 ndash 2 150 290

2003 22 59 8 53 ndash 2 151 295

2004 24 61 9 56 ndash 2 152 305

2005 21 62 10 60 ndash 2 154 309

2006 22 62 12 61 2 2 155 316

2007 23 65 13 60 5 2 157 324

2008 27 68 14 60 7 2 158 337

2009 26 71 15 63 9 2 159 345

2010 25 74 15 65 12 3 161 354

Total 339 1112 173 1225 35 43 3003 5930

1991 199 665 181 41 ndash 34 -277 844

1992 220 687 190 40 ndash 37 -277 897

1993 241 658 185 39 ndash 40 -278 887

1994 272 684 185 39 ndash 42 -278 944

1995 252 646 175 37 ndash 42 -279 874

1996 278 660 182 38 ndash 43 -280 921

1997 269 684 181 36 ndash 53 -281 943

1998 283 720 187 34 ndash 50 -282 993

1999 336 753 214 32 ndash 47 -283 1098

2000 391 746 211 32 ndash 45 -285 1140

2001 456 755 206 31 ndash 43 -287 1204

2002 519 764 202 30 ndash 43 -282 1276

2003 571 794 178 30 ndash 44 -255 1361

2004 614 817 179 32 ndash 44 -254 1432

2005 545 850 182 34 ndash 44 -252 1403

2006 572 851 188 35 -5 45 -258 1428

2007 596 881 177 35 -10 46 -252 1473

2008 613 919 181 35 -15 48 -242 1538

2009 606 950 185 36 -20 48 -237 1569

2010 569 986 186 38 -25 56 -240 1569

Total 8403 15469 3755 704 -75 894 -5358 23793

1991 8 490 -33 1729 ndash 7 3885 6085

1992 8 506 -35 1699 ndash 7 3889 6075

1993 9 485 -33 1659 ndash 8 3890 6018

1994 10 504 -32 1646 ndash 8 3894 6030

1995 9 475 -28 1592 ndash 8 3906 5963

1996 -3 483 -28 1609 ndash 9 3920 5989

1997 -3 458 -28 1546 ndash 11 3933 5916

1998 -3 531 -29 1447 ndash 10 3947 5902

1999 -10 566 -40 1374 ndash 9 3966 5866

2000 -15 579 -36 1350 ndash 9 3989 5876

2001 -25 630 -30 1310 ndash 9 4018 5910

2002 -8 647 -5 1283 ndash 9 4037 5963

2003 -14 677 24 1294 ndash 9 4035 6025

2004 -7 705 49 1376 ndash 9 4064 6195

2005 -15 694 67 1460 ndash 9 4099 6314

2006 -16 697 106 1484 64 9 4138 6481

2007 -23 736 149 1469 128 9 4167 6635

2008 51 786 171 1474 192 10 4192 6876

2009 46 816 178 1528 256 10 4216 7050

2010 66 854 183 1596 320 11 4253 7282

Total 65 12318 566 29927 960 179 80437 124452

Analysis IRR

Base casea 101

Low 108

High 87

20000 80

30000 117

50000 161

70000 189

Elapsed time years

10 146

25 130

Additional files

Funding data

Guidelines data

Additional file 1

Additional file 2

Additional file 3

Additional file 4

16 Jun 2014

1011861741-7015-12-99

2014 1299

Abstract

Background
Methods
Results
Conclusions
Background
Methods
Results
Discussion
Conclusion
Endnotes
Additional files
Abbreviations
Competing interests
Acknowledgements
Author details
References

QALYs (thousands)

1991 8 46 6 71 ndash 2 144 277

1992 9 48 6 70 ndash 2 144 279

1993 10 46 6 68 ndash 2 145 276

1994 11 48 6 67 ndash 2 145 279

1995 10 45 6 65 ndash 2 145 273

1996 11 46 6 66 ndash 2 146 276

1997 11 46 6 63 ndash 3 146 274

1998 11 50 6 59 ndash 2 147 276

1999 13 53 7 56 ndash 2 147 279

2000 15 53 7 55 ndash 2 148 281

2001 17 55 7 54 ndash 2 149 285

2002 20 56 8 53 ndash 2 150 290

2003 22 59 8 53 ndash 2 151 295

2004 24 61 9 56 ndash 2 152 305

2005 21 62 10 60 ndash 2 154 309

2006 22 62 12 61 2 2 155 316

2007 23 65 13 60 5 2 157 324

2008 27 68 14 60 7 2 158 337

2009 26 71 15 63 9 2 159 345

2010 25 74 15 65 12 3 161 354

Total 339 1112 173 1225 35 43 3003 5930

1991 199 665 181 41 ndash 34 -277 844

1992 220 687 190 40 ndash 37 -277 897

1993 241 658 185 39 ndash 40 -278 887

1994 272 684 185 39 ndash 42 -278 944

1995 252 646 175 37 ndash 42 -279 874

1996 278 660 182 38 ndash 43 -280 921

1997 269 684 181 36 ndash 53 -281 943

1998 283 720 187 34 ndash 50 -282 993

1999 336 753 214 32 ndash 47 -283 1098

2000 391 746 211 32 ndash 45 -285 1140

2001 456 755 206 31 ndash 43 -287 1204

2002 519 764 202 30 ndash 43 -282 1276

2003 571 794 178 30 ndash 44 -255 1361

2004 614 817 179 32 ndash 44 -254 1432

2005 545 850 182 34 ndash 44 -252 1403

2006 572 851 188 35 -5 45 -258 1428

2007 596 881 177 35 -10 46 -252 1473

2008 613 919 181 35 -15 48 -242 1538

2009 606 950 185 36 -20 48 -237 1569

2010 569 986 186 38 -25 56 -240 1569

Total 8403 15469 3755 704 -75 894 -5358 23793

1991 8 490 -33 1729 ndash 7 3885 6085

1992 8 506 -35 1699 ndash 7 3889 6075

1993 9 485 -33 1659 ndash 8 3890 6018

1994 10 504 -32 1646 ndash 8 3894 6030

1995 9 475 -28 1592 ndash 8 3906 5963

1996 -3 483 -28 1609 ndash 9 3920 5989

1997 -3 458 -28 1546 ndash 11 3933 5916

1998 -3 531 -29 1447 ndash 10 3947 5902

1999 -10 566 -40 1374 ndash 9 3966 5866

2000 -15 579 -36 1350 ndash 9 3989 5876

2001 -25 630 -30 1310 ndash 9 4018 5910

2002 -8 647 -5 1283 ndash 9 4037 5963

2003 -14 677 24 1294 ndash 9 4035 6025

2004 -7 705 49 1376 ndash 9 4064 6195

2005 -15 694 67 1460 ndash 9 4099 6314

2006 -16 697 106 1484 64 9 4138 6481

2007 -23 736 149 1469 128 9 4167 6635

2008 51 786 171 1474 192 10 4192 6876

2009 46 816 178 1528 256 10 4216 7050

2010 66 854 183 1596 320 11 4253 7282

Total 65 12318 566 29927 960 179 80437 124452

Analysis IRR

Base casea 101

Low 108

High 87

20000 80

30000 117

50000 161

70000 189

Elapsed time years

10 146

25 130

Additional files

Funding data

Guidelines data

Additional file 1

Additional file 2

Additional file 3

Additional file 4

16 Jun 2014

1011861741-7015-12-99

2014 1299

Abstract

Background
Methods
Results
Conclusions
Background
Methods
Results
Discussion
Conclusion
Endnotes
Additional files
Abbreviations
Competing interests
Acknowledgements
Author details
References

QALYs (thousands)

1991 8 46 6 71 ndash 2 144 277

1992 9 48 6 70 ndash 2 144 279

1993 10 46 6 68 ndash 2 145 276

1994 11 48 6 67 ndash 2 145 279

1995 10 45 6 65 ndash 2 145 273

1996 11 46 6 66 ndash 2 146 276

1997 11 46 6 63 ndash 3 146 274

1998 11 50 6 59 ndash 2 147 276

1999 13 53 7 56 ndash 2 147 279

2000 15 53 7 55 ndash 2 148 281

2001 17 55 7 54 ndash 2 149 285

2002 20 56 8 53 ndash 2 150 290

2003 22 59 8 53 ndash 2 151 295

2004 24 61 9 56 ndash 2 152 305

2005 21 62 10 60 ndash 2 154 309

2006 22 62 12 61 2 2 155 316

2007 23 65 13 60 5 2 157 324

2008 27 68 14 60 7 2 158 337

2009 26 71 15 63 9 2 159 345

2010 25 74 15 65 12 3 161 354

Total 339 1112 173 1225 35 43 3003 5930

1991 199 665 181 41 ndash 34 -277 844

1992 220 687 190 40 ndash 37 -277 897

1993 241 658 185 39 ndash 40 -278 887

1994 272 684 185 39 ndash 42 -278 944

1995 252 646 175 37 ndash 42 -279 874

1996 278 660 182 38 ndash 43 -280 921

1997 269 684 181 36 ndash 53 -281 943

1998 283 720 187 34 ndash 50 -282 993

1999 336 753 214 32 ndash 47 -283 1098

2000 391 746 211 32 ndash 45 -285 1140

2001 456 755 206 31 ndash 43 -287 1204

2002 519 764 202 30 ndash 43 -282 1276

2003 571 794 178 30 ndash 44 -255 1361

2004 614 817 179 32 ndash 44 -254 1432

2005 545 850 182 34 ndash 44 -252 1403

2006 572 851 188 35 -5 45 -258 1428

2007 596 881 177 35 -10 46 -252 1473

2008 613 919 181 35 -15 48 -242 1538

2009 606 950 185 36 -20 48 -237 1569

2010 569 986 186 38 -25 56 -240 1569

Total 8403 15469 3755 704 -75 894 -5358 23793

1991 8 490 -33 1729 ndash 7 3885 6085

1992 8 506 -35 1699 ndash 7 3889 6075

1993 9 485 -33 1659 ndash 8 3890 6018

1994 10 504 -32 1646 ndash 8 3894 6030

1995 9 475 -28 1592 ndash 8 3906 5963

1996 -3 483 -28 1609 ndash 9 3920 5989

1997 -3 458 -28 1546 ndash 11 3933 5916

1998 -3 531 -29 1447 ndash 10 3947 5902

1999 -10 566 -40 1374 ndash 9 3966 5866

2000 -15 579 -36 1350 ndash 9 3989 5876

2001 -25 630 -30 1310 ndash 9 4018 5910

2002 -8 647 -5 1283 ndash 9 4037 5963

2003 -14 677 24 1294 ndash 9 4035 6025

2004 -7 705 49 1376 ndash 9 4064 6195

2005 -15 694 67 1460 ndash 9 4099 6314

2006 -16 697 106 1484 64 9 4138 6481

2007 -23 736 149 1469 128 9 4167 6635

2008 51 786 171 1474 192 10 4192 6876

2009 46 816 178 1528 256 10 4216 7050

2010 66 854 183 1596 320 11 4253 7282

Total 65 12318 566 29927 960 179 80437 124452

Analysis IRR

Base casea 101

Low 108

High 87

20000 80

30000 117

50000 161

70000 189

Elapsed time years

10 146

25 130

Additional files

Funding data

Guidelines data

Additional file 1

Additional file 2

Additional file 3

Additional file 4

16 Jun 2014

1011861741-7015-12-99

2014 1299

Abstract

Background
Methods
Results
Conclusions
Background
Methods
Results
Discussion
Conclusion
Endnotes
Additional files
Abbreviations
Competing interests
Acknowledgements
Author details
References

QALYs (thousands)

1991 8 46 6 71 ndash 2 144 277

1992 9 48 6 70 ndash 2 144 279

1993 10 46 6 68 ndash 2 145 276

1994 11 48 6 67 ndash 2 145 279

1995 10 45 6 65 ndash 2 145 273

1996 11 46 6 66 ndash 2 146 276

1997 11 46 6 63 ndash 3 146 274

1998 11 50 6 59 ndash 2 147 276

1999 13 53 7 56 ndash 2 147 279

2000 15 53 7 55 ndash 2 148 281

2001 17 55 7 54 ndash 2 149 285

2002 20 56 8 53 ndash 2 150 290

2003 22 59 8 53 ndash 2 151 295

2004 24 61 9 56 ndash 2 152 305

2005 21 62 10 60 ndash 2 154 309

2006 22 62 12 61 2 2 155 316

2007 23 65 13 60 5 2 157 324

2008 27 68 14 60 7 2 158 337

2009 26 71 15 63 9 2 159 345

2010 25 74 15 65 12 3 161 354

Total 339 1112 173 1225 35 43 3003 5930

1991 199 665 181 41 ndash 34 -277 844

1992 220 687 190 40 ndash 37 -277 897

1993 241 658 185 39 ndash 40 -278 887

1994 272 684 185 39 ndash 42 -278 944

1995 252 646 175 37 ndash 42 -279 874

1996 278 660 182 38 ndash 43 -280 921

1997 269 684 181 36 ndash 53 -281 943

1998 283 720 187 34 ndash 50 -282 993

1999 336 753 214 32 ndash 47 -283 1098

2000 391 746 211 32 ndash 45 -285 1140

2001 456 755 206 31 ndash 43 -287 1204

2002 519 764 202 30 ndash 43 -282 1276

2003 571 794 178 30 ndash 44 -255 1361

2004 614 817 179 32 ndash 44 -254 1432

2005 545 850 182 34 ndash 44 -252 1403

2006 572 851 188 35 -5 45 -258 1428

2007 596 881 177 35 -10 46 -252 1473

2008 613 919 181 35 -15 48 -242 1538

2009 606 950 185 36 -20 48 -237 1569

2010 569 986 186 38 -25 56 -240 1569

Total 8403 15469 3755 704 -75 894 -5358 23793

1991 8 490 -33 1729 ndash 7 3885 6085

1992 8 506 -35 1699 ndash 7 3889 6075

1993 9 485 -33 1659 ndash 8 3890 6018

1994 10 504 -32 1646 ndash 8 3894 6030

1995 9 475 -28 1592 ndash 8 3906 5963

1996 -3 483 -28 1609 ndash 9 3920 5989

1997 -3 458 -28 1546 ndash 11 3933 5916

1998 -3 531 -29 1447 ndash 10 3947 5902

1999 -10 566 -40 1374 ndash 9 3966 5866

2000 -15 579 -36 1350 ndash 9 3989 5876

2001 -25 630 -30 1310 ndash 9 4018 5910

2002 -8 647 -5 1283 ndash 9 4037 5963

2003 -14 677 24 1294 ndash 9 4035 6025

2004 -7 705 49 1376 ndash 9 4064 6195

2005 -15 694 67 1460 ndash 9 4099 6314

2006 -16 697 106 1484 64 9 4138 6481

2007 -23 736 149 1469 128 9 4167 6635

2008 51 786 171 1474 192 10 4192 6876

2009 46 816 178 1528 256 10 4216 7050

2010 66 854 183 1596 320 11 4253 7282

Total 65 12318 566 29927 960 179 80437 124452

Analysis IRR

Base casea 101

Low 108

High 87

20000 80

30000 117

50000 161

70000 189

Elapsed time years

10 146

25 130

Additional files

Funding data

Guidelines data

Additional file 1

Additional file 2

Additional file 3

Additional file 4

16 Jun 2014

1011861741-7015-12-99

2014 1299

Abstract

Background
Methods
Results
Conclusions
Background
Methods
Results
Discussion
Conclusion
Endnotes
Additional files
Abbreviations
Competing interests
Acknowledgements
Author details
References

QALYs (thousands)

1991 8 46 6 71 ndash 2 144 277

1992 9 48 6 70 ndash 2 144 279

1993 10 46 6 68 ndash 2 145 276

1994 11 48 6 67 ndash 2 145 279

1995 10 45 6 65 ndash 2 145 273

1996 11 46 6 66 ndash 2 146 276

1997 11 46 6 63 ndash 3 146 274

1998 11 50 6 59 ndash 2 147 276

1999 13 53 7 56 ndash 2 147 279

2000 15 53 7 55 ndash 2 148 281

2001 17 55 7 54 ndash 2 149 285

2002 20 56 8 53 ndash 2 150 290

2003 22 59 8 53 ndash 2 151 295

2004 24 61 9 56 ndash 2 152 305

2005 21 62 10 60 ndash 2 154 309

2006 22 62 12 61 2 2 155 316

2007 23 65 13 60 5 2 157 324

2008 27 68 14 60 7 2 158 337

2009 26 71 15 63 9 2 159 345

2010 25 74 15 65 12 3 161 354

Total 339 1112 173 1225 35 43 3003 5930

1991 199 665 181 41 ndash 34 -277 844

1992 220 687 190 40 ndash 37 -277 897

1993 241 658 185 39 ndash 40 -278 887

1994 272 684 185 39 ndash 42 -278 944

1995 252 646 175 37 ndash 42 -279 874

1996 278 660 182 38 ndash 43 -280 921

1997 269 684 181 36 ndash 53 -281 943

1998 283 720 187 34 ndash 50 -282 993

1999 336 753 214 32 ndash 47 -283 1098

2000 391 746 211 32 ndash 45 -285 1140

2001 456 755 206 31 ndash 43 -287 1204

2002 519 764 202 30 ndash 43 -282 1276

2003 571 794 178 30 ndash 44 -255 1361

2004 614 817 179 32 ndash 44 -254 1432

2005 545 850 182 34 ndash 44 -252 1403

2006 572 851 188 35 -5 45 -258 1428

2007 596 881 177 35 -10 46 -252 1473

2008 613 919 181 35 -15 48 -242 1538

2009 606 950 185 36 -20 48 -237 1569

2010 569 986 186 38 -25 56 -240 1569

Total 8403 15469 3755 704 -75 894 -5358 23793

1991 8 490 -33 1729 ndash 7 3885 6085

1992 8 506 -35 1699 ndash 7 3889 6075

1993 9 485 -33 1659 ndash 8 3890 6018

1994 10 504 -32 1646 ndash 8 3894 6030

1995 9 475 -28 1592 ndash 8 3906 5963

1996 -3 483 -28 1609 ndash 9 3920 5989

1997 -3 458 -28 1546 ndash 11 3933 5916

1998 -3 531 -29 1447 ndash 10 3947 5902

1999 -10 566 -40 1374 ndash 9 3966 5866

2000 -15 579 -36 1350 ndash 9 3989 5876

2001 -25 630 -30 1310 ndash 9 4018 5910

2002 -8 647 -5 1283 ndash 9 4037 5963

2003 -14 677 24 1294 ndash 9 4035 6025

2004 -7 705 49 1376 ndash 9 4064 6195

2005 -15 694 67 1460 ndash 9 4099 6314

2006 -16 697 106 1484 64 9 4138 6481

2007 -23 736 149 1469 128 9 4167 6635

2008 51 786 171 1474 192 10 4192 6876

2009 46 816 178 1528 256 10 4216 7050

2010 66 854 183 1596 320 11 4253 7282

Total 65 12318 566 29927 960 179 80437 124452

Analysis IRR

Base casea 101

Low 108

High 87

20000 80

30000 117

50000 161

70000 189

Elapsed time years

10 146

25 130

Additional files

Funding data

Guidelines data

Additional file 1

Additional file 2

Additional file 3

Additional file 4

16 Jun 2014

1011861741-7015-12-99

2014 1299

Abstract

Background
Methods
Results
Conclusions
Background
Methods
Results
Discussion
Conclusion
Endnotes
Additional files
Abbreviations
Competing interests
Acknowledgements
Author details
References

1991 199 665 181 41 ndash 34 -277 844

1992 220 687 190 40 ndash 37 -277 897

1993 241 658 185 39 ndash 40 -278 887

1994 272 684 185 39 ndash 42 -278 944

1995 252 646 175 37 ndash 42 -279 874

1996 278 660 182 38 ndash 43 -280 921

1997 269 684 181 36 ndash 53 -281 943

1998 283 720 187 34 ndash 50 -282 993

1999 336 753 214 32 ndash 47 -283 1098

2000 391 746 211 32 ndash 45 -285 1140

2001 456 755 206 31 ndash 43 -287 1204

2002 519 764 202 30 ndash 43 -282 1276

2003 571 794 178 30 ndash 44 -255 1361

2004 614 817 179 32 ndash 44 -254 1432

2005 545 850 182 34 ndash 44 -252 1403

2006 572 851 188 35 -5 45 -258 1428

2007 596 881 177 35 -10 46 -252 1473

2008 613 919 181 35 -15 48 -242 1538

2009 606 950 185 36 -20 48 -237 1569

2010 569 986 186 38 -25 56 -240 1569

Total 8403 15469 3755 704 -75 894 -5358 23793

1991 8 490 -33 1729 ndash 7 3885 6085

1992 8 506 -35 1699 ndash 7 3889 6075

1993 9 485 -33 1659 ndash 8 3890 6018

1994 10 504 -32 1646 ndash 8 3894 6030

1995 9 475 -28 1592 ndash 8 3906 5963

1996 -3 483 -28 1609 ndash 9 3920 5989

1997 -3 458 -28 1546 ndash 11 3933 5916

1998 -3 531 -29 1447 ndash 10 3947 5902

1999 -10 566 -40 1374 ndash 9 3966 5866

2000 -15 579 -36 1350 ndash 9 3989 5876

2001 -25 630 -30 1310 ndash 9 4018 5910

2002 -8 647 -5 1283 ndash 9 4037 5963

2003 -14 677 24 1294 ndash 9 4035 6025

2004 -7 705 49 1376 ndash 9 4064 6195

2005 -15 694 67 1460 ndash 9 4099 6314

2006 -16 697 106 1484 64 9 4138 6481

2007 -23 736 149 1469 128 9 4167 6635

2008 51 786 171 1474 192 10 4192 6876

2009 46 816 178 1528 256 10 4216 7050

2010 66 854 183 1596 320 11 4253 7282

Total 65 12318 566 29927 960 179 80437 124452

Analysis IRR

Base casea 101

Low 108

High 87

20000 80

30000 117

50000 161

70000 189

Elapsed time years

10 146

25 130

Additional files

Funding data

Guidelines data

Additional file 1

Additional file 2

Additional file 3

Additional file 4

16 Jun 2014

1011861741-7015-12-99

2014 1299

Abstract

Background
Methods
Results
Conclusions
Background
Methods
Results
Discussion
Conclusion
Endnotes
Additional files
Abbreviations
Competing interests
Acknowledgements
Author details
References

1991 8 490 -33 1729 ndash 7 3885 6085

1992 8 506 -35 1699 ndash 7 3889 6075

1993 9 485 -33 1659 ndash 8 3890 6018

1994 10 504 -32 1646 ndash 8 3894 6030

1995 9 475 -28 1592 ndash 8 3906 5963

1996 -3 483 -28 1609 ndash 9 3920 5989

1997 -3 458 -28 1546 ndash 11 3933 5916

1998 -3 531 -29 1447 ndash 10 3947 5902

1999 -10 566 -40 1374 ndash 9 3966 5866

2000 -15 579 -36 1350 ndash 9 3989 5876

2001 -25 630 -30 1310 ndash 9 4018 5910

2002 -8 647 -5 1283 ndash 9 4037 5963

2003 -14 677 24 1294 ndash 9 4035 6025

2004 -7 705 49 1376 ndash 9 4064 6195

2005 -15 694 67 1460 ndash 9 4099 6314

2006 -16 697 106 1484 64 9 4138 6481

2007 -23 736 149 1469 128 9 4167 6635

2008 51 786 171 1474 192 10 4192 6876

2009 46 816 178 1528 256 10 4216 7050

2010 66 854 183 1596 320 11 4253 7282

Total 65 12318 566 29927 960 179 80437 124452

Analysis IRR

Base casea 101

Low 108

High 87

20000 80

30000 117

50000 161

70000 189

Elapsed time years

10 146

25 130

Additional files

Funding data

Guidelines data

Additional file 1

Additional file 2

Additional file 3

Additional file 4

16 Jun 2014

1011861741-7015-12-99

2014 1299

Abstract

Background
Methods
Results
Conclusions
Background
Methods
Results
Discussion
Conclusion
Endnotes
Additional files
Abbreviations
Competing interests
Acknowledgements
Author details
References

Analysis IRR

Base casea 101

Low 108

High 87

20000 80

30000 117

50000 161

70000 189

Elapsed time years

10 146

25 130

Additional files

Funding data

Guidelines data

Additional file 1

Additional file 2

Additional file 3

Additional file 4

16 Jun 2014

1011861741-7015-12-99

2014 1299

Abstract

Background
Methods
Results
Conclusions
Background
Methods
Results
Discussion
Conclusion
Endnotes
Additional files
Abbreviations
Competing interests
Acknowledgements
Author details
References

Additional files

Funding data

Guidelines data

Additional file 1

Additional file 2

Additional file 3

Additional file 4

16 Jun 2014

1011861741-7015-12-99

2014 1299

Abstract

Background
Methods
Results
Conclusions
Background
Methods
Results
Discussion
Conclusion
Endnotes
Additional files
Abbreviations
Competing interests
Acknowledgements
Author details
References

Additional files

Funding data

Guidelines data

Additional file 1

Additional file 2

Additional file 3

Additional file 4

16 Jun 2014

1011861741-7015-12-99

2014 1299

Abstract

Background
Methods
Results
Conclusions
Background
Methods
Results
Discussion
Conclusion
Endnotes
Additional files
Abbreviations
Competing interests
Acknowledgements
Author details
References

Additional files

Funding data

Guidelines data

Additional file 1

Additional file 2

Additional file 3

Additional file 4

16 Jun 2014

1011861741-7015-12-99

2014 1299

Abstract

Background
Methods
Results
Conclusions
Background
Methods
Results
Discussion
Conclusion
Endnotes
Additional files
Abbreviations
Competing interests
Acknowledgements
Author details
References

Additional files

Funding data

Guidelines data

Additional file 1

Additional file 2

Additional file 3

Additional file 4

16 Jun 2014

1011861741-7015-12-99

2014 1299

Abstract

Background
Methods
Results
Conclusions
Background
Methods
Results
Discussion
Conclusion
Endnotes
Additional files
Abbreviations
Competing interests
Acknowledgements
Author details
References

16 Jun 2014

1011861741-7015-12-99

2014 1299

Abstract

Background
Methods
Results
Conclusions
Background
Methods
Results
Discussion
Conclusion
Endnotes
Additional files
Abbreviations
Competing interests
Acknowledgements
Author details
References

1011861741-7015-12-99

2014 1299

Abstract

Background
Methods
Results
Conclusions
Background
Methods
Results
Discussion
Conclusion
Endnotes
Additional files
Abbreviations
Competing interests
Acknowledgements
Author details
References

Estimating the returns to UK publicly funded cancer ...

Documents

BARRA’s Risk Models - pdfs.semanticscholar.org · ty...

Estimating the returns to parental time investment in...

Estimating Marginal Returns To Medical Care: Evidence From.....

The Economic Returns to an MBApsarcidi/ReturntoMBA16.pdfThe....

ESTIMATING THE RETURNS OF INVESTING IN RMNCH Malcolm Bryant....

Estimating Returns to Vocational Education at High Schools.....

Estimating the Returns to Education Using a Sample of ...

Estimating the Perceived Returns to College...The estimation...

Estimating Property Holding Period Returns...West 0.0154...

Estimating the Costs of Filing Tax Returns and the Potential...

› english › pri › publication › pp_review › ......

Estimating the Returns to Education: Models, Methods and...

The economic value of key intermediate qualifications...

Estimating Future Stock Market Returns (2)

Trading Costs and Returns for US Equities: Estimating high

Estimating Marginal Returns to...