Top Banner

Click here to load reader

Evidence-based decision making. How would you like that done, Minister ? Tony Ades RSS Avon Group, February 7 2008 Academic Unit of Primary Health Care,

Mar 28, 2015

ReportDownload

Documents

ryan-wyatt

  • Slide 1

Evidence-based decision making. How would you like that done, Minister ? Tony Ades RSS Avon Group, February 7 2008 Academic Unit of Primary Health Care, Department of Community Based Medicine Slide 2 Outline If you were a Minister of Health, and DoH had a decision-making machine (DMM), what properties would you want it to have ? How is evidence used to make decisions ? Some examples (good and bad) Obstacles to building a DMM. Slide 3 Evidence-Based Medicine response to increasing amounts of evidence, arbitrary / selective use of evidence. formal protocol inclusion / exclusion formal synthesis (meta-analysis) of studies Hierarchy of evidence Objective: to help clinicians make decisions Slide 4 Evidence-based decision-making ? Decision Evidence Slide 5 Evidence-based decision-making ? Decision Evidence ? Slide 6 Evidence-based decision-making Model Costs Benefits Net Benefit Decision Evidence Slide 7 Evidence-based decision-making . Model Costs Benefits Net Benefit Decision Evidence needs an evidence-based model Slide 8 How are health care decisions made? 1. Estimate the Costs C j of each strategy, j costs of intervention, downstream treatment costs 2. Estimate the Benefits L j of each strategy quality-adjusted life years 3. Decide the value of an additional Life Year 20,000 - 30,000 4. Work out the Net Benefit of each strategy j NB j = L j. 20000 C j 5. Choose strategy j*, with the highest Net Benefit Slide 9 Desirable properties of a decision making machine Makes complete use of evidence Checks consistency of evidence Propagates uncertainty Slide 10 Complete use of evidence 1.EBM-style Protocol for study inclusion / exclusion. Aim: all relevant evidence 2.Repeatability. ? > 95% overlap between different reviewers would help guarantee completeness 3.Wider inclusion: If parameters a and b are in a model, data on the product ab is relevant 4.But what counts as relevant ? Anything that reduces decision uncertainty is relevant Slide 11 Consistency of evidence 1.All the studies informing the parameter a should agree. Pooling 2.The combined information on a, together with the combined information on b, should agree with the combined (independant) information on the product ab. External validation, triangulation, backward-propagation of evidence Slide 12 Uncertainty propagation Uncertainty in parameters (or data) is propagated through the model to be reflected in uncertainty in the decision Average Net Benefit over the joint distribution of parameters : Choose strategy j* with highest Expected Net benefit. Probabilistic Decision Analysis, usually done by forward Monte Carlo Tells Decision Maker the probability the optimal decision is wrong Should be joint posterior distribution of parameters Slide 13 What kind of machine can do this ? 1.Complete use of evidence achieved through Bayesian or Likelihood methods. Likelihood quite difficult computationally 2.Consistency of evidence: detecting inconsistency is normal practice in statistics but less so in decision modelling. Validation considered important, but formal inference seldom brought to bear 3.Uncertainty propagation : essentially Bayesian idea. Monte Carlo (MC) framework very convenient for decision modelling: hence simulation from a Bayesian posterior distribution. Or try to mimic through bootstrap ? Bayesian Markov chain MC, eg WinBUGS very convenient Slide 14 Completeness, consistency, uncertainty propagation: hardly controversial ! So, whats the problem ? Some modellers are not following this script Script not always easy to follow Understanding the data at all Responding to inconsistencies Inference for computationally complex models Next: Some examples of what is done badly, what can be done that is good, and what is still difficult to do well Slide 15 TOPIC 1: Chlamydia screening ( not following the script) 1.Most common STI. Treatable, but often silent 2.Complications of chlamydia: Pelvic Inflammatory Disease, Ectopic Pregnancy, Infertility 3.=> Prima facie case for screening, But is it cost- effective ? 4.Two UK studies: ClaSS (pro-active), HPA (opportunistic). Both UK Government funded. 5.Each produced models for CEA. Epidemiology / transmission model, and a natural history model, embedded in a cost-effectiveness analysis. Slide 16 Chlamydia complication rates ClaSS study Pr(PID | chlamydia) based on Uppsala Study. GP registers with screening information linked with hospital registers of PID. Pr(PID | chlamydia) estimate based on all PID, not just those screened +ve. Therefore not linked to chlamydia incidence / prevalnce both Over estimates as includes PID not caused by chlamydia And under estimates as misses PID treated by GPs Use of this estimate assumes (i) chlamydia incidence same in UK and Sweden (ii) the over- and under-estimation biases are known to exactly cancel each other out. About 3.6% Slide 17 Chlamydia complication rates HPA study No single, evidence-based, estimate 3 Alternative scenarios: 1%, 10%, 30% But authors observe that 10% is around the maximum credible UK complication rate, based on 1.7% pa PID incidence observed in GPRD A deterministic sensitivity analysis. Useful to know Screening not cost-effective if complication rate < 10%....? .. but not if we have no idea what the true rate is ! Ignores correlations between parameters Slide 18 Chlamydia complication rates: What is the evidence ? Prospective UK chlamydia prevalence p = 1.5 % (Women aged 15-44, NATSAL 2000) Duration of an episode d: 120 days (recent meta-analysis) => UK chlamydia Incidence h = p/d = 1.5 x 365.25/120 = 4.6 % pa Probability Pelvic Inflammatory Disease (PID) per episode of chlamydia : c = 0.15 (meta-analysis of prospective studies) => incidence of PID due to chlamydia: ch = 0.69% pa Retrospective UK Incidence of all PID (GPRD): ch/B = 1.7% pa, but not all PID caused by chlamydia. Proportion of All PID due to chlamydia (retrospective study) : B = 0.39 => incidence of PID due to chlamydia: ch = 0.66 % pa Slide 19 Towards complete use of evidence 1.Meta-analyses of prevalence, duration. 2.Meta-analyses of prospective studies of PID 3.Several retrospective studies of PID 4.Similar prospective & Retrospective studies on other complications: ectopic pregnancy and infertility 5.Further prospective studies following PID forward to infertility and further retrospective studies looking back from infertility to PID. 6.Large network of evidence, to be assembled, synthesised, checked for consistency 7.BUT some difficulties interpreting literature, adjusting for biases, understanding recruitment into studies, taking test sensitivity / specificity into account . etc, etc etc Slide 20 TOPIC 2: How it should be done ! Antenatal screening for HIV: universal or targeted ? a = Proportion Sub-Saharan African (SSA) b = Proportion Injecting Drug Users (IDU) 1-a-b = Proportion Low-Risk c = HIV prevalence SSA d = HIV prevalence IDU e = HIV prevalence Rest (Low Risk) No direct evidence Slide 21 Complete use of evidence: HIV in London, 1998 % Num Denom 1.Proportion Sub-Saharan African (SSA a 10.6 11044 104577 2.Proportion Injecting Drug Users (IDU) b 1.36 12 882 3.HIV prevalence SSA c 1.63 252 15428 4.HIV prevalence IDU d 2.11 10 473 5.HIV prevalence non-SSA [db + e(1-a-b)]/(1-a) 0.054 74 136139 Mixture of Low-risk & IDU: HIV prevalence is a weighted average of Low-Risk prevalence & IDU prevalence provides information on e the Low-Risk prevalence Slide 22 Complete use of evidence: another example value (%) Num Denom 1.Proportion Sub-Saharan African (SSA) a 10.6 11044 104577 2.Proportion Injecting Drug Users (IDU) b 1.36 12 882 3.HIV prevalence SSA c 1.63 252 15428 4.HIV prevalence IDU d 2.11 10 473 5.HIV prevalence non-SSA [db + e(1-a-b)]/(1-a) 0.054 74 136139 6.Overall HIV prevalence ac +db +e(1-a-b) 0.187 254 102287 Weighted average of all 3 risk groups . Possibility of INCONSISTENCY Slide 23 including proportion already diagnosed in the HIV model Slide 24 the larger data set 12 independent data items : 9 parameters 1. proportion Sub-Saharan African (SSA) a 2. proportion Injecting Drug Users (IDU) b 3. HIV prevalence SSA c 4. HIV prevalence IDU d 5. HIV prevalence non-SSA [db + e(1-a-b)]/(1-a) 6. Overall HIV prevalence ac +db +e(1-a-b) 7. Diagnosed HIV in SSA : all diag HIV fca/[fca +gdb +he(1-a-b)] 8. Diagnosed HIV in IDU : non-SSA diag HIV gdb/[gdb + h(1-a-b)] 9. Overall proportion diagnd: [fca+gdb+ he(1-a-b)] / [ca+db+e(1-a-b)] 10. Proportion infected IDUs diagnosed g 11. Proportion serotype B in infected women (SS) w 12. Proportion serotype B in infected women (Non-SSA) [db/[db+e(1-a-b)]]+we(1-a-b)/[db+e(1-a-b)] Slide 25 Uncertainty propagation from posterior Uncertainty in the data is propagated back to the model parameters, and then through the model to the decision. The decision is based on Expected Incremental Net Benefit, difference between Exp NB Universal and Targeted Screening. inb = 105000*(1-a-b) * (M * e * (1-h) - T*(1-e*h)) If Exp[INB] is +ve, choose Universal; if ve, Targeted. M is the Net Benefit of diagnosing an Infected Mother T is the cost of an HIV test Number of pregnant women in London per year Proportion Low-Risk Slide 26 Uncertainty propagation Probability INB +ve = 0.974 Choose universal with confidence e h inb Slide 27 TOPIC 3: Inconsistency - its not so easy after all. 1. proportion Sub-Saharan African (SSA) a 2. proportion Injecting Drug Users (IDU) b 3. HIV prevalence SSA c 4. HIV prevalence IDU d 5. HIV prevalence non-SSA [db + e(1-a-b)]/(1-a) 6. Overall HIV prevalence ac +db +e(1-a-b) 7. Diagnosed HIV in SSA : all diag HIV fca/[fca +gdb +he(1-a-b)] 8. Diagnosed HIV in IDU : non-SSA diag HIV gdb/[gdb + h(1-a-b)] 9. Overall proportion diagnd: [fca+gdb+ he(1-a-b)] / [ca+db+e(1-a-b)] 10. Proportion infected IDUs diagnosed g 11. Proportion serotype B in infected women (SS) w 12. Proportion serotype B in infected women (Non-SSA) [db/[db+e(1-a-b)]]+we(1-a-b)/[db+e