Presenter: Pamela Peele, Ph.D. Vice President, Health Economics February 28, 2012 Learning About Your Clinical Programs
Dec 25, 2015
Presenter: Pamela Peele, Ph.D.Vice President, Health Economics
February 28, 2012
Learning About Your Clinical Programs
• One of the nation’s largest Integrated Delivery Systems• 5th in NIH funding, affiliated University of Pittsburgh• $9.0 billion in total operating revenue• More than 54,000 employees• More than 3,000 employed physicians and 5,000 affiliated
physicians• 21 hospitals: 4,500+ beds and 43 regional cancer centers• 400 clinical locations; home care; rehab, urgent care• 1.8 million members in Insurance Division programs • 20,000+ contracted network providers• Global and Commercial Enterprise (UK; Italy; Qatar; Ireland;
Cyprus)• $500 million commitment to information technology
UPMC Today
2
• 2nd Largest in Nation Provider Led• 3rd Largest Operating in PA• 1.8M Members• Annual Revenues ~$4B• Fastest Growing Medicaid Plan • Fastest Growing Children’s Health• Highest Commercial Satisfaction
J.D. Power• Top 10 Nationally in Medicaid Quality• 4 Star Medicare Plan• Highest Ranked Provider
Satisfaction (PA)• National Business Group on Health
Platinum Winner past three years
UPMC Insurance Services Division Highlights
3
• 1.8 Million covered lives across all insurance products
• Medicare Advantage• Medicare FFS• Commercial (fully insured and ASO)• Medicaid• Special Needs Plan (SNP)• Children’s Health Insurance Plan (SCHIP)• Community Care (behavioral health management plan)
UPMC Health Plan
4
Levels of Analytics Framework
5
Standard ReportsWhat happened?
AlertsWhat actions are needed?
Query DrilldownWhat exactly is the problem?
Ad hoc ReportsHow many, how often, where?
Statistical AnalysisWhy is this happening?
OptimizationWhat’s the best that can happen?
Predictive ModelingWhat will happen next?
ForecastingWhat if these trends continue?
Degree of Intelligence
Com
petit
ive A
dvan
tage
From Tom Farre, “The Analytical Competitor”, in Analytics: The Art and Science of Better, ComputerWorld Technology Briefing.
UPMC HP: 2009
UPMC HP: 2006
The TWO Pieces of the Puzzle
6
1. Program Description
2. Analytic Team
1. What are trying to do or change?
Concrete objectives of the program with outcomes linked to program actions
2. How will we do it?
What are the tasks to be done, who does them, to whom,and when?
3. How will we know if we are successful?
Outcomes, hypothesis testing, measurement
MAKE AN INTERNAL TEMPLATE DOCUMENT
Program Description
7
• Pre/Post evaluations – Confounded by unobservables and time-specific changes – Selection criteria often undermines validity
• Randomized Controlled Trial– Often impractical for regulatory, operational, or member/provider
satisfaction reasons
• Matched cohorts– Requires scientifically sound matching techniques– If continuous enrollment is required, death takes a holiday
Avoiding Regression to the Mean
8
• Intention to Treat– Requires careful thought on the inclusion/exclusion criteria
• Differences in Differences– Powerful method for the real world– Eliminates the concern over uniformly distributed unobservables
Avoiding Regression to the Mean
9
Pediatric Weight Management Program Objectives
10
• A clinic-based program that utilizes a multi-disciplinary team approach to reduce BMI in overweight or obese children
•Collaborative program with Children’s Community Pediatrics and Children’s Hospital of Pittsburgh
•Evaluation focused on children with 2 or more weight measures within 182 days (July 2010 to January 2012) and BMI percentile of 85% or greater
• Research question: Did children enrolled in the program have a greater change in body mass index (BMI) percentile than children not enrolled in the program?
Analysis Design
11
1.Primary analysis: Compared the difference in the change in BMI percentile between the intervention and comparison groups using a matched difference in differences comparison (kernel matching)
• Matching variables: age, gender, baseline BMI percentile, duration between first and last visit, and the number of visits
2.Robustness checks: Compare results of kernel matching to results using exact matching, nearest neighbor propensity score matching, and two OLS regression models
Primary Findings
12
Difference-in –differences: Δintervention-Δcomparison=-0.43-0.02=-0.45
Conclusions and limitations
13
•Primary conclusions• Children enrolled in the Pediatric Weight Management program
experienced significantly greater reductions in their BMI percentile compared to children not enrolled in the program
• These the program and designing a more rigorous study of the outcomepromising findings, if clinically meaningful, provide evidence to warrant continuing s going forward
•Limitations• Program attrition: 86 children (46%) did not have a return program
visit• Very little information is known about the children in the comparison
group
Levels of Analytics Framework
17
Standard ReportsWhat happened?
AlertsWhat actions are needed?
Query DrilldownWhat exactly is the problem?
Ad hoc ReportsHow many, how often, where?
Statistical AnalysisWhy is this happening?
OptimizationWhat’s the best that can happen?
Predictive ModelingWhat will happen next?
ForecastingWhat if these trends continue?
Degree of Intelligence
Com
petit
ive A
dvan
tage
From Tom Farre, “The Analytical Competitor”, in Analytics: The Art and Science of Better, ComputerWorld Technology Briefing.
• Excel
• Access
• Crystal Reports
Staff - 2006
18
Current Staff
19
• Industry Knowledge• Data visualization skills• Data ECTL (extraction, cleaning, transformation, loading) skills• Statistics• Health Services Research • Data Mining• Financial modeling & evaluation• Presentation, writing, and communication skills
• Formally trained but NOT blinded by their training– Challenge deeply held beliefs
Staff Skills and Backgrounds
20
• Database: SQL, Toad• Statistics: SAS, STATISTICA, STATA, R• Data Mining: STATISTICA, SAS Enterprise Miner, R• Modeling & Simulation: MATLAB, Mathematica, Vensim• GIS: ArcGIS
Tools
21
• Data Overload– No Knowledge
• No Learning
• Misleading Data Perspective
Issues
22