Partnering to improve data quality and birth outcomes Extending our reach through partnerships June 2-6, 2013, Phoenix AZ
Dec 16, 2015
Partnering to improve data quality and birth outcomes
Extending our reach through partnershipsJune 2-6, 2013, Phoenix AZ
From Good to Great – Collaborating to Advance Data Quality
The Birth Data Quality Workgroup
Extending our reach through partnershipsJune 2-6, 2013, Phoenix AZ
MembersBirth Data Quality Workgroup
Co-chairs Isabelle Horon (MD), David Justice (NCHS), Joyce Martin (NCHS)
MembersSukhjeet Ahuja (NAPHSIS), Sally Almond (MN), Marie Aschliman (UT), Karyn Backus (CT), Karin Barrett (MA), Krista Bauer (MN), Mary Chase (CO), Bruce Cohen (MA), Cindy Coleman (MN), Greg Crawford (KS), Cheri Denardo (MN), Judy Edwards (MN), Claudia Fabian (IL), Mark Flotow (IL), Coleen Fontana (RI), Kelly Friar (OH), Melissa Gambatese (NYC), Gloria Haluptzok (MN), Cindy Hooley (VT), Renata Howland, (NYC)Priya Kakkar (KS), Larry Nielsen (NAPHSIS), Lloyd Mueller (CT), Ann Madsen (NYC), Carol Moyer (KS), Laura Ninneman (WI), Michelle Osterman (NCHS), Sharon Pagnano (MA), Mariah Pokorny (SD), Phyllis Reed (WA), Matthew Rowe (WY), George Rudis (IL), Margarita Segundo (CO), Neeti Sethi (MN), Shae Sutton (SC), Marie Thoma (NCHS), Elaine Tretter (MD), Regina Zimmerman (NYC)
Origins Birth Data Quality Workgroup
NAPHSIS/NCHS Good to Great collaboration
VSCP Directors’ conference calls
Workgroup establishedone year ago at 2012 Joint Meeting
Mission Birth Data Quality Workgroup
Assess and improve the quality of vital statistics birth and fetal death data
Focus on improving data at the sourcei.e., at the hospital
Priorities Birth Data Quality Workgroup
SubgroupsHospital reportsKaryn Backus (CT)
Engaging hospitalsKelly Friar (OH), Colleen Fontana (RI)
Prenatal care itemsIsabelle Horon (MD)
New subgroupsE-learning training for hospital staffIdentify items to drop from the national birth data file Working with the Good to Great to develop process for dropping
items
Birth Data Quality Workgroup
Meeting Wednesday @ 7:15 amPlease join us!
Data Quality WorkgroupSubgroup 1
Members:• Karyn Backus (CT)• Ann Madsen Straight (NYC)• Phyllis Reed (WA)• Elaine Tretter (MD)• Greg Crawford (KS)• Carol Moyer (KS)• Mary Chase (CO)• Sharon Pagnano (MA)• David Justice (NCHS)• Marie Thoma (NCHS)• Sukhjeet Ahuja (NAPHSIS)
• Charge: Recommend a process (metrics, means of communication, actions) for vital records offices to improve hospital reporting of birth certificate data
• Aim 1: Survey jurisdictions on their current activities to• Measure hospital-level data quality
• Communicate findings to hospitals
• Act on information
Background
NAPHSIS survey participantsCharacteristics of Jurisdictions n/N (%)
Response rate 41/52 (78.8%) – United States2/5 (40.0%) – Territories
2003 Birth Certificate 33/41 (80.5%) – United States1/2 (50.0%) – Territories
Data quality responsibility
Registration Staff 14/41 (34.1%)
Registration and Statistics (shared) 18/41 (43.9%)
Statistics Staff 9/41 (22.0%)
Current Practices:Standardized Worksheets
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
76%61% 58% 64%
30%
24%39% 42% 36%
70%
Yes No
Jurisdictions using the 2003 birth certificate (n=33)
Current Practices: Evaluating Data Quality
Real-time communica-tion (< 1 wk)
Logic checks Regular completeness reports
Medical record audits (rarely or regular)
Other detailed analyses (i.e., trends, patterns)
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
37%
88%
40%
22%
78%
63%
7%
60%
78%
20%
Yes No Unknown
U.S. jurisdictions (n=41)
Current Practices: Evaluating Data Quality
• Does your jurisdiction maintain a desire to perform a more extensive quality review?• 90% indicated “Yes”• 95% indicating limited resources as main impediment
Current Practices: Evaluating Data Quality
Yes63-71%29-37% 100%
Evaluated Impact Detected Improvement
No Yes!Completeness Reports, 63% (n=19)Medical Record Audits, 67% (n=9)Other Detailed Analyses, 71% (n=17)
Current Practices: Raising Awareness
Hospital Staff Only40%
Hospital Staff and Stakeholders
33%
Stakeholders Only5%
No efforts23%
Hospital Staff (total): 72%
Stakeholders (total): 38%
U.S. jurisdictions (n=40)*
* No response for 1 jurisdiction
• Produce Final Summary Report
• Subgroup 1 - Goals:
• Develop metrics to evaluate birth facility data quality that are feasible for jurisdictions to implement
• Develop means of communications to promote birth data quality
• Prepare best practice recommendations for data quality management
• Support overall data quality workgroup and ongoing national initiatives
Future Directions
Data Quality WorkgroupSubgroup 2
Members:• Kelly Friar, Ohio• Colleen Fontana, Rhode Island• Melissa Gambatese, New York City• Matthew Rowe, Wyoming• Sharon Pagano, Massachusetts• Elaine Tretter, Maryland• Shae Sutton, South Carolina• Joyce Martin, NCHS
• Overall Charge• Develop approaches in engaging hospitals to improve
the quality of the birth certificate data provided to the jurisdictions’ Vital Records office
• Goal• Develop a set of standard guidelines for jurisdictions
to follow to engage hospital staff to improve the quality of the birth data.• Can be tailored to any jurisdiction• Can be self-sustaining
Engaging Hospitals
Deliverables• Short List of Deliverables• Recommendations to Engage Hospitals• Introduce intent of project/campaign to high level
hospital key players• Ways to identify key data elements that could/should
be of concern to hospital• Key points to consider when addressing hospital staff• Such as…
A Few Key Points of Consideration• Purpose of birth health data• Contributing factors of birth outcomes• How does quality data help the hospital with the community
it serves• What could be the “burning platform” for the hospital (QI –
PDSA model)• Develop ways for the hospital to access the data• Standardization of definitions• Training• Metrics for vital records offices to improve hospital
reporting – Subgroup 1
Action Items
1) Collaborate with national partners2) Steal shamelessly from other jurisdictions that may
have done this!3) Determine why a hospital should care and how to
convince them they should care4) Determine who in the hospital needs to be convinced5) Develop methods of communication – what and how
1) Collaborating with National Partners• American Congress of Obstetricians and
Gynecologists (ACOG)• March of Dimes• Association of Women’s Health, Obstetric and
Neonatal Nurses (AWHNN)
2) Steal shamelessly from other jurisdictions – we did!
• Ohio – Kelly Friar• Ohio statewide initiative• Improve birth outcomes and reduce infant mortality• Key data elements of focus• Letter of introduction to hospital executives• Dr. Jay D. Iams, Ohio State University Wexner Medical Center• Joined our subgroup conference call to gain doctor’s perspective
– very enlightening and encouraging
• California Maternal Quality Care Collaborative - Dr. Elliott Main
We would love to hear from other jurisdictions – just contact any
member of the workgroup
Thank You !
Members:
• Sukhjeet Ahuja (NAPHSIS)
• Karyn Backus (CT)
• Bruce Cohen (MA)
• Isabelle Horon (MD)
• Renata Howland (NYC)
• Michelle Osterman (NCHS)
• Elaine Tretter (MD)
Prenatal Care Data ItemsSubgroup 3
• To assess the quality of prenatal care data items collected on the U.S. Standard Certificate of Live Birth; and
• Recommend changes for improvement
Charge
1. Determine which prenatal care data items should be studied
2. Prepare summary of why these data items were selected for inclusion on the certificate, and how they are used for public health purposes
3. Compile information available on data quality and barriers to collection of accurate data
4. Compile information on state efforts to improve data quality
5. Prepare best practices document
6. If warranted, recommend changes to data items on certificate
Workgroup plan
• Date of first prenatal care visit
• Date of last prenatal care visit
• Number of prenatal care visits
Data items selected for study
History of collection of PNC data
•1968• Month PNC began to be collected
•1972• Number of PNC visits added
•2000—Panel recommendations• Month PNC began Date of 1st PNC visit• Slight revision to number of PNC visits question
•2001—Addendum to Panel Report• Date of last PNC visit added
Barriers to collecting accurate data
• Missing data• PNC records not sent to hospital • Mothers may change PNC providers
• Inaccurate data• PNC records sent well before delivery and don’t contain current data
• Birth clerks may estimate date of last PNC visit and total number of visits• Some states have placed PNC items on mother’s worksheet• Difficult to evaluate what constitutes:
• A first PNC visit• A PNC visit
• Very time-consuming for birth clerks to collect data items• Providers may provide inaccurate data
Data AccuracyRange of level of agreement in hospitals in 3 states
Data itemLevel of agreement
Medical records and birth certificates
Date of first PNC visit 30% - 90%
Date of last PNC visit 10% - 80%
Number of PNC visits 5% - 90%
**Accuracy rates probably even poorer, since data in medical records often incorrect**
Jurisdictions with efforts aimed at improving PNC data quality
No ef-forts
in place58%
Efforts in
place42%
79% believe efforts aremaking a difference
2013 NAPHSIS Survey
Does poor data quality prevent PNC data items from being useful for public health purposes?
Date of first PNC visit Date of last PNC visit Number of PNC visits0
5
10
15
20
25
30
35
40
45
24
3639
Perc
ent
% of respondents indicating “yes”
2013 NAPHSIS Survey
Summary• Problems with data accuracy• Limited efforts underway to improve data accuracy• Data quality limits usefulness for public health
purposes• Collection/correction requires considerable effort • Medical staff• Birth clerks• Vital Records staff
• Next steps• Can the data be improved?• Should changes be made to certificate?