Traffic Records Assessment Assessor Training October 2015
Jan 19, 2016
Traffic Records Assessment Assessor Training
October 2015
2
What & Where are the Criteria?
The Traffic Records Program Assessment Advisory:
• Provides guidance on the necessary contents, capabilities, and data quality measurements for a comprehensive traffic records system,
• Describes an ideal traffic records system, one that supports high-quality decisions that enable cost-effective improvements to highway and traffic safety,
• Poses a uniform set of questions that reveals the performance of the State traffic records system relative to the ideal.
3
Scope of the Assessment
• TRCC Management
• Strategic Planning
• Data Use & Integration
• Six Core Data Systems:
CRASH DRIVER ROADWAYVEHICLE CITATION/ADJUDICATON
INJURYSURVEILLANCE
4
Assessment QuestionsTRCC Management 19Strategic Planning 16Crash 44Driver 45Vehicle 39Roadway 38Citation / Adjudication 54Injury Surveillance 123*Data Use & Integration 13
Total 391* Injury Surveillance now includes sub-sections on EMS, Emergency Room, Hospital Discharge, Trauma Registry, and Vital Records
5
Standards of Evidence
The Advisory supplies a standard of evidence for each question.
• Describes the information needed to support State assertions that a traffic records system possesses the specific capability referred to in the question
• Determined by subject matter expert panelswith State, Federal, academic, and otherrepresentation.
6
Assessment Ratings
Upon review, assessors will make a determination for each question that reflects how the State’s traffic records systems are performing relative to the ideal detailed in the Advisory.
• MEETS the description of the ideal traffic records system
• PARTIALLY MEETS the description of the ideal traffic records system
• DOES NOT MEET the description of the ideal traffic records system
Assessment ScheduleFinalized Schedule Holiday
Sun Mon Tue Wed Thu Fri Sat
November 26 27 28 29 30 31 1
2 3 4 5 6 7 8 STRAP Training
9 10 11 12 13 14 15 Kickoff Meeting
16 17 18 19 20 21 22 Round 1: Data Collection
23 24 25 26 27 28 29
December 30 1 2 3 4 5 6 Round 1: Analysis
7 8 9 10 11 12 13
14 15 16 17 18 19 20 Round 2: Data Collection
12 22 23 24 25 26 27
January 28 29 30 31 1 2 3
4 5 6 7 8 9 10
11 12 13 14 15 16 17 Round 2: Analysis
18 19 20 21 22 23 24
25 26 27 28 29 30 31 Round 3: Final Data Collection
February 1 2 3 4 5 6 7
8 9 10 11 12 13 14 Round 3: Final Analysis
15 16 17 18 19 20 21
22 23 24 25 26 27 28 Facilitator Finalizes Report
March 1 2 3 4 5 6 7 Final Report Submitted
8 9 10 11 12 13 14
15 16 17 18 19 20 21 Report Out
8
Time and Knowledge Needed
Expect to spend no more than 40 hours working over the course of an assessment for an average module. (40-50 questions)
What qualifies someone as an expert? • “Expertise is a broad knowledge of how a system or process
works; skill, training and experience can give a person a level of expertise, but the ability to analyze various aspects of a program, solve problems and offer solutions is central to being a subject matter expert. This doesn't mean you have to know all there is to know about a subject, more that you are aware of the broader issues such as management principles that foster improvement and excellence.”
9
Average Assessor Time in STRAP
10
Average Hours Per Phase
9.7
5.5
6.1
National Average Assessor Time Spent by Phase
Round 1Round 2Round 3
STRAP Process Flow
State GR submits written request to NHTSA Region
State Respondents answer assigned questions & supply evidence
NHTSA Region forwards written request to NHTSA TR Team
Assessors review answers & evidence; provide findings & ratings
NHTSA TR Team confirms State request & schedules calls
NHTSA TRA team hosts initial call (~4 months prior to kickoff)
Facilitator leads ASSESSMENT KICKOFF MEETING
State Coordinator sends Respondent info & assigns questions
STRAP Tech Support hosts State Coordinator training webinar
STRAP Tech Support Sends State Coordinator STRAP tokens
STRAP Tech Support Launches Data Collection & Analysis Phases
Facilitator hosts call (1 month prior to kickoff)
STRAP Tech Support Sends Respondent STRAP tokens
STRAP Tech Support sends Assessor STRAP tokens
Facilitator review; forwards to NHTSA TR Team
NHTSA TR Team generates Final Report; sends to State
Facilitator leads ASSESSMENTREPORT OUTWEBINAR
Assessors confirm final findings, ratings and summaries
x3
Answers that meet the ideal are not returned to respondents in subsequent rounds
NHTSA TR Team
Facilitator
State Leadership
State Respondents
NHTSA Region
Assessors
STRAP Support
State Traffic Records Assessment Process (STRAP)
STRAP Overview
13
STRAP Overview
STRAP Overview
15
STRAP Overview
16
STRAP Overview
17
STRAP Overview – Module Leader
18
STRAP Overview – Respondent View
19
Sample Q & AModule: TRCC1. Does the State have both an executive and a technical TRCC?Evidence Requirement: Provide a charter and/or MOU. Also provide a roster with all members' names, affiliations, and titles for both the executive and technical TRCC.
State Response: Yes the State's TRCC, is organized as a 3-level tier. The top tier is the Executive level, the second tier is the technical level and the third are the working groups created to address current projects or challenges on a working level. See the attached TRCC charter and executive level roster.
Rating: Meets the Advisory Ideal - Both an executive TRCC and a technical TRCC are in place, and detailed rosters illustrating the structure at each level have been provided.
20
Advice• DO communicate with your co-assessor and/or Module
Leader. (Printouts)
• DON’T use outside information for Ratings/Findings.• Use only information provided by respondents.
• DO use outside information for clarification requests.• Ask the respondent to confirm/corroborate potential outside
information.
• DO use outside information for Module Summaries.• When applicable, note what was provided in-system did not match
external knowledge, but ratings had to be made based on what was actually submitted.
21
Advice
• DON’T restate the question or evidence requirement in the ballot and/or finding.
• DO Provide enough information so someone reading the report at a later date will understand how and why you reached your conclusion.
• DON’T write “State says no.”• Instead write: "The State's crash and driver systems are not linked."
22
Final Report
The result of the assessment is a report with the final question ratings and findings.
Each Module leader also creates a Module Summary similar to the previous assessment that lays out the opportunities and strengths for each module.
23
Final Report
Crash
Vehicle
Driver
Roadway
Citation /
Adjudication
EMS / Injury Surveillance
Description and Contents 96.4% 100.0% 70.0% 100.0% 77.2% 76.5%
Applicable Guidelines 100.0% 100.0% 100.0% 83.3% 75.4% 68.4% Data Dictionaries 73.3% 100.0% 91.7% 80.0% 79.4% 70.0%
Procedures / Process Flow 47.9% 87.9% 97.1% 83.3% 63.0% 67.2%
Interfaces 66.7% 97.0% 90.5% 100.0% 33.3% 42.9% Data Quality Control
Programs 44.2% 53.7% 47.9% 86.8% 46.2% 47.4%
Overall 64.3% 77.6% 77.0% 89.0% 63.0% 58.6%
Overall
Traffic Records Coordinating Committee 88.0% Strategic Planning for the Traffic Records System 76.2%
Data Use and Integration 52.5%
Recommendations are now automatically generated based on the score each module receives. Scores are calculated using a combination of the question importance and rating.
24
National Ratings
25
Becoming an Assessor
If you would like to be considered as an assessor for future assessments of other States’ traffic records systems, please fill out the form provided at the back of the room.
The form and other information is also available at:http://www.tsass.com/opportunities.html
All names are sent to NHTSA for vetting and then put into the Subject Matter Expert Pool for selection.
Questions?