Data Collection Through New On- The-Road Technologies: The Need for Validation David Harkey University of North Carolina Highway Safety Research Center.

Data Collection Through New On-The-Road Technologies: The Need for Validation

David HarkeyUniversity of North Carolina

Highway Safety Research Center

Traffic Records ForumJuly 2003

Denver, Colorado

UNC Highway Safety Research Center Highway Safety Information System

Overview of Presentation

Roadway Inventory: Needs and Methods

Major Missing Element: Alignment History of Recent HSIS Efforts Results from New HSIS Study Conclusions

UNC Highway Safety Research Center Highway Safety Information System

Roadway Inventories

Inventory databases exists in all State DOTs

Used for planning, operations and maintenance

Range of data elementsgeometrics (roadways, intersections and

interchanges)traffic control/operations (segment speed

limits, intersection TCD, signal timings)

UNC Highway Safety Research Center Highway Safety Information System

Roadway Inventory Collection

Manual methodsIn-field acquisition (surveys, observational

runs with DMI or GPS technology) In-office acquisition (video logs, aerial

photos, satellite images) Automated Methods

Instrumented Vans (video logs, pavement conditions, sign inventory, geometrics)

UNC Highway Safety Research Center Highway Safety Information System

Alignment Data

Very few states have curvature and grade data

Only 2 of the 9 HSIS states have good horizontal curvature data

Known relationship between curvature and safety

UNC Highway Safety Research Center Highway Safety Information System

HSIS Research Focus

Improve upon the alignment data available for safety analyses

Automated methods using instrumented vehicles ARAN - Automatic Roadway

Analyzer

Roadware Group, Inc.

UNC Highway Safety Research Center Highway Safety Information System

History of HSIS Efforts

1996/97 - MN Data Collection and Analysis4 2-lane roads with 50+ curves over 21

milesPoor Results - Consistency and Accuracy

1999 - Re-analysis of MN Data with algorithm developed for CT DOTDramatic improvement in consistency and

accuracy

UNC Highway Safety Research Center Highway Safety Information System

New HSIS Effort

Evaluate the CT DOT algorithm that uses ARAN data

Repeatability (Consistency) between multiple runs on the same

segment of roadway Accuracy - Comparison to “ground

truth”As determined from ground surveys

UNC Highway Safety Research Center Highway Safety Information System

Data Collection Tasks

Site Selectionapproximately 50 miles distributed by

number of lanes and level of curvature ARAN Data Collection

5 passes in each direction on every routeazimuth data acquired every 4 meters

Survey AlignmentsCT DOT task from ground surveys

segments

UNC Highway Safety Research Center Highway Safety Information System

Data Processing

Horizontal Curve Classification and Display System (PLV-HC Software)Develop alignments from raw data

Make post-processing adjustmentsRemove unresolved (unknown) segments at

the end of a runConvert spirals to tangents and arcs (split the

difference in length)Remove “true” outliers (not be more than 1 in

the group of 5 runs)

UNC Highway Safety Research Center Highway Safety Information System

PLV-HC Software

UNC Highway Safety Research Center Highway Safety Information System

PLV-HC Software

UNC Highway Safety Research Center Highway Safety Information System

PLV-HC Software

UNC Highway Safety Research Center Highway Safety Information System

Visual Confirmation

Video log produced for each run

Allowed for the removal of induced curves

UNC Highway Safety Research Center Highway Safety Information System

Example Analysis (Rte 190)

Initial Best-Fit AlignmentLog Analysis 1

Log Analysis 2

Log Analysis 3

Log Validation 1

Log Validation 1

Rev Analysis 1

Rev Analysis 2_QC

QC_Harkey

QC_Van Dine

Rev Validation 1

Rev Validation 2

Rev Analysis 3

UNC Highway Safety Research Center Highway Safety Information System

Example Analysis (Rte 190)

Adjustment 1 - removal of unknowns (unresolved

segments)Log Analysis 1

Log Analysis 2

Log Analysis 3

Log Validation 1

Log Validation 1

Rev Analysis 1

Rev Analysis 2_QC

QC_Harkey

QC_Van Dine

Rev Validation 1

Rev Validation 2

Rev Analysis 3

UNC Highway Safety Research Center Highway Safety Information System

Example Analysis (Rte 190)

Adjustment 2 - Conversion of Spirals to Arcs and Tangents

Log Analysis 1

Log Analysis 2

Log Analysis 3

Log Validation 1

Log Validation 1

Rev Analysis 1

Rev Analysis 2_QC

QC_Harkey

QC_Van Dine

Rev Validation 1

Rev Validation 2

Rev Analysis 3

UNC Highway Safety Research Center Highway Safety Information System

Example Analysis (Rte 190)

Kappa Statistics for Log Direction

0.64 0.91 0.91 3 Analyses

0.51 0.83 0.89 2 Validations

0.66 0.87 0.91 3 Analyses + 2 Validations

Kappa Statistics for Reverse Direction

0.83 0.85 0.87 3 Analyses

0.72 0.89 0.84 2 Validations

0.79 0.89 0.89 3 Analyses + 2 Validations

0.91 0.91 0.89 Quality Control Analysis (H&Y)

0.91 0.93 0.92 Quality Control Analysis (H&Y&D)

UNC Highway Safety Research Center Highway Safety Information System

Consistency Kappa Statistics

11 sites (22 sets of results - log and rev)

K > 0.90 9 sets 0.90 > K > 0.75 8 sets K < 0.75 5 sets

UNC Highway Safety Research Center Highway Safety Information System

Example Analysis (Rte 190)

Accuracy AssessmentLog Analysis 1

Log Analysis 2

Log Analysis 3

Log Validation 1

Log Validation 1

Survey

K = 0.92

UNC Highway Safety Research Center Highway Safety Information System

Accuracy Kappa Statistics

11 sites (log direction only)

K > 0.90 2 sites 0.90 > K > 0.75 4 sites K < 0.75 5 sites

UNC Highway Safety Research Center Highway Safety Information System

Conclusions

Are the results consistent?77 percent of the directional groups met the

threshold

Are the results accurate?55 percent of the sites met the threshold

Neither consistent nor accurate enough

UNC Highway Safety Research Center Highway Safety Information System

Conclusions

Need to conduct individual arc analysisvariation in length and radius across

multiple runsvariation in PCs and PTsdifferences with survey results

Critical to validate the results of any automated data acquisition technology

UNC Highway Safety Research Center Highway Safety Information System

For the Latest Info

Visit the HSIS web site

www.hsisinfo.org