Maximizing Data for Reverse 911

CONNECT. TRANSFORM. AUTOMATE.

Maximizing Data for Reverse 911 Amanda Graf Senior Project Manager California CAD Solutions, Inc.

Situation

!  Local municipality needed to leverage their existing GIS system to enable reverse 911 notification of emergency events to mobile phones

!  GIS must interface with existing TENS (Telephone Emergency Notification System)

!  County Emergency Dispatch system had a match rate of less than 65% for mobile numbers

!  County system could not interface with City GIS

CONNECT. TRANSFORM. AUTOMATE.

Complicating Factors

!  Non-repeatable County process meant the updates were excruciatingly tedious and prone to error

!  No documentation of the County geocoding process led to no confidence in the data results

!  Erratic updates to base data used for geocoding !  Jurisdictional battles between City and County

CONNECT. TRANSFORM. AUTOMATE.

Major Factors Impeding Success

!  Multiple Data Vendors with radically different data structures and update methodologies !  AT&T – Monthly updates with a complete listing of

all phone records !  Verizon – Weekly updates with incremental

changes from the prior update delivery !  Inability to get AT&T and Verizon to make

changes to data anomalies (errors) !  Multiple sources of Address information

CONNECT. TRANSFORM. AUTOMATE.

Plan

!  Document address data sources and determine hierarchy of processing

!  Normalize address notations among all the data sources used

!  Process and normalize AT&T data !  Process and normalize Verizon data !  Deliver Geocoded dataset themed by source !  Deliver List of unmatched addresses !  Deliver documentation of entire process

CONNECT. TRANSFORM. AUTOMATE.

Process / Approach

!  Granular approach to the problem was the most effective !  Multiple FME routines

!  1 - Process AT&T Data !  2 – Process Verizon Data !  3 – Combine datasets into single datastore !  4 – Geocode the data

!  Scripted batch files to automate processing

CONNECT. TRANSFORM. AUTOMATE.

1 – Process AT&T Data

!  AT&T Data !  Straight forward CSV file !  “Street Name” included both street name and

street type in a single field !  Liberal use of SubstringExtractors,

AttributeTrimmers, and Testers used to break the information out into separate fields

CONNECT. TRANSFORM. AUTOMATE.

2 – Process Verizon Data

!  Verizon Data !  Fixed Length format requiring use of

SubstringExtractors !  Critically important to process the data sequentially

since a single number can be entered more than once in any particular update file

!  Determine if Insert, Update, or Delete is the appropriate action for each record

CONNECT. TRANSFORM. AUTOMATE.

Insert, Update or Delete?

CONNECT. TRANSFORM. AUTOMATE.

3 - Merge Datasets

!  Massive Data normalization process !  AT&T, Verizon, County Assessor, City Public Works

!  Each organization has their own way of designating (and spelling) addresses

!  1st or First? !  AV or AVE or Ave.? !  Mc Clay or McClay? (Use the MC Hammer) !  Green Oak PL should be Green Oak DR !  Misspellings Agencies won’t fix

CONNECT. TRANSFORM. AUTOMATE.

Normalize Data

CONNECT. TRANSFORM. AUTOMATE.

FME Advantages

!  Update FME routine with known exceptions and the work only needs to be done once

!  Quick and easy to incorporate new exceptions as they are found

!  Original source data is unaltered thereby enabling a viable audit trail of information

CONNECT. TRANSFORM. AUTOMATE.

4 - Geocoding

!  7 data sources used in geocoding process (sources noted in order of priority)

!  County Assessor Data !  City Situs Address Data !  County Assessor Mobile Home Data !  City Situs Mobile Home Address Data !  Street Centerline (Address Range Matching) !  Lat/Lon Lookup Table !  Known Invalid Addresses

CONNECT. TRANSFORM. AUTOMATE.

Verification Process

!  USPS.com !  Matched with LatLon Lookup table.

The lookup table was created by looking up the addresses on www.batchgeocode.com/lookup. All addresses are verified as valid addresses against USPS.com.

!  Loop Back through FME Routine 3 & 4 with edits and additional exceptions

CONNECT. TRANSFORM. AUTOMATE.

Unmatched Examples

!  201 FOREIGN EXCHANGE !  0 AFB !  1 VOIP CALLER !  T-MOBILE@HOME SERVICE

CONNECT. TRANSFORM. AUTOMATE.

Batch Processing

!  FME routines can be run from a batch file !  By using published parameters the FME routines

stay the same even as the source dataset names change each quarter

!  Use a template to create a .bat file for processing the data for the current quarter

!  Input names of source files (published parameters)

!  Run

CONNECT. TRANSFORM. AUTOMATE.

Display Themed Data in Map

CONNECT. TRANSFORM. AUTOMATE.

Pull Reports and Notify Residents

CONNECT. TRANSFORM. AUTOMATE.

Results

FME Saved the Day!! !  RESULTS!!! 99.6% of all records were matched

(100% of all records that had valid addresses were matched)

!  Fast, easy integration with the existing City GIS site

!  Documented, traceable results of worked performed

CONNECT. TRANSFORM. AUTOMATE.

Thank You!

!  Questions?

!  For more information: !  Amanda Graf – amanda.graf@calcad.com

!  California CAD Solutions, Inc. www.calcad.com

CONNECT. TRANSFORM. AUTOMATE.