Maximizing Data for Reverse 911

CONNECT. TRANSFORM. AUTOMATE.

Maximizing Data for Reverse 911 Amanda Graf Senior Project Manager California CAD Solutions, Inc.

Situation

!  Local municipality needed to leverage their existing GIS system to enable reverse 911 notification of emergency events to mobile phones

!  GIS must interface with existing TENS (Telephone Emergency Notification System)

!  County Emergency Dispatch system had a match rate of less than 65% for mobile numbers

!  County system could not interface with City GIS


Complicating Factors

!  Non-repeatable County process meant the updates were excruciatingly tedious and prone to error

!  No documentation of the County geocoding process led to no confidence in the data results

!  Erratic updates to base data used for geocoding !  Jurisdictional battles between City and County


Major Factors Impeding Success

!  Multiple Data Vendors with radically different data structures and update methodologies !  AT&T – Monthly updates with a complete listing of

all phone records !  Verizon – Weekly updates with incremental

changes from the prior update delivery !  Inability to get AT&T and Verizon to make

changes to data anomalies (errors) !  Multiple sources of Address information


Plan

!  Document address data sources and determine hierarchy of processing

!  Normalize address notations among all the data sources used

!  Process and normalize AT&T data !  Process and normalize Verizon data !  Deliver Geocoded dataset themed by source !  Deliver List of unmatched addresses !  Deliver documentation of entire process


Process / Approach

!  Granular approach to the problem was the most effective !  Multiple FME routines

!  1 - Process AT&T Data !  2 – Process Verizon Data !  3 – Combine datasets into single datastore !  4 – Geocode the data

!  Scripted batch files to automate processing


1 – Process AT&T Data

!  AT&T Data !  Straight forward CSV file !  “Street Name” included both street name and

street type in a single field !  Liberal use of SubstringExtractors,

AttributeTrimmers, and Testers used to break the information out into separate fields


2 – Process Verizon Data

!  Verizon Data !  Fixed Length format requiring use of

SubstringExtractors !  Critically important to process the data sequentially

since a single number can be entered more than once in any particular update file

!  Determine if Insert, Update, or Delete is the appropriate action for each record


Insert, Update or Delete?


3 - Merge Datasets

!  Massive Data normalization process !  AT&T, Verizon, County Assessor, City Public Works

!  Each organization has their own way of designating (and spelling) addresses

!  1st or First? !  AV or AVE or Ave.? !  Mc Clay or McClay? (Use the MC Hammer) !  Green Oak PL should be Green Oak DR !  Misspellings Agencies won’t fix


Normalize Data


FME Advantages

!  Update FME routine with known exceptions and the work only needs to be done once

!  Quick and easy to incorporate new exceptions as they are found

!  Original source data is unaltered thereby enabling a viable audit trail of information


4 - Geocoding

!  7 data sources used in geocoding process (sources noted in order of priority)

!  County Assessor Data !  City Situs Address Data !  County Assessor Mobile Home Data !  City Situs Mobile Home Address Data !  Street Centerline (Address Range Matching) !  Lat/Lon Lookup Table !  Known Invalid Addresses


Verification Process

!  USPS.com !  Matched with LatLon Lookup table.

The lookup table was created by looking up the addresses on www.batchgeocode.com/lookup. All addresses are verified as valid addresses against USPS.com.

!  Loop Back through FME Routine 3 & 4 with edits and additional exceptions


Unmatched Examples

!  201 FOREIGN EXCHANGE !  0 AFB !  1 VOIP CALLER !  T-MOBILE@HOME SERVICE


Batch Processing

!  FME routines can be run from a batch file !  By using published parameters the FME routines

stay the same even as the source dataset names change each quarter

!  Use a template to create a .bat file for processing the data for the current quarter

!  Input names of source files (published parameters)

!  Run


Display Themed Data in Map


Pull Reports and Notify Residents


Results

FME Saved the Day!! !  RESULTS!!! 99.6% of all records were matched

(100% of all records that had valid addresses were matched)

!  Fast, easy integration with the existing City GIS site

!  Documented, traceable results of worked performed


Thank You!

!  Questions?

!  For more information: !  Amanda Graf – [email protected]

!  California CAD Solutions, Inc. www.calcad.com


Maximizing Data for Reverse 911

Technology

process verizon data

process att data

data results

themed data

base data

county assessor data

original source data

multiple data vendors