Implementing a Simple Web Application with FME Server

CONNECT. TRANSFORM. AUTOMATE.

Implementing a simple web application with FME ServerLogan PughProgrammer Analyst, City of Austin

Tony CastroGIS Analyst, City of Austin

Ben VanderfordGIS Analyst Senior, City of Austin

Who, what, when, where, why and how?

Who: Small GIS shop in a big (and growing) city

What: Provide zoning verifications (among other services) to property owners and developers

When: 50 to 80 requests a month

Where: City of Austin, Texas

Why: Each request used to take 1-2 hours

How: Tedious manual process of verifying zoning, hand-editing a PDF and sending it to customer

What is zoning verification?

Zoning in Austin started in 1930s

Zoning verifications started in 1960s

Developers and property owners need zoning verification to get financing for (re)development

Old Process was Cumbersome: The Fatigue Factor

Lots of research and keying in values Name of applicant and address Parcel ID Index grid Zoning type, definition and ordinance

Print, sign and mail/fax/hand-deliver letter to sales office for customer pickup

Old letter format New letter format

The Tipping Point

With an ever-increasing workload and not-so-increasing budget/staffing, we need to increase efficiencies

Our staff time stretched, a new zoning verification process became a high priority

Along comes FME Server: we had a new option!

Project Requirements

Accept a parcel number in various formats

Perform zoning info drilldown with complex filtering logic

Allow operator to edit/correct info

Generate PDF with electronic signature automatically

Ability to easily be configured for future changes (new users, jurisdictions, etc.)

Spell checking

Challenges

How to automate what is by definition a human-verified process?

How can we own the process but provide access to people outside of our workgroup?

How can we build this without any additional infrastructure?

FME Server Data Streaming Service

Serves up the results of workspaces directly

Can be HTML, JSON, PDF, etc.

Result must be a single file

Allows everything to work within the context of a web browser

FME Server REST API

Simple protocol for programmatically interacting with FME Server

Uses HTTP requests, e.g. GET, PUT, DELETE

Authentication using simple token service

Tokens are per-user, but can be shared for specific applications

FME REST API Developer Playground: http://fmeserver.com/userweb/sharper/playground/

Workspaces

Zoning info drilldown workspace

Given a parcel number and parcel agency, returns the zoning types, ordinances and case numbers intersecting a given parcel

Negative buffer to reduce erroneous results caused by inaccuracies between zoning and parcel data

Filter out non-applicable zoning ordinances

Zoning Info Drilldown workspace

Workspaces, cont.

ZoningOrdinanceDateCalculator custom transformer ZoningOrdinanceFilter custom transformer

Workspaces, cont.

PDF generation workspace

Given various parameters, produces a complete, signed zoning verification letter in PDF format

CSV to JSON workspaces

Editors

Requestors

Agencies

Workspaces, cont.PDFTextBlockCreator custom transformerPDF Builder workspace

PDFRectangleTextCreator custom transformer

PDFImageCreator custom transformer

Workspaces, cont.

HTML form builder workspace

Serves up an HTML file that ties everything together

Uses JavaScript (Dojo Toolkit) and the FME Server REST API to request, display, edit, and submit data

Can be pre-populated with data via parameters submitted to FME Server

Demo

Behind the scenes

Key Transformers FeatureReader: Performs queries against any

FME-supported format. The queries can have both a spatial and a nonspatial component.

One query is issued to the FME-supported format for each feature that enters the transformer. The results of the query are then output.

This means workspaces can read data dynamically

One of the most powerful transformers in FME!

Key Transformers, cont. PythonCaller: Executes a Python script to

manipulate the feature.Useful when implementing logic too complex to handle with other transformers.

Examples:

Fixed-width word-wrapping, justification and special character escaping for text elements on output PDF

Using regular expressions to replace tokens in HTML file with parameter values

Building SQL expressions and sanitizing user input

Key Transformers, cont. JSONTemplater: Populates a JSON document

with FME feature attribute values{ "status" : xs:int(fme:get-attribute("_statuscode")), "message" : fme:get-attribute("_message"), "data" : { "parcel_agency": fme:get-attribute("_PARCEL_AGENCY"), "parcel_id": fme:get-attribute("_PARCEL_ID"), "zoning_ztypes": [fme:get-list-attribute("_ztype_list{}.ZONING_ZTYPE")], "zoning_ordinance_case_numbers": [fme:get-list-attribute("_case_number_list{}.CASE_NUMBER")], "zoning_ordinance_numbers": [fme:get-list-attribute("_ordinance_number_list{}.ZONING_ORDINANCE_NUMBER")] }}

Key Transformers, cont.

Key Transformers, cont.

{ "status" : xs:int(fme:get-attribute("_statuscode")), "message" : fme:get-attribute("_message"), "data" : { "parcel_agency": fme:get-attribute(“_PARCEL_AGENCY"), "parcel_id": fme:get-attribute("_PARCEL_ID"), "zoning_ztypes": [fme:get-list-attribute("_ztype_list{}.ZONING_ZTYPE")], "zoning_ordinance_case_numbers": [fme:get-list-attribute("_case_number_list{}.CASE_NUMBER")], "zoning_ordinance_numbers": [fme:get-list-attribute("_ordinance_number_list{}.ZONING_ORDINANCE_NUMBER")] }}

Key Transformers, cont.

{ "status" : xs:int(fme:get-attribute("_statuscode")), "message" : fme:get-attribute("_message"), "data" : { "parcel_agency": fme:get-attribute("_PARCEL_AGENCY"), "parcel_id": fme:get-attribute("_PARCEL_ID"), "zoning_ztypes": [fme:get-list-attribute("_ztype_list{}.ZONING_ZTYPE")], "zoning_ordinance_case_numbers": [fme:get-list-attribute("_case_number_list{}.CASE_NUMBER")], "zoning_ordinance_numbers": [fme:get-list-attribute("_ordinance_number_list{}.ZONING_ORDINANCE_NUMBER")] }}

Configuration

Enterprise applications require configurability

Avoid putting file paths, user names, passwords, etc. directly in FME workspaces

Allows workspaces to be shared with confidence, placed under source control (e.g. Git), and configured externally

Uses simple text-based configuration files

Configuration, cont.

FME ServerRepositories

Workspace 1

Workspace 2

Workspace n

dev

SecureNetworkFolders

Workspace 1

Workspace 2

Workspace n

test

Workspace 1

Workspace 2

Workspace n

prod

Configuration, cont.

import os, ConfigParser

global config_dict # Make variable accessible to other scripted parametersconfig_dict = None

# Get path to config file located alongside published workspaceconfig_path_file = os.path.join(FME_MacroValues['FME_MF_DIR'], 'config_path.txt')

# Return config key/value pairs as a dictionarywith open(config_path_file, 'r') as f: config_file = f.readline().strip() config_parser = ConfigParser.RawConfigParser() config_parser.read(config_file) config_dict = dict(config_parser.items('config')) return config_file

Use Python scripted parameters to read in configuration data. In first scripted parameter:

Configuration, cont.

# Just reference dictionary keys to get config valuesreturn config_dict['fmerest_token']

Subsequent scripted parameters are easy:

Conclusions

Reduced processing time from 1-2 hours to ~15 minutes

Ability to email generated PDF to customer, saving on paper use and mailing expenses

Laid groundwork for use by other departments and the public

Demonstrates flexibility of FME Server

Future improvements

Make app directly available to public

Tie in with existing zoning profile web application

Accept payments online

Collaboration with external parcel agencies on data alignment for improved accuracy

Thank You!

Questions?

For more information: Logan Pugh

logan.pugh@austintexas.gov City of Austin

Related web app available for public: http://www.austintexas.gov/gis/zoningprofile/