Uniting traditional GIS and mainstream IT

Tonight’s Agenda, in no particular order:

What’s the problem? Enterprises such as DEFRA want to manage their spatiotemporal data using consolidation, quality and access strategies.

Who cares? Introducing a typical cast of stakeholders (and their diverse cultures) in a shared spatiotemporal data management programme…

Laws of the jungle. Pure “greenfields” are ever more rare – what technical constraints do implementors encounter in large projects?

Meet the Beast. What is IBM’s architectural approach when answering spatiotemporal data management and systems integration challenges?

Where next? With a baseline for data exchange in place, what happens next for an integrated spatiotemporal enterprise, and for our industry?

Questions? Heckle during the presentation, or save up for the end.

What’s the spatial problem?

For GI practitioners, the DEFRA challenge is to make spatiotemporal data management more efficient for both data providers and data recipients. This increased

efficiency will in turn lead to higher quality data and improved policy making.

Drawn from RASTER

DATA

Drawn from VECTOR

DATA

Drawn from NUMERIC,

TEXTUAL, or TEMPORAL

ATTRIBUTES

ORTHOPHOTO

CONTEXTUALBASE MAP

THEMATIC or CHOROPLETH

MAP

Spatiotemporal Data

DEFRA’s 400+ thematic business datasets

Ordnance Survey’sMasterMap & Rasters

UK Perspectives’ Aerial Photography

Spatiotemporal Data at DEFRA

Spatiotemporal Data Capture and Maintenance

Spatiotemporal Application Development

?Spatiotemporal

Data Management&

Systems Integration



Spatial Database(PostgreSQL, Oracle, Informix, DB2, MySQLMicrosoft SQL Server)

Traditional GIS

Traditional GIS

Mainstream IT applied to a

traditional GIS?

Data Provide

r

Data Provide

r

Data Provider

Data Provide

r

Data User

Data User

Data User

Data User

Repository

Data Provide

r

Data Provide

r

Data Provide

r

Data Provider

Data User

Data User

Data User

Data User

Data Provider Pain Remedy Benefit

Repeatedly distributing spatial data to multiple internal and external users in a point-to-point fashion takes time.

Deliver the data once to a central Repository from which multiple internal and external users download data on demand.

Time formerly spent managing our data’s distribution now goes toward improving and maintain our data instead.

Spatiotemporal Data Consolidation

Data User Pain Remedy Benefit

I have to find and manage the data I need for my application, and must ensure that I keep up to date data.

The Repository supplies all the required current data from one access point.

By pulling data from the Repository, I save time formerly spent managing that data myself.

However, central repositories reduce direct contact between people, the same people who used to tell you whether their data was fit for your purposes…or not. How can data users evaluate a

warehouse’s data quality without talking to the people creating and providing the data?

Spatiotemporal Data Quality Validation

Unclosed polygons

Closed polygons

Duplicate vertices

Clockwise exterior and interior rings

Proper exterior and interior ring

rotation

No duplicate vertices

Examples of potentially problematic geometries The same geometries with fixes applied

But… the consolidated datasets in a central repository and the associated quality reports are useless unless they can be searched by both data providers and potential data

recipients.

Spatiotemporal Data Storage and Inventory

Pain Remedy Benefit

It’s time consuming to distribute our spatial data in the many different formats that our users’ heterogenious systems require.

The Repository accepts the data in one standard submission format but distributes it to users in a variety of formats.

Instead of managing our data’s distribution formats, we now have more time to improve and maintain our data.

Pain Remedy Benefit

The data we need is not available in the required format, so we spend time reformatting the data ourselves.

The Repository provides data in the formats most often requested by data recipients.

Instead of reformatting data to suit our system’s needs, we can begin analysing that data immediately upon receiving it.

Spatiotemporal Multi-format Data Access

Primary Customers:

• Departments capturing spatial data for/from public constituents

• Domain specialists analysing spatial data to answer policy questions



?Spatiotemporal

Data Management&

Systems Integration



Do these 4 services comprise Spatio-

temporal Data

Manage-ment?





Spatiotemporal Infrastructure

Spatiotemporal Security

Data Provide

r

Data Provide

r

Data Provide

r

Data Provide

r

Data User

Data User

Data User

Data User

Repository

Data Provide

r

Data Provide

r

Data Provide

rData

Provider

Data User

Data User

Data User

Data User

Point-to-point security model Centralised, administered security model

Offering #9: Usage Metrics and Metering



Spatiotemporal Data Management

Systems Integration







Spatiotemporal Usage Metrics and Metering

Technical Constraints

Pure “greenfields” are ever more rare – what technical constraints do implementors encounter in large projects?

IBM’s technical constraints

IBM’s p-series hardware (64-cpu server)

IBM’s AIX 5.3 Operating System

IBM’s Websphere Application Server (with 64-bit Java Virtual Machine)

IBM’s Websphere Portal Server

Java Development Environment (preferred)

Websphere Application Server

Websphere Portal Server

DEFRA’s technical constraintsGI Clients: CadCorp and ESRI and MapInfo

OracleSpatial

Oracle 9i or 10g Spatial Database

Evaluating DEFRA’s constraints

Version compatibilities of ESRI, Oracle Spatial and OGC Web Services

IBM’s technical constraints

IBM’s p-series hardware (64-cpu server)

IBM’s AIX 5.3 Operating System

IBM’s Websphere Application Server (with 64-bit Java Virtual Machine)

IBM’s Websphere Portal Server

Java Development Environment (preferred)

Websphere Application Server

Websphere Portal Server

DEFRA’s technical constraints

GI Clients: CadCorp and ESRI and MapInfo

OracleSpatial

Oracle 9i or 10g Spatial Database

Geospatial Rendering Engine ??

The tension between traditional GIS, mainstream IT and free open-source geospatial software products

“How is it that groups of computer programmers (sometimes very large groups) made up of individuals separated by geography, corporate boundaries, culture, language, and other characteristics, and connected mainly via telecommunications bandwidth, manage to work together over time and build complex, sophisticated software systems outside the bounaries of a corporate structure and for no direct monetary compensation? And why does the answer to that question matter to anyone who is not a computer programmer?”

“This book explains how the open source software process works. It is broadly a book about technology and society, in the sense that changes in technology uncover hidden assumptions of inevitability in production systems and the social arrangements that accompany them. It is also about computers and software, because the success of open source rests ultimately on computer code, code that people often find more functional, reliable, and faster to evolve than most proprietary software built inside a conventional corporate organization. It is a business and legal story as well. Open source code does not obliterate profit, capitalism, or intellectual property rights. Companies and individuals are creating intellectual products and making money from open source software code, while inventing new business models and notions about property along the way.”

FOSS4G2006 (Free Open Source Software for Geospatial, 2006 Conference

What’s the next (bigger) problem?

For Information Technology practitioners in general, the DEFRA problem is systems integration in support of a more “joined-up” government organisation.

Evidence of success is the British public’s level of satisfaction with government services – for DEFRA, this public is often farmers or anyone involved with food,

livestock, disease or the rural enviroment.

What’s the spatial problem?

For GI practitioners, the DEFRA challenge is to make spatiotemporal data management more efficient for both data providers and data recipients. This increased

efficiency will in turn lead to higher quality data and improved policy making.

Secondary once-removed Customers:

• Government Administrators & Policy-makers

Spatiotemporal Processing Services

• Your postcode intersects a Nitrate Vulnerable Zone.

• Please confirm the following acreages for each field on your farm.

• You are eligible for countryside stewardship scheme funding due to your land’s proximity to a Site of Special Scientific Interest.


Unclosed polygons

Closed polygons

Duplicate vertices

Clockwise exterior and interior rings

Proper exterior and interior ring

rotation

No duplicate vertices

Note: Basic Validation does not test Spatiotemporal Contextual Data Quality. Although their edges should match, polygon data representing moorland (purple diagonal cross hatching) may extend beyond government-established boundaries for ecosystem protection (solid pink) due to data capture at different scales.

No Payment (outside of scheme)

Payment = £16.10/haDisadvantaged Ecosystem

Payment = £29.78/ha Severely Disadvantaged

Ecosystem

Payment = £11.26/haMoorland

Invalid data!RULE: Moorland Line must exist within LFA

Moorland

SeverelyDisadvantagedEcosystem Disadvantaged

Ecosystem

Land Parcel

This farmer’s land parcel may be subject to FOUR different payment tiers, as well as an erroneous payment due to contextually invalid geometric data.

Note: Basic Validation does not test Spatiotemporal Contextual Data Quality. Although their edges should match, polygon data representing moorland (purple diagonal cross hatching) may extend beyond government-established boundaries for ecosystem protection (solid pink) due to data capture at different scales.



Spatial Database(PostgreSQL, Oracle, Informix, DB2, MySQLMicrosoft SQL Server)

Traditional GIS

Traditional GIS

Mainstream IT applied to a

traditional GIS?




Systems Integration











Systems Integration





Spatiotemporal Data Processing




Step 1: An inspector discovers a diseased animal and reports its position. The State Veterinary Svc. creates an initial buffer zone (circle) around the point of disease discovery.

Step 2: SVS then expands the buffer zone (manually) to include intersected and epidemiologically relevant contiguous land known to host livestock management activities.

Step 3: SVS then identifies the livestock keepers contained within the zone and enlists their cooperation in controlling animal movement within it, enforced through a permitting process. In past outbreaks, this series of steps has taken several hours of desktop processing each night, accompanied by a manual results distribution process.

Disease Mapping Scenario for State Veterinary Service (SVS)

Farmer Jones

Farmer Blair

Farmer Smith

Disease Mapping Scenario for State Veterinary Service (SVS)

An alternative: Following disease point discovery, SVS places a point on a map, and sets a radius. The tool automatically selects underlying intersecting fields, allows for easy one-click expansion and automatically identifies the links to a land ownership database. Hours of processing are reduced to minutes and results are automatically available to all members of the service.

Possible technical solutions for Disease Zone polygon extension scenario

Land boundaries via(A) remote

spatial databaseconnection, or

(B) “throttled” WebFeature Services

connection

Local Client

Local Client

Local Client

Database & Web Server

Database & Citrix Server

Database & Web Server

1

2

3

Users’ mouseclicks are sent

to a remote webapplication that

is built for asingle functional

purpose.

Desktop ToolsCadCorp/ESRI/MapInfo

Client Spatial LayersDisease Zones

Desktop ToolsCitrix Client

simulates CacCorp/ ESRI/MapInfo

Client Spatial Layers[None]

Desktop ToolsWeb Browser

Client Spatial Layers[None]

Users’ mouseclicks are sentto Citrix server

which simulatesa local GIS desktopon the local client.

Local GI Client & Remote Spatial Data: Use a local GI client to snap locally stored Disease Zone polygons to remote land boundary geometry via (A) network access to a remote spatial database holding PBL data (e.g. a “mart”), or, (B) a “throttled” PBL Web Feature Service.

Remote Solution-specific GI Application & Remote Spatial Data: Use a web browser and online custom application to add or remove land boundary polygons from the Disease Zone polygon by clicking desired land boundary polygons.

Remote ArcGIS & Remote Spatial Data: Use a remote GI client, provided via Citrix, to snap (remotely stored) Disease Zone polygons to underlying (remotely stored) land boundary polygons. The GI client accesses the remote spatial data via spatial database connections or “throttled” Web Feature Service connections.

Connection

Connection

Connection

Pros: Power users can apply full suite of desktop GI capabilities. Access to features enables local analysis and copy/snap.

Cons: Performance limited by network bandwidth. New data must go back through QC system. Desktop GI client training time and software/licence costs.

Pros: Same pros as (1) above, plus good performance and manageable deployment, licencing, and version control of GI Client tools.

Cons: Citrix licence is an extra expense. Cost of spatial database software & licence. Cost of GI client tool training.

Pros: Same pros as (1) above, plus ease of deployment, no licencing, minimal training, good performance and wide accessibility.

Cons: Time/cost of capturing requirements and building custom solution.

HeavyCPU load

Server Spatial LayersLand boundaries

Aerial Photos

Server Spatial LayersLand boundariesDisease ZonesAerial Photos

Server Spatial LayersLand boundariesDisease ZonesAerial Photos

HeavyNetwork

traffic

HeavyCPU load

LightCPU load

ModerateNetwork

traffic

HeavyCPU load

LightCPU load

LightNetwork

traffic

HeavyCPU load






Spatiotemporal Data Processing





Tonight’s Agenda, in no particular order:

What’s the problem? Enterprises such as DEFRA want to manage their spatiotemporal data using consolidation, quality and access strategies.

Who cares? Introducing a typical cast of stakeholders (and their diverse cultures) in a shared spatiotemporal data management programme…

Laws of the jungle. Pure “greenfields” are ever more rare – what technical constraints do implementors encounter in large projects?

Meet the Beast. What is IBM’s architectural approach when answering spatiotemporal data management and systems integration challenges?

Where next? With a baseline for data exchange in place, what happens next for an integrated spatiotemporal enterprise, and for our industry?

Questions? Heckle during the presentation, or save up for the end.

Uniting traditional GIS and mainstream IT

Documents

data users

data providers

reformatting data

vector data

data exchange

raster data

date data

spatiotemporal data