Introoraspatial

Introduction to Oracle Introduction to Oracle Spatial Using Public DataSpatial Using Public Data

Richard L. FloresRichard L. FloresIsinglass, Inc.Isinglass, Inc.

[email protected]@yahoo.com

About MeAbout Me Schooling in Geology and Business.Schooling in Geology and Business. 16 years working with Geographical 16 years working with Geographical

Information Systems and Computer Information Systems and Computer Mapping in the Oil & Gas Industry.Mapping in the Oil & Gas Industry.

15 years working with Oracle, mostly 15 years working with Oracle, mostly as DBA.as DBA.

Sporadic experience with Oracle Sporadic experience with Oracle Spatial since late Oracle 7.Spatial since late Oracle 7.

Reading MaterialReading Material Clearly written with Clearly written with

wonderful, wonderful, downloadable downloadable examples for you examples for you to work through.to work through.

Covers most Covers most features of Oracle features of Oracle Spatial.Spatial.

Reading MaterialReading Material Wide ranging Wide ranging

topics on the FREE topics on the FREE acquisition of data acquisition of data and tools for the and tools for the processing of processing of geographic data.geographic data.

Great tips and Great tips and insight on the field insight on the field of Electronic of Electronic Cartography.Cartography.

AgendaAgenda

Geographic Data ProcessingGeographic Data Processing Cartography 101Cartography 101 Oracle Spatial ProductsOracle Spatial Products Oracle Spatial Basics by ExampleOracle Spatial Basics by Example

Geographic Data ProcessingGeographic Data ProcessingThe Processing StepsThe Processing Steps

Data AcquisitionData Acquisition Preliminary Data ProcessingPreliminary Data Processing Data Storage and RetrievalData Storage and Retrieval Graphical Display (Visualization), Graphical Display (Visualization),

Analysis, and Interaction.Analysis, and Interaction.

Geogaphic Data ProcessingGeogaphic Data ProcessingData AcquisitionData Acquisition

SourcesSources• Global Positioning Systems (GPS)Global Positioning Systems (GPS)• Published and UnpublishedPublished and Unpublished• Free and CommercialFree and Commercial

FormatsFormats• Vector: shapefiles, DLG, DXF, NTF, etc.Vector: shapefiles, DLG, DXF, NTF, etc.• Raster: TIFF, GIF, etc.Raster: TIFF, GIF, etc.

Geogaphic Data ProcessingGeogaphic Data ProcessingPreliminary Data ProcessingPreliminary Data Processing

May InvolveMay Involve• Digitization of paper mapsDigitization of paper maps• Vectorization of raster imagesVectorization of raster images• Transformation of Mapping CoordinatesTransformation of Mapping Coordinates• Geocoding: calculation of geographic Geocoding: calculation of geographic

coordinates from street addresses.coordinates from street addresses.

Geogaphic Data ProcessingGeogaphic Data ProcessingData Storage and RetrievalData Storage and Retrieval

For OracleFor Oracle• StorageStorage

Use of Oracle’s Geographic Data ModelUse of Oracle’s Geographic Data Model Database design for the integration of spatial and Database design for the integration of spatial and

non-spatial datanon-spatial data SQL and SQL*Loader for loading data in Oracle.SQL and SQL*Loader for loading data in Oracle. Oracle’s “shapefile” to “SDO” utility.Oracle’s “shapefile” to “SDO” utility.

• RetrievalRetrieval It all comes down to SQL using Oracle Spatial stored It all comes down to SQL using Oracle Spatial stored

procedures.procedures.

Geogaphic Data ProcessingGeogaphic Data ProcessingDisplay, Analysis, and Interaction.Display, Analysis, and Interaction.

Not many free programs that work with Not many free programs that work with Oracle Spatial directly, unfortunately.Oracle Spatial directly, unfortunately.

Many commercial programs for displaying Many commercial programs for displaying and interactively querying and editing and interactively querying and editing Oracle Spatial data such as those from ESRI, Oracle Spatial data such as those from ESRI, Intergraph, MapInfo, Autodesk, eSpatial, and Intergraph, MapInfo, Autodesk, eSpatial, and Acquis.Acquis.

Cartography 101Cartography 101The Shape of the Earth: Some FactsThe Shape of the Earth: Some Facts

To accurately place a point on To accurately place a point on the earth, the general shape of the earth, the general shape of the earth must be known. the earth must be known. These shape models are called, These shape models are called, datumsdatums..

The earth is not flat.The earth is not flat.11

The earth is not a perfect The earth is not a perfect sphere.sphere.

The earth’s general shape is The earth’s general shape is best described by a theoretical best described by a theoretical surface called a surface called a geoidgeoid..

For mapping, geoids are For mapping, geoids are approximated by approximated by reference reference ellipsoidsellipsoids..

11Assertions of the Flat Earth Society notwithstanding.

Cartography 101 Cartography 101 The Shape of the Earth: North American DatumThe Shape of the Earth: North American Datum

Older maps used Older maps used NAD27NAD27 Newer maps use the more accurate Newer maps use the more accurate

NAD83NAD83 made using satellites and made using satellites and sophisticated electronic equipment. sophisticated electronic equipment.

Worldwide GPS use a datum Worldwide GPS use a datum compatible to NAD83 called compatible to NAD83 called WGS84WGS84

The coordinates for a point on the The coordinates for a point on the earth can vary significantly earth can vary significantly depending on the datum used.depending on the datum used.

Cartography 101 Cartography 101 Spatial Reference SystemsSpatial Reference Systems

SRSSRS are coordinate-based systems are coordinate-based systems used to locate geographical entities.used to locate geographical entities.

Oracle Geographic SRS TypesOracle Geographic SRS Types• GeodeticGeodetic: Latitude & Longitude: Latitude & Longitude• Projected Coordinate SystemsProjected Coordinate Systems: X & Y : X & Y

where globe is mathematically where globe is mathematically “projected” onto a flat map.“projected” onto a flat map.

Cartography 101Cartography 101Why are SRS important?Why are SRS important?

During analysis, if all spatial data is During analysis, if all spatial data is not in the same Spatial Reference not in the same Spatial Reference System, the geographic layers will System, the geographic layers will not overlay and the analysis will be not overlay and the analysis will be in error.in error.



During analysis, if an inappropriate During analysis, if an inappropriate Spatial Reference System is chosen, Spatial Reference System is chosen, you may introduce unacceptable you may introduce unacceptable distortion into your maps.distortion into your maps.


If you were If you were interested in interested in using a map to using a map to measure land measure land areas, you would areas, you would probably not use probably not use this Spatial this Spatial Reference Reference System.System.

Cartography 101Cartography 101Geodetic Coordinate SystemGeodetic Coordinate System

Latitude/Longitude expressed in Degrees/Minutes/Seconds Latitude/Longitude expressed in Degrees/Minutes/Seconds or Decimal Degreesor Decimal Degrees

Parallels: Lines of latitude North (+) & South (-) of Equator Parallels: Lines of latitude North (+) & South (-) of Equator (-90 to +90)(-90 to +90)

Meridians: Lines of longitude East (+) & West (-) of the Meridians: Lines of longitude East (+) & West (-) of the Prime Meridian (-180 to +180)Prime Meridian (-180 to +180)

Oracle Uses Decimal Degrees in the odd order, Oracle Uses Decimal Degrees in the odd order, Longitude/LatitudeLongitude/Latitude

Cartography 101Cartography 101Geodetic Coordinate SystemGeodetic Coordinate System

Cartography 101 Cartography 101 Projected Coordinate SystemsProjected Coordinate Systems

Cartography 101 Cartography 101 State Plane Coordinate SystemsState Plane Coordinate Systems

•Each state has defined SPCS zones which are popular for local- and state-sized areas because of their accuracy.•“Tall” States, such as TX and CA use the Lambert Conformal Conic Projection.•“Wide” States, such as NY use Transverse Mercator Projection.California Zone III, 0403

Texas South Central Zone, 4204

Cartography 101Cartography 101 Projected Coordinate SystemsProjected Coordinate Systems

Select map projection for display based Select map projection for display based on:on:• How you intend to use the dataHow you intend to use the data• The size and location of the area of interestThe size and location of the area of interest• The metric property of the map you wish to The metric property of the map you wish to

preserve:preserve: ShapeShape AreaArea DistanceDistance DirectionDirection ScaleScale

See See erg.usgs.gov/isb/pubs/MapProjections/projections.htmlerg.usgs.gov/isb/pubs/MapProjections/projections.html

Oracle ProductsOracle ProductsOracle LocatorOracle Locator

IncludesIncludes• Geographic Data ModelGeographic Data Model• Query and Analysis using the Index Query and Analysis using the Index

EngineEngine• Some advanced geometric functionsSome advanced geometric functions

Free with the Standard or Enterprise Free with the Standard or Enterprise EditionEdition

Oracle ProductsOracle ProductsOracle Spatial OptionOracle Spatial Option

IncludesIncludes• All Oracle Locator featuresAll Oracle Locator features• More advanced spatial functionsMore advanced spatial functions• Location enabling geocoderLocation enabling geocoder• Advanced features such as routing, Advanced features such as routing,

network analysis, and georaster.network analysis, and georaster. Added-cost option with the Added-cost option with the

Enterprise EditionEnterprise Edition

Oracle SpatialOracle SpatialGeneralGeneral

•“An integrated set of functions and procedures that enables spatial data to be stored, accessed, and analyzed quickly and efficiently in an Oracle database.”

•A schema (MDSYS) that prescribes the storage, syntax, and semantics of supported geometric data types.•A spatial indexing mechanism•A set of operators and functions for performing area-of-interest queries, spatial join queries, and other spatial analyis operations.•Administrative utilities.

Oracle SpatialOracle SpatialSome DefinitionsSome Definitions

Basic Spatial Elements

Point Line String N-Point Polygon

•The are other, more complex elements, including collections of one or more elements called Geometries. •A Layer is collection of Geometries having similar attributes such a state boundaries, roads, or rivers.

Oracle SpatialOracle SpatialMore on Geometries and LayersMore on Geometries and Layers

ArkansasArkansas

ArizonaArizona

AlaskaAlaska

AlabamaAlabamaTable, STATES = LAYER

A GEOMETRY (outline) of a state is in a single column of each row.

A GEOMETRY may contain more than one element.

Oracle Spatial by ExampleOracle Spatial by ExampleScenarioScenario

You wish to open an upscale beauty You wish to open an upscale beauty salon in central Contra Costa county, salon in central Contra Costa county, California, catering to wealthier, California, catering to wealthier, older women.older women.

You would like to be close to a major You would like to be close to a major thoroughfare for ease of access.thoroughfare for ease of access.

You don’t want to be too close to any You don’t want to be too close to any competitors.competitors.

Oracle Spatial by ExampleOracle Spatial by ExampleIdentify Types and Sources of Data Needed to Support Identify Types and Sources of Data Needed to Support

DecisionDecision

Competitors: Internet Search Engine Competitors: Internet Search Engine Demographic (Age, Gender, Income): U.S. Demographic (Age, Gender, Income): U.S.

Census BureauCensus Bureau Roads: U.S. Geological SurveyRoads: U.S. Geological Survey

Oracle Spatial by ExampleOracle Spatial by ExampleCompetitor Data: TableCompetitor Data: Table

CREATE TABLE beauty (id NUMBER(38), name VARCHAR2(100), full_address VARCHAR2(100), city_state VARCHAR2(50), street_number VARCHAR2(10), street_name VARCHAR2(20), street_type VARCHAR2(15), street_prefix VARCHAR2(10), street_suffix VARCHAR2(10), city VARCHAR2(40), state VARCHAR2(2), postal_code VARCHAR2(16), location MDSYS.SDO_GEOMETRY);

Oracle Spatial by ExampleOracle Spatial by ExampleCompetitor Data: Spatial MetadataCompetitor Data: Spatial Metadata

INSERT INTO user_sdo_geom_metadata VALUES ('BEAUTY', -- Geometry Table 'LOCATION', -- Geometry Column SDO_DIM_ARRY ( SDO_DIM_ELEMENT ('LONGITUDE', -- Longitude Text -180, -- Lower Boundary 180, -- Upper Boundary 0.5), -- Tolerance SDO_DIM_ELEMENT ('LATITUDE', -- Latitude Text -90, -- Lower Boundary 90, -- Upper Boundary 0.5) -- Tolerance ), 8307 -- (SRID) Datum:WGS84 );

Oracle Spatial by ExampleOracle Spatial by ExampleCompetitor Data: Spatial IndexCompetitor Data: Spatial Index

CREATE INDEX beauty_spatial_idx ON beauty (location)INDEXTYPE IS MDSYS.SPATIAL_INDEX;

•R-Tree Index•These are unlike regular Oracle indexes and special steps must be taken with their administration.

Oracle Spatial by ExampleOracle Spatial by ExampleCompetitor Data: SourceCompetitor Data: Source

Name ---------------- ID NAME FULL_ADDRESS CITY_STATE STREET_NUMBER STREET_NAME STREET_TYPE STREET_PREFIX STREET_SUFFIX CITY STATE POSTAL_CODE LOCATION

Extract list of competitors and their addresses from Search Engine.

While very useful, it doesn’t provide any directly mappable data.

Oracle Spatial by ExampleOracle Spatial by ExampleCompetitor Data: GeocodingCompetitor Data: Geocoding

•The Geocoder will •Standardize Address Name and,•Using a database with the coordinates and street addresses of each intersection,•Interpolate the location of the given address.

•Oracle Spatial Option geocoder: added-cost •Third party sells spatial database used to calculate the coordinates


Solution: Use Perl Program against internet geocoding website.

#!/usr/local/bin/perl# simplest_xmlrpc.pluse XMLRPC::Lite;use Data::Dumper;use strict;use warnings;my $where = shift @ARGV or die "Usage: $0 \"1 Main St, Anytown, KS\"\n";my $result = XMLRPC::Lite -> proxy( 'http://rpc.geocoder.us/service/xmlrpc' ) -> geocode ($where) -> result;print Dumper $result;

From Mapping Hacks, Tips & Tools for Electronic Mapping


simplest_xmlrpc.pl “1355 N. Main, Walnut Creek, CA"

$VAR1 = [ { 'number' => '1355', 'street' => 'Main', 'lat' => '37.898365', 'state' => 'CA', 'city' => 'Walnut Creek', 'zip' => '94596', 'suffix' => '', 'long' => '-122.060445', 'type' => 'St', 'prefix' => 'N' } ];

Name ---------------- ID NAME FULL_ADDRESS CITY_STATE STREET_NUMBER STREET_NAME STREET_TYPE STREET_PREFIX STREET_SUFFIX CITY STATE POSTAL_CODE LOCATION

Oracle Spatial by ExampleOracle Spatial by ExampleCompetitor Data: SDO_GEOMETRY Object-Relational Competitor Data: SDO_GEOMETRY Object-Relational

TypeType

UPDATE beauty SET location = SDO_GEOMETRY (2001, -- Geometry Type: 2-D Point 8307, -- SRID, Datum: WGS84 SDO_POINT_TYPE (-122.060445, -- Longitude 37.898365, -- Latitude NULL), NULL, NULL ) WHERE id = 430;

Oracle Spatial by ExampleOracle Spatial by ExampleCompetitor Data: Data DisplayCompetitor Data: Data Display

Martinez

Orinda

Dublin

Clayton

•eSpatial iSmart Explorer free on OTN•OEM Spatial Index Advisor•Oracle Mapviewer•For serious users, many commercial products.

Oracle Spatial by ExampleOracle Spatial by ExampleNon-Spatial Demographic Data: TableNon-Spatial Demographic Data: Table

CREATE TABLE census_data ( CENSUS_TRACT VARCHAR2(10)NOT NULL, MED_HOUSE_INCOME NUMBER(38), GENDER_TOTAL NUMBER(38), FEMALE_GE_40 NUMBER(38));

Oracle Spatial by ExampleOracle Spatial by ExampleNon-Spatial Demographic Data: SourceNon-Spatial Demographic Data: Source

CENSUS_TRACT MED_HOUSE_INCOME FEMALE_GE_40 GENDER_TOTAL--------------- ---------------- ------------ ------------ 3010 44871 975 33553020.02 58769 1467 8475 * * * * * * *

•U.S. Census Bureau•factfinder.census.gov•“Download Center”•Select detailed or summarized data by state, county, and census tract.

Oracle Spatial by ExampleOracle Spatial by ExampleSpatial Census Tract Data: SourceSpatial Census Tract Data: Source

•www.census.gov/geo/www/cob/tr_metadata.html•Has geographic boundaries of Census Tracts which can be loaded into Oracle Spatial.•Choose state and “ARCVIEW Shapefile” format to download file for California. These files are sometimes called “ESRI Shapefiles”.

http://www.census.gov/geo/www/cob/tr_metadata.html

Oracle Spatial by ExampleOracle Spatial by ExampleSpatial Census Tract Data: Pre-processingSpatial Census Tract Data: Pre-processing

• shp2sdo utility downloadable from Oracle will create SQL and SQL*Loader data and control files for creating Spatial objects and loading shapefile data into Oracle Spatial.

./shp2sdo.exe tr06_d00 census_tracts -g geom \-x $-180,180$ -y $-90,90$ -s 8307 -t 0.5 -v

Shapefile Name PrefixTable Name

Column Name

Longitude Limits

Latitude LimitsSRID

Tolerance

Creates: census_tracts.sql, census_tracts.ctl, census_tracts.dat

Oracle Spatial by ExampleOracle Spatial by ExampleSpatial Census Tract Data: LoadingSpatial Census Tract Data: Loading

census_tract.sql

DROP TABLE CENSUS_TRACTS;

CREATE TABLE CENSUS_TRACTS ( AREA NUMBER, PERIMETER NUMBER, TR06_D00_ NUMBER, TR06_D00_I NUMBER, STATE VARCHAR2(2), COUNTY VARCHAR2(3), TRACT VARCHAR2(6), NAME VARCHAR2(90), LSAD VARCHAR2(2), LSAD_TRANS VARCHAR2(50), GEOM MDSYS.SDO_GEOMETRY);

Oracle Spatial by ExampleOracle Spatial by ExampleSpatial Census Tract Data: LoadingSpatial Census Tract Data: Loading

•In SQL*Plus:connect spatial/spatial@census_tracts.sql

•Run SQL*Loader:sqlldr spatial/spatial census_tracts

•In SQL*Plus:connect spatial/spatialEXECUTE SDO_MIGRATE.TO_CURRENT(‘CENSUS_TRACTS’,’GEOM)’

Oracle Spatial by ExampleOracle Spatial by ExampleSpatial Census Tract Data: DisplaySpatial Census Tract Data: Display

•Census tract outlines.•You can “CREATE TABLE SELECT AS” on state=’06’ and county=‘013’ to get just Contra Costa county.

Oracle Spatial by ExampleOracle Spatial by ExampleRoad Data: SourceRoad Data: Source

•seamless.usgs.gov•Bureau of Transportation Statistics from U.S. Geological Survey.•shapefiles

Oracle Spatial by ExampleOracle Spatial by ExampleRoad Data: DisplayRoad Data: Display

Oracle Spatial by ExampleOracle Spatial by ExampleAnalysis: Criteria DefinitionAnalysis: Criteria Definition

•Within 2 miles of census tracts in which•The Median Household Annual Income is greater then $100K and•Over 30% of the people are women 40 years or older

•Within ½ mile of a major thoroughfare•Not within ½ mile of a competitor

Oracle Spatial by ExampleOracle Spatial by ExampleAnalysis: Oracle Spatial BuffersAnalysis: Oracle Spatial Buffers

Original Original GeometryGeometry

Buffered Buffered GeometryGeometry

PointPoint

Line StringLine String

PolygonPolygon

Oracle Spatial by ExampleOracle Spatial by ExampleAnalysis: Target Census Tract BufferAnalysis: Target Census Tract Buffer

CREATE TABLE target_tract_buffer ASSELECT SDO_AGGR_UNION(SDOAGGRTYPE( SDOAGGRTYPE( SDO_GEOM.SDO_BUFFER( a.geom, -- geometry column 2.00, -- Distance 0.5, 'arc_tolerance=0.005 unit=mile'), -- Units 0.5)) geom FROM census_tracts a, census_data bWHERE b.census_tract = a.name AND b.med_house_income >=100000 AND b.female_ge_40/b.gender_total >= 0.30 AND a.state = '06' AND a.county = '013';


SDO_GEOM.SDO_BUFFER

SDO_AGGR_UNIONOriginal Points


Oracle Spatial by ExampleOracle Spatial by ExampleAnalysis: Major Road BufferAnalysis: Major Road Buffer

CREATE TABLE road_buffer ASSELECT prefix, name, type, suffix, SDO_AGGR_UNION( SDOAGGRTYPE( SDO_GEOM.SDO_BUFFER( a.geom, -- geometry column 0.50, -- Distance 0.5, 'arc_tolerance=0.005 unit=mile'), -- Units 0.5)) geom FROM roads a WHERE (name = 'ACALANES' AND type = 'AVE') OR (name = 'ACALANES' AND type = 'RD') * * * * * * * * OR (name = 'YGNACIO VALLEY' AND type = 'RD');

Oracle Spatial by ExampleOracle Spatial by ExampleAnalysis: Major Road BufferAnalysis: Major Road Buffer

Oracle Spatial by ExampleOracle Spatial by ExampleAnalysis: Competitor BufferAnalysis: Competitor Buffer

Oracle Spatial by ExampleOracle Spatial by ExampleAnalysis: Spatial OperationsAnalysis: Spatial Operations

Intersection

Minus


CREATE TABLE target_site_wocomp ASSELECT SDO_AGGR_UNION(SDOAGGRTYPE(c.geom,0.5)) geom FROM (SELECT SDO_GEOM.SDO_INTERSECTION( a.geom, b.geom, 0.5) geom FROM target_tract_buffer a,road_buffer b) c);


CREATE TABLE target_site ASSELECT SDO_AGGR_UNIION(SDOAGGRTYPE(a.geom,0.5)) geom FROM (SELECT SDO_GEOM.SDO_DIFFERENCE( b.geom, c.geom, 0.5) geom FROM target_site_wocomp b,competitor_buffer c) a;

-- Create spatial metadata and index for target_site-- and target_site_wocomp after creation.

Oracle Spatial by ExampleOracle Spatial by ExampleAnalysis: Final DisplayAnalysis: Final Display

Introduction to Oracle Spatial Using Introduction to Oracle Spatial Using Public DataPublic Data

Introoraspatial

Technology

spatial reference system

geographic data processing

spatial demographic data

spatial data

connect spatial spatial

oracle spatial

road data

competitor data