http://www.postgis.us/Presentations/PGOpen2014_Session2.html?print-pdf#/ 1/110 PostGIS 2.1 on Caffeine: Raster, Topology, and pgRouting Regina Obe and Leo Hsu http://www.postgis.us http://www.bostongis.com http://www.paragoncorporation.com http://www.postgresonline.com Except where otherwise , content on these slides is licensed under a . noted Creative Commons Attribution 4.0 International license
110
Embed
PostGIS 2.1 on Caffeine: Raster, Topology, and pgRouting · 2/110 Agenda Beyond geometry and geography Raster Pixelated view of the world
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
RasterRasters are matrixes that you can perform analysis on. They can also be
rendered as pretty pictures. In PostGIS land, they live chopped into tiles in atable row in a column type called raster. They are often found dancing with
Pixel Band TypesBands are classified by the range of numeric data they can store and each
band in a given raster can store a different range type of data.1BB: 1-bit booleanBit Unsigned Integers (BUI): 2BUI, 4BUI, 8BUI, 16BUI, 32BUIBit Signed Integers (SI): 8BSI, 16BSI, 32BSIBit Floats (BF) - 32BF, 64BF
PostGIS raster is built on Gdal and wraps a lot of the functions of GDAL inan SQL wrapper
Commonly used command-line of GDALgdalinfo: inspect a given rastergdal_translate: convert raster from one format to anothergdalwarp: transform raster from one spatial projection to another and alsochanges format
raster2pgsql optionsRELEASE: 2.2.0dev GDAL_VERSION=111 (r12973)USAGE: raster2pgsql [<options>] <raster>[ <raster>[ ...]] [[<schema>.]<table>] Multiple rasters can also be specified using wildcards (*,?).
OPTIONS: -s <srid> Set the SRID field. Defaults to 0. If SRID not provided or is 0, raster's metadata will be checked to determine an appropriate SRID. -b <band> Index (1-based) of band to extract from raster. For more than one band index, separate with comma (,). Ranges can be defined by separating with dash (-). If unspecified, all bands of raster will be extracted. -t <tile size> Cut raster into tiles to be inserted one per table row. <tile size> is expressed as WIDTHxHEIGHT. <tile size> can also be "auto" to allow the loader to compute an appropriate tile size using the first raster and applied to all rasters. -P Pad right-most and bottom-most tiles to guarantee that all tiles have the same width and height. -R Register the raster as an out-of-db (filesystem) raster. Provided raster should have absolute path to the file (-d|a|c|p) These are mutually exclusive options: -d Drops the table, then recreates it and populates it with current raster data. -a Appends raster into current table, must be exactly the same table schema. -c Creates a new table and populates it, this is the
Should give you something likeSupported GDAL raster formats: Virtual Raster GeoTIFF National Imagery Transmission Format Raster Product Format TOC format ECRG TOC format Erdas Imagine Images (.img): Ground-based SAR Applications Testbed File Format (.gff ELAS Arc/Info Binary Grid Arc/Info ASCII Grid GRASS ASCII Grid SDTS Raster DTED Elevation Raster Portable Network Graphics JPEG JFIF In Memory Raster: Graphics Interchange Format (.gif) Envisat Image Format Maptech BSB Nautical Charts X11 PixMap Format MS Windows Device Independent Bitmap SPOT DIMAP AirSAR Polarimetric Image RadarSat 2 XML Product :
Exercise: Load Elevation dataIndexes are important, spatial ref is important, constraints are important tooAnd you want to tile cause it's gonna be a coverage where fast analysis of
small areas is important.raster2pgsql -I -C -e -F -Y -t auto -s 4267 dems/*.DEM po.chicago_dem \| psql -U postgres -d presentation
Output with psqlWrap your query in ST_AsPNG or ST_AsGDALRaster etc.
SELECT oid, lowrite(lo_open(oid, 131072), png) As num_bytes FROM ( VALUES (lo_create(0), ST_AsPNG( (SELECT rast FROM po.chicago_pics WHERE filename = 'Chicago_sunrise_1.jpg') ) ) ) As v(oid,png);
oid | num_bytes---------+----------- 9166618 | 16134052
Exercise: Extracting Select Band andconvert to image type
AKA: How to make your new pictures look really oldSELECT ST_AsPNG(ST_Band(rast,1)) As rast FROM po.chicago_pics WHERE filename='Full_chicago_skyline.jpg';
Exercise: Resorting bandsThere are good uses for this, but this is questionably not one of them.SELECT ST_AsPNG(ST_Band(rast,'{3,2,1}'::integer[])) As png FROM po.chicago_pics WHERE filename='Full_chicago_skyline.jpg';
HillshadeDesigned for elevation data (gives hypothetical illumination), but go ahead
and apply to your pictures and create a stone impression.SELECT ST_AsPNG( ST_HillShade(rast,1, NULL::raster, '8BUI'::text, 90) ) FROM chicago_pics WHERE filename ILIKE 'Mona%';
SELECT ST_AsPNG( ST_HillShade(rast,1, NULL::raster, '8BUI'::text, 315,30,150) ) FROM chicago_pics WHERE filename ILIKE 'Mona%';
Map Algebra with ST_MapAlgebraOperations done on a set of pixels (a neighborhood) where the value
returned by the operation becomes the new value for the center pixel. Thesimplest operation works on a single cell (a 0-neighborhood).
In PostGIS the operation is expressed either as a PostgreSQL algebraicexpression, or a Postgres callback function that takes an n-dimensional
matrix of pixel values.PostGIS has several built-in mapalgebra call-backs, but you can build yourown. PostGIS packaged ones all have 4MA in them: 4MA means for Map
Algebra (ST_Min4MA, ST_Max4MA, ST_Mean4MA, ST_Range4MA (very similar to
ST_Roughness), ST_InvDistWeight4MA, ST_Sum4MA)).Default neighborhood is 0 distance from pixel in x direction, and 0 distance
Setting band nodata valuesSELECT ST_SetBandNoDataValue( ST_SetBandNoDataValue( ST_SetBandNoDataValue( ST_Clip(rast, ST_Buffer(ST_Centroid(rast::geometry),200) ), 1,0),2,0),3,0) FROM chicago_pics WHERE filename='Full_chicago_skyline.jpg';
Resize, Shift, Union with SQLSELECT ST_Union( ST_SetUpperLeft( ST_Resize(rast,i*0.3,i*0.3),i*150,i ) ) FROM chicago_pics CROSS JOIN generate_series(1,3) As i WHERE filename='Full_chicago_skyline.jpg';
Exercise: Elevation at a pointSELECT ST_Value(rast,1,loc) FROM po.chicago_dem As d INNER JOIN ST_Transform( ST_SetSRID(ST_Point(-87.627,41.8819),4326), 4267) As loc ON ST_Intersects(d.rast, loc);
181
Let's compare with Wikipedia's answer http://en.wikipedia.org/wiki/ChicagoElevation [1](mean) 594 ft (181 m)Highest elevation– near Blue Island 672 ft (205 m)Lowest elevation– at Lake Michigan 578 ft (176 m)
Exercise: Histogram of an areaClip first to isolate region of interest
SELECT (h.hist).* FROM ST_Transform( ST_Buffer( ST_Point(-87.627,41.8819)::geography, 5000)::geometry,4267 ) As loc, LATERAL (SELECT ST_Histogram(ST_Union(ST_Clip(rast,loc)),1,5) As hist FROM po.chicago_dem As d WHERE ST_Intersects(d.rast, loc) ) As h;
Exercise: Quantiles of an areaSame exercise as histogram, but with Quants
SELECT (h.quant).* FROM ST_Transform( ST_Buffer( ST_Point(-87.627,41.8819)::geography, 5000)::geometry,4267 ) As loc, LATERAL (SELECT ST_Quantile(ST_Union(ST_Clip(rast,loc)),1, '{0.1,0.5,0.75}'::float[]) As quant FROM po.chicago_dem As d WHERE ST_Intersects(d.rast, loc) ) As h;
TransformationJust like geometries we can transform raster data from one projection to
another. Let's transform to web mercator to match our OSM chicago dataCREATE TABLE po.chicago_dem_wmerc AS SELECT rid, ST_Transform(rast, 900913) As rast FROM po.chicago_dem; SELECT AddRasterConstraints('po'::name, 'chicago_dem_wmerc'::name, 'rast'::name);
Let's check meta dataAfter plain vanilla transformation
SELECT r_table_name, srid, scale_x::numeric(10,5), scale_y::numeric(10,5), blocksize_x As bx, blocksize_y As by, same_alignment As sa FROM raster_columns WHERE r_table_name LIKE 'chicago_dem%';
raster ST_Transform has many formsIn order to union and so forth, we need our tiles aligned
WITH ref As ( SELECT ST_Transform(rast,900913) As rast FROM po.chicago_dem LIMIT 1) SELECT d.rid, ST_Transform(d.rast, ref.rast, 'Lanczos') As rast INTO po.chicago_dem_wmerc from ref CROSS JOIN po.chicago_dem As d;
We arbitrarily picked first transformed raster to align with
Named color mapThere are: grayscale, pseudocolor, fire, bluered predefined in PostGIS.
bluered goes from low of blue to pale white to red.SELECT ST_ColorMap(ST_Union(rast,1),'bluered') As rast_4bFROM po.chicago_dem_wmerc WHERE ST_DWithin( ST_Transform(ST_SetSRID(ST_Point(-87.627,41.8819),4326),900913), rast::geometry,5000);
Make 3D linestringsWITH ref_D AS (SELECT name, way, ST_Transform(ST_SetSRID(ST_Point(-87.627,41.8819),4326),900913) As loc FROM po.planet_osm_roads ORDER BY ST_Transform( ST_SetSRID( ST_Point(-87.627,41.8819),4326),900913) <-> way LIMIT 5 ), ref AS (SELECT name, way FROM ref_d ORDER BY ST_Distance(way,loc) ) SELECT ref.name, ST_AsText( ST_LineMerge( ST_Collect( ST_Translate(ST_Force3D((r.gv).geom), 0,0, (r.gv).val) ) ) ) As wktgeom FROM ref , LATERAL (SELECT ST_Intersection(rast,way) As gv FROM po.chicago_dem_wmerc As d WHERE ST_Intersects(d.rast, ref.way) ) As r GROUP BY ref.name;
Exercise: Overlaying geometries on a grid + colormapping
ST_AsRaster variant 2WITH cte AS (SELECT row_number() OVER() As rn, way As geom, ST_XMax(way) - ST_XMin(way) As width, ST_YMax(way) - ST_YMin(way) As height, ST_Extent(way) OVER() As full_ext FROM po.planet_osm_polygon WHERE admin_level='8' ),ref AS (SELECT ST_AsRaster(ST_SetSRID(full_ext, ST_SRID(geom)), ((ST_XMax(full_ext) - ST_XMin(full_ext))/ (ST_YMax(full_ext) - ST_YMin(full_ext))*600)::integer, 600,ARRAY['8BUI'], ARRAY[255], ARRAY[255]) AS rast FROM cte WHERE rn = 1 )SELECT ST_AsPNG(ST_ColorMap(ST_Union(ST_AsRaster(geom, ref.rast, ARRAY['8BUI'], ARRAY[rn*2], ARRAY[255]) ), 'pseudocolor') )FROM cte CROSS JOIN ref;
Vectorizing portions of a rasterMost popular is ST_DumpAsPolygons which returns a set of geomval
(composite consisting of a geometry named geom and a pixel value calledval).
SELECT (g).val, ST_Union((g).geom) As geomFROM (SELECT ST_DumpAsPolygons(ST_Clip(rast,loc) ) As g FROM po.chicago_dem As d INNER JOIN ST_Transform( ST_Buffer( ST_SetSRID( ST_Point(-87.627,41.8819),4326)::geography, 500 )::geometry, 4267) As loc ON ST_Intersects(d.rast, loc) ) AS fGROUP BY (g).val;
Topology primitives: edges, nodes, and facesTopology views the world as a neatly ordered network of faces, edges, andnodes. Via relationships of these primitive elements, we form recognizable
things like parcels, roads, and political boundaries.
A topogeometry is like a geometry, except it has respect for its foundations.It understands it is not sitting by itself in outerspace. It understands it is
made up of other elements which may be shared by other geometries. Itunderstands its very definition is the cumulative definition of others.
It's nice to be a regular old geometry sometimes, and topogeometry has gota cast to make it so. topo::geometry or geometry(topo).
Tables in a topologytopo_chicago has a set of tables you'll find in all topology generated
schemasedge_data and companion edge view: Holds all the line work that definesthe topological network.node - defines all the points that connect the edges (only connectingpoints are considered) as well as isolated points (isonode) that standalone.face - lists polygons formed by closed line work of edges. The polygon isnot stored, only the minimum bounding boxrelation - lists the set of relationships to edges, nodes, faces, and otherlayers that define a topogoemetry.
Let's create a political boundaries table from scratchCREATE TABLE po.chicago_boundaries(id serial primary key, name text);SELECT AddTopoGeometryColumn('topo_chicago', 'po', 'chicago_boundaries', 'topo', 'POLYGON') As layer_id;
Let's populate our boundariesINSERT INTO po.chicago_boundaries(name, topo)WITH ref AS (SELECT way As ref FROM po.planet_osm_polygon WHERE admin_level = '8' AND name = 'Chicago')SELECT name, toTopoGeom(ST_Union(way), 'topo_chicago', 1) FROM (SELECT name, way FROM po.planet_osm_polygon WHERE admin_level = '8') AS b INNER JOIN refON (ST_Intersects(b.way,ref.ref))GROUP BY name;
osm2pgrouting - not tested on windows.osm2po - works on any OS with a java VM. Other neatfeature is it comes with its own mini-webserver that reads the .pbf filedirectly.
,
osm2pgroutinghttp://osm2po.de
pgRouting OSM data with osm2poWe'll demonstrate osm2po since it's more cross-platform
Prepping data for routing with osm2poExtract osm2po in a folder: we used the 4.8.8 versionMake a copy of the demo.bat or demo.sh and replace with your own pathto pbfIf java is not in your path, you may need to define a path variable at top
You should now have a folder called hh in your osm2po folder, and shouldhave an sql file hh_2po_4pgr.sql
Load this up into your database with psql. Note that osm2po loads data inwgs 84 long lat geometry (4326) so tolerance units for all pgrouting will be in
Analyzing your routespgr_analyzeGraph looks for dead ends and other anomalies and populatesfields in hh_2po_4pgr_vertices_pgr. Tolerance are in units of your spatial ref
sys. Takes a whileSELECT pgr_analyzeGraph('hh_2po_4pgr', 0.000001,'geom_way', 'id', 'source', 'target','true');
pgr_astarShortest Path A-Star function that takes as input:
SQL statement that defines your network or portion of network you wantto inspectnode id of startnode id of enddirected: defaults to true (if your graph has direction)has_rcost: If your edges have costs in different direction. reverse_costmust be provided if this is true
Directed route with astar joined with roadOne-ways are considered
SELECT r.seq, r.id1 As node, s.id As edge, s.osm_name, s.cost, s.km, s.kmh FROM pgr_astar('SELECT id, source, target, cost, x1,y1,x2,y2, reverse_cost FROM vw_routing', 159944, 142934, true, true) As r INNER JOIN vw_routing AS s ON r.id2 = s.id ORDER BY r.seq;
Undirected route with astar joined with roadOne-ways are ignored
SELECT r.seq, r.id1 As node, s.id As edge, s.osm_name, s.cost, s.km, s.kmh FROM pgr_astar('SELECT id, source, target, cost, x1,y1,x2,y2, reverse_cost FROM vw_routing', 159944, 142934, false, false) As r INNER JOIN vw_routing AS s ON r.id2 = s.id ORDER BY r.seq;
The Paragon Logo is copyright Paragon Corporation and may be resized,made transparent and so forth. It should only be used to refer to ParagonCorporation.Many of the raster images used in this presentation were downloadedfrom Wikipedia. The ones of Mona Lisa are resized versions of which is under a public domain license.The aerial clip is a portion of Massachusetts aerial data loaded in PostGISraster format from . This particular clip isborrowed from the PostGIS documentation which is under a
Images of Chicago: were taken from this page and under the respective
licenses of the photographers.Slide were made using which is MIT Licensed