5/31/2011
Consuming Open and Linked Data with Open
Source ToolsOr, how to do something useful
with all this free data
Jo Cook, Geospatial Consultant, Astun Technology
Talk Outline
● The data and the distribution technologies explained
● The software stack explained● Actually getting some data and doing something
with it● Gotchas● Further resources and links
The Data and Technologies
The Data and Technologies
The Data and Technologies
Point in Latitude/Longitude format (WGS84)
The Data and Technologies
The Data and Technologies
The Data and TechnologiesPREFIX naptan: <http://transport.data.gov.uk/def/naptan/>PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>PREFIX skos: <http://www.w3.org/2004/02/skos/core#>PREFIX geo: <http://www.w3.org/2003/01/geo/wgs84_pos#>SELECT DISTINCT ?stopPoint_airport ?name ?iataRef ?lat ?longWHERE { ?item a naptan:StopArea ; naptan:administrativeArea <http://transport.data.gov.uk/id/administrative-area/145> ; skos:prefLabel ?name ; geo:lat ?lat ; geo:long ?long ; naptan:stopPoint [ naptan:airport ?stopPoint_airport ] . ?stopPoint_airport naptan:iataRef ?iataRef . }
The Data and Technologies
The Data and Technologies
The Data and Technologies
http://wallpapertree.com/wallpaper/Sad-Kitten/
The Data and Technologies
The Data and Technologies
The Data and Technologies
The Data and Technologies
http://freeimagesarchive.com
The Data and Technologies
The Data and Technologies
1m4ge.blogspot.com
The Data and Technologies
The Data and Technologies
Actually doing something with the data
Actually doing something with the data
Gotchas
smashingmagazine.com
Gotchas
Mitigation
Keep your data here so you're not relying directly on the service for updates to customers
(Maybe even keep it here too)
A dash of all these, some readily available python libraries, and some relatively simple scriptinggets you useful data for your own applications...
... and by using PostgreSQL to store your data, you can provide a buffer in case of api changes or outages...
... and even do some nifty manipulation to analyse and present the data differently!
Conclusions
Conclusions
Simples!
Conclusions
Not *that* simple, but...
ResourcesLinked Data Resources and Example Queries
Best API for transport data: http://openuplabs.tso.co.uk/O'Reilly Book: http://www.learningsparql.com/Blog: http://talis-systems.com/blog/Airport Query: http://getthedata.org/questions/137/where-can-i-get-a-list-of-all-airports-codes-names-locationsBeginner Guide: http://www.pezholio.co.uk/2011/01/a-beginners-guide-to-sparqling-linked-data-part-1/
NHS Choices- RSS
http://www.nhs.uk/servicedirectories/Pages/ServiceSearch.aspx
Police Data- JSON/REST
API: http://policeapi2.rkh.co.uk/api/docs/CSV/SHP: http://www.police.uk/data
Basic Software Resources
Mapserver dynamic charting: http://mapserver.org/output/dynamic_charting.htmlPython SparQLWrapper: http://sparql-wrapper.sourceforge.net/Python SimpleJSON: https://github.com/simplejson/simplejsonPython PygreSQL: http://www.pygresql.org/Loader: https://github.com/AstunTechnology/Loader
Thank You!
www.archaeogeek.com@[email protected]