Consuming open and linked data with open source tools

5/31/2011

Consuming Open and Linked Data with Open

Source ToolsOr, how to do something useful

with all this free data

Jo Cook, Geospatial Consultant, Astun Technology

Talk Outline

● The data and the distribution technologies explained

● The software stack explained● Actually getting some data and doing something

with it● Gotchas● Further resources and links

The Data and Technologies

The Data and Technologies

The Data and Technologies

Point in Latitude/Longitude format (WGS84)

The Data and Technologies

The Data and Technologies

The Data and TechnologiesPREFIX naptan: <http://transport.data.gov.uk/def/naptan/>PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>PREFIX skos: <http://www.w3.org/2004/02/skos/core#>PREFIX geo: <http://www.w3.org/2003/01/geo/wgs84_pos#>SELECT DISTINCT ?stopPoint_airport ?name ?iataRef ?lat ?longWHERE { ?item a naptan:StopArea ; naptan:administrativeArea <http://transport.data.gov.uk/id/administrative-area/145> ; skos:prefLabel ?name ; geo:lat ?lat ; geo:long ?long ; naptan:stopPoint [ naptan:airport ?stopPoint_airport ] . ?stopPoint_airport naptan:iataRef ?iataRef . }

The Data and Technologies

The Data and Technologies

The Data and Technologies

http://wallpapertree.com/wallpaper/Sad-Kitten/

The Data and Technologies

The Data and Technologies

The Data and Technologies

The Data and Technologies

http://freeimagesarchive.com

The Data and Technologies

The Data and Technologies

1m4ge.blogspot.com

The Data and Technologies

The Data and Technologies

Actually doing something with the data

Actually doing something with the data

Gotchas

smashingmagazine.com

Gotchas

Mitigation

Keep your data here so you're not relying directly on the service for updates to customers

(Maybe even keep it here too)

A dash of all these, some readily available python libraries, and some relatively simple scriptinggets you useful data for your own applications...

... and by using PostgreSQL to store your data, you can provide a buffer in case of api changes or outages...

... and even do some nifty manipulation to analyse and present the data differently!

Conclusions

Conclusions

Simples!

Conclusions

Not *that* simple, but...

ResourcesLinked Data Resources and Example Queries

Best API for transport data: http://openuplabs.tso.co.uk/O'Reilly Book: http://www.learningsparql.com/Blog: http://talis-systems.com/blog/Airport Query: http://getthedata.org/questions/137/where-can-i-get-a-list-of-all-airports-codes-names-locationsBeginner Guide: http://www.pezholio.co.uk/2011/01/a-beginners-guide-to-sparqling-linked-data-part-1/

NHS Choices- RSS

http://www.nhs.uk/servicedirectories/Pages/ServiceSearch.aspx

Police Data- JSON/REST

API: http://policeapi2.rkh.co.uk/api/docs/CSV/SHP: http://www.police.uk/data

Basic Software Resources

Mapserver dynamic charting: http://mapserver.org/output/dynamic_charting.htmlPython SparQLWrapper: http://sparql-wrapper.sourceforge.net/Python SimpleJSON: https://github.com/simplejson/simplejsonPython PygreSQL: http://www.pygresql.org/Loader: https://github.com/AstunTechnology/Loader

Thank You!

www.archaeogeek.com@archaeogeekjocook@astuntechnology.com