Place not Space; Geo without Maps

Place not Space; Geo without MapsFOWA London, October 2009

Gary Gale, Yahoo! Geo Technologies

PLACES, PEOPLE and THINGS

atibens on Flickr : http://www.flickr.com/photos/atibens/2616899638/

Knowing geographic locations,

and the names of places

Knowing where our users are, and the

places that are important to them

Knowing the geographic context of everything we

index, manage and publish

We ConnectPlaces, People and Things

SOME NUMBERS

Koen Vereeken on Flickr : http://www.flickr.com/photos/koenvereeken/2088902012/

80% of all data contains a geo reference

85% of all data stored is unstructured

This doubles every 3 months

Mr Faber on Flickr : http://www.flickr.com/photos/mrfaber/247946146/

Source: Gartner Group

MINE THAT CONTENT

tj blackwell on Flickr : http://www.flickr.com/photos/tjblackwell/3652375290/

Content / URL

+

=

Places & References

http://www.vicchi.org/speaking

UNLOCK PLACEMAKER

bohman on Flickr : http://www.flickr.com/photos/bohman/210977249/

https://developer.apps.yahoo.com/wsregapp/

http://wherein.yahooapis.com/v1/document

Placemaker Parameters

» appid› 100% mandatory

» inputLanguage› en-US, fr-CA, …

» outputType› XML or RSS

» documentContent› text to geoparse

» documentTitle› optional title

» documentURL› URL to geoparse

» documentType› MIME type of doc

» autoDisambiguate› remove duplicates

» focusWoeid› filter around a WOEID

WOEIDs

stevefaeembra on Flickr : http://www.flickr.com/photos/stevefaeembra/3567750853/

» Unique

» Permanent

» Global

» Language Neutral› London = Londra = Londres = ロンドン› United States = États-Unis = Stati Uniti

= 미국» Ensures that geography can be

employed consistently and globally

straup on Flickr : http://www.flickr.com/photos/straup/3504862388/

GeoPlanetA Global Location Repository

Names + Geometry +TopologyWOEIDs for

• cities and towns• postal codes, airports

• admin regions, time zones• telephone code areas

• marketing areas• points of interest• colloquial areas• neighbourhoods

woodleywonderworks on Flickr : http://www.flickr.com/photos/wwworks/2222523978/

// POST to Placemaker

$ch = curl_init();

define('POSTURL', 'http://wherein.yahooapis.com/v1/document');define('POSTVARS', 'appid='.$key.'&documentContent='.urlencode($content).

'&documentType=text/plain&outputType=xml'.$lang); $ch = curl_init(POSTURL);curl_setopt($ch, CURLOPT_POST, 1);curl_setopt($ch, CURLOPT_POSTFIELDS, POSTVARS);curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); $placemaker = curl_exec($ch);curl_close($ch);

PLACES

that_james on Flickr : http://www.flickr.com/photos/that_james/496797309/

<placeDetails><place><woeId>44418</woeId><type>Town</type><name><![CDATA[London, England, GB]]></name><centroid><latitude>51.5063</latitude><longitude>-0.12714</longitude></centroid></place><matchType>0</matchType><weight>1</weight><confidence>10</confidence></placeDetails>

One place for WOEID 44418

REFERENCES

misterbisson on Flickr : http://www.flickr.com/photos/maisonbisson/117720946/

<reference><woeIds>44418</woeIds><start>1079</start><end>1089</end><isPlaintextMarker>1</isPlaintextMarker><text><![CDATA[London, UK]]></text><type>plaintext</type><xpath><![CDATA[]]></xpath></reference><reference><woeIds>44418</woeIds><start>1116</start><end>1126</end><isPlaintextMarker>1</isPlaintextMarker><text><![CDATA[London, UK]]></text><type>plaintext</type><xpath><![CDATA[]]></xpath></reference>

Two references for WOEID 44418

Two references for WOEID 44418

// turn into an PHP object and loop over the results

$places = simplexml_load_string($placemaker, 'SimpleXMLElement',

LIBXML_NOCDATA); if($places->document->placeDetails){

$foundplaces = array();

// create a hashmap of the places found to mix with// the references found

foreach($places->document->placeDetails as $p){$wkey = 'woeid'.$p->place->woeId;$foundplaces[$wkey]=array(

'name'=>str_replace(', ZZ','',$p->place->name).'', 'type'=>$p->place->type.'', 'woeId'=>$p->place->woeId.'', 'lat'=>$p->place->centroid->latitude.'', 'lon'=>$p->place->centroid->longitude.'’

);}

}

// loop over references and filter out duplicates

$refs = $places->document->referenceList->reference;$usedwoeids = array();foreach($refs as $r){

foreach($r->woeIds as $wi){if(in_array($wi,$usedwoeids)){

continue;} else {

$usedwoeids[] = $wi.'';}$currentloc = $foundplaces["woeid".$wi];if($r->text!='' && $currentloc['name']!='' &&

$currentloc['lat']!='' && $currentloc['lon']!=''){

$text = preg_replace('/\s+/',' ',$r->text);$name = addslashes(str_replace(', ZZ’,

$currentloc['name']));$desc = addslashes($text);$lat = $currentloc['lat'];$lon = $currentloc['lon'];$class = stripslashes($desc)."|$name|$lat|$lon";$placelist.= "<li>".

}}

select * from flickr.photos.info where photo_id in(select id from flickr.photos.search where woe_id=44418)and license=4;

<placeDetails><place><woeId>44418</woeId><type>Town</type><name><![CDATA[London, England, GB]]></name><centroid><latitude>51.5063</latitude><longitude>-0.12714</longitude></centroid></place><matchType>0</matchType><weight>1</weight><confidence>10</confidence></placeDetails>

ragewear on Flickr : http://www.flickr.com/photos/ragewear/3367122893/

http://developer.yahoo.com/yui/

http://developer.yahoo.com/yql/

THE INTERNET IS BROKEN

Nesster on Flickr : http://www.flickr.com/photos/nesster/3168425434/

// load the URL, using YQL to filter the HTML// and fix UTF-8 nasties

$url = 'http://www.vicchi.org/speaking';

$realurl = 'http://query.yahooapis.com/v1/public/yql’.'?q=select%20*%20'.'from%20html%20where%20url%20%3D

%20%22'.urlencode($url).'%22&format=xml';

$ch = curl_init(); curl_setopt($ch, CURLOPT_URL, $realurl); curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); $c = curl_exec($ch); curl_close($ch);if(strstr($c,'<')){

$c = preg_replace("/.*<results>|<\/results>.*/",'',$c);$c = preg_replace("/<\?xml version=\"1\.0\"".

" encoding=\"UTF-8\"\?>/",'',$c);$c = strip_tags($c);$c = preg_replace("/[\r?\n]+/"," ",$c);

}

MINOR ANNOYANCES

swooshthesnail on Flickr : http://www.flickr.com/photos/swooshthesnail/3281681399/

50,000 BYTES

ASurroca on Flickr : http://www.flickr.com/photos/asurroca/147049402/

NO JSON

X

POST NOT GET

sludgegulper on Flickr : http://www.flickr.com/photos/sludgeulper/2645478209/

WANT TO KNOW MORE?

selva on Flickr : http://www.flickr.com/photos/selva/24604141/

http://github.com/codepo8/PlaceEarth

Earth

http://isithackday.com/hacks/placeearth/

http://developer.yahoo.com/geo/placemaker

http://developer.yahoo.com/geo/geoplanet

http://slideshare.net/vicchi

THANK YOU FOR LISTENING

quinn.anya on Flickr : http://www.flickr.com/photos/quinnanya/3118626057/

www.ygeoblog.com

twitter.com/yahoogeo

twitter.com/vicchi