Internalizing location services with geo names

Internalizing loca.on services with GeoNames

John Marc ImbresciaSenior Software Engineer - Etsy

Wednesday, May 1, 13

Internalizing location services with GeoNames

May 2 2013


The world’s online handmade marketplace.What is Etsy.com?


What is Etsy.com?•20 million unique items•18 million daily item searches•800,000 sellers•28 million unique views per month•Developer blog: codeascraft.etsy.com•450 worldwide employees


Our ProblemLocation names were only in English

•Search based on English names•Display and search needed to be i18n friendly.•API limits and speed concerns meant we needed a new solution.


What do we use Location for?More than just search

•Display•Local Search

•No Mapping•No Bounding boxes


What do we use Location for?Item Search






What do we use Location for?Location Display


How did this use to work?•Yahoo API•Every lookup was an API call•Stored user input and API response•Searched based on text match of API response•Not radius using lat/lon•No way to Internationalize


What Services did Etsy need to Internalize location services?

•Lookup - Autosuggest•Update - Scripts to refresh data•Display - Built into the php stack•Search - Existing, modified for new pattern


What we have now•GeoNames as a data source•Feeds “geonamessuggest” Solr Core•Sqlite database for place name lookup•GeoName IDs used for local search•Leverages GeoName hierarchy data•Built in Internationalization


How did we get here?•Mapped old locations to GeoNames•Added Geoname ID hierarchy to listing search•Pushed out Sqlite database to webservers•Slowly transitioned lookup and search services•Did side by side testing to look for anomalies


What are the data types?GeoNames


SchemasGeoNames

•775k Entries•1.4m alternate spellings


SchemasGeoNames


GeonamessuggestOur autosuggest for place names

•Localized•GeoIP•Population


GeonamessuggestSchema


Distance and population come firstSort function


GeonameId HierarachyLocal listing search

•Each listing gets a hierarchy of geonameids•Local search is a filter on this ID•Fast & Reliable•Enables powerful functionality•Kept old data fields


GeonameId CollectionLocal listing search

•Each listing gets a hierarchy of geonameids•Local search is a filter query on this ID•Fast & Reliable•Enables powerful functionality•Kept old data fields


CONFERENCE PARTYThe Tipsy Crow: 770 5th AveStarts after Stump The ChumpYour conference badge gets you in the door

TOMORROW Breakfast starts at 7:30Keynotes start at 8:30

CONTACT John Marc Imbrescia@[email protected]


Internalizing location services with geo names

Education

Internalizing location services with geo names