1 Authoring an ontology of place semantics using volunteered geographic information Alistair Edwardes and Ross Purves Department of Geography University of Zurich
Feb 21, 2016
1
Authoring an ontology of place semantics using volunteered
geographic information
Alistair Edwardes and Ross Purves
Department of GeographyUniversity of Zurich
2
Overview
• Motivation for considering place• Why this is useful in the context of image
retrieval• Where can we find place descriptions• How might we build semantic resources
from these
3
Motivation
• GI bias towards spatial representations of Geography– BUT
• Not all geographic information is spatial
4
Motivation
5
Motivation
• GI bias towards spatial representations of Geography– BUT
• Not all geographic information is spatial• Doesn’t reflect how people experience, remember
and talk about geography
• What else is there?
6
Place
• Space-Place Continuum– Objective-Subjective– Universal-Personal– Machine-Human
7
Where is place important?• Location-Based Services
– Movement towards technologies closer to everyday, direct experience, activity based.
“The historical demarcation in psychological and behavioural geography between direct and indirect experience blurs when handheld devices are used as an adjunct to reality in the field.” (Longley, 2004)
• Web 2.0– Social interaction, user generated information, personal memories
8
Where is place important?
[Baostar] documents place in a way that embodies neogeography, where human perspective and social interaction supercede latitude and longitude.
9
Where is place important?• Location-Based Services
– Movement towards technologies closer to everyday, direct experience, activity based.
“The historical demarcation in psychological and behavioural geography between direct and indirect experience blurs when handheld devices are used as an adjunct to reality in the field.” (Longley, 2004)
• Web 2.0– Social interaction, user generated information, personal memories
• Geographic Information Retrieval– Vernacular geography, organising activities
• Photographs
“GI theory articulates the idea of absolute Euclidean spaces quite well, but the socially-produced and continuously changing notion of place has to date proved elusive to digital description except, perhaps, through photography and film.” (Fisher and Unwin, 2006)
10
Place and photographs
• Observer/Viewpoint– Different from universal perspective of maps
• Information is perceptual– Closer to direct experience– Pre-cognitive– Many ways to interpret
• Highly ephemeral– moment in time
11
Problems
• Many such moments in space and time– How do we sort through them?– Image Retrieval
1. How do we access a description of the contents of an image?
2. What do we describe about an image?
12
Image Retrieval Approaches (CBIR)
• Content Based Image Retrieval
• “Natural” for format– Use primitive
features like colour, shape and texture
Smeulders et al, 2000
13
The Semantic Gap
“The semantic gap is the lack of coincidence between the information that one can extract from the visual data and the interpretation that the same data have for a user in a given situation.” (Smeulders et al, 2000)
• Concept based image retrieval– Define high-level semantic concepts
• Defined in loosely structured word lists (LSCOM)– Detect using low-level feature vectors
14
Image Retrieval Approaches (TBIR)
• Text-based Image Retrieval– Describe contents in text
1. How do you access this description?2. What should be described?
15
How do you access a description?
• Manual annotate– Expensive and time consuming
• Definitively won’t scale• Need to automate
– Inconsistency amongst annotators (Markey, 1984)
• Inter-annotator agreement (e.g. Ahn and Dabbish, 2004)
• Controlled Vocabulary– Getty Images 12,000 keywords with 45,000 synonyms
(Bjarnestam, 1998)
– Specialist knowledge
16
Tripod Approach
• Describe location instead– Spatial data– Geographic knowledge– Web resources
17
Obligatory Project Slide• European Commission Sixth Framework Programme Project• 3 years (started January 2007)• €3,150,000• Partners
– University of Sheffield, United Kingdom– University of Zurich, Switzerland– Dublin City University, Ireland– Otto-Friedrich-Universität, Bamberg, Germany– Cardiff University, United Kingdom– Ordnance Survey, United Kingdom– Centrica, Italy– Geodan, The Netherlands– Fratelli Alinari Istituto Edizioni Artistiche, Italy– Tilde, Latvia
• Focus on image retrieval by users of professional stock photo libraries• Focus on particularly geographic images
– e.g. Natural landscapes
18
What should be described? ModesFacets
Specific Of Generic Of About
Who?
Individually named persons, animals, things
Kinds of persons, animals, things
Mythical beings, abstraction manifested or symbolised by objects or beings
What?Individually named events
Actions, conditions Emotions, Abstractions manifested by actions
Where?Individually named geographic locations
Kind of place geographic or architectural
Places symbolised, abstractions manifest by locale
When?Linear time; dates or periods
Cyclical time; seasons, time of day
Emotions or abstraction symbolised by or manifest by
Panofsky-Shatford facet matrix – Shatford (1986)
19
What should be described?GenericOf: EngineeringAbout: Innovation, technical brilliance, complexitySpecificOf: [Da Vinci Chambord staircase]
20
What should be described?
• Advertising– Mystery– Isolation– Chocolate
21
Dimensions of Place
• Theoretical Dimensions of Place– Physical setting, activities, meanings Relph (1976)
– location, material form, investment in meaning Gieryn (2000)
– location (spatial distribution activities), locale (the setting), sense of place Agnew (1987)
• Similar to Shatford – SpecificOf – Location/Identity– GenericOf – Setting– About – Sense of place, meanings, activities
22
Organisation in Tripod
• Concept Ontology– GenericOf
• Scene types• Elements
– About• Sense of Place
– Affective, Cognitive, Conative• Qualities and Activities
• Toponym ontology– SpecificOf
• Identity, location
23
How can we elicit place descriptions?
• Inductively– Ask people
• Adjective Check List• Category Norms / Basic Levels
– e.g. Mountains, Parks, Beaches, Cities– Attributes, Activities, Parts
– Pick terms from a dictionary and validate– Code unstructured domain knowledge– Data mine web resources
• Deductively– Look at structured semantic resources
• Use a combination of these approaches
24
Empirical Elicitation
• Online interactive experiments • Database of 150 landscape photographs from Switzerland, Germany, Holland, Italy, Portugal and the UK.
Free description
Controlled vocabulary Sort and describe
25
landscape village valley countryside wall field top sunlight hills cliffs castle peak fortress narrow birch
town plain land hillside lawn farm creek orchard harbour fall rocks fields flowers buildings ridge
bushes woods track stone ruins meadows undergrowth dunes deciduous cross coast brush wooden weather walls villa monument lane city waterfall vineyard valleys tundra trail terraced
temple sunrise station ship shelter rolling riverbank pylons peninsula pathway park outcrop munros loch lightly historical hilltop ground gardens foot
flood dock cycle croft covered catwalk cargo canopy canal burned brook boulder bay arm angler
altitude acre
wooded country surrounded scene mountainous low dense coastline young world wonderland wintry waterside variation uniform two twilight tuscany
tide three sunlit suburban structure streaming snowland sicily sedimented seaside seacoast savanna ruined route romantic rollign roadside riverside riverbed rising rigi region reach ranch prospect populated pond plateau photographed peeking
pastures passage overgrown outback november mountaintops moor montains mist mediterran lonley lodge leafy lakeside junction heaven hanging gurgling greek
france foreground following floor featureless falling dive development dawn countour copse cliif babbling area alice
mediterranean summer quiet winter hilly calm isolated rural steep open mediteranian cold hot
blurry windy free spring remote lonely clean dangerous cool tranquil arid well rugged rough
picturesque dark vista urban snowy sandy rotting northern medieval intervention home
hike fertile end dusk back
running rowing touring biking looking enjoying bycicle parking live estate
Overall
Overall ∩ Qualities
Overall ∩ Elements
Overall ∩ Activities
27
Subjectivity
28
How can we elicit place descriptions?
• Inductively– Ask people
• Adjective Check List• Category Norms / Basic Levels
– e.g. Mountains, Parks, Beaches, Cities– Attributes, Activities, Parts
– Pick terms from a dictionary and validate– Code unstructured domain knowledge– Data mine web resources
• Deductively– Look at structured semantic resources
• Use a combination of these approaches
29
Volunteered Resources
Gatliff Trust Hostel on Berneray: Picture taken from the beach on Berneray of the historic Gatliff Trust Hostel. Visited in the 1990s, shortly before the causeway linking Berneray to North Uist was built.
30
Word lists(WordNet)
Scene types
Examine frequency and co-occurrence of scene types and terms with respect to a database of image captions
www.geograph.org.uk
Validation of terms
31
Activities Elements Adjectives Activities Elements Adjectives Beach n=2824 Village n=12707
Surfing Shingle Sandy Conservation Pub Deserted Bathing Sand Deserted Reading Shop Pretty
Defence Cliff Eroded Fishing Inn Green Swimming Headland Soft Playground Church Quiet
Tourism Bay Rocky Defence Housing Lovely Wading Sea Warm Bowling Edge Pleasant
Protection Rock Glacial Tourism Cottage Beautiful Sport Coast Low Football Main Road Remote
Shipping Shore Beautiful Entertainment Village green Unusual Golf Island Lovely Sitting Stone Large
Hill n=16232 Mountain n=1256 Climbing Fort Steep Biking Peak Distant
Skiing Top Distant Kayaking Summit Black Holidays Summit Wooded Outing Ridge Remote
Observation Horizon Black Mountaineering Moorland Rocky Sitting Ridge Rough Escape Quarry Grassy
Walking Sheep Grassy Walks Stream Steep Running Valley Round Fun Sheep Natural Cycling Side Big Racing Forest Dark
Preservation Trees White Climbing Top Broad Escape Track Broad Cycling Path Running
Scene Type Descriptions
32
How can we elicit place descriptions?
• Inductively– Ask people
• Adjective Check List• Category Norms / Basic Levels
– e.g. Mountains, Parks, Beaches, Cities– Attributes, Activities, Parts
– Pick terms from a dictionary and validate– Code unstructured domain knowledge– Data mine web resources
• Deductively– Look at structured semantic resources
• Use a combination of these approaches
33
Data Mining • Analysed Nouns (those used >100 in captions)
– Aim• Identify similar element concepts (equivalence relationships)
– Analyse noun co-occurrence with the landscape adjectives of Craik• Identify related element concepts (associative relationships)
– Analyse noun-noun co-occurrence– Methodology (vector space analysis)
• Identify a list of nouns (inter-annotator agreement)• Form co-occurrence vectors for each noun with nouns or adjectives• Remove insignificant occurrences (tested with chi-squared p>0.01) • Filter out vectors with few occurrences (<3)• Analyse (cosine) similarity between idf-weighted co-occurrence
vectors • Visualise using hierarchical clustering and multi-dimensional
scaling– Throw out largest cluster (noise)
34
Element SimilaritiesHeadland, outcrop, coastline, knoll, shoreline, promontory, outcrops, foreshore
Valley(s), ravine, gorge
Gully, channel(s), cuttingditch(es), holes, pool, fordshaft
hill, bank, slopes, hillside, slope, cliffs, banks, crag, crags, incline, descent, fall, ascent, coombe, gradient
Landforms
land, forest, fields, farmland, moor, moorland, countryside, heathland, grassland, downland
Areas, block, granite, blocks, shed, boulder(s), expanse, slab(s), pieces
Land cover
35
Related ElementsChurch, tower, clock, nave, porch, font, aisle
House, hall, stone, wall, home, manor, brick, grounds, roof, walls, door, structure, floor, parts, window(s), period, glass, mansion, lady, storey, gable(s), architect, doorway, moat, material, façade, wing, doors, bricks, materials rubble, columns, foundations, keys, wings, village, entrance, castle, pub, cottages, inn, avenue
Built Environment
Trees, edge, wood, forest, woodland, plantation, forestry, oak, beech, inclosure, birch, pine, heathland, ponies, conifers, pines, plantations, holly, oaks, lawn, conifer, pony, spruce, sitka, larch, commoners
view, farm, lane, hill, footpath, valley, farmland, bridleway, hedge, heath, hillside, horse, stile, walkers, copse, leaves, chalk, picnic, warren, cyclists, spinney, trails
Nature
railway(s), line, station, train(s), branch, viaduct, cutting, embankment, rail(s), stations, trackbed, goods, passenger(s), terminus, mainline, gauge, platform(s), locomotive, overbridge, sidings, freight, diesel
Road, way, track, junction(s), route(s), section, access, crossing, course, traffic, direction, mile, pass, roundabout, motorway, camera, links, yards, bypass, carriageway, network, lights, pedestrian(s), loop, barrier, flyover
Transport
Bridge, river(s), water(s), bank, burn, stream(s), brook, footbridge, dam, ford, pool, flood, banks, drain, plain, weir, tributary, fish, source, waterfall, bed, meadow(s), levels, gorge, fen, aqueduct, sewage confluence, riverside, reservoir(s), pipe, sluice, salmon, pools, meanders, trout, floods, waterfalls, springs, anglers, channels, table, fishery, outflow, watercourse, wharfe, otter, dike, floodplain, watershed,
Water
36
Flickr
cityart, graffiti, graffitiart, graffitti, grafitti, graphiti, stencil, streetart, urbanart, paint, spray, stickers, street, wall(s), mural
Street Artcity, cityscape, downtown, court, skyline, skyscraper, urban, capitalcity, innercity innerlondon, capitol
countycourthouse, county courthouse, courthouses, cityhall, capitolbuilding, texascountycourthouses, texascourthouses, court,building(s), architecture
alley, billboard, brick(s), castle, cathedral, centre, centreville, chimney, church, clock, county, door, entrance, façade, glass, houses, interior, metro, roof schloss, shop, sign(s), stairs, steps, store, streets, suburb, subway, tower(s), town, underground, ville, wall, window(s), wires
Built environment
butterfly, insect(s), insectes, landscape, landschaft, natur, nature, scenery, bloom, blossom, cherry, flora, flower, flowers, garden, gardens, orchid, plant, plants, rose, wildflowers
barn, bench, countryside, environment, farm, fence, field, flood, fog, grass, meadow, mist, moss, mud, parks, path, pine, rain, rural, storm, weather, wiese, wood, baum, fall, forest, leaf, leaves, tree(s), woodland, woods
Nature
atlantic, beach, coast, ocean, pacific, pier, plage, playa, sand, sea, seagull, seaside, shore, surf, wave(s), water
Sun, bluesky, cloud, clouds, dusk, horizon, sky, sunshine, himmel insel, meer, sonne, strand sonnenuntergang, wasser, wolken
Beaches
Weather
aeroplane , aircraft, airline, airplane, airport, aviation, boeing, flughafen, flugzeug, plane, air, apache, flight, helicopter
Aviation
Work with Christian Matyas of Bamberg University
37
Concepts
Land cover Landforms
Agricultural land crops farmland agriculture
Forest plantation wood woods …
Arable land field fields wheat …
Topographic eminences
Mountains beinn mountain sgurr
....
Hills hill down cnoc
....
Develop Taxonomies
Taxonomy groups
ActualNouns
Conceptcategories
38
Conclusions• Place
– Facets– Importance to geographical semantics
• Eliciting place• Volunteer sources
– Usefulness– Potential biases
• Left open– Infrastructure– Concept detection
39
Acknowledgements
• We would like to gratefully acknowledge contributors to Geograph British Isles, see http://www.geograph.org.uk/credits/2007-02-24, whose work is made available under the following Creative Commons Attribution-ShareAlike 2.5 Licence (http://creativecommons.org/licenses/by-sa/2.5/).
• This research reported in this paper is part of the project TRIPOD supported by the European Commission under contract 045335.