- 1. Protg4US: Harvesting Ontology Authoring Data with Protg
Markel Vigo, Caroline Jay, Robert Stevens
[email protected] @markelvigo, @CarolineEJay,
@stevensrd65 Workshop on Human-Semantic Web Interaction, HSWI 2014
May 26. Crete (Greece)
2. Little is known about the human factors of ontology authoring
Protg4US: Harvesting Ontology Authoring Data with Protg HSWI @ ESWC
2014 What we know is based on anecdotal evidence We asked about
problems and strategies Making sense Search and retrieval Efficient
population On-the-fly reasoning Overloaded explanations Lack of
evaluation methods Design insights for the next wave ontology
authoring tools. CHI 2014 http://dx.doi.org/10.1145/2556288.2557284
Introduction > Protg4US > Results > Conclusion 3. Lack of
sound HCI research in the Semantic Web Protg4US: Harvesting
Ontology Authoring Data with Protg HSWI @ ESWC 2014 HCI does not
pervade all computing disciplines Instruments to run user studies
are scarce Consequences for the OWL realm No understanding about
the authoring process Authoring tools are not human-centered What
if we want to go further? Automatic detection of authoring patterns
Intelligent support for authoring Introduction > Protg4US >
Results > Conclusion 4. Protg4US: a step towards having
observational instruments Protg4US: Harvesting Ontology Authoring
Data with Protg HSWI @ ESWC 2014 Protg4US: Protg for User Studies
Logging capabilities of: Interaction events: click, hover, expand
hierarchy... Authoring events: add siblings, add restrictions...
Environment commands: reason, search, undo...
76585,2,Classes,Element edited,Juliette subclass of: Potato and
hasCroppingTime some Main cropping 77786,3,Classes,Save
ontology,http://owl.cs.manchester.ac.uk/ontology/start-here.owl
80204,3,Classes,Reasoner invoked,HermiT 1.3.8 80647,1,Classes,Mouse
entered, Class hierarchy (inferred) 82910,1,Classes,Element
hovered,Early_cropping_potato 83049,1,Classes,Element
selected,Early_cropping_potato 83661,1,Classes,Hierarchy
expanded,Early_cropping_potato Introduction > Protg4US >
Results > Conclusion 5. User study to show the strengths of
Protg4US Protg4US: Harvesting Ontology Authoring Data with Protg
HSWI @ ESWC 2014 Experimental design: Participants: 15 expert
authors Stimuli: a potato ontology and Protg4US 3 authoring tasks
with an increased complexity Collected data (apart from Protg4US
logs): Completion times Self reported expertise Perceived task
difficulty Introduction > Protg4US > Results > Conclusion
6. Protg4US in action Protg4US: Harvesting Ontology Authoring Data
with Protg HSWI @ ESWC 2014 Introduction > Protg4US > Results
> Conclusion 7. Data analysis to check the intention of the
experimental design Protg4US: Harvesting Ontology Authoring Data
with Protg HSWI @ ESWC 2014 Tasks given had an increased complexity
Completion times significantly different: T1 Protg4US > Results
> Conclusion 8. Some events may be indicators of problematic
situations Protg4US: Harvesting Ontology Authoring Data with Protg
HSWI @ ESWC 2014 Positive and significant correlations between task
completion time and: Expansions of the class hierarchy Running the
reasoner Renaming entities Log analysis allows to profile users
based on their tab use Introduction > Protg4US > Results >
Conclusion 9. Reconstructing the interaction allows to identify
patterns through visualisation Protg4US: Harvesting Ontology
Authoring Data with Protg HSWI @ ESWC 2014 Web diagrams show the
most frequent transitions between statesP15 log Back Check property
Class addition Convert into defined class:finished Convert into
defined class:start Entity renamed Explanation invoked Entity
deleted Entity dragged Entity edited:finishEntity edited:start
Entity selected Link clicked Property addition Reasoner finished
Reasoner invoked Save Set active ontology Hierarchy collapsed
Hierarchy expanded Undo P9 log Back Check property Class addition
Convert into defined class:finished Convert into defined
class:start Entity renamed Explanation invoked Entity deleted
Entity dragged Entity edited:finishEntity edited:start Entity
selected Link clicked Property addition Reasoner finished Reasoner
invoked Save Set active ontology Hierarchy collapsed Hierarchy
expanded Undo Introduction > Protg4US > Results >
Conclusion 10. Reconstructing the interaction allows to identify
patterns through visualisation Protg4US: Harvesting Ontology
Authoring Data with Protg HSWI @ ESWC 2014 Time diagrams show the
authoring rhythm Introduction > Protg4US > Results >
Conclusion 11. Reconstructing the interaction allows to identify
patterns through visualisation Protg4US: Harvesting Ontology
Authoring Data with Protg HSWI @ ESWC 2014 The analysis of diagrams
across users allows to sketch decision trees Reasoner invoked after
1. A class is converted into a defined class 2. Ontology is saved
3. An entity is selected Hierarchy is expanded after 1. Reasoner
finishes 2. Hierarchy is expanded 3. Entity addition is invoked
Introduction > Protg4US > Results > Conclusion 12. Future
work and Conclusions Protg4US: Harvesting Ontology Authoring Data
with Protg HSWI @ ESWC 2014 Future work Statistical analysis of
patterns and identification of strategies Incorporation of
eye-tracking data Ontology authoring into the wild Conclusions
Protg4US may give urgency to a more human- centred Semantic Web
Better tools for ontology and linked data authoring Introduction
> Protg4US > Results > Conclusion 13. Protg4US: Harvesting
Ontology Authoring Data with Protg Markel Vigo, Caroline Jay,
Robert Stevens [email protected] @markelvigo,
@CarolineEJay, @stevensrd65 Workshop on Human-Semantic Web
Interaction, HSWI 2014 May 26. Crete (Greece) WhatIf: Answering
What if... questions for Ontology Authoring. EPSRC reference
EP/J014176/1