1 l July 2, 2022 1 l Detecting, Analyzing and Representing Vagueness in Ontologies for Facilitating Data Reuse Dr. Panos Alexopoulos Semantic Applications Research Manager iSOCO S.A. University of Aberdeen 1/4/2014
Dec 06, 2014
1 l April 10, 20231 l
Detecting, Analyzing and Representing Vagueness in Ontologies for Facilitating Data Reuse
Dr. Panos AlexopoulosSemantic Applications Research ManageriSOCO S.A.
University of Aberdeen1/4/2014
What will I talk about
Research Questions and (some) Answers
Ongoing & Future Work
Vagueness and Ontologies
What will I talk about
Research Questions and (some) Answers
Ongoing & Future Work
Vagueness and Ontologies
Vagueness
„Vagueness is a semantic phenomenon where predicates admit borderline cases, namely cases where it is not determinately true that the predicate applies or not“
[Shapiro, 2006]
This happens when predicates have blurred boundaries:
•What’s the threshold number of years separating old and not old films?
•What are the exact criteria that distinguish modern restaurants from non-modern?
Where does vagueness appear in an ontology?
●A class is vague if it admits borderline cases, i.e. if there are (or could be) individuals for which it is indeterminate whether they instantiate the class.
●“TallPerson”, “StrategicClient”
Vague Classes
●Datatypes whose value range consists of a set of vague terms.
●E.g. “RestaurantPriceRange” when this comprises the terms “cheap”, “moderate” and “expensive”.
Vague Datatypes
●A relation is vague if there are (or could be) pairs of individuals for which it is indeterminate whether they stand in the relation.
●“isExpertInSubject”, “belongsToGenre”
Vague Relations
Vagueness in human communication
I am telling you this is a strategic client for the firm with large-budget projects!
Come on, $300,000 in two years is hardly a
large budget!
When is vagueness in an ontology a problem?
● Defining instances: Vagueness will cause disagreement among experts when defining class and relation instances.
● Utilizing Vague Facts in Ontology-Based Systems: Reasoning results might not meet users’ expectations.
● Integrating Vague Semantic Information: The merging of particular vague elements can lead to data that will not be valid for all its users.
● Reusing Vague Datasets: The intended meaning of the dataset’s vague elements might not be compatible to the one needed in a particular application context.
What will I talk about
Research Questions and (some) Answers
Ongoing & Future Work
Vagueness and Ontologies
Research Questions
Can we automatically detect vague ontological definitions?
Vagueness Detection
Vagueness Measurement
Vagueness Explicitness
Can we quantify the amount and importance of vagueness in an ontology and/or a dataset?
Can we make the intended meaning of vague elements in ontologies more explicit than it typically is?
Q1: Vagueness Detection
● Can we automatically determine whether an ontology entity (class, relation etc.) is vague or not?
● For example, “StrategicClient” as “A client that has a high value for the company” is vague!
● “AmericanCompany” as “A company that has legal status in the Unites States” is not!
Problem Definition
Q1: Vagueness Detection
● We train a supervised classifier that may distinguish between vague and non-vague word senses, using corresponding examples from Wordnet.
● We use this classifier to determine whether a given ontology element definition is vague or not.
Approach
Q1: Vagueness Detection
● 2,000 adjective senses from WordNet. ● 1,000 vague● 1,000 non-vague
● Inter-agreement of vague/non-vague annotation among 3 human judges was 0.64 (Cohen’s Kappa)
Data
Vague Senses Non Vague Senses
• Abnormal: not normal, not typical or usual or
regularor conforming to a norm
• Compound: composed of more than one part
• Impenitent: impervious to moral persuasion • Biweekly: occurring every two weeks.
• Notorious: known widely and usually
unfavorably
• Irregular: falling below the manufacturer's
standard
• Aroused: emotionally aroused • Outermost: situated at the farthest possible
point from a center.
Q1: Vagueness Detection
● 80% of the data used to train a multinomial Naive Bayes classifier.
● We removed stop words and we used the bag of words assumption to represent each instance.
● The remaining 20% of the data was used as a test set.
● Classification accuracy was 84%!
Training & Evaluation
Q1: Vagueness Detection
● We also used a subjective sense classifier to classify the dataset’s senses as subjective or objective.
● From the 1000 vague senses, only 167 were classified as subjective while from the 1000 non-vague ones 993.
● This shows that treating vagueness in the same way as subjectiveness is not really effective.
Comparison with Subjectivity Analyzer
Q1: Vagueness Detection
● As an ontology use case we considered CiTO, an ontology that enables characterization of the nature or type of citations.
● CiTO consists primarily of relations, many of which are vague (e.g., plagiarizes).
● We selected 44 relations and we had 3 human judges manually classify them as vague or not.
● Then we applied the Wordnet-trained vagueness classifier on the textual definitions of the same relations.
Detecting Vagueness in CiTO Ontology
Q1: Vagueness DetectionDetecting Vagueness in CiTO Ontology
Vague Relations Non Vague Relations
• plagiarizes: A property indicating
that the author of the citing entity
plagiarizes the cited entity, by
including textual or other elements
from the cited entity without formal
acknowledgement of their source
• sharesAuthorInstitutionWith:
Each entity has at least one author
that shares a common institutional
affiliation with an author of the other
entity
• citesAsAuthority: The citing entity
cites the cited entity as one that
provides an authoritative
description or definition of the
subject under discussion.
• providesDataFor: The cited entity
presents data that are used in work
described in the citing entity.
Q1: Vagueness Detection
● Classification Results:● 82% relations were correctly classified as vague/non-vague ● 94% accuracy for non-vague relations.● 74% accuracy for vague relations.
● Again, we classified the same relations with the subjectivity classifier:● 40% of vague/non-vague relations were classified as
subjective/objective respectively.● 94% of non-vague were classified as objective.● 7% of vague relations were classified as subjective.
Detecting Vagueness in CiTO Ontology
Q2: Vagueness Measurement
● Can we use metrics to quantify the existence and importance of vagueness in ontologies and semantic data?
● E.g. can we somehow claim that ontology A is more vague than ontology B and have a number to support this claim?
● Some initial metrics we have thought:
● Vagueness Spread
● Vagueness Intensity
Problem Definition & Proposal
Q2: Vagueness Measurement
● The ratio of the number of ontological elements (classes, relations and datatypes) that are vague, divided by the total number of elements.
● This metric reflects the extent to which vagueness is present in the ontology.
● It also provides an indication of the ontology's potential comprehensibility and shareability:
● An ontology with a high value of vagueness spread is less explicit and shareable than an ontology with a low value.
● Calculation of this metric can be done semi-automatically using the vagueness detector we have described.
Vagueness Spread
Q2: Vagueness Measurement
● The degree to which the ontology's users disagree on the validity of the (potential) instances of the ontology elements.
● The higher this disagreement is for an element, the more problems the element is likely to cause.
● Metric Calculation:● Consider a sample set of vague element instances, ● Have potential ontology users denote whether and to what extent
they believe these instances are valid● Measure the inter-agreement between users (e.g. by using Cohen’s
kappa)
Vagueness Intensity
Q3: Vagueness Explicitness
● Can we make the intended meaning of vague elements in ontologies more explicit than it typically is?
● For example, ontologies typically do not explicitly state which of their elements are vague and which are not.
● Why is this a problem?
● “Fat Person” as “A person weighing many kilos” is vague!● “Fat Person” as “A person with a BMI greater than 25” is not!
● What are the odds that a typical ontology user will immediately see this distinction without prior notice?
Problem Definition
Q3: Vagueness ExplicitnessOur Proposal: Vagueness Aware Ontologies
Vagueness-aware ontologies are “ontologies
whose vague entities are accompanied by
meta-information that describes the nature
and characteristics of their vagueness”
Q3: Vagueness ExplicitnessWhat should be made explicit?
E.g. “Low Budget” has quantitative vagueness while “Expert Consultant”
qualitative.
“E.g. “Strategic Client" is vague in the dimension of the generated revenue
”
“E.g. “Strategic Client" is vague in the dimension of the generated revenue in the context of Financial Reporting”
E.g. “Strategic Client" is vague in the dimension of the generated revenue according to the Financial Manager.
Vagueness Type
Vagueness dimensions
Applicability Context
Entity Creator
E.g. “Tall Person” is vague.Vagueness Existence
Q3: Vagueness ExplicitnessThe Vagueness Ontology
Vagueness Ontology
● Q1: What entities have been explicitly defined either as vague or non-vague?
● Q2: What entities are vague, in what contexts and according to whom?
● Q3: What entities have been defined both as vague and non-vague at the same time and why?
● Q4: What entities of a specific type (e.g., classes) have been defined either as vague or non-vague?
● Q5: What entities are characterised by a specific vagueness type?
● Q6: What entities have quantitative vagueness, in what dimensions and what is the context of their dimensions (if any)?
Supported Competency Questions
Vagueness Ontology
● The object property isExpertInResearchArea is considered vague by John Doe in the context of researcher hiring.
● Moreover, it is quantitatively vague since, for him, expertise is judged by the number of publications and projects.
● These two different dimensions he thinks are applicable in different contexts:
● Number of relevant publications in Academia
● Number of relevant projects in Industry.
Example Scenario
Vagueness OntologyExample Scenario
Vagueness Ontology
● Vagueness Ontology is meant to be used by both producers and consumers of ontologies and semantic data.
● The former to annotate the vague part of their produced ontologies with relevant vagueness metainformation
● The latter to query this metainformation and use it to make a better use of the data.
● The annotation task should ideally take place in the course of the ontology's construction and evolution process.
Intended Usage
Vagueness OntologyUsage and Benefits
Vagueness Ontology Usage Expected Benefits
Structuring Data with a Vague Ontology
• Communicate the meaning of the vague
elements to the domain experts.
• Use the metamodel to characterize the created
data's vagueness.
• Make the job of the experts
easier and faster and
reduce disagreements
among them.
Utilizing Vague Semantic Data in an Ontology-Based System
• Check which data is vague.
• Use the properties of the vague elements to
provide vagueness-related explanations to the
users.
• Know a priori which data
may affect the system's
effectiveness.
Integrating Vague Semantic Datasets
• Compare same vague elements across datasets
according to their vagueness type and
dimensions
• Know a priori which data
may affect the system's
effectiveness.
Evaluating Vague Semantic Datasets for Reuse
• Query the metamodel to check the vagueness
compatibility of the dataset with the intended
domain and application scenario.
• Avoid re-using (parts of)
datasets that are not
compatible to own
interpretation of vagueness.
Vagueness OntologyDocumentation and Resources
Documentation: http://www.essepuntato.it/2013/10/vagueness/documentation.html Examples: http://www.essepuntato.it/2013/10/vagueness/examples.html
What will I talk about
Research Questions and (some) Answers
Ongoing & Future Work
Vagueness and Ontologies
Ongoing and Future Work
● Develop ontology authoring tool that:
1. Takes as input an ontology.
2. Detects automatically vague entities.
3. Guides the user into annotating them with the Vagueness Ontology in a Q&A manner.
4. Gives as output a vagueness annotation for the given ontology.
Creation of Vagueness-Aware Ontologies
Ongoing and Future Work
● Refine/expand/evaluate the current vagueness metrics we have defined.
● Devise methods and tools for their effective and efficient calculation.
Vagueness-Based Evaluation of Ontologies
● Identify vagueness-related reasoning tasks that are useful when working with vague ontologies.
● Expand/revise the Vagueness Ontology accordingly.
Reasoning with Vagueness-Aware Ontologies
Thank you for your attention!
Dr. Panos AlexopoulosSemantic Applications Research Manager
Email: [email protected]
Web: www.panosalexopoulos.com
LinkedIn: www.linkedin.com/in/panosalexopoulos
Twitter: @PAlexop