Taxonomy & Ontology Impact Taxonomy & Ontology Impact on Search Infrastructure on Search Infrastructure John R. McGrath Sr. Director, Fast Search & Transfer
Mar 27, 2015
Taxonomy & Ontology ImpactTaxonomy & Ontology Impacton Search Infrastructureon Search Infrastructure
John R. McGrath
Sr. Director, Fast Search & Transfer
2
Mar
ket
Po
ten
tial
Search InnovationConventional Innovative
Ma
ture
Ma
rke
tG
row
th M
ark
et
General Site Search, Intranet
eTailing
BusinessIntelligence
EnterpriseeCommerce
Premium Content Delivery
InformationManagement
KnowledgeManagement
Complianceand ContentGovernance
Detection, Surveillance and Enforcement
Predictive Search• alerting through
business rules• tracking and monitoring• high order analytics• pattern matching
Next Generation Search• 360º View of Enterprise• Hyper-personalization• Contextual synergy• Technology transparency• Central governance
Conventional Search• search bar w/ results• fixed relevancy model• static navigation• few data sources
Advanced Search• tunable relevancy and navigation• wide array of sources (structured, unstructured)• static navigation
Extended Search• Extended platform (desktop, mobile)• Intelligent use of context (Web, geospatial)• rich media integration
The Real Search Solution SpaceEvolving more powerful capability
3
Improving Search through Contextual AnalysisThe Importance of Context in Search
Usage PatternsQueriesInterest ProfilesLocation
USER CONTEXT
APPLICATION CONTEXTINFORMATION CONTEXTWhat does the data indicate is most important?
What matters most to the user?
What matters most to the business or organization providing the data?
Statistics
MetadataOrganization
Business RulesEditorial ControlProgram Control
4
Improving Search through Conceptual AnalysisEmbedded Application Semantics
FAST Answers
Extreme precision applications:– Self-service, NLP, Mobile, Compliance– Information discovery, Intelligence– Web 2.0 – The semantic web
Rich Media
The Adaptive Information Warehouse
– Scene level discovery– Podcasts: Extreme precision access
– Linguistic cleansing– On-the-fly fact mining– 10-200 X query speed-ups– Low latency access to extreme volumes
When was D-day?
– Visualize implicit facts– Visualize uncertainty– Use embedded semantics:
–Ex: Patent claims–Ex: Blogs
5
Applying Ontology to the Search Architecture
COLLECTIONS CONNECTORS REFINEMENT SEARCH & ALERT ENGINES PROCESSING SEARCH PROFILE USER
SEARCH & ADMINISTRATION MANAGEMENT
ALERT
CONTENTREFINEMENT
QUERY PROCESSING
RESULT PROCESSING
SEARCH
ST
RU
CT
UR
ED
DA
TA
RIC
H M
ED
IAU
NS
TR
UC
TU
RE
DD
AT
A
SEARCH
App. LogicTagging
ExtractionClassificationRelationship
PersonalizationTerm ExpansionQuery Control
Source SelectionRefinement Logic
PresentationSecurity
App logicNavigation
6
Primary applications today in search
Classify documents – facilitate simple, keyword-based retrieval
Provide a common language, or thesaurus – offer terms to refine a search from a consistent, controlled vocabulary
Create browse-able directories – facilitate rapid navigation through defined hierarchies of information
Promote meaningful clustering – establish ‘fixed points’ for clustering results
Generate pick-list elements – select or combine terms to limit/define your search domain
Expedite query refinement – refine/exclude on similarly tagged items
7
FAST Relevancy Framework
Business Rules
User Profiles
Core Algorithmic Model
Application Model•Sorting •Navigation•Feedback
Accessible to… Control Mechanisms
End Users
Business Managers
•Alert Parameters•Page “boosting”
Administrator •Rank Profile•Concept Security
Developer •Dynamic Algorithm “weights”
Levels o
f co
ntro
l
Multiple levels of control
8
Human Factors Considerations
Limited user capacitance• Most business users do not navigate
deeper than 4 levels in a taxonomy• More than 10 choices/nodes per level
impacts willingness to move deeper, to next level
Multiple perceptions of value• Provides navigation for discovery
(40%)• Organizes disparate info (18%)• Structures KM repository (16%)• Automates classification & alerting
(14%)• Enhances searching(12%)
Distribution of expected benefits• Increased productivity (21%)• Reduced search time ( 20% )• Increased knowledge sharing (18%)• Shortened time to decision (16%)• Improved collaboration (13%)• Discover new opportunities (10%) Source : Delphi Group 2004 Survey – 300 respondents
Taxonomy Levels Navigated
17%
68%
11%
4%
0% 50% 100%
Source:Delphi Group 2004 Survey
>65-63-41-2
9
Observed Trends
Taxonomies and ontologies are expense to develop and maintain.
• Published works and services• Social network product• Automatic generation
10
Observed Trends
Primary application is “smart” navigation.
• Simple knowledge bases• Supervised clustering
11
Near Future?
Application of Knowledge bases is increasing to support advanced features.
• Extraction and association• Relationship analysis• Advanced personalization