The future is federated Ruben Verborgh
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
The future
is federated
Ruben Verborgh
Big DataI think
is boring.
Big Data thriveson centralization.
Knowledgeis inherently distributed.
Knowledgeis inherently heterogeneous.
Knowledge on the Webis inherently linked.
Centralizationskips
interestingthe most
problems
Where to find data you need?
How to access them?
How to integrate them?
Let’s create smart appsover VIVO and Web data.
a light interface to VIVO data
queries over that interface
an app built on such queries
You’ll get to see 3 things:
We can integratemultiple data sourceson the live Web,but we need to setour expectations right.
The future
is federated
Big Data fails at Web scaleLight interfaces ruleEngineer for serendipity
The future
is federated
Big Data fails at Web scaleLight interfaces ruleEngineer for serendipity
RDFTHE DATA LANGUAGE
<subject> <predicate> <object>.
triple
SPARQLTHE QUERY LANGUAGE
SPARQLTHE PROTOCOL
clientSPARQL
endpointSPARQL protocol
SPARQLquery
SELECT ?person ?name WHERE { ?person a dbo:Scientist. ?person rdfs:label ?name. ?person dbo:birthPlace dbp:Denver. }
Hey, SPARQL endpoint…
Sure!
SELECT DISTINCT ?drug ?drug1 ?drug2 ?drug3 ?drug4 ?d1 WHERE { ?drug1 <http://www4.wiwiss.fu-berlin.de/drugbank/resource/drugbank/drugCategory> <http://www4.wiwiss.fu-berlin.de/drugbank/resource/drugcategory/antibiotics> . ?drug2 <http://www4.wiwiss.fu-berlin.de/drugbank/resource/drugbank/drugCategory> <http://www4.wiwiss.fu-berlin.de/drugbank/resource/drugcategory/antiviralAgents> . ?drug3 <http://www4.wiwiss.fu-berlin.de/drugbank/resource/drugbank/drugCategory> <http://www4.wiwiss.fu-berlin.de/drugbank/resource/drugcategory/antihypertensiveAgents> . ?drug4 <http://www4.wiwiss.fu-berlin.de/drugbank/resource/drugbank/drugCategory> <http://www4.wiwiss.fu-berlin.de/drugbank/resource/drugcategory/anti-bacterialAgents> . ?drug1 <http://www4.wiwiss.fu-berlin.de/drugbank/resource/drugbank/target> ?o1 . ?o1 <http://www4.wiwiss.fu-berlin.de/drugbank/resource/drugbank/genbankIdGene> ?g1 . ?o1 <http://www4.wiwiss.fu-berlin.de/drugbank/resource/drugbank/locus> ?l1 . ?o1 <http://www4.wiwiss.fu-berlin.de/drugbank/resource/drugbank/molecularWeight> ?mw1 . ?o1 <http://www4.wiwiss.fu-berlin.de/drugbank/resource/drugbank/hprdId> ?hp1 . ?o1 <http://www4.wiwiss.fu-berlin.de/drugbank/resource/drugbank/swissprotName> ?sn1 . ?o1 <http://www4.wiwiss.fu-berlin.de/drugbank/resource/drugbank/proteinSequence> ?ps1 . ?o1 <http://www4.wiwiss.fu-berlin.de/drugbank/resource/drugbank/generalReference> ?gr1 . ?drug <http://www4.wiwiss.fu-berlin.de/drugbank/resource/drugbank/target>?o1 . ?drug2 <http://www4.wiwiss.fu-berlin.de/drugbank/resource/drugbank/target> ?o2 . ?o1 <http://www4.wiwiss.fu-berlin.de/drugbank/resource/drugbank/genbankIdGene> ?g2 . ?o2 <http://www4.wiwiss.fu-berlin.de/drugbank/resource/drugbank/locus> ?l2 . ?o2 <http://www4.wiwiss.fu-berlin.de/drugbank/resource/drugbank/molecularWeight> ?mw2 . ?o2 <http://www4.wiwiss.fu-berlin.de/drugbank/resource/drugbank/hprdId> ?hp2 . ?o2 <http://www4.wiwiss.fu-berlin.de/drugbank/resource/drugbank/swissprotName> ?sn2 . ?o2 <http://www4.wiwiss.fu-berlin.de/drugbank/resource/drugbank/proteinSequence> ?ps2 . ?o2 <http://www4.wiwiss.fu-berlin.de/drugbank/resource/drugbank/generalReference> ?gr2 . ?drug <http://www4.wiwiss.fu-berlin.de/drugbank/resource/drugbank/target>?o2 . ?drug3 <http://www4.wiwiss.fu-berlin.de/drugbank/resource/drugbank/target> ?o3 .
Hey, SPARQL endpoint…
Sure!
SPARQL endpointstry to be the Web’sBig Data processors.
for free
few endpoints exist
the average endpoint isdown for 1.5 days/month
Can I SPARQLyour endpoint?
Big Data failsat Web scalebecause Web Scaleis much bigger.
SEMANTIC WEBSHOULDN’T TRY TO COMPETE WITH
BIG DATA
WEBI WANT TO PUT THE
BACK INTO SEMANTIC WEB
IT’S OUR MAIN DIFFERENTIATORFROM BIG DATA
WEBIF IT’S NOT
I’M NOT INTERESTED
That’s why I thinkBig Data is boring.
The future
is federated
Big Data fails at Web scale
Light interfaces ruleEngineer for serendipity
AVERAGEHUMANWhat would the
do?
SELECT ?person ?name WHERE { ?person a dbo:Scientist. ?person rdfs:label ?name. ?person dbo:birthPlace dbp:Denver. }
AVERAGE HUMAN
You can use only Wikipedia.
AVERAGE HUMAN
Which scientists were born in Denver?
You can use only Wikipedia.
AVERAGE HUMAN1. visit the page about Denver 2. make a list of people born there 3. read their pages to see if they’re a scientist
You can use only Wikipedia.
WEB LINKINGIS UNIDIRECTIONALa Denver person’s page links to DenverDenver doesn’t necessarily link to that person
AVERAGE HUMAN1. visit the page about Denver 2. make a list of people born there 3. read their pages to see if they’re a scientist
You can use only Wikipedia.
AVERAGEHUMANWe need to empower the
but please not with a SPARQL endpoint because they’re so expensive to keep up.
SIMPLESTCOMPLEXITYWHAT IS THE ?
THE ESSENCEOF RDF
<subject> <predicate> <object>.
THE ESSENCEOF LINKED DATA
?subject <predicate> <object>.
THE ESSENCEOF LINKED DATA
Denver <predicate> <object>.
THE ESSENCEOF TPF
?subject ?predicate ?object.
THE ESSENCEOF TPF
?subject ?predicate Denver.
TRIPLEPATTERNFRAGMENTS
Clients can askthe server onlyfor triple patterns.
AVERAGE HUMAN
Which scientists were born in Denver?
You can only use a TPF interface of DBpedia.
AVERAGE HUMAN1. “?people birthPlace Denver.” 2. “?person type Scientist.” 3. “?person fullName ?name.”
You can only use a TPF interface of DBpedia.
AVERAGE MACHINE1. “?person birthPlace Denver.” 2. “?person type Scientist.” 3. “?person fullName ?name.”
You can only use a TPF interface of DBpedia.
SELECT ?person ?name WHERE { ?person a dbo:Scientist. ?person rdfs:label ?name. ?person dbo:birthPlace dbp:Denver. }
AVERAGE MACHINE
You can only use a TPF interface of DBpedia.
The future
is federated
Big Data fails at Web scaleLight interfaces rule
Engineer for serendipity
Engineer for serendipity.—Roy T. Fielding
If 1 endpoint is downfor 1.5 days each month, then 2 endpoints might be for 3 days each month.
Federated queries withSPARQL endpointspose a problem.
Just ask each of the questions to different TPF servers.
Federated queries arenative to TPF clients.
But in federated scenarios,performance can be on par with SPARQL endpoints!
TPF trades server cost for query performance.
TPF is not the final solution —no API will ever be— but an excellent starting point.
Lightweight interfacesare easy to extend and combine with others.
The Memento protocolbrings time to the Web.
Ask for representations at a certain point in the past.
TPF and Mementoare a great match.
We combined them in collaboration with Herbert Van de Sompel & team at the Los Alamos National Laboratory.
The future
is federated
Big Data fails at Web scaleLight interfaces ruleEngineer for serendipity
VIVO
client SPARQL
VIVO today
TPFserver
VIVO
client TPF
VIVO tomorrow?
Federationis a game changer.
Federationis a game changer.
with the TPF interface
powerWith great
responsibilitycomes great
realisticWe need
expectationsabout our
to be
Some queries willalways be hardon an open Web.You might need centralization if you want answers fast.
*
*Terms and conditions apply.
…and streaming!
Many more queriesthan you’d thinkare pretty fast…
OPEN SOURCElinkeddatafragments.org
@RubenVerborgh
and it
starts today
The future
is federated
Related Documents
