Neo4j- or, why graph dbs are teh
AWESOME!!!11
oh, and also... USE CASES!!!
Emil EifremNeo Technology
#neo4j@[email protected]
What's the plan?
An intro to graph databases and Neo4j
Brief review of some use cases
Conclusion and Q&A
Graph DBs& Neo4j intro
The Graph DB model: representationCore abstractions:
NodesRelationships between nodesProperties on both
name = “Emil”age = 29sex = “yes”
type = KNOWStime = 4 years
type = carvendor = “SAAB”model = “95 Aero”
11 22
33
Example: The Matrix
name = “Thomas Anderson”age = 29
11
name = “The Architect”
4242
CODED_BY
disclosure = public
name = “Cypher”last name = “Reagan”
disclosure = secretage = 6 months
name = “Agent Smith”version = 1.0blanguage = C++
33
1313
KNOWS KNOWS
name = “Morpheus”rank = “Captain”occupation = “Total badass”
age = 3 days
name = “Trinity”
77
22
KNOWS
KNOWS
KNO
WS
Code (1): Building a node spaceGraphDatabaseService graphDb = ... // Get factory
// Create Thomas 'Neo' AndersonNode mrAnderson = graphDb.createNode();mrAnderson.setProperty( "name", "Thomas Anderson" );mrAnderson.setProperty( "age", 29 );
// Create MorpheusNode morpheus = graphDb.createNode();morpheus.setProperty( "name", "Morpheus" );morpheus.setProperty( "rank", "Captain" );morpheus.setProperty( "occupation", "Total bad ass" );
// Create a relationship representing that they know each othermrAnderson.createRelationshipTo( morpheus, RelTypes.KNOWS );// ...create Trinity, Cypher, Agent Smith, Architect similarly
Code (1): Building a node spaceGraphDatabaseService graphDb = ... // Get factoryTransaction tx = graphDb.beginTx();
// Create Thomas 'Neo' AndersonNode mrAnderson = graphDb.createNode();mrAnderson.setProperty( "name", "Thomas Anderson" );mrAnderson.setProperty( "age", 29 );
// Create MorpheusNode morpheus = graphDb.createNode();morpheus.setProperty( "name", "Morpheus" );morpheus.setProperty( "rank", "Captain" );morpheus.setProperty( "occupation", "Total bad ass" );
// Create a relationship representing that they know each othermrAnderson.createRelationshipTo( morpheus, RelTypes.KNOWS );// ...create Trinity, Cypher, Agent Smith, Architect similarly
tx.commit();
Code (1b): Defi ning RelationshipTypes// In package org.neo4j.graphdbpublic interface RelationshipType{ String name();}
// In package org.yourdomain.yourapp// Example on how to roll dynamic RelationshipTypesclass MyDynamicRelType implements RelationshipType{ private final String name; MyDynamicRelType( String name ){ this.name = name; } public String name() { return this.name; }}
// Example on how to kick it, static-RelationshipType-likeenum MyStaticRelTypes implements RelationshipType{ KNOWS, WORKS_FOR,}
Whiteboard friendly
Björn Big Car
DayCare
Björn
owns
drivesbuild
The Graph DB model: traversalTraverser framework for high-performance traversing across the node space
name = “Emil”age = 31sex = “yes”
type = KNOWStime = 4 years
type = carvendor = “SAAB”model = “95 Aero”
11 22
33
Example: Mr Anderson’s friends
name = “Thomas Anderson”age = 29
11
name = “The Architect”
4242
CODED_BY
disclosure = public
name = “Cypher”last name = “Reagan”
disclosure = secretage = 6 months
name = “Agent Smith”version = 1.0blanguage = C++
33
1313
KNOWS KNOWS
name = “Morpheus”rank = “Captain”occupation = “Total badass”
age = 3 days
name = “Trinity”
77
22
KNOWS
KNOWS
KNO
WS
Code (2): Traversing a node space
// Instantiate a traverser that returns Mr Anderson's friendsTraverser friendsTraverser = mrAnderson.traverse(
Traverser.Order.BREADTH_FIRST,StopEvaluator.END_OF_GRAPH,ReturnableEvaluator.ALL_BUT_START_NODE,RelTypes.KNOWS,Direction.OUTGOING );
// Traverse the node space and print out the resultSystem.out.println( "Mr Anderson's friends:" );for ( Node friend : friendsTraverser ){
System.out.printf( "At depth %d => %s%n",friendsTraverser.currentPosition().getDepth(),friend.getProperty( "name" ) );
}
$ bin/start-neo-exampleMr Anderson's friends:
At depth 1 => MorpheusAt depth 1 => TrinityAt depth 2 => CypherAt depth 3 => Agent Smith$
friendsTraverser = mrAnderson.traverse( Traverser.Order.BREADTH_FIRST, StopEvaluator.END_OF_GRAPH, ReturnableEvaluator.ALL_BUT_START_NODE, RelTypes.KNOWS, Direction.OUTGOING );
name = “Thomas Anderson”age = 29
name = “Morpheus”rank = “Captain”occupation = “Total badass”
name = “The Architect”
disclosure = public
age = 3 days
name = “Trinity”
name = “Cypher”last name = “Reagan”
disclosure = secretage = 6 months
name = “Agent Smith”version = 1.0blanguage = C++
77
22
33
1313
4242
11KNOWS KNOWS CODED_BYKNOWS
KNOWSKN
OW
S
Example: Friends in love?
name = “Thomas Anderson”age = 29
name = “Morpheus”rank = “Captain”occupation = “Total badass”
name = “The Architect”
disclosure = public
name = “Trinity”
name = “Cypher”last name = “Reagan”
disclosure = secretage = 6 months
name = “Agent Smith”version = 1.0blanguage = C++
77
22
33
1313
4242
11KNOWS KNOWS CODED_BYKNOWS
KNOWS
KNO
WS
LOVES
Code (3a): Custom traverser
// Create a traverser that returns all “friends in love”Traverser loveTraverser = mrAnderson.traverse(
Traverser.Order.BREADTH_FIRST,StopEvaluator.END_OF_GRAPH,new ReturnableEvaluator(){
public boolean isReturnableNode( TraversalPosition pos ){
return pos.currentNode().hasRelationship( RelTypes.LOVES, Direction.OUTGOING );
}},RelTypes.KNOWS,Direction.OUTGOING );
Code (3a): Custom traverser
// Traverse the node space and print out the resultSystem.out.println( "Who’s a lover?" );for ( Node person : loveTraverser ){
System.out.printf( "At depth %d => %s%n",loveTraverser.currentPosition().getDepth(),person.getProperty( "name" ) );
}
new ReturnableEvaluator(){ public boolean isReturnableNode( TraversalPosition pos) { return pos.currentNode(). hasRelationship( RelTypes.LOVES, Direction.OUTGOING ); }},
$ bin/start-neo-exampleWho’s a lover?
At depth 1 => Trinity$
name = “Thomas Anderson”age = 29
name = “Morpheus”rank = “Captain”occupation = “Total badass”
name = “The Architect”
disclosure = public
name = “Trinity”
name = “Cypher”last name = “Reagan”
disclosure = secretage = 6 months
name = “Agent Smith”version = 1.0blanguage = C++
77
22
33
1313
4242
11KNOWS KNOWS CODED_BYKNOWS
KNOWSKN
OW
SLOVES
Bonus code: domain modelHow do you implement your domain model?Use the delegator pattern, i.e. every domain entity wraps a Neo4j primitive:
// In package org.yourdomain.yourappclass PersonImpl implements Person{ private final Node underlyingNode; PersonImpl( Node node ){ this.underlyingNode = node; }
public String getName() { return (String) this.underlyingNode.getProperty( "name" ); } public void setName( String name ) { this.underlyingNode.setProperty( "name", name ); }}
Domain layer frameworksQi4j (www.qi4j.org)
Framework for doing DDD in pure Java5Defi nes Entities / Associations / Properties
Sound familiar? Nodes / Rel’s / Properties!Neo4j is an “EntityStore” backend
Jo4neo (http://code.google.com/p/jo4neo)
Annotation drivenWeaves Neo4j-backed persistence into domain objects at runtime
S pring Data G raph (http://www.springsource.org/spring-data)
Collaboration with SpringSourceAnnotation driven, AspectJ based
Neo4j system characteristicsDisk-based
Native graph storage engine with custom binary on-disk format
TransactionalJTA/JTS, XA, 2PC, Tx recovery, deadlock detection, MVCC, etc
Scales up
Many billions of nodes/rels/props on single JVMRobust
7+ years in 24/7 production
M'kay... but what are some
use cases?
Social data (customer: e.g. EU largest social network)
name = “Mike”age = 29
11
disclosure = public
name = “Charlie”last_name = “Runkle”
name = “Dani”last_name = “California”age = 27
33
1313
KNOWS KNOWS
name = “Hank”last_name = “Moody”age = 42
age = 3 days
name = “Karen”
77
22
KNOWS
KNOWSKN
OW
S
name = “Marcy Runkle”
4242
KNOWS
Just a social graph?
Spatial data (customer: large telecom company)
name = “Omni Hotel”lat = 3492848long = 283823423
11
length = 7 miles
name = ...lat, long = ...
name = “Swedland”lat = 23410349long = 2342348852
33
1313
ROAD ROOOAD
name = “The Tavern”lat = 1295238237long = 234823492
length = 3 miles
name = ...
77
22
ROAD
ROADR
OAD
name = ...
4242
ROAD
Social? Spatial? … Social AND spatial!
Social AND spatial data (customer: LBS)
name = “Omni Hotel”lat = 3492848long = 283823423
11
weight = 10
name = “Emil”beer_qual = expert
name = “Maria”age = 30beer_qual = non-existant
33
1313
LIKES SIBLING
name = “The Tavern”lat = 1295238237long = 234823492
length = 3 miles
name = ...
77
22
ROAD
ROADR
OAD
name = “Peter”
4242
KNOWS
Financial data (customer: international bank)
name = “Mr Godfather”karma = veeeery-lowcash = more-than-you
11
amount = $1000
name = “Emil”cash = always-too-li'l
title = “ATM @ Wall St”id = 230918484233cash_left = 384204
33
1313
TRANSFER WITHDRAW
name = “The Tavern”lat = 1295238237long = 234823492
amount = $1000
name = ...
77
22
OWNS
DEPOSITTR
ANSF
ER
name = ...
4242
WITHDRAW
Cute. But what about performance?
# nodes query timeRelational database (MySQL) 1 000 2 000 msGraph database (Neo4j) 1 000 2 msGraph database (Neo4j) 1 000 000 2 ms
pathExists(a, b, 5)
name = “Mr Godfather”karma = veeeery-lowcash = more-than-you
11
amount = $1000
name = “Emil”cash = always-too-li'l
title = “ATM @ Wall St”id = 230918484233cash_left = 384204
33
1313
TRANSFER WITHDRAW
name = “The Tavern”lat = 1295238237long = 234823492
amount = $1000
name = ...
77
22
WITHDRAW
DEPOSITTR
ANSF
ER
name = ...
4242
WITHDRAW
How ego are you? (aka other impls?)Franz’ A lleg roG raph (http://agraph.franz.com)
Proprietary, Lisp, RDF-oriented but real graphdb
Sones g raphDB (http://sones.com)
.NET, open source version available
Twitter's FlockDB (http://github.com/twitter/flockdb)
Twitter's graph database for large and shallow graphs
Sparsity Technologies Dex (http://sparsitytechnologies.com)
New on the scene, spun out of Spanish university
Objectivity InfiniteG raph (http://infinitegraph.com)
Graph db on-top-of OODB
ConclusionGraphs && Neo4j => teh awesome!Available NOW under AGPLv3 / commercial license
AGPLv3: “if you’re open source, we’re open source”If you have proprietary software? Must buy a commercial licenseBut the first one is free!
Downloadhttp://neo4j.org
Feedbackhttp://lists.neo4j.org
The N eo4j teamis hiring !
Do you see the Matrix? Apply now.
Questions?
Image credit: lost again! Sorry :(
http://neotechnology.com