3/15 - Intro to Spring Data Neo4j

WebinarIntroduction to

Spring Data Neo4j 2.0

Michael Hunger, Neo Technology

Thursday, March 15, 12

Me & You ?

๏Me: Michael Hunger, Neo Technology

• passionate software developer / Neo4j German Division

• Spring Data Neo4j Project Lead

•Neo4j Cloud Hosting

•[email protected] / @mesirii

๏You:

• Java & Spring !

• Enterprise ?

• Rich Domain ?

•NOSQL ?


mailto:[email protected]

mailto:[email protected]

About the Webinar

๏Covers Spring Data Neo4j only

๏too much content anyway

๏so it will be just a whirlwind tour

๏no detailed introductions to NOSQL and Neo4j in particular

๏ for those go to http://neo4j.org

๏Q&A at the end

๏all your questions will be answered in a follow up blog post, don‘t hesitate to ask


http://video.neo4j.org


What‘s up?

๏Spring Data overview

๏Networks are everywhere

๏Neo4j introduction

๏Spring Data Neo4j introduction

4


Spring Data

5


What is NOSQL?


What is NOSQL?

It’s not “No to SQL”


What is NOSQL?


It’s not “Never SQL”


What is NOSQL?



It’s “Not Only SQL”


What is NOSQL?



It’s “Not Only SQL”

NOSQL \no-seek-wool\ n. Describes ongoing trend where developers increasingly opt for non-relational databases to help solve their problems, in an effort to use the right tool for the right job.


NOSQL categories

We see four main categories in the NOSQL space:


NOSQL categories


•Redis

•Riak

•Voldemort

Key-Value


NOSQL categories


•Cassandra

•HBase

Column-family/BigTable

•Redis

•Riak

•Voldemort

Key-Value


NOSQL categories


•Cassandra

•HBase


•Redis

•Riak

•Voldemort

Key-Value

•MongoDB

•CouchDB

Document


NOSQL categories


•Cassandra

•HBase


•Redis

•Riak

•Voldemort

Key-Value

•MongoDB

•CouchDB

Document

•Neo4j

•InfiniteGraph

•OrientDB

•DEX

Graph


Scaling to size vs. Scaling to complexity

8

Size

Complexity

Key-Value stores

Bigtable clones

Document databases

Graph databases



8

Size

Complexity

Key-Value stores

Bigtable clones

Document databases

Graph databases

Billions of nodesand relationships



8

Size

Complexity

Key-Value stores

Bigtable clones

Document databases

Graph databases

> 90% of use cases

Billions of nodesand relationships


Why NOSQL now?

๏Trend 1: Size of data is growing

๏Trend 2: Data is increasingly connected

๏Trend 3: Data is increasingly semi-structured

๏Trend 4: Change in architecture

We have observed four trends since the 90‘s:


What is Spring Data?

10

๏VMWare/SpringSource initiative to give Spring developers easy access to the emerging world of NOSQL, including:

•Non-relational databases

•Grails NOSQL support

•Cross-store persistence

•Object Persistence Mapping Infrastructure

•Generic Repository Infrastructure

• upcoming Spring Roo add-ons


Spring Data projects

๏Code is in SpringSource git repository:

• http://springsource.org/spring-data

• https://github.com/SpringSource/spring-data-*

๏ Includes:

• data-commons

• spring-data-graph-neo4j

• spring-data-{redis,riak}

• spring-data-mongo

• spring-data-jdbc

• spring-data-jpa


http://springsource.org/spring-data

http://springsource.org/spring-data

https://github.com/SpringSource/spring-data-graph


Spring Data Neo4j

๏Focus on Spring Data Neo4j

๏VMWare is collaborating with Neo Technology, the company behind the Neo4j graph database.

๏Improved programming model: Annotation-based programming model for applications with rich domain models

๏Cross-store persistence: Extend existing JPA application with NOSQL persistence

๏Spring Roo support: Add graph persistence with Roo add-on


13

Graphs are everywhere


Even in the Matrix- everything is a graph

14

Google Image Search: „graph OR network“


Graphs Everywhere

๏Relationships in

• Politics, Economics, History, Science, Transportation

๏Biology, Chemistry, Physics, Sociology

•Body, Ecosphere, Reaction, Interactions

๏ Internet / IT

•Hardware, Software, Interaction

๏Social Networks

• Family, Friends

•Work, Communities

•Neighbours, Cities, Society 15


Good Relationships

๏World is rich, messy and related data

๏Relationships are as least as important as the things they connect

๏Graphs = Whole > Sum of all parts

๏Complex interactions

๏Always changing, change of structures as well

๏Graph => Relationships are part of the data

๏RDBMS => Relationships part of the fixed schema

16


Questions and Answers

๏Complex Questions

๏Answers lie between the lines (things)

๏Locality of the information

๏Global searches / operations very expensive

๏Constant query time, regardless of data volume

17


Categories ?

๏Categories == Classes, Trees ?

๏What if more than one category fits?

๏Tags

๏Categories via relationships like „IS_A“

๏Any number, easy to change

๏„Virtual“ Relationships - Traversals

๏Category dynamically derived from queries

18


Neo4j introduction


What is Neo4j?data model


What is Neo4j?

A Graph Databasedata model


What is Neo4j?



What is Neo4j?



What is Neo4j?

Node Node



What is Neo4j?

Node Node

A Graph Database

Typed relationship

data model


What is Neo4j?

NodeNode

A Graph Database

Typed relationship

data model

key: value key: value


For example


For example

name: Oliver


For example

name: Oliver name: Michael


For example

name: Oliver name: MichaelWORKS_WITHproject: spring-data


For example


company: VMware

EMPLOYED_BY

city: Dresdencountry: DE

LIVE

S_IN


For example


company: VMware

EMPLOYED_BY


LIVE

S_IN

name: Rod

KNOWS KNOWS


For example


company: VMware

EMPLOYED_BY


LIVE

S_IN

name: Rod

KNOWS KNOWS

food: Chinese

LIKE

S


Facts about Neo4j

•Written in Java •Embeddable or standalone Server•Schema-free - perfect for rich domains•Fully transactional (ACID)•Persistent to custom on-disk file structure•Traversal speeds of 1,000,000+ hops per second•Integrated indexing•Plenthora of language bindings•24/7 production since 2003


Show me some code, please

Node david = graphDb.createNode();Node andreas = graphDb.createNode();

david.setProperty(“name”, “David Montag”);andreas.setProperty(“name”, “Andreas Kollegger”);

Relationship presentedWith = david.createRelationshipTo( andreas, PresentationTypes.PRESENTED_WITH);presentedWith.setProperty(“date”, System.currentTimeMillis());

GraphDatabaseService graphDb = new EmbeddedGraphDatabase(“var/neo4j”);


Show me some code, please

Node david = graphDb.createNode();Node andreas = graphDb.createNode();

david.setProperty(“name”, “David Montag”);andreas.setProperty(“name”, “Andreas Kollegger”);

Relationship presentedWith = david.createRelationshipTo( andreas, PresentationTypes.PRESENTED_WITH);presentedWith.setProperty(“date”, System.currentTimeMillis());

Transaction tx = graphDb.beginTx();try {

tx.success();} finally { tx.finish();}

GraphDatabaseService graphDb = new EmbeddedGraphDatabase(“var/neo4j”);


Traversal framework


Traversal frameworkExample: a dependency graph

A

BDC



A

BDC

Query: Find all dependencies of A, transitively



A

BDC




A

BDC




A

BDC




A

BDC


TraversalDescription desc = Traversal.description() .relationships(ExampleTypes.DEPENDS_ON, Direction.OUTGOING);

Node a = ...;

for (Node dependency : desc.traverse(a).nodes()) { print(dependency);}


So how do I find a node to traverse from?

?



?By us

ing in

dexin

g!



?name: DavidBy us

ing in

dexin

g!



?Davidname: DavidBy

using

inde

xing!



?Davidname: David

Ed

Allison

Michael

Andreas

By us

ing in

dexin

g!


Cypher Query Language

๏Declarative query language

•Describe what you want, not how

•Based on pattern matching

๏Examples:

26

START david=node:people(name=”David M”) # index lookupMATCH david-[:knows]-friends-[:knows]-new_friendsWHERE new_friends.age > 18RETURN new_friends

START user=node(5, 15, 26, 28) # node IDsMATCH user--friendRETURN user, COUNT(friend), SUM(friend.money)


Beyond basics๏Traversal API / DSL

• Express powerful Graph Traversals succinctly

๏Graph algorithm library

•Cheapest path (Dijkstra, A*)

• Shortest path

• Simple paths

•All paths

๏REST API access to Neo4j Standalone Server

๏High availability and online backups

27


Spring Data Neo4j 2.0


Spring Framework Conveniences

๏default enterprise development framework

๏ future Java Cloud stack

๏POJO centric application design

๏made AspectJ aspects socially acceptable (tx-config, @Configurable)

๏Template programming model

๏ Inversion of Control / Dependency Injection

๏Spring Java Config, configuration XML-namespaces

๏ lots of powerful libraries and library abstractions

๏existing, well-known persistence framework for JDBC

๏Spring Data project


Programming model overview

๏SDN is the brain child of Rod Johnson and Emil Eifrém

•Wrote functional initial prototype

•Developed by Neo Technology and SpringSource teams

๏Uses annotations to define graph entities

๏Entity state backed by graph database

๏Two modes of Object Graph Mapping

• POJO Graph Mapping

• seamless AspectJ backed Object Graph Mapping

๏Spring Roo add-on available


Spring Data Neo4j features

๏ Annotation-based programming model

๏ Spring Data Commons Repository support

๏ Neo4j Query (Cypher, Gremlin) and Traversal support

• on dynamic fields and via repository methods

๏ Neo4j indexing support (includes fulltext and numeric range queries)

๏ Entity types stored in the graph database as well

๏ Dynamic type projection (duck typing)

๏ Cross-store support for partial JPA / graph entities

๏ Support for JSR-303 (bean validation)

๏ Support for the Neo4j Server (remote server and in server extension)

๏ Neo4jTemplate with exception translation, optional transaction management and more

31


public class Actor { private final Node underlyingNode;

Actor( final Node node ) { underlyingNode = node; }

public Node getUnderlyingNode() { return underlyingNode; }

public final String getName() { return (String) underlyingNode.getProperty( “name” ); }

public void setName( final String name ) { underlyingNode.setProperty( “name”, name ); }}

Classic Neo4j domain class


Spring Data Neo4j domain class

@NodeEntitypublic class Actor { @Indexed private String name;}


What about relationships

public Iterable<Movie> getMovies() { final List<Movie> movies = new LinkedList<Movie>(); for ( Relationship rel : underlyingNode.getRelationships( RelTypes.ACTS_IN, Direction.OUTGOING ) ) { movies.add( new Movie( rel.getEndNode() ) ); } return movies; }

@NodeEntityclass Actor { ...

Old @RelatedTo(type="ACTS_IN", elementCl

ass = Movie.class)

private Set<Movie> movies; public Iterable<Movie> getMovies() {

return movies; }

New


Defining entity classes



•@NodeEntity

• Represents a node in the graph

• Fields saved as properties on node

• References stored as relationships between nodes

• Instantiated using Java ‘new’ keyword, like any POJO

• Also returned by lookup mechanisms

• Type information stored in the graph





•@RelationshipEntity

• Represents a relationship in the graph

• Fields saved as properties on relationship

• Special fields for start- and end-nodes

• Only returned by lookup methods


Fields in node entities (@NodeEntity)

• Primitive types and strings are directly persisted

• For all other types, Spring conversion support can be used

• Enum and Date conversion is provided out-of-the-box

•Transient fields not persisted

37

@NodeEntitypublic class Actor { private String name; private int age; private HairColor hairColor; private transient String nickname;}



• Fields of types that represent a node (@NodeEntity)

38




38

@NodeEntitypublic class Movie {}

@NodeEntitypublic class Person { private Movie favoriteMovie;}




38


@NodeEntitypublic class Person { private Movie favoriteMovie;}

@NodeEntitypublic class Movie { private Actor topActor;}

@NodeEntitypublic class Actor { // Mirrors topActor in Movie @RelatedTo(type = ”topActor”, direction = Direction.INCOMING) private Movie wasTopActorIn;}



• Fields of collections of node entities: @RelatedTo

39


@NodeEntitypublic class Actor { @RelatedTo(type = “ACTS_IN”) private Set<Movie> movies;}



๏Fields of collections of relationship entities: @RelatedToVia

๏Read only view of relationship entities40

@RelationshipEntitypublic class Role { @StartNode private Actor actor; @EndNode private Movie movie; private String roleName;}

@NodeEntitypublic class Actor { @RelatedToVia(type = “ACTS_IN”) private Iterable<Role> roles;}


Fields in relationship entities (@RelationshipEntity)

• Primitive and convertible types work just the same

• @StartNode and @EndNode provide access to the start and end node entities for the relationship entity

@RelationshipEntitypublic class Role { @StartNode private Actor actor; @EndNode private Movie movie; private String title;}


Indexing

@NodeEntitypublic class Actor { @Indexed private String name; @Indexed private HairColor hairColor;

GraphRepository<Actor> actorRepo = template.repositoryFor(Actor.class);Actor kevin = actorRepo.findByPropertyValue(“name”, “Kevin Bacon”);Iterable<Actor> allBlondActors = actorRepo.findAllByPropertyValue(“hairColor”, “blond”);

It can then be looked up:

By annotating an entity field with @Indexed it becomes searchable:


Indexing

@NodeEntitypublic class Actor { @Indexed(type = FULLTEXT) private String name;

๏ Index name defaults to domain class name

๏ Index key defaults to field name

๏Fulltext and spatial index types

๏Repository query methods for any Lucene query, including ranges:

Iterable<Actor> allKevinsOlderThan32 = actorRepo.findAllByQuery(“name:Kevin* AND age>32”);Iterable<Actor> youngActors = actorRepo.findAllByRange(“age”, 3, 18);


Traversal

44

@NodeEntitypublic class Actor { @GraphTraversal( traversalBuilder = CoactorsTraversalDescriptionBuilder.class) private Iterable<Actor> coactors;}

public class CoactorsTraversalDescriptionBuilder implements FieldTraversalDescriptionBuilder { public TraversalDescription build(...) { return Traversal.description() .evaluator(Evaluators.atDepth(2)) .relationships(RelTypes.ACTS_IN); }}

Example for dynamic field computation


Cypher query language

45

@NodeEntitypublic class Actor { @Query(“START actor=node({self}) ” + “MATCH actor-[:ACTS_IN]->movie<-[:ACTS_IN]-coactor ” + “RETURN coactor”) private Iterable<Actor> coactors;}

@NodeEntitypublic class User { @Query(“START actor=node({self}) ” + “MATCH actor-[:ACTS_IN]->movie<-[:ACTS_IN]-coactor ” + “RETURN movie.title, coactor.name”) private List<Map<String, Object>> movieCoactorPairs;}


POJO-Mapping

46

๏Based on Spring-Data-Commons infrastructure

• Extract Mapping Meta Information

• Entity-Converter for Object-Graph-Mapping

‣Type-resolution

‣Entity construction

‣Transfer properties

‣Load Cache

‣Load Policies + recurse for eagerly fetched relationships

•Neo4j-Template for Graph-Interaction


AspectJ

47

๏ Introduces interface to entities:

•NodeBacked into @NodeEntity classes

• RelationshipBacked into @RelationshipEntity classes

๏NodeBacked introduces methods such as:

• relateTo

• findByQuery

• findAllByTraversal

• ...

๏going to be pulled out in separate Active-Record-Mixin


AspectJ - Tooling

48

๏ IDE‘s

• latest versions of STS / Eclipse with current AJDT plugin

• IntelliJ IDEA 10.5 compile + run, some editor quirks

‣full AspectJ support in IDEA 11

๏Build Systems

•Maven

•Gradle

•Ant / Ivy

• ...


AspectJ - NodeBacked.relateTo

49

<T extends RelationshipBacked> T NodeBacked.relateTo( NodeBacked node, Class<T> relationshipEntityType, String relationshipType);

@NodeEntitypublic class Actor { public Role actsIn(Movie movie, String roleName) { Role role = relateTo(movie, Role.class, “ACTS_IN”); role.setName(roleName); return role; }}

usage:


Interface based Repositories๏ based on Repository infrastructure in Spring Data Commons

๏ just define the interface and the namespace configuration

๏provide out-of-the-box support for

•CRUD-Operations

• Index-Lookups

•Traversal-Execution

•Annotated Graph-Queries (Cypher, Gremlin)

•Derived Queries

• Spatial Queries

๏extensible via custom methods with provided implementations50


Repositories

51

interface MovieRepository extends GraphRepository<Movie> { Movie findById(String id); Page<Movie> findByTitle(String title, Pageable page);}

<neo4j:repositories base-package=“com.example.repositories„/>

@Controllerpublic class MovieController { @Autowired MovieRepository moviesRepository;

@RequestMapping(value = "/movies/{movieId}",...) public String show(Model model, @PathVariable String movieId) { Movie movie = moviesRepository.findByPropertyValue("id", movieId); Movie movie = moviesRepository.findById(movieId); model.addAttribute("movie", movie); return "/movies/show"; }}


Repository-Query-Support

52

interface MovieRepository extends GraphRepository<Movie> {

@Query("start m=({movie}) match m-[ACTS_IN]-actor return actor") List<Actor> getActorsCypher(@Param("movie") Movie m); @Query("start movie =({0}) match (movie)<-[role:ACTS_IN]-(actor) return actor.name, role.title") Iterable<Map<String,Object>> getCast(Movie m);

@Query(value = "g.v(movie).out('ACTS_IN')", type = QueryType.Gremlin) Set<Person> getActorsGremlin(@Param("movie") Movie m);

Page<Person> findByTitleAndActorsName(String title, String name, Pageable page); }


Neo4j-Template (I)

๏well known Spring Template Pattern

๏Resource / Transaction Management

๏Convenience Methods

๏Nodes and Entities handling & conversion

๏Fluent Query Result Handling

๏Works also via REST with Neo4j-Server

๏Exception Translation

53


Neo4j-Template (II)

54

template.lookup("node", "name", "David") .to(String.class, new PropertyContainerNameConverter()).single()

template.traverse(node, traversal) .to(Integer.class,new ResultConverter<Path,Integer>() { public Integer convert(Path path, Class<String> type) { return path.length(); }})

template.query("start movie=(Movies,title, {m}) match movie-->actor return actor", map("m","Matrix")).to(Actor.class)

template.execute("g.v(ref).outE", map("ref",0)).to(Relationship.class)


REST-Client-Support

55

<bean id="restGraphDatabaseService" class="org.sf.data.neo4j.rest.SpringRestGraphDatabase"> <constructor-arg value="http://localhost:7473/db/data" /> </bean>

<datagraph:config graphDatabaseService="restGraphDatabaseService"/>

๏drop-in replacement for the embedded GraphDatabase

๏works transparently with POJO-Entity-Mapping and

Neo4j-Template


http://localhost:7473/db/data

http://localhost:7473/db/data

REST-Server-Support

56

public class HelloWorldInitializer extends SpringPluginInitializer { public HelloWorldInitializer() { super(new String[]{"spring/helloWorldServer-Context.xml"}, Pair.of("worldRepository", WorldRepository.class), Pair.of("graphRepositoryFactory", GraphRepositoryFactory.class)); }}

๏ integrate Spring Data Neo4j config with already running

Graph-Database in Neo4j-Server

๏expose Spring Beans as Jersey Injectables


Cross-store persistence


A scenario...

You have a traditional web app using JPA to persist data to a relational database


Option C: Introducing cross-store persistence

๏ JPA data and NOSQL data can share a data model

๏Could be the entire entity, or some fields of an entity

๏We call this cross-store persistence

•One transaction manager to coordinate the NOSQL database with the JPA relational database

•AspectJ support to manage the NOSQL entities and fields


The JPA data model

Restaurant@Entitypublic class Restaurant {

@Id @GeneratedValue private Long id; private String name; private String city; private String state; private String zipCode;

UserAccount@Entity@Table(name = "user_account")public class UserAccount { @Id @GeneratedValue private Long id; private String userName; private String firstName; private String lastName; @Temporal(TemporalType.TIMESTAMP) private Date birthDate; @ManyToMany(cascade = CascadeType.ALL) private Set<Restaurant> favorites;


Adding to the data model

Recommendation

@RelationshipEntitypublic class Recommendation { @StartNode private UserAccount user; @EndNode private Restaurant restaurant; private int stars; private String comment;

Restaurant@Entity@NodeEntity(partial = true)public class Restaurant { @Id @GeneratedValue private Long id; private String name; private String city; private String state; private String zipCode;

UserAccount@Entity@Table(name = "user_account")@NodeEntity(partial = true)public class UserAccount { @Id @GeneratedValue private Long id; private String userName; private String firstName; private String lastName; @Temporal(TemporalType.TIMESTAMP) private Date birthDate; @ManyToMany(cascade = CascadeType.ALL) private Set<Restaurant> favorites;

@GraphProperty String nickname; @RelatedTo(type = "friends", elementClass = UserAccount.class) Set<UserAccount> friends; @RelatedToVia(type = "recommends", elementClass = Recommendation.class) Iterable<Recommendation> recommendations;



Recommendation







Recommendation







Recommendation







Recommendation






Spring Data Neo4j Roo add-on

62

๏Roo adding support for non-JPA persistence providers

๏Spring Data Neo4j was the first NOSQL persistence Roo Add-On

๏See the chapter on Spring Data Neo4j in the latest O’Reilly Roo book, Getting Started with Roo.


Spring Data Neo4j Roo add-on

63

roo> project --topLevelPackage org.neo4j.imdbroo> graph setup --provider NEO4J --databaseLocation target/imdb

roo> graph entity --class ~.model.Movieroo> field string titleroo> field number --fieldName year --type java.lang.Integer --permitReservedWords --primitive

roo> graph entity --class ~.model.Actorroo> field string name

roo> graph relationship --to Movie --from Actor --fieldName movies --type ACTS_IN --cardinality ONE_TO_MANYroo> graph relationship --via ~.model.Role --to Movie --from Actor --fieldName roles --type ACTS_IN --cardinality ONE_TO_MANYroo> graph relationship --from Movie --to Actor --type ACTS_IN --fieldName actors --direction INCOMING --cardinality MANY_TO_ONEroo> field string --fieldName title --class ~.model.Role

roo> controller scaffold --class ~.web.ActorController --entity ~.model.Actorroo> controller scaffold --class ~.web.MovieController --entity ~.model.Movie


http://web.ActorController

http://web.ActorController

http://web.MovieController

http://web.MovieController

Spring Data Neo4j Guidebook“Good Relationships”

๏Spring Data Neo4j comes with a great Guide Book, featuring:

• Forewords by Rod Johnson and Emil Eifrem

•An easy to read, narrative tutorial walkthrough for cineasts.net

•A comprehensive reference for all the details

•Check it out here: http://spring.neo4j.org/guide

64

“I’m excited about Spring Data Neo4j.... Spring Data Neo4j makes working with Neo4j amazingly easy, and therefore has the potential to make you more successful as a developer.”

Rod Johnson, founder of Spring


http://cineasts.net/

http://cineasts.net/

http://bit.ly/das-book

http://bit.ly/das-book

The end (and the beginning!)๏See the Spring Data Neo4j site for more info:

http://spring.neo4j.org

๏Check out the developer notes at GitHub:http://spring.neo4j.org/notes

๏Again, don’t miss our fantastic e-book on Spring Data Neo4j:http://spring.neo4j.org/guide

๏Spring Data Forum at http://spring.neo4j.org/discussions

๏All about Neo4j: http://neo4j.org

๏Neo4j videos and webinars: http://video.neo4j.org


http://git.springsource.org/spring-data

http://git.springsource.org/spring-data



http://bit.ly/sdg-book

http://bit.ly/sdg-book

http://forum.springsource.org/forumdisplay.php?f=80

http://forum.springsource.org/forumdisplay.php?f=80





Cineasts.net

Check Out: http://spring.neo4j.org/tutorialThursday, March 15, 12

https://github.com/jexp/cineasts

https://github.com/jexp/cineasts

3/15 - Intro to Spring Data Neo4j

Real Estate

cardinality

private restaurant

private int

private long

private string

private string

private string

private string