Unless otherwise indicated, these slides are © 2016 Pivotal Software, Inc. and licensed under aCreative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Using Docker, Neo4j, and Spring Cloud for Developing Microservices
Kenny Bastani, Spring Developer Advocate, Pivotal @kennybastani
https://github.com/kbastani/spring-boot-graph-processing-example
Speaker Intro - Kenny Bastani
2
Ranking Twitter ProfilesUsing PageRank
3 https://github.com/kbastani/spring-boot-graph-processing-example
https://github.com/kbastani/spring-boot-graph-processing-example
PageRank algorithm
4
https://github.com/kbastani/spring-boot-graph-processing-example
PageRank algorithm
5
https://github.com/kbastani/spring-boot-graph-processing-example6
https://github.com/kbastani/spring-boot-graph-processing-example7
https://github.com/kbastani/spring-boot-graph-processing-example8
https://github.com/kbastani/spring-boot-graph-processing-example
Tools we’ll be using
! Spring Boot
! Neo4j
! Apache Spark
! Docker
! RabbitMQ
9
https://github.com/kbastani/spring-boot-graph-processing-example
Containerize all the things!
10
https://github.com/kbastani/spring-boot-graph-processing-example11
https://github.com/kbastani/spring-boot-graph-processing-example12
https://github.com/kbastani/spring-boot-graph-processing-example13
https://github.com/kbastani/spring-boot-graph-processing-example14
Connecting Neo4j and Apache Spark…to submit PageRank jobs
15 https://github.com/kbastani/spring-boot-graph-processing-example
https://github.com/kbastani/spring-boot-graph-processing-example
Request new Apache Spark job
16
Algorithm TypeRelationship Type
https://github.com/kbastani/spring-boot-graph-processing-example
Export Neo4j graph to HDFS
17
New Job Request
HDFS Path: /../../graph.csv
Job Type: pagerank
https://github.com/kbastani/spring-boot-graph-processing-example
Encoding a graph as an edge list
18
edge list
G B
H B
I B
K B
E B
F B
J B
D B
G E
H E
I E
K E …
https://github.com/kbastani/spring-boot-graph-processing-example
Import edge list to Apache Spark
19
Process Job Request
HDFS Path: /../../graph.csv
Job Type: pagerank
graph.csv
0 1
1 2
2 3
0 3
2 1 …
https://github.com/kbastani/spring-boot-graph-processing-example
Apply results back to Neo4j
20
Completed Job
HDFS Path: /../../results.csv
Job Type: pagerank
results.csv
0 .56
1 .42
2 .14
3 .25 …
https://github.com/kbastani/spring-boot-graph-processing-example
Graph processing platform
21
Algorithm TypeRelationship Type
https://github.com/kbastani/spring-boot-graph-processing-example
docker-compose.yml
! Demo
22
Building Microservices
23 https://github.com/kbastani/spring-boot-graph-processing-example
https://github.com/kbastani/spring-boot-graph-processing-example
Building Microservices
24
25 https://github.com/kbastani/spring-boot-graph-processing-example
Building Microservices
26
Creating Spring Data Neo4j Repositories
https://github.com/kbastani/spring-boot-graph-processing-example
https://github.com/kbastani/spring-boot-graph-processing-example
What our application needs
! Repositories
• User repository (to manage and import users)
• Follows repository (to manage and import following relationships)
• Custom Cypher queries mapped to repository methods
! Domain model
• User — (our node entity)
• Follows — (our relationship entity)
! REST API
• Expose the resources of the domain as a REST API
27
Creating Spring Data Neo4j Repositories
28 https://github.com/kbastani/spring-boot-graph-processing-example
Creating Spring Data Neo4j Repositories
29 https://github.com/kbastani/spring-boot-graph-processing-example
Creating Spring Data Neo4j Repositories
30 https://github.com/kbastani/spring-boot-graph-processing-example
Creating Spring Data Neo4j Repositories
31 https://github.com/kbastani/spring-boot-graph-processing-example
Creating Spring Data Neo4j Repositories
32 https://github.com/kbastani/spring-boot-graph-processing-example
Creating Spring Data Neo4j Repositories
33 https://github.com/kbastani/spring-boot-graph-processing-example
Exposing repository APIs using Spring Data REST
34 https://github.com/kbastani/spring-boot-graph-processing-example
Exposing repository APIs using Spring Data REST
35 https://github.com/kbastani/spring-boot-graph-processing-example
Exposing repository APIs using Spring Data REST
36 https://github.com/kbastani/spring-boot-graph-processing-example
Connecting to the Twitter API
37 https://github.com/kbastani/spring-boot-graph-processing-example
38 https://github.com/kbastani/spring-boot-graph-processing-example
Connecting to the Twitter API
Connecting to the Twitter API
39 https://github.com/kbastani/spring-boot-graph-processing-example
Connecting to the Twitter API
40 https://github.com/kbastani/spring-boot-graph-processing-example
We can override these properties as environment variables at runtime
41 https://github.com/kbastani/spring-boot-graph-processing-example
Connecting to the Twitter API
42 https://github.com/kbastani/spring-boot-graph-processing-example
Connecting to the Twitter API
Scheduling new PageRank jobs
43 https://github.com/kbastani/spring-boot-graph-processing-example
https://github.com/kbastani/spring-boot-graph-processing-example
Scheduling PageRank jobs from Neo4j
44
Scheduling new PageRank jobs
45 https://github.com/kbastani/spring-boot-graph-processing-example
Ranking Dashboard
46 https://github.com/kbastani/spring-boot-graph-processing-example
47 https://github.com/kbastani/spring-boot-graph-processing-example
Ranking dashboard
48 https://github.com/kbastani/spring-boot-graph-processing-example
49 https://github.com/kbastani/spring-boot-graph-processing-example
Ranking dashboard
Adding static web content
50 https://github.com/kbastani/spring-boot-graph-processing-example
Ranking dashboard
51 https://github.com/kbastani/spring-boot-graph-processing-example
https://github.com/kbastani/spring-boot-graph-processing-example
Ranking dashboard
52
https://github.com/kbastani/spring-boot-graph-processing-example
Add seed profiles
53
https://github.com/kbastani/spring-boot-graph-processing-example
Choose 3 seed profiles
54
Creating Spring Data Neo4j Repositories
55 https://github.com/kbastani/spring-boot-graph-processing-example
https://github.com/kbastani/spring-boot-graph-processing-example
Discover new users and update rankings
56
https://github.com/kbastani/spring-boot-graph-processing-example57
Learn More. Stay Connected.
! Twitter: @kennybastani
! Spring: spring.io/team/kbastani
! Blog: kennybastani.com
! GitHub: github.com/kbastani
Twitter: twitter.com/springcentral
YouTube: spring.io/video
LinkedIn: spring.io/linkedin
Google Plus: spring.io/gplus