Page 1
Unless otherwise indicated, these slides are © 2016 Pivotal Software, Inc. and licensed under aCreative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Using Docker, Neo4j, and Spring Cloud for Developing Microservices
Kenny Bastani, Spring Developer Advocate, Pivotal @kennybastani
Page 2
https://github.com/kbastani/spring-boot-graph-processing-example
Speaker Intro - Kenny Bastani
2
Page 3
Ranking Twitter ProfilesUsing PageRank
3 https://github.com/kbastani/spring-boot-graph-processing-example
Page 4
https://github.com/kbastani/spring-boot-graph-processing-example
PageRank algorithm
4
Page 5
https://github.com/kbastani/spring-boot-graph-processing-example
PageRank algorithm
5
Page 6
https://github.com/kbastani/spring-boot-graph-processing-example6
Page 7
https://github.com/kbastani/spring-boot-graph-processing-example7
Page 8
https://github.com/kbastani/spring-boot-graph-processing-example8
Page 9
https://github.com/kbastani/spring-boot-graph-processing-example
Tools we’ll be using
! Spring Boot
! Neo4j
! Apache Spark
! Docker
! RabbitMQ
9
Page 10
https://github.com/kbastani/spring-boot-graph-processing-example
Containerize all the things!
10
Page 11
https://github.com/kbastani/spring-boot-graph-processing-example11
Page 12
https://github.com/kbastani/spring-boot-graph-processing-example12
Page 13
https://github.com/kbastani/spring-boot-graph-processing-example13
Page 14
https://github.com/kbastani/spring-boot-graph-processing-example14
Page 15
Connecting Neo4j and Apache Spark…to submit PageRank jobs
15 https://github.com/kbastani/spring-boot-graph-processing-example
Page 16
https://github.com/kbastani/spring-boot-graph-processing-example
Request new Apache Spark job
16
Algorithm TypeRelationship Type
Page 17
https://github.com/kbastani/spring-boot-graph-processing-example
Export Neo4j graph to HDFS
17
New Job Request
HDFS Path: /../../graph.csv
Job Type: pagerank
Page 18
https://github.com/kbastani/spring-boot-graph-processing-example
Encoding a graph as an edge list
18
edge list
G B
H B
I B
K B
E B
F B
J B
D B
G E
H E
I E
K E …
Page 19
https://github.com/kbastani/spring-boot-graph-processing-example
Import edge list to Apache Spark
19
Process Job Request
HDFS Path: /../../graph.csv
Job Type: pagerank
graph.csv
0 1
1 2
2 3
0 3
2 1 …
Page 20
https://github.com/kbastani/spring-boot-graph-processing-example
Apply results back to Neo4j
20
Completed Job
HDFS Path: /../../results.csv
Job Type: pagerank
results.csv
0 .56
1 .42
2 .14
3 .25 …
Page 21
https://github.com/kbastani/spring-boot-graph-processing-example
Graph processing platform
21
Algorithm TypeRelationship Type
Page 22
https://github.com/kbastani/spring-boot-graph-processing-example
docker-compose.yml
! Demo
22
Page 23
Building Microservices
23 https://github.com/kbastani/spring-boot-graph-processing-example
Page 24
https://github.com/kbastani/spring-boot-graph-processing-example
Building Microservices
24
Page 25
25 https://github.com/kbastani/spring-boot-graph-processing-example
Building Microservices
Page 26
26
Creating Spring Data Neo4j Repositories
https://github.com/kbastani/spring-boot-graph-processing-example
Page 27
https://github.com/kbastani/spring-boot-graph-processing-example
What our application needs
! Repositories
• User repository (to manage and import users)
• Follows repository (to manage and import following relationships)
• Custom Cypher queries mapped to repository methods
! Domain model
• User — (our node entity)
• Follows — (our relationship entity)
! REST API
• Expose the resources of the domain as a REST API
27
Page 28
Creating Spring Data Neo4j Repositories
28 https://github.com/kbastani/spring-boot-graph-processing-example
Page 29
Creating Spring Data Neo4j Repositories
29 https://github.com/kbastani/spring-boot-graph-processing-example
Page 30
Creating Spring Data Neo4j Repositories
30 https://github.com/kbastani/spring-boot-graph-processing-example
Page 31
Creating Spring Data Neo4j Repositories
31 https://github.com/kbastani/spring-boot-graph-processing-example
Page 32
Creating Spring Data Neo4j Repositories
32 https://github.com/kbastani/spring-boot-graph-processing-example
Page 33
Creating Spring Data Neo4j Repositories
33 https://github.com/kbastani/spring-boot-graph-processing-example
Page 34
Exposing repository APIs using Spring Data REST
34 https://github.com/kbastani/spring-boot-graph-processing-example
Page 35
Exposing repository APIs using Spring Data REST
35 https://github.com/kbastani/spring-boot-graph-processing-example
Page 36
Exposing repository APIs using Spring Data REST
36 https://github.com/kbastani/spring-boot-graph-processing-example
Page 37
Connecting to the Twitter API
37 https://github.com/kbastani/spring-boot-graph-processing-example
Page 38
38 https://github.com/kbastani/spring-boot-graph-processing-example
Connecting to the Twitter API
Page 39
Connecting to the Twitter API
39 https://github.com/kbastani/spring-boot-graph-processing-example
Page 40
Connecting to the Twitter API
40 https://github.com/kbastani/spring-boot-graph-processing-example
We can override these properties as environment variables at runtime
Page 41
41 https://github.com/kbastani/spring-boot-graph-processing-example
Connecting to the Twitter API
Page 42
42 https://github.com/kbastani/spring-boot-graph-processing-example
Connecting to the Twitter API
Page 43
Scheduling new PageRank jobs
43 https://github.com/kbastani/spring-boot-graph-processing-example
Page 44
https://github.com/kbastani/spring-boot-graph-processing-example
Scheduling PageRank jobs from Neo4j
44
Page 45
Scheduling new PageRank jobs
45 https://github.com/kbastani/spring-boot-graph-processing-example
Page 46
Ranking Dashboard
46 https://github.com/kbastani/spring-boot-graph-processing-example
Page 47
47 https://github.com/kbastani/spring-boot-graph-processing-example
Page 48
Ranking dashboard
48 https://github.com/kbastani/spring-boot-graph-processing-example
Page 49
49 https://github.com/kbastani/spring-boot-graph-processing-example
Ranking dashboard
Page 50
Adding static web content
50 https://github.com/kbastani/spring-boot-graph-processing-example
Page 51
Ranking dashboard
51 https://github.com/kbastani/spring-boot-graph-processing-example
Page 52
https://github.com/kbastani/spring-boot-graph-processing-example
Ranking dashboard
52
Page 53
https://github.com/kbastani/spring-boot-graph-processing-example
Add seed profiles
53
Page 54
https://github.com/kbastani/spring-boot-graph-processing-example
Choose 3 seed profiles
54
Page 55
Creating Spring Data Neo4j Repositories
55 https://github.com/kbastani/spring-boot-graph-processing-example
Page 56
https://github.com/kbastani/spring-boot-graph-processing-example
Discover new users and update rankings
56
Page 57
https://github.com/kbastani/spring-boot-graph-processing-example57
Learn More. Stay Connected.
! Twitter: @kennybastani
! Spring: spring.io/team/kbastani
! Blog: kennybastani.com
! GitHub: github.com/kbastani
Twitter: twitter.com/springcentral
YouTube: spring.io/video
LinkedIn: spring.io/linkedin
Google Plus: spring.io/gplus