Meetup Paris 19 Octobre, 2016 1
Meetup Paris
19 Octobre, 2016
1
GraphConnect San Francisco:April 26 in London
Tonight:
PARIS AFTERGLOW
Emil EifremCEO & Co-founder, Neo Technology
LOOKS_AT
Returns
Purchase History
Price-range
Home delivery
Inventory
Express goods
Complaints
reviews
TweetsEmails
Category
Promotions
Bundling
Location
KITCHEN AID SERIESHow to remain
competitive?The Connected Enterprise
Emil’ Product UpdateCEO & Co-founder, Neo Technology
Neo4j 3.1: now in BETA The Graph Foundation for the Enterprise
Octobre 2016
12
Neo4j 3.0 Recap APRIL 2016 RELEASE Adoption-Focused Release Delivering New Graph Capabilities
DevelopersDevelopers
Develop applications
faster and easier
ArchitectsArchitects
Design bigger and
faster applications
AdministratorsAdministrators
Deploy Neo4janywhere easily
Neo4j 3.0 enables and accelerates large-scale graph initiativesNeo4j 3.0 enables and accelerates large-scale graph initiatives
Giant graphs, fast performance
Easy full-stack development
Cloud, containerand on-premise
14
Introducing Neo4j 3.1New Security and Clustering Architecture
Build and deploy graph applications acrossan entire enterprise• Compliance with internal and external
enterprise Information Security needs• Robust and flexible new clustering
architecture for diverse operational scenarios and application needs
A foundation that enables mainstream enterprise solutions on-premises and in the cloud
ENTERPRISE GRAPH FOUNDATIONENTERPRISE GRAPH FOUNDATION
Operational, Analytic, and Transactional UsesOperational, Analytic, and Transactional Uses
SecuritySecurity ClusteringClustering OperabilityOperability
EnterpriseGraph
Applications
EnterpriseGraph
Applications
15
The Graph Foundation for the Enterprise
Neo4j 3.1 Highlights
SecurityFoundat
ion
Database Kernel and Operations Advances
16
IBM Power8
CAPI Flash Support
SchemaViewer
CausalClusteringState-of-the-
ArtCluster
Architecture
Consistency with Graphs: A Different ChallengeKeeping Your Graph Intact is Essential for Graph Operations
Graphs with Causal Consistency
Graphs with Eventual Consistency
17
The graph transaction moves together as one ACID transaction
with built-in safetyGuarantees Graph Consistency
Without ACID transactions, the view of the graph & the data
itself becomes corruptedNot Good Enough for Graphs
Non-Native Graph DB
Raft-based architecture • Continuously available• Consensus commits• Third-generation cluster architecture
Cluster-aware stack• Seamless integration among drivers,
Bolt protocol and cluster• Eliminates need for external load balancer• Stateful, cluster-aware sessions with
encrypted connections
Streamlined development• Relieves developers from complex infrastructure
concerns• Faster and easier to develop distributed graph
applications
Neo4j Causal Clustering ArchitectureResilient, Modern, Fault-Tolerant. Guarantees Graph Safety.
18 ENTERPRISE EDITION
How Causal Clustering Works
19
Replica Servers
Query, View
Core ServersSynced Cluster
ReadReplicaRead
Replica
Read-WriteRead-Write
ReadReplicaRead
Replica
Read-WriteRead-WriteRead
Replica
ReadReplic
a
Read Replica
Reportingand Analysis
Read Replica
Reportingand Analysis
Gra
ph
App
Gra
ph
App
Driver
Driver B O L TB O L T
W ri teW ri teR e a dR e a d
ReadReplic
a
ReadReplic
a
ReadReplic
a
ReadReplic
a
ReadReplic
a
ReadReplic
a
Built-in load balancing• Spreads reads to core and replica
servers• Spreads writes across core servers
Causal consistency• Always-consistent view of data at any
scale• Stronger than eventual consistency• Supports varying app SLAs• Best model for graph transactions
Large heterogeneous clusters• 1000+ instance clusters• No dependence on master
avoids bottleneck• Mix and match instance types
App servers, reporting servers, IoT devices…
ENTERPRISE EDITION
R E P L I C A Q U E R I E SR E P L I C A Q U E R I E S C O R E Q U E R I E SC O R E Q U E R I E S
Causal Clustering Architecture Optimizes forCost-Consistency at Query Time
ReadAny
ReadAny
20
ReadYour Own
Writes
ReadYour Own
Writes
ReadAny
ReadAny
ReadYour Own
Writes
ReadYour Own
Writes
QUORATE
Q U E R Y C O S T Q U E R Y C O S T
ENTERPRISE EDITION
How Causally Consistent Reads Work
App ServerApp Server
DriverDriver
3: Review Profile
4: Create an order
AsyncReplicatio
n
RaftReplicatio
n
1: Read Product Catalog
Core Server
Core Server
Replica Server
App ServerApp Server
DriverDriver
App ServerApp Server
DriverDriver
ENTERPRISE EDITION
2. Create Account
5: Review orders
How it Works:• Application chooses a consistency level “Read Any” vs “Read your own writes”• Cluster chooses appropriate membersDefault optimizes for scalability (i.e. read replica server for reads)
Causal Clustering Enables:• Application-driven SLAs• Optimizing for freshness vs. cost• Tunability within an application
On an application & session basis
1: Read any replica | 2: Write [Tx 101] | 3: RYOW*[Tx 101] | 4: Write [Tx 102] | 5: RYOW [Tx 102]
Consistency with Causal Clustering
22
Expected Consistency BehaviorNoSQL
Eventual Consistency
Neo4j Causal
Consistency
Every single server is eventually updated View of related data is always consistent
Users reading and re-reading data always see the same data Unless there have been intervening updates by others
Users writing and updating data always see the latest data Unless there have been intervening updates by others
ENTERPRISE EDITION
Satisfy enterprise admin and database security requirements • Flexible authentication options ActiveDirectory/LDAP or Native users• Role-based Authorization• List and kill running queries• Access controls for User-Defined
Procedures Enables subgraph access control
• Query logging and Security event logging
Passes through originating end user• Extendable Auth plugin Architecture
Official Kerberos support in near future
23
Neo4j Security FoundationEnterprise-Class Security and Control
P R E D E F I N E D R O L E S
PrivilegesReade
rPublish
erArchite
ctAdmin
Change own password • • • •
Read data • • • •
View own details • • • •
Terminate own query • • • •
Write/update/delete data • • •
Manage index/constraints • •
Terminate others’ queries •
ENTERPRISE EDITION
Choose from several authentication and authorization methods• Built-in native users repository
Enabled by default• LDAP connector to Active
Directory or openLDAP
• Custom auth provider plugins
24
Neo4j Security FoundationEnterprise-Class Security and Control
CustomPlugin
CustomPlugin
CustomPlugin
CustomPlugin
Active DirectoryActive Directory openLDAPopenLDAP
LDAP connecto
r
LDAP connecto
r
Auth PluginExtension Module
Auth PluginExtension Module
Built-inNative Users
Built-inNative Users
Neo4jNeo4j
ENTERPRISE EDITION
Neo4j Security FoundationManage Running Queries with Admin UI in Neo4j Browser
25
List all running queries with :qs (soon to be :queries)
List all running queries with :qs (soon to be :queries)
List query string with parameters and transaction
metadata
List query string with parameters and transaction
metadata
Users can only see and terminate their own
queries
Users can only see and terminate their own
queries
Terminate selected
query
Terminate selected
query
Admins can view and terminate all running
queries across the cluster
Admins can view and terminate all running
queries across the cluster
Track elapsed time for queriesTrack elapsed
time for queries
ENTERPRISE EDITION
Neo4j Security FoundationAdminister Users & Roles with Admin UI in Neo4j Browser
26
Add new user with:server user addAdd new user with:server user add
List all native users with :server
user list command
List all native users with :server
user list command
Assign roles when
creating users
Assign roles when
creating users
Suspend or activate
users
Suspend or activate
users
DeleteusersDeleteusers
Add or remove assigned roles to
existing users
Add or remove assigned roles to
existing users
Enforce user password change
at first login
Enforce user password change
at first login
ENTERPRISE EDITION
Neo4j 3.1Database Kernel and Operational ImprovementsExecution guard• Terminates long-running end-user queries• Lets admins specify max query duration per
instance
Node and relationship ID allocation (ENTERPRISE EDITION)
• Reallocates space from deleted records while the database cluster is online
27
(a)-->(b)
New command-line interface • Based on Bolt• Enables ad-hoc querying and scripting• Extends Neo4j Browser development to command
line• Can run remotely with encrypted communication by
default
Neo4j 3.1Schema Viewer
• Neo4j Schema Viewer displays the graph model at the current point in time
• Uses exact database statistics for instant responsiveness
• Helps developers, admins and users understand and communicate the database contents
28
Built into the Neo4j Browser tool
Neo4j 3.1User-Defined Functions
• Allows users to create their own functions and use them with Cypher• Useful for expressing common computations, rules, conversions,
predicates• Extends the Neo4j 3.0 Stored Procedure mechanism
Creating a UUID with a Stored Procedure vs. with a Function
Neo4j 3.1: The Graph Foundation for the Enterprise
Questions and Answers
30
Emil’ OpenCypher UpdateCEO & Co-founder, Neo Technology
SQL for Graphs
Users Love Cypher
Why openCypher?
CompanyVendor Independence
IndividualCypher as a transferable skill set
openCypher Supporters
Q4 2015Announcement
2016 2017 2018
Tooling Enablement DBMS Enablement
Process Language Evolution
Open StandardProposed Open Standard
Tooling Providers DBMS Early Adopters Everyone
openCypher Roadmap
DataInterchange
Graph Storage & Processing
RealTime/Operational (“OLTP”)
Analytic/Compute(“OLAP”)
+Cypher
Emil’ Startup ProgramUpdateCEO & Co-founder, Neo Technology
Anybody with an idea and a purpose,
can now use a technology that is powerful enough to change an industry
Tiers, Neo4j Startup Program
≤ 20 employeesFree Neo4j Enterprise Subscription(Unsupported)
≤ 50 employeesStartup Pricing (Includes Support)
349
From selffunded to funded by the top VCs such as a16z, Accel and Sequoia
Facts, Neo4j Startup Program
349 startups, 32 Countries
Security, Machine Learning, IoT, eCommerce, Social Networks…
Numbers
Field of Innovation
Funding
Prominent Neo4j Startup Program Alumnis
Neo4j Startup Program
Technology SupportMentorship MarketingThrough Neo4jChannels
EcosystemSupport
Talent Acquisition
Backup slides:Neo4j’s Next Generation Cluster Architecture
Ultra Large Clusters Raft & Consensus Commit
Full Stack Cluster Awareness Causal Consistency
The Next Generation Cluster Architecture
ReadWriteReadWrite
ReadWriteReadWrite
ReadWriteReadWrite
Read ReplicaRead Replica
Read ReplicaRead Replica
Read ReplicaRead Replica
Read ReplicaRead Replica
Read ReplicaRead Replica
Read ReplicaRead Replica
Read ReplicaRead Replica
Read ReplicaRead Replica
Read ReplicaRead Replica
Read ReplicaRead Replica
Read ReplicaRead Replica
Read ReplicaRead Replica
Replica ServersQuery, View
Core ServersSynced ClusterUltra Large
Clusters
1
ReadWriteReadWrite
ReadWriteReadWrite
ReadWriteReadWrite
Read ReplicaRead Replica
Read ReplicaRead Replica
Read ReplicaRead Replica
Read ReplicaRead Replica
Read ReplicaRead Replica
Read ReplicaRead Replica
Read ReplicaRead Replica
Read ReplicaRead Replica
Read ReplicaRead Replica
Read ReplicaRead Replica
Read ReplicaRead Replica
Read ReplicaRead Replica
Replica ServersQuery, View
Core ServersSynced ClusterUltra Large
Clusters
1
Read ReplicaRead Replica
Read ReplicaRead Replica
Read ReplicaRead Replica
Read ReplicaRead Replica
Read ReplicaRead Replica
Read ReplicaRead Replica
Read ReplicaRead Replica
Read ReplicaRead Replica
Read ReplicaRead Replica
Read ReplicaRead Replica
Read ReplicaRead Replica
Read ReplicaRead Replica
Read ReplicaRead ReplicaRead
ReplicaRead Replica
Read ReplicaRead Replica
Read ReplicaRead Replica
Read ReplicaRead Replica
Read ReplicaRead Replica
Read ReplicaRead Replica
Read ReplicaRead Replica
Read ReplicaRead Replica
Read ReplicaRead Replica
Read ReplicaRead Replica
Read ReplicaRead Replica
Read ReplicaRead Replica
Read ReplicaRead Replica
Read ReplicaRead Replica
Read ReplicaRead ReplicaRead
ReplicaRead Replica
Read ReplicaRead Replica
Read ReplicaRead Replica
Read ReplicaRead Replica
Read ReplicaRead Replica
Read ReplicaRead Replica
Read ReplicaRead Replica
Read ReplicaRead Replica
Read ReplicaRead Replica
Read ReplicaRead ReplicaRead
ReplicaRead Replica
Raft and Consensus Commits
2
ReadWrite
ReadWrite
ReadWrite
ReadWrite
ReadWrite
Write Request
WriteAcknowledgement
Core ServersSynced Cluster
Write
Read Replica
Read Replica
Read
Read Replica
Replica ServersQuery, View
Read Replica
Application
Driver
Read Replica
Read Replica
ReadWrite
ReadWrite
Full Stack Cluster Awareness
3
Causal Consistency
4
EventualWrite Read
<Sleep = 100 ms>
propagation
Application Application
ReadWrite
propagation
Application Application
Application
Read AnyCore
Write Write Read-Your-Own-Write
Raft ReplicationAsyncReplication
Core Server Core Server Read Replica
Driver
Application
Driver
Application
Driver
Causal Consistency
4
New Security Foundation
Federated Authenticationwith LDAP and AD(Next up: Kerberos!)
RoleBased Authorization
New Security Foundation
Fine-grained access control, maybe show a user calling a stored procedure where there is a larger part of the graph below the stored proc, and above the stored proc what comes back to the user is a subset of the graph (i.e. fewer relationships, fewer nodes, and fewer properties on the returned nodes)
FineGrained Access Control(ProcedureBased)
The New Security Foundation EnablesSarbanesOxley, HIPAA, PCIDSS, et al