Page 1
Unless otherwise indicated, these slides are © 2013-2014 Pivotal Software, Inc. and licensed under a
Creative Commons Attribution-NonCommercial l icense: http://creativecommons.org/licenses/by-nc/3.0/
Unless otherwise indicated, these slides are © 2013-2014 Pivotal Software, Inc. and licensed under a
Creative Commons Attribution-NonCommercial l icense: http://creativecommons.org/licenses/by-nc/3.0/
Day 2 Keynote: Platform Reinvented
Juergen Hoeller, Mark Fisher, Dave Syer, Jon Brisbin
1
Page 2
Unless otherwise indicated, these slides are © 2013-2014 Pivotal Software, Inc. and licensed under a
Creative Commons Attribution-NonCommercial l icense: http://creativecommons.org/licenses/by-nc/3.0/
Spring IO Platform
2
Jobs, Steps,
Readers, Writers
Ingestion, Export,
Orchestration, Hadoop
Controllers, REST,
WebSocket
Channels, Adapters,
Filters, Transformers
WEB INTEGRATION BATCH BIG DATA
SPRING CORE
FRAMEWORK SECURITY GROOVY REACTOR
DATA
RELATIONAL
DATA ACCESS
NON-RELATIONAL
DATA ACCESS
BOOT
Bootable, Minimal, Ops-Ready
GRAILS Full-stack,
Web
XD Stream, Taps,
Jobs
IO EXECUTION
IO FOUNDATION
IO COORDINATION SPRING CLOUD
Page 3
Unless otherwise indicated, these slides are © 2013-2014 Pivotal Software, Inc. and licensed under a
Creative Commons Attribution-NonCommercial l icense: http://creativecommons.org/licenses/by-nc/3.0/
Spring IO Core: Spring Framework 4.1 GA
• Annotated JMS listener methods
• Comprehensive support for JCache (JSR-107) annotations
• Flexible resolution and transformation of static web resources
• MVC views: declarative resolution + Groovy markup templates
• Performance: SpEL compiler mode + concurrency fine-tuning
3
Page 4
Unless otherwise indicated, these slides are © 2013-2014 Pivotal Software, Inc. and licensed under a
Creative Commons Attribution-NonCommercial l icense: http://creativecommons.org/licenses/by-nc/3.0/
Spring 4.1: Annotated JMS listener methods
4
@Component
public class MyService {
@JmsListener(destination = "myQueue")
public void processOrder(Order order, @Header("order_type") String type) {
...
}
}
Page 5
Unless otherwise indicated, these slides are © 2013-2014 Pivotal Software, Inc. and licensed under a
Creative Commons Attribution-NonCommercial l icense: http://creativecommons.org/licenses/by-nc/3.0/
Spring 4.1: Support for JCache annotations
5
@Component
public class MyService {
@CacheResult(cacheName = "book")
public Book findBook(@CacheKey ISBN isbn, boolean checkWarehouse) {
...
}
}
Page 6
Unless otherwise indicated, these slides are © 2013-2014 Pivotal Software, Inc. and licensed under a
Creative Commons Attribution-NonCommercial l icense: http://creativecommons.org/licenses/by-nc/3.0/
Spring 4.1: Flexible web resource handling
• Flexible resource resolution chain:path resolution, caching,
compression, versioning
• Flexible resource transformation chain:CSS links, caching,
HTML5 AppCache manifests
• Talk: “Resource Handling in Spring MVC 4.1”
by Brian Clozel and Rossen Stoyanchev - Wednesday 8:30
• Showcase: https://github.com/bclozel/spring-resource-handling
6
Page 7
Unless otherwise indicated, these slides are © 2013-2014 Pivotal Software, Inc. and licensed under a
Creative Commons Attribution-NonCommercial l icense: http://creativecommons.org/licenses/by-nc/3.0/
Spring 4.1: Declarative MVC view resolution
7
@Configuration
@EnableWebMvc
public class WebConfig extends WebMvcConfigurerAdapter {
public void configureViewResolvers(ViewResolverRegistry registry) {
registry.enableContentNegotiation(new MappingJackson2JsonView());
registry.freeMarker().cache(false);
}
}
Page 8
Unless otherwise indicated, these slides are © 2013-2014 Pivotal Software, Inc. and licensed under a
Creative Commons Attribution-NonCommercial l icense: http://creativecommons.org/licenses/by-nc/3.0/
Spring 4.1: Many further improvements...
• Java 8's Optional for injection points and MVC handler
parameters
• Annotated @Lookup methods with support for method arguments
• Direct field binding support with auto-growing for nested fields
• WebSocket scope, SockJS client support, WebSocket stats
• Declarative SQL scripts and programmatic transactions for tests
8
Page 9
Unless otherwise indicated, these slides are © 2013-2014 Pivotal Software, Inc. and licensed under a
Creative Commons Attribution-NonCommercial l icense: http://creativecommons.org/licenses/by-nc/3.0/
Spring XD – Big Data, Little Time
9
Page 10
Unless otherwise indicated, these slides are © 2013-2014 Pivotal Software, Inc. and licensed under a
Creative Commons Attribution-NonCommercial l icense: http://creativecommons.org/licenses/by-nc/3.0/
Spring XD: Unified Platform for Big Data
10
Spring XD Runtime
BIDIRECTIONAL
Compute
HDFS
RDBMS
NoSQL
R, SAS
Streams Jobs
ingest workflow
export
taps
Predictive Modelling
>_
Redis
Page 11
Unless otherwise indicated, these slides are © 2013-2014 Pivotal Software, Inc. and licensed under a
Creative Commons Attribution-NonCommercial l icense: http://creativecommons.org/licenses/by-nc/3.0/
Spring XD: Quality of Service
• High Availability
• Cluster Management built upon ZooKeeper
• Leader Election for Admin Nodes
• Module Redeployment across Container Nodes
• Customizable Deployment
• Module Count and Criteria for placement
• Data Partitioning by Key
• Direct Binding for Co-located Modules
11
Page 12
Unless otherwise indicated, these slides are © 2013-2014 Pivotal Software, Inc. and licensed under a
Creative Commons Attribution-NonCommercial l icense: http://creativecommons.org/licenses/by-nc/3.0/
Demo Domain: Smart Grid Data
12
http://www.cse.iitb.ac.in/debs2014 *
Inspired by ACM DEBS 2014 Grand Challenge* Demonstrate the applicability of event-based systems to provide scalable, real-time analytics over high volume sensor data to compute load forecasting.
Field Name Description
ID Unique ID of the measurement
Timestamp Number of seconds since epoch
Load Load in watts
House ID The house where the plug is located
Household ID The household inside the house
Plug ID The unique ID of the smart plug
Page 13
Unless otherwise indicated, these slides are © 2013-2014 Pivotal Software, Inc. and licensed under a
Creative Commons Attribution-NonCommercial l icense: http://creativecommons.org/licenses/by-nc/3.0/
Demo Objectives
• Ingest household energy data to HDFS via HTTP
• Real-time aggregation and visualization
• Real-time model evaluation to predict energy demand
13
ingest: http | hdfs
count: tap:stream:ingest > aggregate-counter
predict: tap:stream:ingest > pmml | aggregate-counter
partition on house
Page 14
Unless otherwise indicated, these slides are © 2013-2014 Pivotal Software, Inc. and licensed under a
Creative Commons Attribution-NonCommercial l icense: http://creativecommons.org/licenses/by-nc/3.0/
XD Admin
Zoo Keeper Zoo Keeper Zoo Keeper
Load
Balancer Generator
XD Container
HTTP
XD Container
HTTP
XD Container
HDFS
XD Container
HDFS
XD Container
HDFS Hadoop
Hadoop
Hadoop
Rabbit Redis
Demo Topology
Page 15
Unless otherwise indicated, these slides are © 2013-2014 Pivotal Software, Inc. and licensed under a
Creative Commons Attribution-NonCommercial l icense: http://creativecommons.org/licenses/by-nc/3.0/
Demo:
Unless otherwise indicated, these slides are © 2013-2014 Pivotal Software, Inc. and licensed under a
Creative Commons Attribution-NonCommercial l icense: http://creativecommons.org/licenses/by-nc/3.0/
Spring XD
15
Page 16
Unless otherwise indicated, these slides are © 2013-2014 Pivotal Software, Inc. and licensed under a
Creative Commons Attribution-NonCommercial l icense: http://creativecommons.org/licenses/by-nc/3.0/
Looking Ahead
• Programming Model
• Further Unification of the Batch and Stream Models
• Reactive Streams
• Developer Experience
• Support for Java Config and Spring Integration DSLs
• Spring Boot-based Module Deployment and Packaging
• Deployment Targets
• Cloud Foundry Service
• Docker
• Mesos
16
Page 17
Unless otherwise indicated, these slides are © 2013-2014 Pivotal Software, Inc. and licensed under a
Creative Commons Attribution-NonCommercial l icense: http://creativecommons.org/licenses/by-nc/3.0/
Spring XD Sessions
17
TUE 2:30 PM Develop powerful Big Data Applications easily with Spring XD
TUE 4:30 PM Spring XD - A Guided Tour
WED 10:30 AM Building a Recommendation Engine with Spring and Hadoop
WED 2:30 PM IoT Realized - The Connected Car
THU 10:30 AM Spring XD for Real-time Hadoop Workload Analysis
THU 12:45 PM Implementing the Lambda Architecture with Spring XD
Page 18
Unless otherwise indicated, these slides are © 2013-2014 Pivotal Software, Inc. and licensed under a
Creative Commons Attribution-NonCommercial l icense: http://creativecommons.org/licenses/by-nc/3.0/
Spring IO Platform
18
Grails
Spring IO Platform Distribution
Spring XD
Spring Foundation
Apps
3rd Party Dependencies
Spring Boot
Spring Cloud
Page 19
Unless otherwise indicated, these slides are © 2013-2014 Pivotal Software, Inc. and licensed under a
Creative Commons Attribution-NonCommercial l icense: http://creativecommons.org/licenses/by-nc/3.0/
No Man (Microservice) is an Island
It's excellent to be able to implement a microservice really easily
(Spring Boot), but building a system that way surfaces
"non-functional" requirements that you otherwise didn't have.
There are laws of physics that make some problems unsolvable
(consistency, latency), but brittleness and manageability can be
addressed with generic, boiler plate patterns.
19
Page 20
Unless otherwise indicated, these slides are © 2013-2014 Pivotal Software, Inc. and licensed under a
Creative Commons Attribution-NonCommercial l icense: http://creativecommons.org/licenses/by-nc/3.0/
Emergent features of micro-service systems
Coordination of distributed systems leads to boiler plate patterns
• Distributed/versioned configuration
• Service registration and discovery
• Routing
• Service-to-service calls
• Load balancing
• Circuit Breaker
• Asynchronous
• Distributed messaging
20
Page 21
Unless otherwise indicated, these slides are © 2013-2014 Pivotal Software, Inc. and licensed under a
Creative Commons Attribution-NonCommercial l icense: http://creativecommons.org/licenses/by-nc/3.0/
Spring Cloud Components
21
Client Bindings
Service
Discovery
Config
Server
Circuit
Breakers
Edge
Service
Intelligent
Router
Leader
Election
One-time
Tokens
Global
Locks
Cluster
State
Event Bus
Page 22
Unless otherwise indicated, these slides are © 2013-2014 Pivotal Software, Inc. and licensed under a
Creative Commons Attribution-NonCommercial l icense: http://creativecommons.org/licenses/by-nc/3.0/
Demo:
Unless otherwise indicated, these slides are © 2013-2014 Pivotal Software, Inc. and licensed under a
Creative Commons Attribution-NonCommercial l icense: http://creativecommons.org/licenses/by-nc/3.0/
Spring Cloud
Components
22
Page 23
Unless otherwise indicated, these slides are © 2013-2014 Pivotal Software, Inc. and licensed under a
Creative Commons Attribution-NonCommercial l icense: http://creativecommons.org/licenses/by-nc/3.0/
Spring Cloud Components
• Common features of distributed systems developed by users as
Spring Boot apps
• Highly configurable and customizable
• Always do something sensible out of the box
• Works nicely as standalone apps on developer box
• Easy migration to Cloud and PCF
23
Page 24
Unless otherwise indicated, these slides are © 2013-2014 Pivotal Software, Inc. and licensed under a
Creative Commons Attribution-NonCommercial l icense: http://creativecommons.org/licenses/by-nc/3.0/
Spring Cloud Sessions
24
TUE 4:30 PM Security for Microservices with Spring and OAuth2
WED 12:45 PM Simplify Cloud Applications using Spring Cloud
WED 12:45 PM Spring Cloud, Spring Boot and Netflix OSS
WED 2:30 PM Developing Microservices for PaaS with Spring and Cloud Foundry
Page 25
Unless otherwise indicated, these slides are © 2013-2014 Pivotal Software, Inc. and licensed under a
Creative Commons Attribution-NonCommercial l icense: http://creativecommons.org/licenses/by-nc/3.0/
Demo:
Unless otherwise indicated, these slides are © 2013-2014 Pivotal Software, Inc. and licensed under a
Creative Commons Attribution-NonCommercial l icense: http://creativecommons.org/licenses/by-nc/3.0/
Reactor
25
Page 26
Unless otherwise indicated, these slides are © 2013-2014 Pivotal Software, Inc. and licensed under a
Creative Commons Attribution-NonCommercial l icense: http://creativecommons.org/licenses/by-nc/3.0/
Spring IO Core: Reactor
• Microservices proliferate in modern architectures
• More Runtime Resources required
• More Runtime Resources means more $$$
• Microservices have acute need for efficiency
• Runtime Resources aren’t free/cheap
• Microservice Architecture introduces boundaries
• Boundaries introduce latency
26
Page 27
Unless otherwise indicated, these slides are © 2013-2014 Pivotal Software, Inc. and licensed under a
Creative Commons Attribution-NonCommercial l icense: http://creativecommons.org/licenses/by-nc/3.0/
Spring IO Core: Reactor
• What is Reactive Programming?
• Event-driven
• Modular
• Scalable
• Highly-available
• Non-blocking/async
o Blocking IO (BIO) is expensive
o Non-blocking IO (NIO) is more efficient
o NIO means async patterns, which can be hard
27
Page 28
Unless otherwise indicated, these slides are © 2013-2014 Pivotal Software, Inc. and licensed under a
Creative Commons Attribution-NonCommercial l icense: http://creativecommons.org/licenses/by-nc/3.0/
Spring IO Core: Reactor
Microservices are natural fit for Reactive Architecture
28
Page 29
Unless otherwise indicated, these slides are © 2013-2014 Pivotal Software, Inc. and licensed under a
Creative Commons Attribution-NonCommercial l icense: http://creativecommons.org/licenses/by-nc/3.0/ 29
Reactive Architecture
• Reactive is usually Asynchronous
• Asynchronous !== Concurrent
• Non-blocking versus Blocking (usually IO)
• Provides Isolation
• Resource usage
• Failure
• Events are replayable
• Reactive can be complex
• Too low-level for most end users
Page 30
Unless otherwise indicated, these slides are © 2013-2014 Pivotal Software, Inc. and licensed under a
Creative Commons Attribution-NonCommercial l icense: http://creativecommons.org/licenses/by-nc/3.0/
Reactor
Spring IO: Components → Reactor
30
Task Queue
Micro
Service
Micro
Service
Micro
Service
Micro
Service
Event
Event Event
Page 31
Unless otherwise indicated, these slides are © 2013-2014 Pivotal Software, Inc. and licensed under a
Creative Commons Attribution-NonCommercial l icense: http://creativecommons.org/licenses/by-nc/3.0/
Reactive Stream Contributors
31
ccv
Doug Lea – SUNY Oswego
Page 32
Unless otherwise indicated, these slides are © 2013-2014 Pivotal Software, Inc. and licensed under a
Creative Commons Attribution-NonCommercial l icense: http://creativecommons.org/licenses/by-nc/3.0/
Spring IO Core: Reactor
What is Reactive Streams?
32
“...provide a standard for asynchronous stream processing with
non-blocking backpressure.”
github.com/reactive-streams/reactive-streams
Available as a library
org.reactivestreams:reactive-streams:0.4.0.M2
Page 33
Unless otherwise indicated, these slides are © 2013-2014 Pivotal Software, Inc. and licensed under a
Creative Commons Attribution-NonCommercial l icense: http://creativecommons.org/licenses/by-nc/3.0/
Reactive Sessions
33
WED 12:45 PM Asynchronous design with Spring and RTI: 1M events per second
THU 8:30 AM Building Reactive applications with Reactor and the Reactive Streams
standard