Top Banner
Reactive Stream Processing with WildFly Swarm and Apache Kafka Ken Finnigan Red Hat
42

Reactive Stream Processing - RainFocus...Reactive Stream Processing with WildFly Swarm and Apache Kafka Ken Finnigan Red Hat Who is Ken Finnigan? • Project lead for WildFly Swarm

May 20, 2020

Download

Documents

dariahiddleston
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: Reactive Stream Processing - RainFocus...Reactive Stream Processing with WildFly Swarm and Apache Kafka Ken Finnigan Red Hat Who is Ken Finnigan? • Project lead for WildFly Swarm

Reactive Stream Processingwith WildFly Swarmand Apache Kafka

Ken Finnigan Red Hat

Page 2: Reactive Stream Processing - RainFocus...Reactive Stream Processing with WildFly Swarm and Apache Kafka Ken Finnigan Red Hat Who is Ken Finnigan? • Project lead for WildFly Swarm

Who is Ken Finnigan?• Project lead for WildFly Swarm

• Contributor to Eclipse MicroProfile

• Author of several books

• Twitter: @kenfinnigan

Page 3: Reactive Stream Processing - RainFocus...Reactive Stream Processing with WildFly Swarm and Apache Kafka Ken Finnigan Red Hat Who is Ken Finnigan? • Project lead for WildFly Swarm
Page 4: Reactive Stream Processing - RainFocus...Reactive Stream Processing with WildFly Swarm and Apache Kafka Ken Finnigan Red Hat Who is Ken Finnigan? • Project lead for WildFly Swarm
Page 5: Reactive Stream Processing - RainFocus...Reactive Stream Processing with WildFly Swarm and Apache Kafka Ken Finnigan Red Hat Who is Ken Finnigan? • Project lead for WildFly Swarm
Page 6: Reactive Stream Processing - RainFocus...Reactive Stream Processing with WildFly Swarm and Apache Kafka Ken Finnigan Red Hat Who is Ken Finnigan? • Project lead for WildFly Swarm
Page 7: Reactive Stream Processing - RainFocus...Reactive Stream Processing with WildFly Swarm and Apache Kafka Ken Finnigan Red Hat Who is Ken Finnigan? • Project lead for WildFly Swarm
Page 8: Reactive Stream Processing - RainFocus...Reactive Stream Processing with WildFly Swarm and Apache Kafka Ken Finnigan Red Hat Who is Ken Finnigan? • Project lead for WildFly Swarm

<plugin><groupId>org.wildfly.swarm</groupId><artifactId>wildfly-swarm-plugin</artifactId><version>${version.wildfly-swarm}</version><executions><execution><id>package</id><goals><goal>package</goal></goals></execution></executions></plugin>

Page 9: Reactive Stream Processing - RainFocus...Reactive Stream Processing with WildFly Swarm and Apache Kafka Ken Finnigan Red Hat Who is Ken Finnigan? • Project lead for WildFly Swarm

<dependency><groupId>org.wildfly.swarm</groupId><artifactId>jaxrs</artifactId></plugin>

Page 10: Reactive Stream Processing - RainFocus...Reactive Stream Processing with WildFly Swarm and Apache Kafka Ken Finnigan Red Hat Who is Ken Finnigan? • Project lead for WildFly Swarm
Page 11: Reactive Stream Processing - RainFocus...Reactive Stream Processing with WildFly Swarm and Apache Kafka Ken Finnigan Red Hat Who is Ken Finnigan? • Project lead for WildFly Swarm

WildFly Swarm figures out what bits

you need

Page 12: Reactive Stream Processing - RainFocus...Reactive Stream Processing with WildFly Swarm and Apache Kafka Ken Finnigan Red Hat Who is Ken Finnigan? • Project lead for WildFly Swarm
Page 13: Reactive Stream Processing - RainFocus...Reactive Stream Processing with WildFly Swarm and Apache Kafka Ken Finnigan Red Hat Who is Ken Finnigan? • Project lead for WildFly Swarm

<dependency><groupId>org.wildfly.swarm</groupId><artifactId>bom</artifactId><version>${version.wildfly-swarm}</version><type>pom<type><scope>import</scope></plugin>

Page 14: Reactive Stream Processing - RainFocus...Reactive Stream Processing with WildFly Swarm and Apache Kafka Ken Finnigan Red Hat Who is Ken Finnigan? • Project lead for WildFly Swarm

<dependency><groupId>org.wildfly.swarm</groupId><artifactId>jaxrs</artifactId></plugin>

Page 15: Reactive Stream Processing - RainFocus...Reactive Stream Processing with WildFly Swarm and Apache Kafka Ken Finnigan Red Hat Who is Ken Finnigan? • Project lead for WildFly Swarm

$mvnwildfly-swarm:run

Run it!

Page 16: Reactive Stream Processing - RainFocus...Reactive Stream Processing with WildFly Swarm and Apache Kafka Ken Finnigan Red Hat Who is Ken Finnigan? • Project lead for WildFly Swarm

$mvncleanpackage

Package it!

Page 17: Reactive Stream Processing - RainFocus...Reactive Stream Processing with WildFly Swarm and Apache Kafka Ken Finnigan Red Hat Who is Ken Finnigan? • Project lead for WildFly Swarm

$java-jarmyapp-swarm.jar

Run it!

Page 18: Reactive Stream Processing - RainFocus...Reactive Stream Processing with WildFly Swarm and Apache Kafka Ken Finnigan Red Hat Who is Ken Finnigan? • Project lead for WildFly Swarm
Page 19: Reactive Stream Processing - RainFocus...Reactive Stream Processing with WildFly Swarm and Apache Kafka Ken Finnigan Red Hat Who is Ken Finnigan? • Project lead for WildFly Swarm

swarm:datasources:data-sources:MyDS:driver-name:h2connection-url:jdbc:h2:mem:test;DB_CLOSE_DELAY=-1;DB_CLOSE_ON_EXIT=FALSEuser-name:sapassword:sa

project-defaults.yaml

Page 20: Reactive Stream Processing - RainFocus...Reactive Stream Processing with WildFly Swarm and Apache Kafka Ken Finnigan Red Hat Who is Ken Finnigan? • Project lead for WildFly Swarm
Page 21: Reactive Stream Processing - RainFocus...Reactive Stream Processing with WildFly Swarm and Apache Kafka Ken Finnigan Red Hat Who is Ken Finnigan? • Project lead for WildFly Swarm

Keycloak<dependency><groupId>org.wildfly.swarm</groupId><artifactId>keycloak</artifactId></plugin>

Page 22: Reactive Stream Processing - RainFocus...Reactive Stream Processing with WildFly Swarm and Apache Kafka Ken Finnigan Red Hat Who is Ken Finnigan? • Project lead for WildFly Swarm

swarm:keycloak:secure-deployments:time.war:realm:servicebearer-only:trueauth-server-url:http://localhost:9090/authssl-required:externalresource:time-serviceenable-cors:truedeployment:time.war:web:security-constraints:-url-pattern:/*roles:[time-access]

Page 23: Reactive Stream Processing - RainFocus...Reactive Stream Processing with WildFly Swarm and Apache Kafka Ken Finnigan Red Hat Who is Ken Finnigan? • Project lead for WildFly Swarm
Page 24: Reactive Stream Processing - RainFocus...Reactive Stream Processing with WildFly Swarm and Apache Kafka Ken Finnigan Red Hat Who is Ken Finnigan? • Project lead for WildFly Swarm

Java EEis an

awesomely fantasticway to write microservices.

Page 25: Reactive Stream Processing - RainFocus...Reactive Stream Processing with WildFly Swarm and Apache Kafka Ken Finnigan Red Hat Who is Ken Finnigan? • Project lead for WildFly Swarm
Page 26: Reactive Stream Processing - RainFocus...Reactive Stream Processing with WildFly Swarm and Apache Kafka Ken Finnigan Red Hat Who is Ken Finnigan? • Project lead for WildFly Swarm
http://MP.io
Page 27: Reactive Stream Processing - RainFocus...Reactive Stream Processing with WildFly Swarm and Apache Kafka Ken Finnigan Red Hat Who is Ken Finnigan? • Project lead for WildFly Swarm
Page 28: Reactive Stream Processing - RainFocus...Reactive Stream Processing with WildFly Swarm and Apache Kafka Ken Finnigan Red Hat Who is Ken Finnigan? • Project lead for WildFly Swarm
Page 29: Reactive Stream Processing - RainFocus...Reactive Stream Processing with WildFly Swarm and Apache Kafka Ken Finnigan Red Hat Who is Ken Finnigan? • Project lead for WildFly Swarm

Event Ledger Distributed Redundant

A distributed commit log

Page 30: Reactive Stream Processing - RainFocus...Reactive Stream Processing with WildFly Swarm and Apache Kafka Ken Finnigan Red Hat Who is Ken Finnigan? • Project lead for WildFly Swarm

Kafka Semantics

• Based off Messaging systems

• Producers

• Consumers

• Topics

Page 31: Reactive Stream Processing - RainFocus...Reactive Stream Processing with WildFly Swarm and Apache Kafka Ken Finnigan Red Hat Who is Ken Finnigan? • Project lead for WildFly Swarm

Four Parts to Kafka

Page 32: Reactive Stream Processing - RainFocus...Reactive Stream Processing with WildFly Swarm and Apache Kafka Ken Finnigan Red Hat Who is Ken Finnigan? • Project lead for WildFly Swarm

Kafka Record

• Consists of a Key / Value / Timestamp

• Each Record

• Immutable

• Persisted (up to a retention limit)

Page 33: Reactive Stream Processing - RainFocus...Reactive Stream Processing with WildFly Swarm and Apache Kafka Ken Finnigan Red Hat Who is Ken Finnigan? • Project lead for WildFly Swarm

Kafka Topic• Logical name for one or more data partitions

• Append Only

• Ordering within a partition guaranteed

• Business key on Record is critical

• Offers Partition Replication for resiliency

Page 34: Reactive Stream Processing - RainFocus...Reactive Stream Processing with WildFly Swarm and Apache Kafka Ken Finnigan Red Hat Who is Ken Finnigan? • Project lead for WildFly Swarm
Page 35: Reactive Stream Processing - RainFocus...Reactive Stream Processing with WildFly Swarm and Apache Kafka Ken Finnigan Red Hat Who is Ken Finnigan? • Project lead for WildFly Swarm

Kafka Consumers• Reads Records from Topics

• Can start at any offset, inc. replay entire stream

• Consumer Groups are a logical grouping of Consumers

• Ensures a Record is only read by a single Consumer within a Consumer Group

• Provides load balancing

Page 36: Reactive Stream Processing - RainFocus...Reactive Stream Processing with WildFly Swarm and Apache Kafka Ken Finnigan Red Hat Who is Ken Finnigan? • Project lead for WildFly Swarm

CDI Extension for Kafka

• Utilizes Kafka APIs underneath

• Removes boilerplate code for configuring Kafka

• Converts “pull” of Kafka API into “push” for CDI

• https://github.com/aerogear/kafka-cdi

Page 37: Reactive Stream Processing - RainFocus...Reactive Stream Processing with WildFly Swarm and Apache Kafka Ken Finnigan Red Hat Who is Ken Finnigan? • Project lead for WildFly Swarm

Demo

Page 38: Reactive Stream Processing - RainFocus...Reactive Stream Processing with WildFly Swarm and Apache Kafka Ken Finnigan Red Hat Who is Ken Finnigan? • Project lead for WildFly Swarm

Currently writing

https://www.manning.com/books/enterprise-java-microservices

Page 39: Reactive Stream Processing - RainFocus...Reactive Stream Processing with WildFly Swarm and Apache Kafka Ken Finnigan Red Hat Who is Ken Finnigan? • Project lead for WildFly Swarm

WildFly Swarm Resourceshttp://github.com/wildfly-swarm/wildfly-swarm/

http://wildfly-swarm.io/

#wildfly-swarm on irc.freenode.net

issues.jboss.org/browse/SWARM

https://howto.wildfly-swarm.io/

https://reference.wildfly-swarm.io/

https://groups.google.com/forum/#!forum/wildfly-swarm

Page 40: Reactive Stream Processing - RainFocus...Reactive Stream Processing with WildFly Swarm and Apache Kafka Ken Finnigan Red Hat Who is Ken Finnigan? • Project lead for WildFly Swarm
Page 41: Reactive Stream Processing - RainFocus...Reactive Stream Processing with WildFly Swarm and Apache Kafka Ken Finnigan Red Hat Who is Ken Finnigan? • Project lead for WildFly Swarm

Demo Code

https://github.com/kenfinnigan/javaone2017

Page 42: Reactive Stream Processing - RainFocus...Reactive Stream Processing with WildFly Swarm and Apache Kafka Ken Finnigan Red Hat Who is Ken Finnigan? • Project lead for WildFly Swarm

Thanks!