Introduction to Really.io

INTRODUCING REALLY.IO

Ahmed Solimanأحمد سليمانA reactive horizontally-scalable observable data backend

• CAT Reloaded Co-founder

• Life-long architect, software, and systems engineer.

• Focusing on systems reliability, scalability, and clean code.

Conictus

WHAT IS REALLY.IO?

BADASS ELASTIC OPEN-SOURCE BACKEND FOR YOUR REALTIME

APPLICATIONS

REACTIVE BY DESIGN

SHARDING/CLUSTERING/NO SPF

BATTLE-TESTED TECHNOLOGY STACKJVM, Scala, akka, websockets, kafka, mongodb, etc.

GRAPH-LIKE DATA MODELING

sample from pyreally

really.get('/products/1234/reviews/1234')

• yaml-based model definition

• directory structure that maps the model hierarchy

• model is versioned for easier data-migration

version: 3fields: firstName: type: string required : true lastName: type: String fullName: type: Calculated valueType: String dependsOn: firstName, lastName value: | function calculate(firstName, lastName) { var fullName = firstName if (lastName != undefined) fullName += " " + lastName return fullName; } locale: type: String validation: | value.length <= 3

ARCHITECTURE

CORE DESIGN DECISIONS• Event-Sourcing Architecture (CQRS) based in Akka Persistence

• Offload Data Storage Engines

• Strong Consistency with relaxed write performance constraints (read-my-own-writes)

• Strong optimization for read-performance and update-propagation performance

• At Least-Once delivery semantics.

• Share-nothing, horizontally scalable, automatic rebalancing, bounded-latency, highly-concurrent

• Easy Integration with Big-Data streaming systems (Spark, Storm, Flume, etc.)

TRANSPORT AND PROTOCOL

• Native WebSockets (full-duplex)

• Plain JSON protocol, fully asynchronous.

• Objects has revisions (versioned) “_rev”: 245

• Only deltas are propagated on the wire

{ tag: 1, cmd: "get", cmdOpts: { fields: ["firstname", "@displayInfo"] }, r: '/users/114243'}

{ tag: 1, meta: { fields: ["firstname", "lastname"] }, r: '/users/114243', body: { "_r": "/users/114243/", "_rev": 323 "firstname": "Ahmed", "lastname": "Soliman", "creator": { value: "/users/234234", ref: { //reference field -- dereferenced "_r": "/users/234234/", "_rev": 20 //revision must be set "name": "Hamada Imam", } } }}

Request Response

{ tag: 1, cmd: "update", cmdOpts: { }, r: "/users/66123/", rev: 42, body: { ops: [ { op: 'set', key: 'lastName', value: 'Ahmed' }, { op: "addNumber", key: 'age', value: 1 //adding one } ] }}

Request Response

{ tag: 1 r: '/users/66123/', rev: 43}

READS AND QUERIES

• Reads are really fast!

• Welcome to the RQL (Really Query Language™). But Why?

firstName = $name AND (age BETWEEN $range OR group IN $groups)

{ name: "Ahmed", range: [10, 20], groups: ["admins", "developers"]}

COLLECTION ACTOR AND VIEWS

Event Log

CMD

CollectionActor

RESP

EVT EVT

ViewMaterializer

EVT

Database(mongodb)

Read Handler

CMD

ERR

RESP

Model Validation

Gorilla

GORILLA EVENT CENTER

FROM ITS SCALA-DOCS• The gorilla event centre is an actor that receives events from the

Collection View Materializer and store it in a persistent ordered store (H2 database)

• It ensures that we are not storing the same revision twice for the same object (by ignoring the event)

• Publishes the event on the Gorilla PubSub asynchronously for real-time event distribution

Gorilla Event Center Log

Replayer

EVTViewMaterializer

Object SubscriberValidation

In-Memory Volatile

Revision Log

EVT

Distributed PubSub

PUSH

Subscription Manager

ABOUT OUR LOGO

Peregrine

Idea and Design by Mohamed Gamal

THETEAM

ROADMAP• Bug fixing (a lot of them)

• A detailed documentation on the architecture, protocol, and client-development

• Administration interface (API/web-based)

• Performance tuning

• Checkout https://reallyio.atlassian.net/ for the issue tracking

MISSING YET.

• Authentication sub-module for username/password combination

• A fully-functional entirely-reactive JavaScript low-level driver

• Higher-level libraries for integration with AngularJs, EmberJs, ReactJs.

• Android and iOS client libraries for mobile apps integration

CALL FOR CONTRIBUTORS

!github.com/reallylabs/really

http://really.io

THANK YOU!