Job queue in b2b with Akka

Job Queue in B2Bwith Akka

long versionYasuki Okumura

● Shanon, Inc. - Marketing company

● Scala developer for 2 years

twitter: @busterdayogithub: https://github.com/buster84

About me

Shanon hiring Scala developers

● Job Queueing manager● https://github.com/Shanon/Malba

Malba

● Most problematic thing in B2B is job order.● Most of B2B service cases, normal FIFO or

priority Queue don’t fit their business model.

● We need our own Custom Queue.

Motivation

Example - Email service

1. A attempt to send 100,000 emails.2. B attempt to send 100 emails.

Does B have to wait until finishing to send 100,000 emails?

Balancing Queue

A’s 100,000 emails

FIFO

B’s 100 emails

1

32

4

100,001

100,100

Balancing Queue

1 2

100,100

200

4

199

3

● Breadth-first search -> FIFO Queue● Depth-first search -> FILO Queue● A* search -> Priority Queue

Inspired by Graph search

What makes difference among the algorithms is Queue.

● Premium users can send mails faster than normal users.

● It is guaranteed that sending less than 1,000 mails finish in 1 hour.

Other cases

How we use

1. AddTaskhttps://github.com/Shanon/Malba/blob/master/Malba-protocol/src/main/scala/jp/co/shanon/malba/worker/MalbaProtcol.scala#L9-L17

2. AddWorkerhttps://github.com/Shanon/Malba/blob/master/Malba-

protocol/src/main/scala/jp/co/shanon/malba/worker/MalbaProtcol.scala#L151-L158

3. GetTaskhttps://github.com/Shanon/Malba/blob/master/Malba-protocol/src/main/scala/jp/co/shanon/malba/worker/MalbaProtcol.scala#L56-L63

Structure + Protocol

Demo

https://github.com/typesafehub/activator-akka-distributed-workershttp://letitcrash.com/post/29044669086/balancing-workload-across-nodes-with-akka-2

You can compare Malba with above.

Strongly inspired by

● CI environment● Make and Publish Packages● Support HTTP● Tests● Documents● Take snapshots● GUI - using PersistentView

TODO

We welcome your PR or comments.https://github.com/Shanon/Malba

Thank you