Job Queue by Abhinav Lal

8/9/2019 Job Queue by Abhinav Lal

1/19

JOB QUEUE

web is more than req

Abhinav Lal

est and response

8/9/2019 Job Queue by Abhinav Lal

2/19

WHO AM I

CTO at Naabo

TurboDoc.in

Cooking PHP for 4 years

8/9/2019 Job Queue by Abhinav Lal

3/19

Job Queue

In computer scqueues and mailb

engineering componen

communication,or for inter-

the sam

http://en.wikipe

Any long runningworkw

feedback shouldbe

i

ence, messageoxes are software-

ts used for interprocess

hread communicationwithin

process.

dia.org/wiki/Message_queue

ichdoes not require user

done inbackground.

8/9/2019 Job Queue by Abhinav Lal

4/19

Order processi

queue

New

Store data

Genera

Provision new

Email user wi

Send thank you me

g without job

Order

in database

e receipt

settings for user

h order details

sage back to browser

8/9/2019 Job Queue by Abhinav Lal

5/19

Order processi

queue

New Order

Store data in database

Add receipt generation job

Add Provision user job

Add email user job

Send thank you message back to browser

g with job

Job Queue

Generate Receipt

Provision user

Email user

8/9/2019 Job Queue by Abhinav Lal

6/19

Jobs to run in backg

Sending email

Fetching RSS feedperiodic

Interactingwithother app

Complex reportgeneratio

Loganalysis

cleaning file system

o periodically or after cert

r

ound

lly

icationAPI

ainaction.

8/9/2019 Job Queue by Abhinav Lal

7/19

But I can use c

Asynchronous

Lowlatency

Reliable

Scalable

on for that

8/9/2019 Job Queue by Abhinav Lal

8/19

Options out there

Activ

Beanstalkd

AmazonSQS

Dropr Rabbi

MQ

t MQ

Gearman

Zero MQ

MemcacheQueue

OpenAMQ

8/9/2019 Job Queue by Abhinav Lal

9/19

Options out there

Active MQ

Beanstalkd

AmazonSQS

Dropr Rabbit MQ

Active MQ

Rabbit MQ

earman

ero MQ

MemcacheQueue

OpenAMQ

8/9/2019 Job Queue by Abhinav Lal

10/19

Gearman = Ma

Developedby Danga (P

Open source C daemon

Native PHP extension

Persistence drizzle,po

Distributed

No single pointof failur

ager

ople behindmemcache)

stgres,memcache, sqllite

8/9/2019 Job Queue by Abhinav Lal

11/19

Beanstalkd

Open source C daemon

Fast

Persistence binlog file

Distributed

PHP library pheanstal

Simple protocol

,beanstalk

8/9/2019 Job Queue by Abhinav Lal

12/19

Simple setup

Client script

Job

Serve

r

DISTRIBUTED SETUP

Job

Serve

rWorker script

DISTRIBUTED SETUP

8/9/2019 Job Queue by Abhinav Lal

13/19

Beanstalkd

PUT

Ready

Reser e

Reserved

DELETE

Poof

8/9/2019 Job Queue by Abhinav Lal

14/19

Beanstalkd clie t

8/9/2019 Job Queue by Abhinav Lal

15/19

Beanstalkd Wo ker

8/9/2019 Job Queue by Abhinav Lal

16/19

Beanstalkd Go

Named Tubes max 20

Priority

Delayed job

Bury

Kick

dness

bytes

8/9/2019 Job Queue by Abhinav Lal

17/19

Advantages

Speed upwork

Parallelandasynchrono

Scales easily

Call functionality inoth

Does notblock your ap

us work

r programminglanguage

che process.

8/9/2019 Job Queue by Abhinav Lal

18/19

Running worke

Use supervisord

GearmanManager

Gear up

Check for memory leaks

Restart your worker scri

runs.

script

inworker scripts

pts after certainnumber of

8/9/2019 Job Queue by Abhinav Lal

19/19

QueAbBlo

Twi

Slid

tionsinavLal: abhinavlal.wordpress.com

ter: @abhinavlal

eshare: slideshare.net/abhinavlal