Job Queue - web is more than request and response

JOB QUEUEweb is more than request and responseAbhinav Lal

WHO AM I

CTO at Naabo TurboDoc.in Cooking PHP for 4 years

Job Queue

In computer science, message queues and mailboxes are software-

engineering components used for interprocess communication, or for inter-thread communication within

the same process.http://en.wikipedia.org/wiki/Message_queue

Any long running work which does not require user feedback should be done in background.

Order processing without job queue

Order processing with job queue

Jobs to run in background

• Sending email

• Fetching RSS feed periodically

• Interacting with other application API

• Complex report generation

• Log analysis

• cleaning file system

o periodically or after certain action.

But I can use cron for that

Asynchronous

Low latency

Reliable

Scalable

Options out there

Options out there

Gearman = Manager

Developed by Danga (People behind memcache) Open source C daemon Native PHP extension Persistence – drizzle, postgres, memcache, sqllite Distributed No single point of failure

Beanstalkd

Open source C daemon Fast Persistence – bin log file Distributed PHP library – pheanstalk, beanstalk Simple protocol

Simple setup

Client script

JobServe

r

Worker script

Beanstalkd

Beanstalkd client

Beanstalkd Worker

Beanstalkd Goodness

Named Tubes – max 200 bytes Priority Delayed job Bury Kick

Advantages

Speed up work Parallel and asynchronous work Scales easily Call functionality in other programming language Does not block your apache process.

Running worker script

Use supervisord Gearman Manager Gear up Check for memory leaks in worker scripts Restart your worker scripts after certain number of

runs.

QuestionsAbhinav LalBlog: abhinavlal.wordpress.comTwitter: @abhinavlalSlideshare: slideshare.net/abhinavlal