Which messaging layer to use in a loosely coupled distributed app (no speaker notes)

Which messaging layer to use in a loosely coupled

distributed app?!

Narahari (Hari) Allamraju

!

java developer; python enthusiast working with messaging systems for 9.5 years

!

http://github.com/supercoderz [email protected]

Messeveryone talks to everyone

Messagingeveryone talks to everyone through a single medium

Work Queueeveryone queues their tasks at the same place and waits

for the results

So you want to build a loosely

coupled app!

!

what do you need to do?

The basics• Will the application benefit by

splitting up into components?

• What are the components? How are they dependent on each other?

• Do we need more than one instance of each to cope up with the load?

• Do we need newer components to handle certain cases that arise from the new messaging layer?

Identify the patterns• What sort of interface do we

need - request-reply, publish subscribe or queuing?

• Do we need to separate out messages into various types?

• Do we need schemas for the messages or free form?

• Are there any components that have a higher or lower throughput than others?

Latency, routing, priority• Which components need the

message yesterday; and which can wait till end of day?

• Do we need to expire messages when they are too late?

• How easily can we route messages?

• Can we make some messages take highest priority?

Build, Manage and Support

• Do we have a suitable API?

• Easy to provision and manage

• Good monitoring

• Guaranteed delivery of messages

• Should be persist messages to disk for crash recovery?

• Cleanup in case of issues

A few options

• RabbitMQ

• ZeroMQ

• Redis

• RDBMS or NOSQL database

• RPC

Considering future growth

• What if the loads increased by 15%?

• What if we had a consistent heavy run for 24 hours or more?

• Storage costs etc

• If we have to switch to a different underlying - how hard is it?

Tokyo Metro

Questions!• This presentation can be

found on slideshare at - http://goo.gl/KVJxoE !

• The version with speaker notes will be available soon !

• you can also email me at [email protected]