0MQ And Mongrel2 Overview

5/12/2018 0MQ And Mongrel2 Overview - slidepdf.com

http://slidepdf.com/reader/full/0mq-and-mongrel2-overview 1/21

0MQ and Mongrel2The future is far too simple.

5/12/2018 0MQ And Mongrel2 Overview - slidepdf.com

http://slidepdf.com/reader/full/0mq-and-mongrel2-overview 2/21

ZeroMQ

Developed by iMatrixBuilt AMQP for a bank

Didn't like what they madeBlogged about itTried to fix it

Did a pretty good job

5/12/2018 0MQ And Mongrel2 Overview - slidepdf.com

http://slidepdf.com/reader/full/0mq-and-mongrel2-overview 3/21

ZeroMQ is Not

AMQPAMQP "Light"

A job/task queueJust lazy socketsMagic bullet that fixes every

problem you've ever had withdata distribution.

5/12/2018 0MQ And Mongrel2 Overview - slidepdf.com

http://slidepdf.com/reader/full/0mq-and-mongrel2-overview 4/21

ZeroMQ is

A message passing libraryWith bindings for everything

Atomic messagesNo message fragmentationYou can ask for it if you

wantYour chunks remain atomicPeer is notified

Asyncronous message delivery

5/12/2018 0MQ And Mongrel2 Overview - slidepdf.com

http://slidepdf.com/reader/full/0mq-and-mongrel2-overview 5/21

ZeroMQ is

Simple

Similar API to BSD sockets,similar terminologyAround 20 total API calls

zmq_init, zmq_socket, zmq_connect, zmq_bind, zmq_setsockopt,

zmq_getsockopt, zmq_close, zmq_term, zmq_poll, zmq_recv, zmq_send,zmq_device, zmq_msg_init, zmq_msg_init_data, zmq_msg_init_size,zmq_msg_close, zmq_msg_size

Easy to keep entirely in yourhead

5/12/2018 0MQ And Mongrel2 Overview - slidepdf.com

http://slidepdf.com/reader/full/0mq-and-mongrel2-overview 6/21

ZeroMQ is

N:NSockets are more than pairs

Can connect and bind a socketto many addresses over manytransports at the same time

Behavior governed by sockettype

5/12/2018 0MQ And Mongrel2 Overview - slidepdf.com

http://slidepdf.com/reader/full/0mq-and-mongrel2-overview 7/21

ZeroMQ is

Transport AgnosticCan use multiple transports

transperantly andsimultaniouslytcp:// udp:// ipc:// inproc://

IPC are unix pipesinproc are internal memorybuffers

5/12/2018 0MQ And Mongrel2 Overview - slidepdf.com

http://slidepdf.com/reader/full/0mq-and-mongrel2-overview 8/21

ZeroMQ really is

A set of message distributionpatterns.

Different socket types promisedifferent delivery semantics

A set of chosen socketsenforce the promised behavior

5/12/2018 0MQ And Mongrel2 Overview - slidepdf.com

http://slidepdf.com/reader/full/0mq-and-mongrel2-overview 9/21

ZeroMQ Sockets

Request/ReplyZMQ_REQ/ZMQ_REP

Alternate zmq_send()/zmq_recv()

Requests are load balancedReplies are matched to requesterAnything else is an error

ZMQ_ROUTER/ZMQ_DEALER

Advanced REQ/REPUnrestricted send/recvMore requirements on messageformat

5/12/2018 0MQ And Mongrel2 Overview - slidepdf.com

http://slidepdf.com/reader/full/0mq-and-mongrel2-overview 10/21

ZeroMQ Sockets

Pub/Sub

ZMQ_PUB/ZMQ_SUB

ZMQ_SUBSCRIBE/ZMQ_UNSUBSCRIBE socketoptionsOne side can only recv(), the other can onlysend()Messages are filtered by subscriptionDelivered to all connected peersIf no one is listening, messages are dropped.If you don't subscribe, you don't get messages.

5/12/2018 0MQ And Mongrel2 Overview - slidepdf.com

http://slidepdf.com/reader/full/0mq-and-mongrel2-overview 11/21

ZeroMQ Sockets

Work PipelineZMQ_PUSH/ZMQ_PULLZMQ_PUSH produces work, and can only send()ZMQ_PULL consumes work, can only recv()Outgoing messages are fair-queued to allconnected peers.No connected peers causes send() to block,unless you ask nicey not to.

No messages are dropped.

5/12/2018 0MQ And Mongrel2 Overview - slidepdf.com

http://slidepdf.com/reader/full/0mq-and-mongrel2-overview 12/21

ZeroMQ Sockets

Exclusive PairZMQ_PAIR

Exclusive pair for inter-threadcommunicationOnly "works" on inproc://

transportIf you think you need thissocket, you're probably wrong.

5/12/2018 0MQ And Mongrel2 Overview - slidepdf.com

http://slidepdf.com/reader/full/0mq-and-mongrel2-overview 13/21

That's cute, but what

does it have to dowith web servers?

5/12/2018 0MQ And Mongrel2 Overview - slidepdf.com

http://slidepdf.com/reader/full/0mq-and-mongrel2-overview 14/21

Mongrel2

"Application, language, andnetwork architecture agnostic

web server"

Correct, FSM-driven HTTP

parserLong-pollingZeroMQ based

5/12/2018 0MQ And Mongrel2 Overview - slidepdf.com

http://slidepdf.com/reader/full/0mq-and-mongrel2-overview 15/21

Mongrel2

Long pollingSuperpoll

Hipster compliant pollingmechanismUses load to determine

polling mechanismChoice based on statisticalperfomance analysis

5/12/2018 0MQ And Mongrel2 Overview - slidepdf.com

http://slidepdf.com/reader/full/0mq-and-mongrel2-overview 16/21

Mongrel2

ZeroMQ HandlersRequests sent to a knownZMQ_PUSH addressResponses receieved at a

known ZMQ_SUB address

Wrapper protocol for HTTPrequests based onnetstrings+JSON, tnetstrings

5/12/2018 0MQ And Mongrel2 Overview - slidepdf.com

http://slidepdf.com/reader/full/0mq-and-mongrel2-overview 17/21

Mongrel2

Wait, what?

Request and Reply are decoupledA client is identified by an integerMongrel2 doesn't have to keep a pile of state around while a request is processedA reply mentions client ids that shouldreceive itAnything that can send a ZMQ messageto the server can generate an HTTP

response

5/12/2018 0MQ And Mongrel2 Overview - slidepdf.com

http://slidepdf.com/reader/full/0mq-and-mongrel2-overview 18/21

Mongrel2

N:M RepliesEach reply mentions the UUID of the handlerEach reply mentions every client id that shouldreceive itEach reply can come from anywhereSo each reply can target multiple clients with asingle message just by adding more clients to theresponse list

Usefull for propogation of information to manyinterested peersHTTP transported PUB/SUB

5/12/2018 0MQ And Mongrel2 Overview - slidepdf.com

http://slidepdf.com/reader/full/0mq-and-mongrel2-overview 19/21

Mongrel2 Wire

Protocol[T]Netstrings for everythingRequest

uuid id path size:headers,size:body,Response

uuid size:id id id, bodyEverything is ASCII

UUID identifies a handlerSizes are in bytesIDs are client id integers

Request ID identifies a browser that sent a requestResponse IDs target the reply to the given

browsers

5/12/2018 0MQ And Mongrel2 Overview - slidepdf.com

http://slidepdf.com/reader/full/0mq-and-mongrel2-overview 20/21

Mongrel2

That really is it.Everything else is a detail or fluff 

Disconnect messages state when a peer youreplied to went away

Large uploads are stored on disk and sent as a0mq message with a path to the file

Handler messages at start and finishCan be terminated in-flight by a handler

Configuration through sqlite3, zeromq configuration

in progressCluster response routing can be handled with 0mqPUB/SUBOpen SourceI'm your local contributor with repo keys

5/12/2018 0MQ And Mongrel2 Overview - slidepdf.com

http://slidepdf.com/reader/full/0mq-and-mongrel2-overview 21/21

Questions?RTFM

ZeroMQ

http://zguide.zeromq.org/page:allhttp://api.zeromq.org/

Mongrel2

http://mongrel2.org/

Me

http://twitter.com/sshirokovslava hackin ibs ns.c m