øMQ Vortrag

a practical introduction

structure1. Introduction

2. Why use ?

3. Architecture

4. Hands on!

5. Projects using

6. Future?

1. Introduction

What is ?

is a high-performance asynchronous messaging library

developed by the “iMatix Corporation“ and an open sourcecommunity

it is a “message queue” system, unlike “message oriented middleware” (no broker architecture)

developed as a simple way contrary to AMQP (Advanced Message Queuing Protocol)

LGPL/GPLv3 Software

written in C++

1. Introduction

1. Introduction

: it allows designing complex communication

systems without much effort

it is not a complete messaging system

no out of the box product

like unwrap it, configure it, start it up

it is a higher level socket interface to build up

your own messaging system

History

the original designer of AMQP (Pieter

Hintjens, CEO from iMatrix) started ZeroMQ on

March 30, 2010

he left AMQP working group and now supports

AMQP 1.0 with the new project ZeroMQ

now the latest version is 2.0.10

1. Introduction

entity 1queue

entity 2

1. Introduction

2.Why use ?

2. Why use ?

“If you've done any work with

threads, protocols, or networks,

you'll realize this is pretty much

impossible. It's a dream.

Even connecting a few programs across a

few sockets is plain nasty,

when you start to handle real life

situations.”

Scalability◦ one ZeroMQ socket can connect to multiple endpoints

◦ it automatically provides load balancing

Simplicity◦ the API looks simple

◦ compared with raw sockets it is really simple to deliver

messages

◦ no buffer management is needed

Performance◦ no overhead of an over-engineered protocol

2. Why use ?

language bindings

Ada

Basic

C

C++

C# (.NET & Mono)

Common Lisp

D

Erlang

Go

Haskell

2. Why use ?

language bindings

node.js Objective-C Perl PHP Python Racket Ruby Ruby Tcl Lua

2. Why use ?

3. Architecture

3. Architecture

general architectureapplication

Socket_base.cpp

ctx.cpp

tcp_connecter.cpp tcp_socket.cpp

zmq.cpp/zmq.h

3. Architecture

concurrency model

ØMQ's concurrency model may a bit confusing

at first

ØMQ is a multithreaded application without

◦ mutexes

◦ condition variables

◦ semaphores

instead, each object will live in its own thread

other threads never touch them

3. Architecture

concurrency model

threads communicate with the aid of messages

they called 'commands' to distinguish from user-

level ØMQ-messages

same way the objects can speak to other

objects

3. Architecture

threading model

ZeroMQ

Application

I/O ThreadsApplication

Thread

3. Architecture

threading model

ZeroMQ

Application

I/O ThreadsApplication

Threadcreated outside of

ØMQ to access

the API

created inside of

ØMQ to send

messages

3. Architecture

mailbox system

threads are only objects with a 'mailbox„

basically mailboxes are queues to store

commands

threads retrieve

commands from

the mailbox

mailbox.cpp

3. Architecture

API from zmq.cpp

3. Architecturezmq.cpp

3. Architecturezmq.cpp

3. Architecture

3 steps to set up ZeroMQ

1. choose a transport mechanism

2. set up the infrastructure

3. select a messaging pattern

3. Architecture

choose a transport mechanism

3. Architecture

possible ways to deliver a message:

INPROC:◦ an In-Process communication model

IPC:◦ an Inter-Process communication model

MULTICAST:◦ multicast via PGM, possibly encapsulated in UDP

TCP:◦ a network based transport

3. Architecture

PGM (Pragmatic General Multicast)

is a reliable multicast transport protocol

provides a reliable sequence of packets to

multiple recipients simultaneously

making it suitable for applications like multi-

receive

3. Architecture

set up the infrastructure

3. Architecture

possible infrastructures

QUEUE:◦ a forwarder for the request/response messaging

pattern

FORWARDER:◦ a forwarder for the publish/subscribe messaging

pattern

STREAMER:◦ a forwarder for the pipelined messaging pattern

3. Architecture

select a messaging pattern

3. Architecture

REQUEST/REPLY:◦ bidirectional, load balanced and state based

PUBLISH/SUBSCRIBE:◦ publish to multiple recipients at once

UPSTREAM / DOWNSTREAM:◦ distribute data to nodes arranged in a pipeline

PAIR:◦ communication exclusively between peers

◦ old pattern only for specific applications

3. Architecture

valid socket combinations

PUB and SUB REQ and REP REQ and XREP XREQ and REP XREQ and XREP XREQ and XREQ XREP and XREP PUSH and PULL PAIR and PAIR zmq.h

3. Architecture

REQUEST/REPLY:

3. Architecture

PUBLISH/SUBSCRIBE:

3. Architecture

UPSTREAM / DOWNSTREAM:

4. Hands on!

4. Hands on!

basic concepts

hwserver.cpp

4. Hands on!

basic concepts

hwserver.py

4. Hands on!

basic concepts

hwserver.cpp

4. Hands on!ctx.cpp

4. Hands on!ctx.cpp

4. Hands on!

basic concepts

ctx.h

4. Hands on!

basic concepts

ctx.h

4. Hands on!

basic concepts

zmq.cpp

4. Hands on!

basic concepts

hwserver.cpp

4. Hands on!

basic concepts

zmq.cpp

4. Hands on!Socket_base.cpp

4. Hands on!Socket_base.cpp

4. Hands on!

config.hpp

4. Hands on!

example!!

4. Hands on!

get : (only source code version)

1. download source code (git://github.com/zeromq/zeromq2.git)

2. build libraries (Windows MSVS or your favorite

Compiler on Linux )

3. install on system

4. additional language bindings only available as

source package

5. Projects using ZeroMQ

5. Projects using ZeroMQ

Projects using ZeroMQ

Dripdrop (Javascript Reactor pattern) Zeromqt (Qt) Soaplib (Python) C++ wrapper of zmq::poll zdevices devices project JeffMQ (peer to peer queue framework) 0MQ plugin for RabbitMQ TCP thin streams ZMQMachine (Ruby) (reactor pattern)

6. Future?

Future?

ØMQ 3.0 Roadmap

TCP virtual services / virtual endpoints

Pattern Checking

API Simplifikation

Removals

Future?

ØMQ 3.0 Roadmap

context configuration

it is not possible to dynamically configure a

context at this time

thanks for your attention!