Brokered Messaging in Windows Azure

NEIL MACKENZIESATORY GLOBAL

Brokered Messagingin Windows Azure

Who Am I?

Neil MackenzieWindows Azure MVPBook:

Microsoft Windows Azure Development Cookbook

Blog: http://convective.wordpress.com/Twitter: @mknz

Content

Messaging overviewConceptsService Bus (digression)Brokered Messaging APIDemo

Messaging

Facilitates service-oriented architecture

Windows Azure Platform supports: Direct

Push - WCF Relayed

Push – Service Bus Relayed Messaging Brokered

Pull – Service Bus Brokered Messaging

Brokered Messaging

Windows Azure Service Bus Brokered Messaging Provides persistent store for messages Exposes

Send endpoint Listen endpoint

Supports: Structured message content Serializable message body Sophisticated message receipt semantics Message sessions Correlated messages

Scenarios

Temporal decoupling Senders and receivers don’t need to be active

simultaneouslyLoad leveling

Receiver processes messages at its own paceLoad balancing

Scale up number of receivers to handle loadPub/Sub - Multicasting

Multiple subscriptions

Message Store

Managed by Brokered MessagingExposed through a service path on the

Service BusAccess controlled through Service Bus ACSMaximum size of queue/topic 1, 2, 3, 4, 5 GBMaximum message size: 256KBMessage comprises:

Header (name/value pairs) < 64KB Body (serializable)

At-Most Once or At-Least Once

Brokered Messaging supports: At-most once semantics

ReceiveMode.ReceiveAndDelete At-least once semantics

ReceiveMode.PeekLock (default)

Message disposition with PeekLock: Abandon Complete DeadLetter Defer

Queues

S

R

R

S

CompetingReceivers

MultipleSenders

Queue

S

Topics & Subscriptions

S

R

RS

IndependentlyCompetingReceivers

MultipleSenders

R

R

R

R

Subscriptions

Rules(filters & actions)

Topic

S

Rules, Filters and Actions

Rule comprises a Filter and an Action Filter uses Properties bag of a subscription to filter

messages in the subscription Action uses SQL92 syntax to modify the Properties

bag of a subscriptionFilters:

CorrelationFilter - uses CorrelationId FalseFilter - no messages SqlFilter - SQL 92 syntax TrueFilter - all messages (default)

Aside: Service Bus Namespaces

Format: [namespace].servicebus.windows.net/[ServicePath]

Examples: gress.servicebus.windows.net/services/echo

gress.servicebus.windows.net/interestingtopic

gress.servicebus.windows.net/news/subscriptions/subscriber Namespace: gress Topic name: news Subscription name: subscriber

Aside: Service Identity

“Owner” – admin account for entire namespaceFollowing claims used for each service path:

manage send listen

Create specific service identity for service pathUse minimal set of claims

e.g. only “send” claim needed for a send-only applicationSBAzTool

SDK sample – application to manage service identities

Token Providers

TokenProvider exposes factory methods creating various authentication token providers: Saml token Shared secret token Simple web token

Uri serviceUri = ServiceBusEnvironment.CreateServiceUri( "sb", serviceNamespace, String.Empty);TokenProvider tokenProvider = TokenProvider.CreateSharedSecretTokenProvider( issuer, issuerKey);

Managing Queues, Topics and Subscriptions

NamespaceManager exposes factory methods to create and delete: queues topics subscriptions

NamespaceManager namespaceManager = new NamespaceManager( serviceUri, tokenProvider);namespaceManager.CreateQueue("queueName");

Queue, Topic and Subscription Metadata

QueueDescription, TopicDescription and SubscriptionDescription used to configure queues, topics and subscriptions.

Properties (e.g.): DefaultMessageTimeToLive (qts) LockDuration (qs) (Max 300s, def

60s) MessageCount (qs) Name (s) Path (qt) RequiresSession (qs)

Client Classes

MessagingFactory exposes factory methods to create the MessagingClientEntity derived objects used to send and receive brokered messages: MessageReceiver MessageSession MessageSender QueueClient SubscriptionClient TopicClientMessagingFactory messagingFactory = MessagingFactory.Create( serviceUri, tokenProvider);

Class: Brokered Message

BrokeredMessage Represents the brokered message to be sent to a

queue or topic, or retrieved from a queue or subscription.

Comprises a header with various properties and a body created from a serializable class.

Properties Properties bag can be used, instead of the message

body, to store the content of a brokered message Very powerful technique enabling rules, filters and

actions

Class: QueueClient

QueueClient Handles sending and receiving brokered messages for

a queue. Implements standard message disposition methods

(Abandon, Complete, DeadLetter and Defer) when using ReceiveMode.PeekLock

Supports synchronous and asynchronous versions of all methods

No support for rules, filters and actions

Class: TopicClient

TopicClient sends brokered messages to a topic. Supports synchronous and asynchronous versions of

all methods

TopicClient sender = messagingFactory.CreateTopicClient("topicName");BrokeredMessage brokeredMessage = new BrokeredMessage(serializableObject);sender.Send(brokeredMessage);

Class: SubscriptionClient

SubscriptionClient receives brokered messages from a subscription implements standard message disposition methods

(Abandon, Complete, DeadLetter and Defer) when using ReceiveMode.PeekLock

supports synchronous and asynchronous versions of all methods

Classes: MessageReceiver & MessageSender

MessageSender Abstracts functionality of QueueClient and TopicClient

so that it can send messages to either queues or topics.

MessageReceiver Abstracts functionality of QueueClient and

SubscriptionClient so that it can receive messages from either queues or subscriptions.

Easiest way to access the deadletter queue.

Class: MessageSession

MessageSession provides specialized MessageReceiver used to handle

sessions receives messages only for a particular session

Sessions are supported only if queue or subscription created with RequiresSession property

Session is a sequence of brokered messages with the same SessionId

Deadletter Subqueue

Subqueue containing “failed” messagesMessages are added to deadletter subqueue

When explicitly deadlettered On message expiration (if configured) On filter evaluation failure (if configured)

Deadletter queue processed like any other queue

Named by extending service path, e.g.: /queue/$deadLetterQueue /topic/Subscriptions/subscription/$deadLetterQueue

WCF Interface

WCF implementation of brokered messaging uses: NetMessagingBinding

Binding: Folds brokered message into a WCF message Handles message polling for receiver

See: Rick Hollander post on MSDN http://bit.ly/s2Zagg

REST Interface

REST interface supports: Send Receive Filters More senders and receivers than .NET API

REST interface does not support: Sessions Client batching

Comparison With Azure Storage Queues

Feature Azure Queues Brokered Messaging

API REST, .NET .NET, REST, WCF

Authentication Storage Service HMAC

Service Bus ACS

Maximum queue size 100TB 5GB

Maximum message size

64KB 256KB

Maximum message TTL

7 days 10,675,199 days

At most once delivery

No Yes

At least once delivery

Yes Yes

Maximum message lock

7 days 5 minutes

Hosted service affinity

Yes No

Receive behavior Non-blocking Long polling (<24 days)

Throughput 5,000 msgs/second 800-3,000 msgs/sec

See: http://bit.ly/tY96CZ

Performance

Throughput: 800-3000 1KB messages / second through connection Scales down by number of subscriptions

Control performance through: Client-side batching (default: on - factory)

send/complete Database access (default: on - client entity)

Server caching Prefetching (default: off – client entity)

Use multiple queues and topics if necessary

Best Practices

Use: Non-blocking asynchronous senders & receivers

Asynchronous programming model (BeginX/EndX) Transient Fault Handling Framework Single message factory (connection) Single client entity (thread safe) Client-side batching

Do not use: Default version of Task Parallel Library

See http://bit.ly/t5jUdm

References

Alan Smith: Developers Guide to AppFabric http://bit.ly/uB0aHS

Rick Garibay: Code magazine article http://bit.ly/sRm9iK

Clemens Vasters: Build 2011 http://bit.ly/t5ip6k

Neil Mackenzie: blog post http://bit.ly/oZdewm

MSDN documentation http://bit.ly/v9dcPD