Building With Open MQ

Linda SchneiderTechnical LeadSun Microsystems, Inc.

Using OpenMQ

Copyright Sun Microsystems Inc. All Rights Reserved. [2]

What will be covered ?

An introduction to OpenMQ. A Customer example. Basic customer requirements. Building part of the infrastructure.

Warnings:No in depth coverage Assumes basic JMS knowledge


What is OpenMQ ?


Allows heterogenous applications to reliability and asynchronously pass data between each other.Open Source Java Message Service (JMS) implementation (+ additions)Default Messaging Provider for GlassfishUseful on its own for standalone JMS applicationsEnterprise level quality (>8 years in development)Open Source since JavaOne 2006Available as a supported product: Sun Java System Message Queue (SJSMQ)

Overview


Tell Me More

Developer and User discussion forumsStable builds with product releasesEarly access, promoted builds availableNew features, and fixes

Dual license support (GPL v2 and CDDL)Open source version of Java MQ is available from http://mq.dev.java.net

http://mq.dev.java.net/


Using OpenMQ ?


An example

To mimic problems faced in designing applications, an example:Represents a complex system with loosely connected applicationsUtilizes various types of messagingIs easy to understandIs at least minimally interesting


Our Example: Santa Claus, Inc

Why, you ask ?

Even if you don't believe in Santa Claus, you must still understand that delivering all those presents would be a daunting taskAnd while its not Christmas which comes but once a year, but requires year round planning and preparation.Just because Santa Claus lives at the North Pole, doesn't imply he can't use technology


Overall System Requirements

Santa Claus, Inc. software applications need do the following:Handle gift selection and deliveryManage resources e.g.giftsreindeerElves

Track general status informationhow many days before Christmasetc.


What are we doing:

Focusing on handling christmas gift processingSteps to design it include:Determining the high level operationComing up with the name and type of destinationsDetermining models used for the messagingDetermining load characteristicsLooking at code for some components


Defining the High Level Operation of the System


What do we need to do ?


What do we need to do ? (cont.)


What are the destinations ?


The Child Queue


The Naughty/Nice Queues


The Wrap Queue


The Stuff to Pack Queue


Select-a-gift Queues


Topic LogChild


A quick overview to design patterns


Some basic design patterns:Pattern Description

Request/Reply

Step OperationsBroadcast One message goes to many consumers

ConduitBatch Messages are processed in a chunk

Time Critical/Sensitive

Message is sent to another application who sends back a responseMessages go through several iterations, the message is persisted at key points where processing it again would be expensive

Multiple consumers send messages to a single destination

Messages must be processed within a short period of time (e.g. under an hour) and can not be lost


More things to think about:

Use persistent messages if it can not afford to be lostUse non-persistent messages for:

non-critical step messages (when it can be repeated)Request/ReplyAnytime a message can be lost on a server crash

Use durables for Topics when it may need to be retrieved laterUse normal or XA transactions when multiple things must process together:

XA if it includes other resources like databases


Processing Queue ChildConduit: many producers to one queuePersistent: would be time consuming to lose message


Processing Naughty and NiceStep Pattern: one step of itNaughty Queue: Non-Persistent

its OK if a child who is bad misses their coal

Nice Queue: Persistent. They must get their present.


Processing NiceStep Pattern: more steps of itMultiple resources so XA


Processing Nice (Select a gift)Request/Reply PatternNon-persistentAction repeated on failure


The Wrap Queue Step operationPersistent: end of an expensive set of steps that they don't want to repeat


The Log Child Topic

Broadcast PatternPersistent because santa wants his database accurate


In this example:

The batch pattern was not usedSanta does use It for processing HR updates for the elves

The time sensitive/critical data pattern was not used:Santa does use it during present delivery on christmas eve

to track where he is

Because he has no time sensitive/critical data, reliability is important however data availability isn't


Performance Requirements

22 billion kids364 days for preparation (since christmas is taken)

31,526,000 seconds

70 children/second must be processedAssume 60% are “nice”Assume 40% downtime to cover outages and normal processing (so goal is approx 100 kids/second)


Performance Requirements (cont)

Naughty Kids use1 Persistent queue (child)1 Non-persistent queue (naughty)1 Persistent Topic (log child)

Nice Kids Use:Persistent Queue (child)Persistent Queue (nice)2 Non-Persistent Queues (Inventory request and reply

queues)Non-persistent queue (Wrap)1 Persistent Topic (log child)


The cold hard requirements

Messages:Child: 100 msgs/second (persistent)Naughty: 40 msgs/second (non-persistent)Log Child: 100msgs/second (persistent)Nice: 60 msgs/second (persistent in XA transaction)Inventory request/reply: 60 msgs/second *2 (non-

persistent)Wrap: 60 msgs/second (persistent)

TOTALS:Persistent: 380 msgs/secondNon-persistent: 160 msgs/second


Some Sample Code


Sending the “Child” messagepublic void doPost(HttpServletRequest request,

HttpServletResponse response) throws ServletException, IOException {

// retrieve initial context (ic) QueueConnectionFactory qcf = (QueueConnectionFactory) ic.lookup("MyConnectionFactory"); Queue destQueue = (Queue)ic.lookup("Child"); QueueConnection connection = qcf.createQueueConnection(); try { QueueSession session = connection.createQueueSession( False, Session.AUTO_ACKNOWLEDGE); QueueSender sender = session.createSender(destQueue); MapMessage msg = session.createMapMessage(); // Set each item msg.setString(“firstname”, request.getParameter(“firstname”)); // … retrieve other properties … ; sender.send(msg); } finally { connection.close(); }}


Processing the “nice” queuepublic void onMessage(Message inMessage) { TextMessage msg = null; try { //Message is of type text and has a unique child id msg = (TextMessage) inMessage; String id = msg.getText(); String[] list = db.getList(id); // makes SQL call String item = null; if (list == null) { // no list, send request String item = getListItem(); //next slide } else { item = list[0]; } //update inventory db.updateInventory(item, id);//makes SQL call // put on packing list pack(item, id); } catch (Exception e) { // things went wrong roll back mdc.setRollbackOnly(); }}


Processing the “nice” queue (step 2)public String getListItem(String childid) throws Exception { QueueConnectionFactory factory = jndiContext.lookup(“MyQueueFactory”); QueueConnection qc = factory.createQueueConnection(); qc.start(); QueueSession session = qc.createSession(true, Session.AUTO_ACKNOWLEDGED); Queue q = session.createQueue(“RandomPresent”); Queue reply = session.createTemporaryQueue(); // get sender and receiver QueueSender sender = session.createSender(q); QueueReceiver receiver = session.createReceiver(q); //send message and wait TextMessage m = session.createTextMessage(childid); m.setJMSReplyTo(reply); //send the message sender.send(m); TextMessage back = (TextMessage) receiver.receive(60*1000); // wait a minute if (back == null) { didn't get anything throw new Exception(“Nothing”); return back.getText();}


Processing the “nice” queue (step 3)

public String pack(String item, String child_id) throws JMSException{ QueueConnectionFactory factory = jndiContext.lookup(“MyQueueFactory”); QueueConnection qc = factory.createQueueConnection(); QueueSession session = qc.createSession(true, Session.AUTO_ACKNOWLEDGED);); Queue q = session.createQueue(“Pack”);

// get sender QueueSender sender = session.createSender(q);

//send message MapMessage m = session.createMapMessage(childid); m.setString(“child_id”, child_id); m.setString(“present”, item); sender.send(m);}


You'll need to fill in the rest


More Information


OpenMQ -- More Information

Visit the product webpagehttp://sun.com/software/products/message_queue

Join the Open Message Queue projecthttps://mq.dev.java.net

Browse the product documentationhttp://docs.sun.com/app/docs/coll/1307.3

Take the free technical traininghttp://www.sun.com/training/catalog/courses/WMT-SMQ-1491.xml

http://sun.com/software/products/message_queue

https://mq.dev.java.net/

http://www.sun.com/training/catalog/courses/WMT-SMQ-1491.xml


Related Information

Java Composite Application Platform Suitehttp://sun.com/software/javaenterprisesystem/javacaps/

Java System Identity Managerhttp://sun.com/software/products/identity

Project GlassFish https://glassfish.dev.java.net/

The Aquarium, A community forumhttp://blogs.sun.com/theaquarium/

Using OpenMQ

Linda [email protected]

Thank You!

Building With Open MQ

Technology

system11 copyright

xa26 copyright

use technology8 copyright

naughtynice queues16

wrap queue17 copyright

queue persistent

child queue

naughty queue