BLIP Basic Lightweight Information Protocol Philosophy, Design Decisions, and Experimental Results Matt Jensen blip.org mailing list: blip-subscribe@makelist.com.

BLIPBLIP Basic Lightweight Information ProtocolBasic Lightweight Information Protocol

Philosophy, Design Decisions, Philosophy, Design Decisions, and Experimental Resultsand Experimental Results

Matt Jensen

blip.org

www.blip.org mailing list: blip-subscribe@makelist.com

FOR MORE INFO...

BLIP DescriptionBLIP Description

A federated, server-based protocol for near real-time, robust, scalable, subject-based publish-and-subscribe messaging. Useful for user-oriented or system-oriented communication.

Project GoalsProject Goals

Open standard for P&S messagingPrototyping and experimentingAggressive timing goals

Project GoalsProject Goals

Open standard for P&S messaging– Platform and Language Neutral

• run anywhere• handhelds to enterprise server farms

– Simple, text-based wire protocol– Leverage existing standards– Robust, can be extended

Project GoalsProject Goals

Prototyping and experimenting– Provide freeware tools– Provide source code– Let a thousand flowers bloom

Project GoalsProject Goals

Aggressive timing goals– Early tools available now– Finalize first-phase protocol in Aug.– End user tools by September– Fold lessons learned into next standard

OriginsOrigins

Limitations of “push”

OriginsOrigins Limitations of “push”

– No open standard; big buy-in, ads– Little control over info sent– Little control over notification techniques– Polling is slow, congests corporate network– Synchronous - miss messages when offline– Limited user-created topics– Limited data formats– Limited security, reliability

OriginsOrigins

Requirements for a Good System

OriginsOrigins

Requirements for a Good System– Open system– Notification is the essence– User has fine-grained control over what they receive, when, and how.– Publishers can send any MIME type data.– Persistent messages needed (offline, etc.)– User accounts needed

• Publishers want to control access• Subscribers need to keep track of what they’ve seen

OriginsOrigins

Formed blip.org – Non-profit– Encourages freeware– New web site, mailing list

OriginsOrigins Moving beyond Push

– General Notification• Buddy lists/collaboration tools• Printers• System monitoring, security

– Reliability• Once, only once, in-order, guaranteed delivery• Support transactions• You’re already 90% there• Opens new opportunities

BLIP - Possible ApplicationsBLIP - Possible Applications

Apps for People– News– Buddy lists– Monitoring (email, web pages)– Security (where’s my kid, my car?)– Persistent chat– Auctions– Stock trading (personal program trading)

BLIP - Possible ApplicationsBLIP - Possible Applications

Apps for Organizations Intranet/Extranet news delivery Custom business events Track inventory levels Workflow apps (e.g., DAV) Transactions

Process orders, coordinate databases, financial transactions

Distributed/parallel processing

ReliabilityReliability

Publish & Subscribe

+ Reliable Message Queueing

+ Open Internet Protocol

= BLIP

Reliability - Message QueueingReliability - Message Queueing

– Uses• Tie systems together

– distributed processing– financial transactions– only send news if you can send bill, too

• Personal Assurance– personal apps can be mission-critical

Reliability - Message QueueingReliability - Message Queueing– Uses

• Tie systems together– distributed processing– financial transactions– only send news if you can send bill, too

• Personal Assurance– personal apps can be mission-critical

– Implementation Costs• Add two-phase commit• Optional

– All servers must support it– A topic can turn it off

Reliability - Existing ToolsReliability - Existing Tools

– Publish & Subscribe• Proprietary tools• Many new Java companies• No interoperability or openness

– Message Queueing• Very proprietary (MQSeries, MSMQ, etc.)• Slow efforts at cooperation (BMQ)

Reliability - JMSReliability - JMS

– Java Messaging Service (JMS)• Provides event services• Limitations• Sun-directed; not open ?• Java only

JMS JMS

Java Client Java Server

BLIP BLIP

JMS

Reliability - JMSReliability - JMS

– Java Messaging Service (JMS)• Provides event services

• Limitations

• Sun controlled - not open

• Java only

• JMS on top of BLIP/other standard?

JMS is an API for Java, which could use BLIP/other

as underlying protocol.

JMS

Java Client Java Server

BLIP BLIP

Other API Other API

Java Client Java Server

Reliability - JMSReliability - JMS

– Java Messaging Service (JMS)• Provides event services

• Limitations

• Sun controlled - not open

• Java only

• JMS on top of BLIP/other standard?

BLIP BLIP

Other API Other API

Java Client Java Server

public interface SimpleBlipAwareness { public void fireBlipMessageReceived(

int aTopicID, int aMessageID, String headline, String completeMessage);

public void fireBlipError(String e); public void fireBlipConnectionStateChange(

int newState, int oldState);}

Reliability - DatabasesReliability - Databases

Very useful for persistent messages– Offloads queries from event server– Leverage existing tools

• SQL for queries• Proven reliability -- transactions, recoverability• High performance (depending on domain)• Integration w/ other systems

– Middleware community moving this way

DB

Reliability - DatabasesReliability - Databases

Not required everywhere– Perceived as slower– Power not always needed

DBServer

Custom Filing System

DBTopicMgrDBTopicMgr

SimpleTopicMgrSimpleTopicMgr

Topics

Reliability & PeopleReliability & People

People want notification systems that are reliable.

“Almost everything can be mission critical to someone.”

BLIP Software - CurrentBLIP Software - Current

Java server Java client classes Java applets Win95 native client Perl send/receive scripts Simple client OCX VB demos

BLIP Software - UpcomingBLIP Software - Upcoming

Palm OS client WinCE client Native Linux server Native NT server More applets Bridges to MSMQ, MQSeries ? Possible JMS wrapper ?

The BLIP ProtocolThe BLIP Protocol

The BLIP ProtocolThe BLIP Protocol

What BLIP doesn’t do

– Presentation• topic or client specific

– ACLs• looking at other standards

– Topic discovery (?)• ACAP/LDAP

– Advanced security• Open hook to other systems

The BLIP Protocol - The BLIP Protocol - SubscriptionsSubscriptions URL-based ACLs are out of band Different cases:

– messages from now on– messages from #1 on– start at 10 messages ago

Default policies– e.g. - buddy lists

The BLIP Protocol - MessagesThe BLIP Protocol - Messages

SMTP-style headers

Any MIME type data– text– GIF– HTML– XML– etc. ...

The BLIP Protocol - Filters?The BLIP Protocol - Filters?

Server-side rules– XML, but how to script?

The BLIP Protocol - Filters?The BLIP Protocol - Filters?

Server-side rules– XML, but how to script?

Currently, repackage data as new topic– like “Event Refinement” in Keryx ?

The BLIP ProtocolThe BLIP Protocol

Multiple access modes– Improves scaling– Offers levels of service

The BLIP ProtocolThe BLIP Protocol

Multiple access modes– ALIVE

ClientServer

The BLIP ProtocolThe BLIP Protocol

Multiple access modes– ALIVE

ClientServer

The BLIP ProtocolThe BLIP Protocol

Multiple access modes– ALIVE

ClientServer

The BLIP ProtocolThe BLIP Protocol

Multiple access modes– ALIVE

ClientServer

The BLIP ProtocolThe BLIP Protocol

Multiple access modes– AWARE

The BLIP ProtocolThe BLIP Protocol

Multiple access modes– AWARE

ClientServer

The BLIP ProtocolThe BLIP Protocol

Multiple access modes– AWARE

ClientServer

tag

IP

The BLIP ProtocolThe BLIP Protocol

Multiple access modes– AWARE

ClientServer

tag

IP

The BLIP ProtocolThe BLIP Protocol

Multiple access modes– AWARE

ClientServer

tag

IP

The BLIP ProtocolThe BLIP Protocol

Multiple access modes– AWARE

ClientServer

tag

IP

The BLIP ProtocolThe BLIP Protocol

Multiple access modes– AWARE

ClientServer

tag

IP

The BLIP ProtocolThe BLIP Protocol

Multiple access modes– AWARE

ClientServer

tag

IP

The BLIP ProtocolThe BLIP Protocol

Multiple access modes– AWARE

ClientServer

tag

IP

The BLIP ProtocolThe BLIP Protocol

Multiple access modes– ALIVE

• Fast - maintains connection• Expensive - maintains connection

– AWARE• Slower - create TCP connection• Scalable - only use needed connections

The BLIP ProtocolThe BLIP Protocol

Multiple access modes– ALIVE

• Fast - maintains connection• Expensive - maintains connection

– AWARE• Slower - create TCP connection• Scalable - only use needed connections

– Other modes?• Multicast -- Reliable Multicast very delicate

– RFC 2357

The BLIP ProtocolThe BLIP Protocol

Modes allow flexible scaling– High-priority users get ALIVE mode

• ISPs, paying subscribers

– Low-priority users get AWARE mode

Server

ALIVE

AWARE

The BLIP ProtocolThe BLIP Protocol

Another example

Server “Real” Clients

200 ALIVE clients per level3 levels...= 8 million ALIVE clientswith 5-10 second delay(?)

can it be coordinated?

Proxies/Relayers

The BLIP Protocol - Client StatesThe BLIP Protocol - Client States

Unauthorized Ready

Receiving

Sending

Login

The BLIP Protocol - Client StatesThe BLIP Protocol - Client States

Unauthorized Ready

ReceivingSending

C: Acknowledge

S:

Tra

ns

ac

tio

nC: Acknowledge

S:

Tra

ns

ac

tio

n

C: Acknowledge

S:

Tra

ns

ac

tio

n

S: UPDATE TXN=1S: START ITEM #322 61S: Headline: Inflation is upS: Posted 05 Jul 1998 12:49:55 GMTS: Content-Length: 145S:S: The Federal Reserve. . .S: END ITEM (additional items)S: END UPDATES:C: !A002 ACK 1C:S: !A002 200 - OK.S:

The BLIP Protocol - A SessionThe BLIP Protocol - A SessionS: BLIP 0.46 MODES=ALIVES:C: LOGIN joeuser mypassword mode=ALIVEC:S: 200 - OK. Logged in.S:C: !A001 UPDATES ONC:S: !A001 200 - OK.S:

either immediately, or after some time…

The BLIP Protocol - VariationsThe BLIP Protocol - Variations

S: BLIP 0.46 MODES=ALIVE

S: BLIP 0.46 MODES=ALIVE,AWARE

C: LOGIN joeuser mypassword mode=ALIVE

C: LOGIN joeuser mypassword mode=ALIVE,AWARE

C: LOGIN joeuser mypassword mode=ALIVEC:

C: LOGIN joeuser auth=XYZ mode=ALIVEC: Key=de5te4545665r65e4ddhkjdkasiudy753sdC:

Modes - Server variations

Modes - Client variations

Security - Client variations

Issues - Server vs. Point-to-PointIssues - Server vs. Point-to-Point

BLIP is server-based– Not point-to-point– Could be used w/ p-to-p tools

Issues - Server vs. Point-to-PointIssues - Server vs. Point-to-Point

With few clients, p-to-p is nice– reduces server load– can reduce latency– could enhance privacy

With many clients, client gets burdened– Needs sophisticated programming– Especially for real-time collaboration

Issues - Server vs. Point-to-PointIssues - Server vs. Point-to-Point

Example - Spot Demo

Issues - Server vs. Point-to-PointIssues - Server vs. Point-to-Point

Issues - Server vs. Point-to-PointIssues - Server vs. Point-to-Point

Old user sees this... New user sees this...

Clients

Del

ay (

ms)

0

50

100

150

200

250

300

350

400

450

500

2 10 20 30 40

Point-to-Point

Client/Server

Issues - Server vs. Point-to-PointIssues - Server vs. Point-to-Point

IssuesIssues

Roaming users and AWARE mode– multiple ‘active’ client devices per

person?• “Location Server” from SIP?

(Session Initiation Protocol)

IssuesIssues

Do Subscribers get messages sent before their subscription started?

– “Quenching”– Multicast

– Flexibility

IssuesIssues

Dynamic filter control– How to fine-tune what you receive, on

the fly, without disturbing subscriptions?

C: !A0002 UPDATES ONC: - /nasdaq/quotes/*C: + /nasdaq/quotes/msftC:

EndEnd

Matt Jensen

mattj@blip.org

www.blip.org mailing list: blip-subscribe@makelist.com

FOR MORE INFO...