Event Processing – Vision and Reality BPM-BAM-CEP Conference Regensburg, June 19, 2006 Keynote Address. Dr. Opher Etzion, STSM Lead Architect, Event Processing Technologies IBM Software Group. Outline. Event Processing – market view. Event Processing - segments. - PowerPoint PPT Presentation
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Software Group – Event Processing Technology and Architecture
Nothing is really new We process events for many years (e.g. exception handling in OS).
However, recently: Significant amount of events – types, sources, instances Variety of application need to process events Some traditionally stand-alone applications need to be integrated with regular
information systems (simulation, real-time) Functionality requires sophistication – e.g. temporal capabilities, spatio-temporal
capabilities.These all contributed to COTS tools
It is not cost-effective to develop this functionality for a single application It is recognized as middleware level capability, thus customer preference for COTS. Drive for standards is next step
.
Software Group – Event Processing Architecture and Technologies
Event-Based Application PlatformsDefinition: Application platforms that offer a programming model for event-driven computing. Business application vendors will need complex event processing capabilities as part of their evolving service-oriented architectures (SOAs) to deliver adaptability and flexibility of their platforms.Justification for Hype Cycle Position/Adoption Speed: Some leading platform vendors have proposed, but none has strategically committed to, an event model as primary for business application design. JavaBeans is the early, and nearly failed, attempt at this model. The model has an intrinsic power that makes its gradual adoption likely, but not in the near term.Business Impact Areas: Event-based application platforms see the software environment as a relationship of events, as opposed to a relationship of programs in prevailing models. Business application vendors will design and develop event-driven principles into their traditional SOAs. This evolution will make unique application styles possible.
Benefit Rating: High.Market Penetration: Less than 1 percent of target audience.Maturity: Embryonic.
Software Group – Event Processing Architecture and Technologies
Complex Event Processing for Trade FacilitationDefinition: A component of business activity monitoring, complex event processing (CEP) entails the receipt or extraction of events as they are occurring in real- or near-to-real time, processing (analysis, filtering, aggregation and cleansing) to determine if the event requires action, and producing an output that can be used to trigger an appropriate action.Justification for Hype Cycle Position/Adoption Speed: Current implementations typically involve a limited number of feeds and relatively simple filtering or analysis. CEP is one component of the enterprise nervous system concept, and to straight-through processing and real-time operations.Business Impact Areas: Applications include algorithmic trading, quality of service monitoring, best execution assessment.Benefit Rating: High.Market Penetration: Five percent to 20 percent of target audience.Maturity: Adolescent.Example Vendors: Atsmai, iSphere, Progress Software (acquired Apama), Streambase.Recommended Reading:• “Clarifying the Terms 'Event-Driven' and 'Service-Oriented Architecture'”• “Innovative Vendors in Business Event Management”
Software Group – Event Processing Architecture and Technologies
Main functionality: Eliminate redundant events – filtering, remove duplication
Event correlation to determine symptoms
Problem/symptoms relations modelingBoth real-time and post-mortem (log files) May need high throughput Time interval typically short Assumption that events can be lost:
No fault tolerance is required.
Partial pattern satisfaction may be useful. Pattern language is focused on – sequence within short time-
frames, thresholds, filtering condition. Typical users: system administrators, developers, product support
team.
Software Group – Event Processing Architecture and Technologies
Main functionality: Eliminate redundant events – filtering, remove duplication
Event correlation to determine symptoms
Problem/symptoms relations modelingBoth real-time and post-mortem (log files) May need high throughput Time interval typically short Assumption that events can be lost:
No fault tolerance is required.
Partial pattern satisfaction may be useful. Pattern language is focused on – sequence within short time-
frames, thresholds, filtering condition. Typical users: system administrators, developers, product support
team.
Software Group – Event Processing Architecture and Technologies
Personalized and specialized form of the publish/subscribe paradigm
Typically loosely-coupled from the operational system, Subscription can be by individuals, “communities of
interests” or applications – routing decisions and management of subscriptions is important
Subscription can be to derived events and not only to raw events, which requires pattern detection – trends, content filtering combined with basic relations (conjunction, disjunction, negation, sequence etc..).
Requires end-user interfaces for subscriptions in some cases, development interfaces in cases of applications’ subscriptions.
Software Group – Event Processing Architecture and Technologies
Requires predictive tools embedded in event processing Requires dependency model of events and entities Requires mining tools to find causality relations requires rich causality model with possible uncertainty
handling (e.g. probabilistic engine) Rich pattern language – different type of condition
causality.
Software Group – Event Processing Architecture and Technologies
Context: A set of criteria to partition the space of events according to temporal (e.g. time window), spatial (e.g. space window) and partitioning entity (e.g. platinum customers)
Event Stream: A collection of events that arrive to a single consumer over a single event topic within a single context
Event cloud: A collection of all event streams that a single consumer receives
Software Group – Event Processing Architecture and Technologies
Pattern Detector: A software artifact that obtains an event cloud as an input, and creates a “complex event” –Example: the stock quote is monotonically decreasing within 2 hours. Return the collection of stock quotes.
Complex event: An event that contains reference to all events that participate in the pattern
Software Group – Event Processing Architecture and Technologies
Derived Event: An event that is calculated as a function of other events.
Event Processor: A software artifact that obtains a composite event and creates collection of derived events using some function on the input events, such as: Enrichment, Transformation, Aggregation, Split.
Software Group – Event Processing Architecture and Technologies
There are several different approaches to event processing. Rule-based approach Extended SQL-based approach (stream and active database) Script oriented approach Other approaches
Our aim is to have a model-based approach that is in a “platform independent” level, have appropriate tooling, and consistent with standard approaches.
Probably there is no “one fits all” solution in the run-time level, but from the model and tooling level everything should be seamless
We should strive for open architecture with: Separation of concerns:
o each function can be implemented by various run-time artifacts, each of them satisfies a subset of the appropriate functions, and given set of non-functional requirements.
o We should own the “integration platform” for event processing: the biggest market for event processing will be in “application integration through event processing”.
o We should have our own run-time artifacts, but allow easy integration with other run-time artifacts, mainly specialized ones.
Software Group – Event Processing Architecture and Technologies