Natural Rule Language Christian Nentwich, Model Two Zero Immo Hüneke, Zühlke Engineering Presented at the British Computer Society 8 th of June 2011
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Natural Rule Language
Christian Nentwich, Model Two Zero Immo Hüneke, Zühlke Engineering
Presented at the British Computer Society
8th of June 2011
Agenda
• Issues in System Integration
• The goal of executable specifications
• Quick intro to the Natural Rule Language
• Project Review: Soft Trades Synchronizer (Immo)
• Additional experience report from financial services
Area of Concern
• We are dealing with large-scale system integration
ESB
Problem Context
• Hundreds or thousands of systems
• Lots of variety
– Off-the-shelf
– In-house
– “Tactical” (hacks)
• Great complexity
– Tens of thousands of data items
– Complex processes
Widespread Practice
Business Requirement
“We could save money by pre-populating SMALL SYSTEM from BIG SYSTEM
every day. Do it, we are allocating £200,000 to it.”
Analysis
• Analyse the data and semantics in SMALL SYSTEM and BIG SYSTEM
• Write a big specification
• (Usually extremely data centric, with some process on top)
Implementation
Testing and Go-Live
• Sometimes performed by separate teams
• Errors found here are guaranteed to delay the schedule
• Interpret and implement the specification
• Done by programmers using proprietary or open source tools
Readability By Non-Technical Users
• Critical for gap analysis on future projects
• Important for bridging communication gap between developers / business
(Not readable)
Precise and Formal Semantics
• Plain English is too ambiguous
• Must control that you are talking about right things in system context
> ird-12 (Mandatory)
> English Description:
> Context: CalculationPeriodDates (complex type)
>The frequency specified in calculationPeriodFrequency must divide the [period] precisely.
> This means that by stepping through the period from the start date at the specified frequency,
> it must be possible to reach the end date.
What does happen with stubs? Shouldn't the rule include the cases with stubs?
Microsoft Word/Excel
Are you sure
this exists?
What does
“gives any
value” mean?
“Is any of this
currently in
production??”
Is this as good as it gets?
• Poor processes
• Huge maintenance problems
– No transparency of data flows
– Specification useless after go-live
– Too much code to maintain
• Inefficient use of differently skilled people
• Communication barriers
– Off-shoring problems
• System integration is a specialised discipline – but discipline is what is lacking
Changing Direction
Instead of writing informal specifi-
cations and then code, create
executable specifications.
Executable Specifications
• Critical features:
– Readable by non-technical users
– Independent of a proprietary tool
– Precise and formal semantics
– Easy to write code generators or interpreters
• Desirable features:
– Writeable by non-technical users
– Amenable to easy analysis using tools
Microsoft Word/Excel
• Readable by non-technical users
• Independent of a proprietary tool
• Precise and formal semantics
• Easy to write code generators or interpreters
• Writeable by non-technical users
• Amenable to easy analysis using tools
Java / XSLT / Schematron
• Readable by non-technical users
• Independent of a proprietary tool
• Precise and formal semantics
• Easy to write code generators or interpreters
• Writeable by non-technical users
• Amenable to easy analysis using tools
Natural Rule Language
• An open language for expressing:
– Validation rules (constraints)
– Action rules (e.g. Enrichment rules)
– Transformation / mapping
• Aimed at the core problem areas in integration
• Goals
– Read like an English sentence wherever possible
– Require no customisation to get going
– Offer symbolic and textual alternatives
• Specification: http://nrl.sourceforge.net
The Origins: Validation
public void validate(Transaction transaction) {
if (transaction.getDebit() != null && transaction.getCredit() != null) {
if (transaction.getDebit().getValue() == transaction.getCredit()
.getValue() && getChangeInAmount() != null) {
exceptions.add(new ValidationError(
"Must not specify changeInAmount if a debit is equal to a credit"));
}
}
}
context: Transaction
inv: (self->debit = self->credit) implies (self->changeInAmount->empty())
<rule context="ns:transaction">
<assert test="ns:debit != ns:credit or not(ns:changeInAmount)">
Must not specify changeInAmount if a debit is equal to a credit
</assert>
</rule>
Java
OCL
Schematron NRL
Context: Transaction
Validation Rule "Our Sample Rule"
If a debit is present and a credit is present then
no changeInAmount is present
Report 'Must not specify changeInAmount if a debit is equal to a credit'
NRL
A quick practical introduction!
How to use this?
• NRL started out as a “Swiss army knife”
– “Use it for anything”
• Lesson: don’t build Swiss army knives!
• Here is where we got to eventually...
Proprietary
Format Validation
(NRL Constraints)
Mapping (NRL Actions)
Enrichment (NRL Actions)
Canonical
Format
POJO Classes
Spring Mule ESB
WebSphere
Logical Adapter
Physical Adapter
Weblogic
Camel
JBoss
NRL Parser
• The NRL parser is free and open source, also on sourceforge
• Designed for processing / code generation
ConstraintRuleDeclaration
IfThenStatement
ExistsStatement ExistsStatement
1. Text in NRL concrete syntax 2. Parser-generated AST
3. Code generators (from AST)
Model Specifications
Executable Specification using NRL
UML
XML
Schema
Rule Specifications
CSV
Fixed
Width
XML
SWIFT
MT
NRL
Constraints
NRL
Actions
*SWIFT SDK
Code
Guaranteed Consistency
Plain Java
Datastage
Drools
Project Review
Over to Immo!
What else have we done?
• Almost entire take-up is for mapping – Central reference data publishing service of a bank
– Corporate action processing
– SWIFT messaging
– 60+ smaller projects, world-wide
Lesson: Dates are Important
• 83 out of 207 (40%) of FpML validation rules are about dates
• 220 out of 751 rules (29%) in reference data export adapter about dates
• Very important to treat dates as a primitive type
If DebtInstrumentCharacteristics.LegalMaturityDateTime is present
then set to DebtInstrumentCharacteristics.LegalMaturityDateTime;
Lesson: Rationale is External
Context: InterestRateStream
Validation Rule "ird-2"
paymentDates.paymentFrequency [is an integer multiple
of] calculationPeriodDates.calculationPeriodFrequency
Why???
Pragmatics != Semantics
Lesson: Testing
• How to test this?
• People still want to do TDD, even with executable specs
• Tests form part of the specification!
• We’re looking into it
Context: InterestRateStream
Validation Rule "ird-2"
paymentDates.paymentFrequency [is an integer multiple
of] calculationPeriodDates.calculationPeriodFrequency
Lesson: Analysis
• There are big benefits to be had from metadata management
– See session this afternoon!
• Turing-complete languages are too hard to analyse
• Think hard about how expressive your language needs to be
Traceability Both Ways
Natural
Rule
Language
UML XML
Schema
Generated
Code
Metadata
Tools
Published
Specifications
Lesson: Analysis
public boolean validate(CalculationPeriodDates context) throws Exception {
CalculationPeriodDates current = context;
return current.getTerminationDate().getUnadjustedDate().getValue()
.toGregorianCalendar().compareTo(current.getEffectiveDate().
getUnadjustedDate().getValue().toGregorianCalendar()) > 0;
}
Context: CalculationPeriodDates
Validation Rule "ird-14"
terminationDate.unadjustedDate is after effectiveDate.unadjustedDate
Lesson: Don’t Abuse Rule Engines
• We wrote a code generator for the JESS engine
• Hand-wrote NRL
• Artificially split rules into left-hand and right-hand side for RETE matching
• Didn’t work too well (execution model too hard, and doesn’t fit)
Lesson: Programmers...
... like to turn things into code!
If ([getStaticData] 'getSystemDate') = 'Y' then
Set header.DateOfCreationAmendment to
([padField] ([currentDate]'yyyyMMdd') and 8 and 'B' and '0')
Else
Set header.DateOfCreationAmendment to
([padField] headerrecord.DateOfCreationAmendment and 8 and 'B' and '0')
Camel-case instead of English sentence
Mixing syntax and semantics
Lesson: Discipline
• What makes one model better than another?
• How do a good data architects distinguish themselves?
• What is the best way to gather tacit rules?
• What are best practices for large-scale modelling?
• What is the standard ontology of artefacts and activities in integration?
Lesson: Discipline
• Some inroads
• System integration still broadly lacks discipline
• Many topics are off the radar of computer science curriculum
• Good luck finding the best people!
Related Documents
