Twin Cities BizTalk User Group Rule Engines: BRE vs WF Rules November 2006 Stephen Kaufman.

Twin Cities BizTalk User Group

Rule Engines: BRE vs WF Rules

November 2006

Stephen Kaufman

Agenda

• RETE Algorithm

• WF Rules Processing

• BRE Rules Processing

• Compare/Contrast

• Forward Chaining

• Iterating

• Overview

To RETE or Not to RETE

• Is that a question we should care about?• Which rules engine implements RETE?• What is RETE

– It is an optimization algorithm– It reduces the number of predicate evaluations

by remembering the most recent value and re-using it on subsequent re-evaluations

• You do not need a RETE engine to implement forward chaining

RETE

• There are times that rule sets are structured so that the optimization would make no difference or even hurt performance– If each rule evaluates independent

expressions against different facts, the RETE algorithm will provide no improvement.

– The RETE algorithms intrinsic overhead may result in a net loss of performance

Agenda

• RETE Algorithm





• Iterating

• Overview

WF Rules Processing

• WF provides a policy class – This policy class uses an internal executor

class to execute the rule set

• The policy activity sends the rule set to the executor which creates a sorted list of RuleState objects– Each RuleState presents an individual rule

and its associated state– The RuleState objects are sorted by priority

WF Rules Processing

– Each rule is executed in sequence• If two or more rules have the same priority the execution

ends up based on an implementation dependent choice – currently on rule name (however, that is not something to count on)

• Each rule can have two sets of actions– One of these sets is executed depending on the

overall evaluation of the rule condition – thus allowing if/then/else functionality

• The executor detects when actions perform changes to properties

WF Rules Processing

• The executor detects when actions perform changes to properties which can lead to forward chaining

Agenda

• RETE Algorithm





• Iterating

• Overview

BRE Rule Processing

• The BRE uses a RETE engine to process the rule execution– The RETE engine is a compiled

representation of a rule set

• The BRE executes it RETE algorithm by asserting a number of facts to the engine

• These facts are turned into rule objects which may end up on the agenda

BRE Rule Processing

• All rule objects are evaluated based on their priority– Once rules are placed on the agenda they are

then processed in order

• The BRE follows the Match-Resolve-Act phases.– This is discussed on pages 314-316 in the book

“Artificial Intelligence – a modern approach”, v1 by Russell and Norvig

BRE Rule Processing• The agenda is created with the rule objects in the

order they will run• As rules are pulled off the agenda and executed

they may;– cause conditions to be re-evaluated

• and the agenda being modified before the next rule is pulled off the agenda

– update existing facts, – assert new facts,– re-assert existing facts– Remove facts from the agenda

Agenda

• RETE Algorithm





• Iterating

• Overview

Compare/Contrast - Agenda

• WF does not use an agenda nor does it have the concept of a vocabulary

• BRE uses an agenda and the developer has the ability to modify how the engine operates at run time

Compare/Contrast - Assert• WF acts against one object

– At run-time there is no mechanism for adding or removing individual items

– It is only possible to update the values – which may lead to a re-evaluation of the rules (forward chaining)

• With the BRE you can assert as many facts as necessary (short term and long term) as well as retract facts– You need to manually remove certain types

Compare/Contrast - Assert

• Retracting Facts Manually– This is necessary if you manually add facts

during rule processing and don’t want to reuse that fact in future processing

– These facts can be preserved if you are using the same instance of the engine and want to maintain the state across execution calls

– Typically manually added facts are used for specific instances of rule processing and therefore need to be retracted

Compare/Contrast - Verbs

• WF’s Update verb is used in rule actions to invoke forward chaining where the value of a property has not been changed.

• BRE’s Update verb works differently in that you are telling the engine to re-evaluate what should be added to the agenda

• WF provides both explicit and implicit updates whereas the BRE only provides explicit updates.

Compare/Contrast - Verbs

• Implicit Update– Each rule is analyzed and relationships

between rules are identified automatically.– The engine ensures that the rule will be re-

evaluated whenever a dependant rule is executed

• Explicit Update– The developer tell the engine to re-evaluate

rules with a specified condition

Compare/Contrast - Instances

• BRE requires you think and design in a non procedural fashion

• One rule may result in multiple facts of the same type being added to the agenda– The BRE allows you to associate an instance

number with the terms that reference a given fact type

• factType(1).item == factType(2).item

• This will cause every possible combination of one instance being evaluated against every possible combination of the other instance

Multi Instance Support

• What is it?– BRE rules are expressed against types, not

instances or variables. At execution time, the engine matches instances that are asserted against the rules written against their type.

– This makes it possible to assert as many instances of a given type as needed and have them be processed independently.

• Be Careful.


• Why should I be careful?• Example

IF Customer.AccountType == “Residential”AND Order.Value > 500THEN Order.Discount = 5

• If two instances of Customer type and Order were asserted, then the outcome would be:

– Instance 1 of Customer and Instance 1 of Order– Instance 1 of Customer and Instance 2 of Order– Instance 2 of Customer and Instance 1 of Order– Instance 2 of Customer and Instance 2 of Order


• Why should I be careful?• Example

IF Customer.AccountType == “Residential”AND Order.Value > 500THEN Order.Discount = 5

• If two instances of Customer type and Order were asserted, then the outcome would be:

– Instance 1 of Customer and Instance 1 of Order– Instance 1 of Customer and Instance 2 of Order– Instance 2 of Customer and Instance 1 of Order– Instance 2 of Customer and Instance 2 of Order

• You need to add an additional condition to the rulesIF Customer.AccountType == “Residential”AND Order.Value > 500AND Order.CustomerID = Customer.IDTHEN Order.Discount = 5

Types vs Instances

• There are times when writing rules against instances have advantages over types– Scenario

• An employee is requesting a higher security clearance. If the employees manager has authorization to grant the requested level or a higher level and the employee is within one level of the requested security clearance then grant the request

Types vs Instances

• WFIF this.manager.AuthorizationLevel >= this.accessRequest.RequestedLevelAND this.employee.SecurityLevel == this.accessRequest.RequestedLevel – 1AND this.accessRequest.IsManagerApproved == trueTHEN this.employee.SecurityLevel = this.accessRequest.RequestedLevel

IF this.employee.SecurityLevel > this.accessRequest.RequestedLevel – 1AND this.accessRequest.IsManagerApproved == trueTHEN this.director.RequestSecurityClearance(this.employee, this.accessRequest)

• this.employee, this.manager and this.director are all objects of type Employee

Types vs Instances

• BREIF Employee.AuthorizationLevel >= AccessRequest.RequestedLevelAND Employee.SecurityLevel == AccessRequest.RequestedLevel – 1AND AccessRequest.IsManagerApproved == trueTHEN Employee.SecurityLevel = AccessRequest.RequestedLevel

IF Employee.SecurityLevel > AccessRequest.RequestedLevel – 1AND AccessRequest.IsManagerApproved == trueTHEN Employee.RequestSecurityClearance(Employee, AccessRequest)

Types vs Instances



Will this work? Is there something missing?

Types vs Instances



Will this work? Is there something missing?

You need to distinguish between the various instances of Employee

Types vs Instances

• BRE

IF Employee(0).AuthorizationLevel >= AccessRequest.RequestedLevelAND AccessRequest.EmployeeID == Employee(0).IDAND AccessRequest.IsManagerApproved == trueAND Employee(1).SecurityLevel == AccessRequest.RequestedLevel – 1AND Employee(0).ManagerID == Employee(1).IDTHEN Employee(1).SecurityLevel = AccessRequest.RequestedLevel

IF Employee(0).SecurityLevel > AccessRequest.RequestedLevel – 1AND AccessRequest.EmployeeID == Employee(0).IDAND AccessRequest.IsManagerApproved == trueAND Employee(1).Title == “Director”THEN Employee(1).RequestSecurityClearance(Employee(0), AccessRequest)

Types vs Instances

• There are also times when writing rules against types have advantages over instances– The BRE algorithm lets you decide the

manager relationship in rules rather in the calling code

– If you wanted to change the rule so that the manager’s manager can be used when the manager is out of the office you could do this entirely in the rules with BRE

Types vs Instances

– With WF you would need to change the calling code

• You would need to add an extra property• You would also need to get the additional employee

object

• All in all they really are doing different things.– BRE is executing over a collection of employee

objects, – WF is executing over a single object with 2

properties.

Agenda

• RETE Algorithm





• Iterating

• Overview

Forward Chaining Explained

• In forward chaining, rule actions operate against the object state where rule actions change the value of items in the object state

• This causes the engine to automatically re-evaluate the relevant rules against the new values which in turn continues until no additional re-evaluation is needed.

Forward Chaining - WF

• Rules run directly on the sorted list of the RuleState objects and execute the then or else action

• The RuleState object has a pending flag which starts off as true– As each rule is processed the pending flag is

set to false– The flag is set no matter which action may, or

may not, have fired


• When the then or else is processed, the engine analyzes the action and determines if they invoke code which will have side effects.– A side effect is code that may alter the state

of the object in a way that could affect the evaluation of conditions in other rules


• If side effects are encountered, the engine searches the collection of RuleState objects looking for rules whose conditions are affected AND whose pending flag is set to false

• If the engine finds an affected rule with a higher priority than the rule which caused the side effect, the higher priority rule is evaluated and its actions are processed


• Execution continues from the higher priority rule that just ran

• If there are no higher priority rules affected then the engine continue to process the rules in order from the rule that caused the side effect

• So, forward chaining is tied to prioritization and can cause a loop back to higher priority rules


• For simpler rule sets , forward chaining can sometimes be eliminated by re-prioritizing the rules so that they are evaluated in a different order.– Or for rule sets with no prioritization, creating

the rules in a different order MAY also be a possibility to eliminate forward chaining

– Remember that rules are evaluated arbitrarily

Forward Chaining Example

• What does this rule do?IF

Truck.ShippingCharge < 2.5

THEN

Truck.ShippingCharge = 1;

Forward Chaining - Looping

• WF provides the RuleReevaluationBehavior at the rule level as well as the RuleChainingBehavior

public enum RuleChainingBehavior

{

Full, // default; implicit, explicit and attribute-based chaining

None, // do not chain; purely sequential execution

UpdateOnly, // chain only on explicit Update statements

}

• BRE provides the Maximum Execution Loop Depth at the ruleset level (default of 65536)

Agenda

• RETE Algorithm





• Iterating

• Overview

Iterating in the BRE

• Given the following XML, how will the BRE loop through the item nodes?

<Package>

<Items>

<Item>Manifest</Item>

<Item>Pro BizTalk 2006 by Apress</Item>

<Item>Essential Windows Workflow by Addison-Wesley</Item>

</Items>

</Package>

• Default Composer Settings– XPath Field: *[local-name()='Item' and namespace-uri()=''] – XPath Selector: /*[local-name()='Package' and namespace-uri()='']/*[local-

name()='Items' and namespace-uri()='']

Iterating in the BRE<Package>

<Items>

<Item>Manifest</Item>

<Item>Pro BizTalk 2006 by Apress</Item>

<Item>Essential Windows Workflow by Addison-Wesley</Item>

</Items>

</Package>

• The developer must manually modify the properties in the Composer

– XPath Field: '.' (without the single quotes)– XPath Selector: /*[local-name()='Items' and namespace-uri()='']/*[local-

name()='Item' and namespace-uri()='']'

Iterating in WF Rules

• WF requires additional methods/properties be placed on the object that is submitted.

• It is possible to iterate using the WF rules when your source is an xml document

• i.e., this.document.SelectNodes()


• Create ruleset with at least 4 rules as well as decorate your class with extra properties

• Create an enumerator property of type System.Collections.Generic.IEnumerator<T> or System.Collections.IEnumerator

• Create an items collection to contain your collection of items which can be of type List<T> or anything that supports IEnumerator

• Create a property to hold the current item


• Rules Iteration Pattern - Four Rules– Rule 1 (Priority = 2) //Always execute this rule once to create the enumerator.

IF 1==1

THEN this.enumerator = this.myCollection.GetEnumerator()

– Rule2 (Priority = 1)IF this.enumerator.MoveNext()

THEN this.currentInstance = this.enumerator.Current

– Rules 3-N (Priority = 0).... //Additional rules written against this.currentInstance


• Rules Iteration Pattern - Four Rules– Rule N+1 (Priority = -1)

// can be any condition as long as it is evaluated every time;

// this.currentInstance will be evaluated each time this.currentInstance changes, whereas

// "1==1" would only be evaluated once.

IF this.currentInstance == this.currentInstance

THEN ... Update("this/enumerator") //this will cause Rule 2 to be reevaluated

ELSE ...

Update("this/enumerator")

• This pattern works for a single collection– Repeat this as needed or in a hierarchy for multiple collections

Agenda

• RETE Algorithm





• Iterating

• Overview

Tools

• BRE Provides– An in-process execution engine that runs on the server– The ability to collect references into a concise library to provide

domain specific terms – a vocabulary. This is a feature that will be added in future version of WF rules

– A centralized database for the storage and management of rulesets. This includes audit information on ruleset changes

– A windows service to monitor changes and provide ruleset deployment

– A tracking infrastructure when called from BizTalk. This tracking data is available through the BTS tracking DB as well as the Health and Activity Tracking tool (HAT) and also file based. There is also a public interface to write your own tracking interceptor

Tools

• WF Provides– A rules UI that can be re-hosted in your own application– An in-process execution engine that runs on the client– The ability to track the rules – Rules located and executed on the client

BRE vs. Workflow Rules

• BizTalk Rules– Rete Based Analysis– XML/DB support in addition to

the CLR Type support– If/Then– Vocabulary– File and DB storage of policies– Dynamic Policy-level updates– Caching Architecture– Implicit multi-instance support– Working Memory (stateful)– Rules Tracking– Server Based– Security

• Workflow Rules– Forward Execution Algorithm – Rules written against workflow

variables – CLR Types– If/Then/Else– Short circuiting evaluation– Implicit chaining – More chaining control– Client based

Questions?

Twin Cities BizTalk User Group Rule Engines: BRE vs WF Rules November 2006 Stephen Kaufman.

Documents

agenda slide

wf rules processing

rule objects

rule set

rules engine

rule execution

rete engine

rule condition