Top Banner
Advanced Rules & Feeders
41

Advanced Rules & Feeders - OLAPLINE€¦ · Advanced Rules & Feeders. Outline •Rules Basics •Feeders Completeness •Performance •Rules vs. TI. 1 Rule Basics TM1 Calculations

Jul 29, 2018

Download

Documents

ledien
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: Advanced Rules & Feeders - OLAPLINE€¦ · Advanced Rules & Feeders. Outline •Rules Basics •Feeders Completeness •Performance •Rules vs. TI. 1 Rule Basics TM1 Calculations

Advanced Rules & Feeders

Page 2: Advanced Rules & Feeders - OLAPLINE€¦ · Advanced Rules & Feeders. Outline •Rules Basics •Feeders Completeness •Performance •Rules vs. TI. 1 Rule Basics TM1 Calculations

Outline•Rules Basics

•Feeders Completeness

•Performance

•Rules vs. TI

Page 3: Advanced Rules & Feeders - OLAPLINE€¦ · Advanced Rules & Feeders. Outline •Rules Basics •Feeders Completeness •Performance •Rules vs. TI. 1 Rule Basics TM1 Calculations

1 Rule Basics

TM1 Calculations

Cell Calculations

View Calculations

Feeders Basics

Page 4: Advanced Rules & Feeders - OLAPLINE€¦ · Advanced Rules & Feeders. Outline •Rules Basics •Feeders Completeness •Performance •Rules vs. TI. 1 Rule Basics TM1 Calculations

1.1 TM1 Calculations

Calculations are On Request and Recursive

Cell by Cell Calculation

View Calculation

Intermediate results are stored for re-use.

Page 5: Advanced Rules & Feeders - OLAPLINE€¦ · Advanced Rules & Feeders. Outline •Rules Basics •Feeders Completeness •Performance •Rules vs. TI. 1 Rule Basics TM1 Calculations

1.1 TM1 Calculations (cont’d)

Why Calculations on Demand: Real Time Performance.

OLAP Geometric Explosion - One change affects many cells.

Example: 10 Dimensions, 3 Levels each. Change one leaf cell affects 310 -1 calculated cells = 59,048.

In-memory architecture and special algorithms permit fast last-minute calculations.

Only relevant cells are calculated.

Page 6: Advanced Rules & Feeders - OLAPLINE€¦ · Advanced Rules & Feeders. Outline •Rules Basics •Feeders Completeness •Performance •Rules vs. TI. 1 Rule Basics TM1 Calculations

1.2 Cell Calculations

Get Cell

Simple Cell

Rule

Interpret Expression

Get CellsGet Value from Cube

Done

Calculate Expression

Consolidation

Go through nonempty cells

Get Cells

Accumulate

Determine Cell Type

Page 7: Advanced Rules & Feeders - OLAPLINE€¦ · Advanced Rules & Feeders. Outline •Rules Basics •Feeders Completeness •Performance •Rules vs. TI. 1 Rule Basics TM1 Calculations

1.3 Views Calculations

View consolidations are calculated “wholesale” using an algorithm called “Stargate”.

Performance ~ n times faster than cell by cell, where n is the number of cells. Get n for ~ the price of 1.

Page 8: Advanced Rules & Feeders - OLAPLINE€¦ · Advanced Rules & Feeders. Outline •Rules Basics •Feeders Completeness •Performance •Rules vs. TI. 1 Rule Basics TM1 Calculations

1.3 Rules in Views

When rule

Is evaluated in the context of a view

Requests a consolidated cell

Cell some cell members occur in the axes of the view.

Then

Consolidation is done with Stargate and result saved for use by other cells in view.

Page 9: Advanced Rules & Feeders - OLAPLINE€¦ · Advanced Rules & Feeders. Outline •Rules Basics •Feeders Completeness •Performance •Rules vs. TI. 1 Rule Basics TM1 Calculations

1.3 Storage of Views

VMT and VMM control storage of views.

When view requested is included in stored view, stored view is used.

When view requested overlaps stored view, the axes are “joined” and a new expanded view is constructed and stored.

Page 10: Advanced Rules & Feeders - OLAPLINE€¦ · Advanced Rules & Feeders. Outline •Rules Basics •Feeders Completeness •Performance •Rules vs. TI. 1 Rule Basics TM1 Calculations

1.4 Feeders Basics

Consolidation speed derives from “skipping” empty areas of cube.

In most applications “density” is < 10-6.

Thus need to “tag” cells that have values calculated by rules.

Otherwise calculation could take more than 1 million times longer. The price of forgetting SKIPCHECK!

Page 11: Advanced Rules & Feeders - OLAPLINE€¦ · Advanced Rules & Feeders. Outline •Rules Basics •Feeders Completeness •Performance •Rules vs. TI. 1 Rule Basics TM1 Calculations

1.4 Feeders Basics – Q&A

Are there any effects of feeders if SKIPCHEK is forgotten?

No, feeders are still done, but ignored. Instead every cell is evaluated.

Would that happen if feeder and rules are in different cubes?

Same

Even an empty rule file switches sparsity algorithm off?

Yes - VERY IMPORTANT

Page 12: Advanced Rules & Feeders - OLAPLINE€¦ · Advanced Rules & Feeders. Outline •Rules Basics •Feeders Completeness •Performance •Rules vs. TI. 1 Rule Basics TM1 Calculations

1.4 Feeders Basics (cont’d)

Feeder statements have the form:TriggerRange => Target.

Feeders are triggered by data input to an empty cell

If A = k * B - Feed A from B.

If A = B +C - Feed from B AND C

If A = B * C - Feed from either B OR C

Feeders propagate recursively.

Page 13: Advanced Rules & Feeders - OLAPLINE€¦ · Advanced Rules & Feeders. Outline •Rules Basics •Feeders Completeness •Performance •Rules vs. TI. 1 Rule Basics TM1 Calculations

1.4 Feeders Basics – Q&A

• Concerning B*C, should you feed from B or from C? – from whichever is more likely to be zero.

• If Cube X feeds Y, and Y feeds Z, would it make sense to feed Z directly from X

– No, because rules will be hard to understand and maintain, and you still have to feed Y

• If what if X, Y and Z are portions of the same cube? – Answer unchanged

• If Y has rules that use values from X, where are feeders written?

– always in the source cube, i.e., in X.

Page 14: Advanced Rules & Feeders - OLAPLINE€¦ · Advanced Rules & Feeders. Outline •Rules Basics •Feeders Completeness •Performance •Rules vs. TI. 1 Rule Basics TM1 Calculations

1.4 Feeders Basics (Cont’d)

Zero Suppression

Feeders are necessary for proper zero suppression.

Even if you do not use SKIPCHECK, zero suppression will not work without feeders.

Feeding Strings

String cells are normally not fed.

Must use FEEDSTRINGS statement to enable.

Page 15: Advanced Rules & Feeders - OLAPLINE€¦ · Advanced Rules & Feeders. Outline •Rules Basics •Feeders Completeness •Performance •Rules vs. TI. 1 Rule Basics TM1 Calculations

1.4 Feeders Basics (Cont’d)

Feeding from Strings

Feeders from string cells re-fire whenever value changes.

Re-fire does not propagate – if target cell is already fed (string or numeric) it will not fire.

Watch out when feeding from strings – Entering a string at a consolidated level will cause overfeeding.

Page 16: Advanced Rules & Feeders - OLAPLINE€¦ · Advanced Rules & Feeders. Outline •Rules Basics •Feeders Completeness •Performance •Rules vs. TI. 1 Rule Basics TM1 Calculations

1.4 Feeders Basics (Cont’d)

Feeding from and to consolidations

To consolidation: feed all leaf components. Can result in overfeeding.

From consolidation: feed from each leaf component.

Page 17: Advanced Rules & Feeders - OLAPLINE€¦ · Advanced Rules & Feeders. Outline •Rules Basics •Feeders Completeness •Performance •Rules vs. TI. 1 Rule Basics TM1 Calculations

1.4 Feeders Basics - Case

Q. What if Y uses values from X, and the time dimension in X (Month) goes down to months, but the one in Y (Year) only down to years?

A. The months in X must feed the corresponding years in Y. However, you cannot write !Year on the right had side. You must use ELPAR or preferably an attribute that maps month to year.

Page 18: Advanced Rules & Feeders - OLAPLINE€¦ · Advanced Rules & Feeders. Outline •Rules Basics •Feeders Completeness •Performance •Rules vs. TI. 1 Rule Basics TM1 Calculations

2 Feeders Completeness

Locating Missing Feeders

String-Driven Feeders

Page 19: Advanced Rules & Feeders - OLAPLINE€¦ · Advanced Rules & Feeders. Outline •Rules Basics •Feeders Completeness •Performance •Rules vs. TI. 1 Rule Basics TM1 Calculations

2.1 Locating Missing Feeders

Using Rules Tracer

Needle in a haystack. Must examine every cell.

Judiciously look at subareas.

Page 20: Advanced Rules & Feeders - OLAPLINE€¦ · Advanced Rules & Feeders. Outline •Rules Basics •Feeders Completeness •Performance •Rules vs. TI. 1 Rule Basics TM1 Calculations

2.2 String Driven Feeders

Strings parameters are often used to direct calculations an feeders.

Care must be used to define feeders so they re-trigger when the string parameters are changed.

Page 21: Advanced Rules & Feeders - OLAPLINE€¦ · Advanced Rules & Feeders. Outline •Rules Basics •Feeders Completeness •Performance •Rules vs. TI. 1 Rule Basics TM1 Calculations

2.2 String Driven Feeders – A common trap

Feeders are often written in such a way that the one of the elements in the target (written using a db() expression) are specified by a string stored in some cube.TriggerRange => DB( …, DB(stringcubecell, ...), …)

The problem with this is that if the string in question is changed, the feeder will not re-fire because the cells in TriggerRange are already populated. And the consolidations will be WRONG.

To correct the situation, the trigger of the feeder must use the string in question which, as mentioned before, will fire whenever the string is changed.

Page 22: Advanced Rules & Feeders - OLAPLINE€¦ · Advanced Rules & Feeders. Outline •Rules Basics •Feeders Completeness •Performance •Rules vs. TI. 1 Rule Basics TM1 Calculations

3. Performance

Missing SKIPCHEKs

Overfeeding

Other

Page 23: Advanced Rules & Feeders - OLAPLINE€¦ · Advanced Rules & Feeders. Outline •Rules Basics •Feeders Completeness •Performance •Rules vs. TI. 1 Rule Basics TM1 Calculations

3.1 Missing Skipchecks

Symptom: Very slow performance

Approach:

Check all rule files and put them in.

Put in appropriate feeders

An empty rule will cause sparse consolidations to be turned off and slow performance

Page 24: Advanced Rules & Feeders - OLAPLINE€¦ · Advanced Rules & Feeders. Outline •Rules Basics •Feeders Completeness •Performance •Rules vs. TI. 1 Rule Basics TM1 Calculations

3.2 Overfeeding• Why worry about overfeeding?

• Locating Overfeeds

• Conditional Feeders

Page 25: Advanced Rules & Feeders - OLAPLINE€¦ · Advanced Rules & Feeders. Outline •Rules Basics •Feeders Completeness •Performance •Rules vs. TI. 1 Rule Basics TM1 Calculations

3.2.1 Why worry about overfeeding?

• Slow cube loading.

• Unnecessary memory consumption.

• Slow Performance.

Which is the most significant?

Page 26: Advanced Rules & Feeders - OLAPLINE€¦ · Advanced Rules & Feeders. Outline •Rules Basics •Feeders Completeness •Performance •Rules vs. TI. 1 Rule Basics TM1 Calculations

3.2.2 Technique for locating Overfeeds

Create cube OVERFEED with the same dimensions as subject cube.

Add rule to OVERFEED as follows:SKIPCHECK;

[]=N:IF(DB(’01Basic’,!dim1 , !dim2, ...)=0,1,0);

Add feeder to subject cube as follows:[]=>DB(‘OVERFEED’, !dim1 , !dim2, ...);

Page 27: Advanced Rules & Feeders - OLAPLINE€¦ · Advanced Rules & Feeders. Outline •Rules Basics •Feeders Completeness •Performance •Rules vs. TI. 1 Rule Basics TM1 Calculations

3.2.3.1 Conditional Feeders Basics

Purpose - to minimize number of “overfeeds”.

Trick: Write if expression for the target cube name that yields a blank when feeders are NOT to take place.

Classic example:

A = B * C

B feeds A if C is nonzero

C feeds A if B is nonzero

Page 28: Advanced Rules & Feeders - OLAPLINE€¦ · Advanced Rules & Feeders. Outline •Rules Basics •Feeders Completeness •Performance •Rules vs. TI. 1 Rule Basics TM1 Calculations

3.2.3.2 Different Dimensionalities

= *

Conditional

AxB A B

Baux

Page 29: Advanced Rules & Feeders - OLAPLINE€¦ · Advanced Rules & Feeders. Outline •Rules Basics •Feeders Completeness •Performance •Rules vs. TI. 1 Rule Basics TM1 Calculations

3.2.3.2 Different Dimensionalities - ExampleTo Calculate:

Commission( Product, SalesPerson, Time) = Sales( Product, SalesPerson, Time) * CommissionRate( Product, Time)

Create CommissionRateAux(Product, SalesPerson, Time)

In CommissionRate:[] => DB(‘CommissionRateAux’, !Product, ‘All Salespersons’, !Time);

In CommissionRateAux:[] => DB( IF( DB(‘Sales’, !Product, !SalesPerson, !Time) = 0, ‘’,‘Commission’), !Product, !SalesPerson, !Time);

In Sales:[] => DB( IF( DB(‘CommissionRate’, !Product, !Time) = 0. ‘’,‘Commission’), !Product, !SalesPerson, !Time);

Page 30: Advanced Rules & Feeders - OLAPLINE€¦ · Advanced Rules & Feeders. Outline •Rules Basics •Feeders Completeness •Performance •Rules vs. TI. 1 Rule Basics TM1 Calculations

3.3 Other Methods

• Avoiding Long Expressions

• Avoiding Ratio Feeders

• Avoiding Matrix Multiplication Feeders

• Freezing Rules

• Miscellaneous

Page 31: Advanced Rules & Feeders - OLAPLINE€¦ · Advanced Rules & Feeders. Outline •Rules Basics •Feeders Completeness •Performance •Rules vs. TI. 1 Rule Basics TM1 Calculations

3.3.1 Avoiding Long Expressions

Complex expression (with identical data) used for multiple cells - Calculation is done over and over.

Approach: “Factor out” by defining calculation as cells in auxiliary cube.

Page 32: Advanced Rules & Feeders - OLAPLINE€¦ · Advanced Rules & Feeders. Outline •Rules Basics •Feeders Completeness •Performance •Rules vs. TI. 1 Rule Basics TM1 Calculations

3.3.2 Avoiding Ratio FeedersRatio calculations such as:

[‘Sales Percent of Year'] = ['Sales']\['Year','Sales']*100;

apply at all levels of hierarchy. I.e., rule is not preceded by N: or C:

Therefore no feeders are needed for consolidations because leaf values are never consolidated. Problem is, how to support zero suppression.

Answer: Define “Sales Percent of Year” as a consolidation of “Sales”. This effectively acts as a feeder from “Sales”, but costs no memory.

Page 33: Advanced Rules & Feeders - OLAPLINE€¦ · Advanced Rules & Feeders. Outline •Rules Basics •Feeders Completeness •Performance •Rules vs. TI. 1 Rule Basics TM1 Calculations

3.3.3 Avoiding Matrix Multiplication Feeders

= *AxB A B

d2

d1d3

d2

d3

d1

d2

Page 34: Advanced Rules & Feeders - OLAPLINE€¦ · Advanced Rules & Feeders. Outline •Rules Basics •Feeders Completeness •Performance •Rules vs. TI. 1 Rule Basics TM1 Calculations

3.3.3 Case– “Multicurrency” Company

Large reporting cube

Need all data in 12 currencies

Long time to load

Hardware memory limits exceeded

Client not happy

Page 35: Advanced Rules & Feeders - OLAPLINE€¦ · Advanced Rules & Feeders. Outline •Rules Basics •Feeders Completeness •Performance •Rules vs. TI. 1 Rule Basics TM1 Calculations

3.3.3 Approach

Currency rate varies only with TIME - not with other dimensions.

Can express currency rule for both “C” and “N” cells.

All consolidations would be right except TIME.

Express TIME consolidation using rules (ConsolidateChildren)

No feeders needed.

But need to express calculated currencies as consolidation of base currency in order to support zero suppression.

Page 36: Advanced Rules & Feeders - OLAPLINE€¦ · Advanced Rules & Feeders. Outline •Rules Basics •Feeders Completeness •Performance •Rules vs. TI. 1 Rule Basics TM1 Calculations

3.3.4 Freezing Rules

One way to improve performance of a TM1 application is to “freeze” rule calculated values for periods or versions that are not subject to change. Typically history or frozen scenarios.

This way, the cells thus frozen need not be recalculated when other values in the same cube are updated.

Page 37: Advanced Rules & Feeders - OLAPLINE€¦ · Advanced Rules & Feeders. Outline •Rules Basics •Feeders Completeness •Performance •Rules vs. TI. 1 Rule Basics TM1 Calculations

3.3.4 Freezing Rules - Method

User selects a cube, a dimension within the cube, and leaf members in the dimension to be frozen.

The leaf values for those members are extracted and output to an ASCII file, including leaf values calculated by rules.

The rules that apply to leaf cells are deactivated for the members frozen.

The values that were output to the ASCII file are re- imported into the cube.

Page 38: Advanced Rules & Feeders - OLAPLINE€¦ · Advanced Rules & Feeders. Outline •Rules Basics •Feeders Completeness •Performance •Rules vs. TI. 1 Rule Basics TM1 Calculations

3.3.4 Freezing Rules - Continued

In order to disable the rules for the frozen members automatically, rather than editing the rule file, do as follows:

Put disabled members in a parallel dimension.

Put rules that do not apply only to leaf cells at the beginning.

Follow it by a conditional expression of the form:

[] = IF( DIMIX(‘Dimx_Freeze', !Dimx) =0, CONTINUE, STET);

One for each dimension with frozen members.

Page 39: Advanced Rules & Feeders - OLAPLINE€¦ · Advanced Rules & Feeders. Outline •Rules Basics •Feeders Completeness •Performance •Rules vs. TI. 1 Rule Basics TM1 Calculations

3.3.5 Miscellaneous

Inadequate optimization of consolidations in rules (rare cases).

Rules not taking full advantage of “Stargate”.

Can easily make performance worse by a factor of 10.

Report to support / development.

Page 40: Advanced Rules & Feeders - OLAPLINE€¦ · Advanced Rules & Feeders. Outline •Rules Basics •Feeders Completeness •Performance •Rules vs. TI. 1 Rule Basics TM1 Calculations

Rules TIEase of writing / maintaining + Declarative expression more

intuitive for user.- Feeders a challenge for all.- Limited ability for complex calculations.

+ Procedural calculations more familiar to developers.- Requires additional setup to trigger calculations.+ Able to handle iterative calculations.

Error proneness -Locating incomplete feeders-Value-directed feeders

- User must remember to trigger.+ Procedural calculations easier to test and fix.

Performance + Only necessary cells are calculated- Invalidated when values change.O Calculation speed comparable

- Must calculate all cells. No trigger based on change.O Calculation speed comparable

Dynamism + Dynamic - always up to date - Not Dynamic

Memory O No significant difference O No significant difference

4 Rules vs. TI

Page 41: Advanced Rules & Feeders - OLAPLINE€¦ · Advanced Rules & Feeders. Outline •Rules Basics •Feeders Completeness •Performance •Rules vs. TI. 1 Rule Basics TM1 Calculations

Thank You

Advanced Rules and Feeders