Orthogonal Orthogonal ArraysArrays TestingTesting StrategyStrategy(OATS)(OATS)
Mohamed FathyMohamed FathyQA Engineer
Arx ICTSeptember 4, 2009
AgendaAgenda
• OATS Overview.• How to use Orthogonal Arrays?• OATS Examples.• Case Study.• Where OATS could be applied?• OATS Benefits.• References & Resources.• Questions?
OATS Overview
• Taguchi's orthogonal arrays are highly fractional orthogonal designs proposed by Dr. Genichi Taguchi.
• OATS is one such combinatorial testing method.
• OATS is a systematic, statistical way of testing pair-wise interactions.
OATS Overview (Cont’d)
• It addresses generation of test cases for problems involving multiple parameters and combinations.
• It can be used to reduce the number of combinations and provide maximum coverage with a minimum number of test cases.
How to use Orthogonal Arrays?
Lruns(Levels^Factors) or Ln(s^k): Orthogonal array of N runs where k factors take on any value from a set of s symbols/level.
• Runs: The number of rows or number of test cases in the array that will be
generated by Orthogonal Array technique. Each row represents a test case.
• Factors: The number of columns or the number of parameters/variables in an array that need to be tested in the System.
• Levels: The maximum number of values in an Orthogonal Array that can be taken on by any single factor.
OATS Examples
Example 1
OATS Examples (Cont’d)
The borders and shading dialog box :• 5 settings• 5 styles (showing)• 9 colors• 9 widths
5 x 5 x 9 x 9 = 2025 combinations!L81(94) array will suffice
OATS Examples (Cont’d)
Example 2
OATS Examples (Cont’d)
• 3 toolbars (pictures, text, or both).• 3 choices for launch on startup (browser, mail or news).• 3 choices for startup page (blank home page, home page
names a valid file, or home page name has a syntax error).• 2 choices for underlining of links (don’t underline, or
underline).• 2 choices for expiration of links (never expire, expire after 30
days).
3 x 3 x 3 x 2 x 2 = 108 combinations!
OATS Examples (Cont’d)
OATS Examples (Cont’d)
OATS Examples (Cont’d)
OATS Examples (Cont’d)
• Example 3:
Factors Levels
terminal_capability 1 - 2 - 3
pos_entry_mode SWIPED - KEYED
card_present Yes - No
card_type Visa – MC
OATS Examples (Cont’d)
• Orthogonal array to be used: L9000001120221101111201202202221012210
Case Study
Optimization
• Remove invalid runs.• Optimization need not necessarily mean the
reduction of tests. You can add some combinations that are critical for testing, but missed by OATS.
Where OATS could be applied?
OATS could be applied in: User interface testing. System testing. Regression testing. Configuration testing. Performance testing. This method is extremely valuable for testing complex
applications. Useful for testing products with a large number of
configuration possibilities.
Benefits of OATS
• Provides uniformly distributed coverage of the test domain.
• Concise test set with fewer test cases is created.
• All pair-wise comibnations of test set created.• Arrives at complex combinations of all the
variables.
Benefits of OATS (Cont’d)
• Simpler to generate and less error prone than test sets created manually.
• Reduces testing cycle time.
References & Resources
• ReferencesReferences::1- R Krishnan, S Murali Krishna, P Siva Nandhan - Motorola:
Combinatorial Testing: Learnings from our Experience .2- Madhav S. Phadke , Quality Engineering Using Robust Design.
• Resources:Resources:1- http://www.research.att.com/~njas/oadir/index.html2- http://www.mathdox.com/nguyen/orthogonal_array.jsp
