Finding Incremental Solutions for Evolving Requirements

Finding Incremental Solutions for Evolving Requirements

Neil  Ernst,  Alexander  Borgida,  Ivan  Jureta

[email protected]  -­‐  [email protected]  -­‐  [email protected]

1

The problem

• Changing  requirements  are  costly  and  a  major  source  of  software  errors.  

• Requirements  drift  from  implementation.  • Lack  of  tool  support  for  requirements  evolution.• If  we  don't  know  what,  or  more  importantly,  

why  we  are  doing  something,  "how"  we  do  it  is  inconsequential.  

2

The solution

• Focus  on  the  case  of  evolving  requirements  due  to  unanticipated  change.

• Introduce  a  simple  propositional  formalism  on  a  case  study.

• Define  solutions  to  requirements  problems.• Define  the  requirements  evolution  problem.• Use  a  functional  interface  to  determine  what  

changes  to  make  and  why.• Evaluate  the  performance  of  solution-­‐finding.

3

Requirements problems:Goals, tasks, and assumptions• Requirements  describe  stakeholder  desires  for  the  new  

system  (e.g.,  “protect  cardholder  data”).• These  desired  states  we  call  goals.• Goals  are  iteratively  refined  until  operationalized  by  an  

implementation  task.• A  goal  model  defines  a  space  of  alternative  designs  for  

satisfying  goals,  constrained  by  domain  assumptions.

4

The  requirements  problem:  given  a  set  of  goals,  which  tasks  and  assumptions  satisfy  those  goals?1

[1] [1] P. Zave and M. Jackson, “Four Dark Corners of Requirements Engineering,” TOSEM, vol. 6, pp. 1-30, 1997.

PCI Data Security Standard (PCI-DSS)

1. Build  and  Maintain  a  Secure  Network  2. Protect  Cardholder  Data3. Maintain  a  Vulnerability  Management  Program4. Implement  Strong  Access  Control  Measures5. Regularly  Monitor  and  Test  Networks6. Maintain  an  Information  Security  Policy

5

Increase revenues

Accept credit card

Avoid financial losses and penalties

Be PCI compliant

No money for new servers

Implement only one primary function per

server

Virtualize server

instances

Use Verifone POS

Use multiple servers

Use Moneris POS

Accept payment

Accept cash

Buy strongbox

PCI-DSS model

6

Increase revenues

Accept credit card

Avoid financial losses and penalties

Be PCI compliant

No money for new servers

Implement only one primary function per

server

Virtualize server

instances

Use Verifone POS

Use multiple servers

Use Moneris POS

Accept payment

Accept cash

Buy strongbox

PCI-DSS model

6

Increase revenues

Accept credit card

Avoid financial losses and penalties

Be PCI compliant

No money for new servers

Implement only one primary function per

server

Virtualize server

instances

Use Verifone POS

Use multiple servers

Use Moneris POS

Accept payment

Accept cash

Buy strongbox

PCI-DSS model

6

Increase revenues

Accept credit card

Avoid financial losses and penalties

Be PCI compliant

No money for new servers

Implement only one primary function per

server

Virtualize server

instances

Use Verifone POS

Use multiple servers

Use Moneris POS

Accept payment

Accept cash

Buy strongbox

PCI-DSS model

6

Goal

Increase revenues

Accept credit card

Avoid financial losses and penalties

Be PCI compliant

No money for new servers

Implement only one primary function per

server

Virtualize server

instances

Use Verifone POS

Use multiple servers

Use Moneris POS

Accept payment

Accept cash

Buy strongbox

PCI-DSS model

6

Refinement

Increase revenues

Accept credit card

Avoid financial losses and penalties

Be PCI compliant

No money for new servers

Implement only one primary function per

server

Virtualize server

instances

Use Verifone POS

Use multiple servers

Use Moneris POS

Accept payment

Accept cash

Buy strongbox

PCI-DSS model

6

Task

Increase revenues

Accept credit card

Avoid financial losses and penalties

Be PCI compliant

No money for new servers

Implement only one primary function per

server

Virtualize server

instances

Use Verifone POS

Use multiple servers

Use Moneris POS

Accept payment

Accept cash

Buy strongbox

PCI-DSS model

6

Domain assumption

Increase revenues

Accept credit card

Avoid financial losses and penalties

Be PCI compliant

No money for new servers

Implement only one primary function per

server

Virtualize server

instances

Use Verifone POS

Use multiple servers

Use Moneris POS

Accept payment

Accept cash

Buy strongbox

PCI-DSS model

6

Alternatives

Increase revenues

Accept credit card

Avoid financial losses and penalties

Be PCI compliant

No money for new servers

Implement only one primary function per

server

Virtualize server

instances

Use Verifone POS

Use multiple servers

Use Moneris POS

Accept payment

Accept cash

Buy strongbox

PCI-DSS model

6

Conflict

Formalizing the requirements problem

• D,  S  ⊢  G1

• A  flavour  of  the  Techne  RE  language.Concepts• Goals  G• Domain  assumptions  D• Sets  of  implementation  tasks  S

Relations• implication  (Horn  clause)  • conflict  (A,  B,  A  ⋀  B  →⊥)

7

[1] Jureta, Borgida, Ernst, Mylopoulos, “Techne: Towards a New Generation of Requirements Modeling Languages with Goals, Preferences, and Inconsistency Handling.” RE2010, 115-124.

The REKB

• Store  requirements  problem  elements  as  logical  sentences.• i.e.,  goals,  tasks,  domain  assumptions  and  relations

• Define  a  TELL/ASK  interface  for  solving  requirements  problems.

• TELL:  atoms,  sentences.• UNTELL:  asserted  atoms.• ASK:  solutions  to  requirements  problem  in  REKB.

8

G1 G2

G

T3T2T1

Standard ASK questions

1. Are  goals  achieved  from  tasks?  (linear  time  complexity)

2. Find  minimal  sets  of  tasks  that  achieve  goals    (NP-­‐hard  complexity)

9

G1 G2

G

T3T2T1

Standard ASK questions

1. Are  goals  achieved  from  tasks?  (linear  time  complexity)

2. Find  minimal  sets  of  tasks  that  achieve  goals    (NP-­‐hard  complexity)

9

G1 G2

G

T3T2T1

Standard ASK questions

1. Are  goals  achieved  from  tasks?  (linear  time  complexity)

2. Find  minimal  sets  of  tasks  that  achieve  goals    (NP-­‐hard  complexity)

9

X

G1 G2

G

T3T2T1

Standard ASK questions

1. Are  goals  achieved  from  tasks?  (linear  time  complexity)

2. Find  minimal  sets  of  tasks  that  achieve  goals    (NP-­‐hard  complexity)

9

G1 G2

G

T3T2T1

Standard ASK questions

1. Are  goals  achieved  from  tasks?  (linear  time  complexity)

2. Find  minimal  sets  of  tasks  that  achieve  goals    (NP-­‐hard  complexity)

9

Smin = {T1, T3} {T2, T3}

Increase revenues

Accept credit card

Avoid financial losses and penalties

Be PCI compliant

No money for new servers

Implement only one primary function per

server

Virtualize server

instances

Use Verifone POS

Use multiple servers

Use Moneris POS

Accept payment

Accept cash

Buy strongbox

10

Increase revenues

Accept credit card

Avoid financial losses and penalties

Be PCI compliant

No money for new servers

Implement only one primary function per

server

Virtualize server

instances

Use Verifone POS

Use multiple servers

Use Moneris POS

Accept payment

Accept cash

Buy strongbox

10

Increase revenues

Accept credit card

Avoid financial losses and penalties

Be PCI compliant

No money for new servers

Implement only one primary function per

server

Virtualize server

instances

Use Verifone POS

Use multiple servers

Use Moneris POS

Accept payment

Accept cash

Buy strongbox

10

The requirements evolution problem

• Given  an  existing  solution  Si  which  satisfiesD,  Si  ⊢  G,  and

• modified  entities  (δ(G),  δ(D),  δ(S));• Find  Ŝ  so  that  δ(D),  Ŝ  ⊢  δ(G),  such  that  this  

satisfies  some  desired  property  π,  relating  Ŝ  to  Si.

11

Increase revenues

Accept credit card

Avoid financial losses and penalties

Be PCI compliant

No money for new servers

Implement only one primary function per

server

Virtualize server

instances

Use Verifone POS

Use multiple servers

Use Moneris POS

Accept payment

Accept cash

Buy strongbox

12

Increase revenues

Accept credit card

Avoid financial losses and penalties

Be PCI compliant

No money for new servers

Implement only one primary function per

server

Virtualize server

instances

Use Verifone POS

Use multiple servers

Use Moneris POS

Accept payment

Accept cash

Buy strongbox

12

Si

Increase revenues

Accept credit card

Avoid financial losses and penalties

Be PCI compliant

No money for new servers

Implement only one primary function per

server

Virtualize server

instances

Use Verifone POS

Use multiple servers

Use Moneris POS

Accept payment

Accept cash

Buy strongbox

Use Secure Hash on CC #

12

Si

Increase revenues

Accept credit card

Avoid financial losses and penalties

Be PCI compliant

No money for new servers

Implement only one primary function per

server

Virtualize server

instances

Use Verifone POS

Use multiple servers

Use Moneris POS

Accept payment

Accept cash

Buy strongbox

Use Secure Hash on CC #

12

Si

New Requirement

Increase revenues

Accept credit card

Avoid financial losses and penalties

Be PCI compliant

No money for new servers

Implement only one primary function per

server

Virtualize server

instances

Use Verifone POS

Use multiple servers

Use Moneris POS

Accept payment

Accept cash

Buy strongbox

Use Secure Hash on CC #

12

New Requirement

Increase revenues

Accept credit card

Avoid financial losses and penalties

Be PCI compliant

No money for new servers

Implement only one primary function per

server

Virtualize server

instances

Use Verifone POS

Use multiple servers

Use Moneris POS

Accept payment

Accept cash

Buy strongbox

Use Secure Hash on CC #

12

Ŝ New Requirement

Useful properties π

1. Minimal  implementation  effort.2. Minimal  change  effort  solutions.3. Maximal  familiarity  solutions.4. Solution  reuse  in  family  of  solutions.

13

ed

cbahgf fd

ca

gd

cba

SaSi Sb Sc

Useful properties π

1. Minimal  implementation  effort.2. Minimal  change  effort  solutions.3. Maximal  familiarity  solutions.4. Solution  reuse  in  family  of  solutions.

13

ed

cbahgf fd

ca

gd

cba

SaSi Sb Sc

Useful properties π

1. Minimal  implementation  effort.2. Minimal  change  effort  solutions.3. Maximal  familiarity  solutions.4. Solution  reuse  in  family  of  solutions.

13

ed

cbahgf fd

ca

gd

cba

SaSi Sb Sc

Useful properties π

1. Minimal  implementation  effort.2. Minimal  change  effort  solutions.3. Maximal  familiarity  solutions.4. Solution  reuse  in  family  of  solutions.

13

ed

cbahgf fd

ca

gd

cba

SaSi Sb Sc

Useful properties π

1. Minimal  implementation  effort.2. Minimal  change  effort  solutions.3. Maximal  familiarity  solutions.4. Solution  reuse  in  family  of  solutions.

13

ed

cbahgf fd

ca

gd

cba

SaSi Sb Sc

Useful properties π

1. Minimal  implementation  effort.2. Minimal  change  effort  solutions.3. Maximal  familiarity  solutions.4. Solution  reuse  in  family  of  solutions.

13

ed

cbahgf fd

ca

gd

cba

SaSi Sb Sc

Useful properties π

1. Minimal  implementation  effort.2. Minimal  change  effort  solutions.3. Maximal  familiarity  solutions.4. Solution  reuse  in  family  of  solutions.

13

ed

cbahgf fd

ca

gd

cba

SaSi Sb Sc

Implementing the REKB

• REKB  is  a  functional  description.• Leverages  an  Assumption-­‐based  Truth  

Maintenance  System1  (ATMS).  • Incremental,  minimal,  conflict-­‐tolerant.

• Other  options:  SAT  solvers,  SMT,  PsB.• Problem  solver  tracks  implemented  tasks,  

versions  elements,  retracts  atoms.

14[1] J. de Kleer, “An assumption-based TMS,” Artificial Intelligence, 28, 1986, pp. 127-162.

Evaluating the REKB

• Evaluate  the  REKB  approach  on• the  PCI-­‐DSS  • random  requirements  

models

15

6000 50 100 150 200 250 300 350 400 450 500 550

100

0.01

0.1

1

10

# Nodes

Tim

e (s

)

Evaluating the REKB (2)

• Test  incremental  performance  on  three  evolution  scenarios        (400  nodes)

16

Naive add

0.1

1

Tim

e (s)

High%level

Conflict

New%task

Incremental add

Conclusions

• The  REKB  is  a  framework  for  managing  goals,  tasks,  and  domain  assumptions.

• Find  incremental  solutions,  and  find  optimal  solutions.FIX

• Solving  requirements  problems  provide  the  “why”  for  maintenance  activity.

17

Thanks!

18

http://neilernst.net@neilernst

github.com/neilernst