Improving Error Discovery using Guided Search

Improving Error Discovery using Guided Search

Neha Rungta & Eric MercerComputer Science Department

Brigham Young University,

Provo UT

Verification and Validation, CS Dept, BYU 2

Software Model Checking Motivation

Ariane 5 Comair debacle

Verifying Software Models A transition graph for the model is created A predefined property is verified ex. Reachability

Problem Number of behaviors is exponential with every increment This causes a state explosion problem


Approaches Traditional techniques to counter it

Parallel or Distributed Model Checking Predicate Abstraction Disk based Algorithm Heuristics for Guided search

Heuristics Find a counterexample before memory runs out Property based heuristics Structure based heuristics

Structure of program can be use to guide the search


Current Structural heuristics Stefan Edelkamp and Tilman Mehler Finds a short and easy to understand Error trail Minimal operations to reach g from s is FSM distance This distance is admissible and consistent Build control flow graph (CFG) with just PC values

Willem Visser and Alex Groce Specific only to Java


01mainmain:01: ldx #102: call foo03: add x,104: call foo05: check for errorfoo:06: pshx07: pulx08: rts

Underestimation Example


01

02

main


main:01: ldx #102: call foo03: add x,104: call foo05: check for errorfoo:06: pshx07: pulx08: rts


01

02

main

foo


06



01

02

03

04

05

main

foo

error


06

07

08



error


01

02

03

04

05

main

foo

error

06

07

08


Edelkamp’s FSM Heuristic:Shortest Distance from current state to checking for error in the CFG


error


01

02

03

04

05

main

foo

error

06

07

08




error


01

02

03

04

05

main

foo

error

06

07

08







error

01

02

03

04

05

main

foo

error

06

07

08



foo

3 steps



error

01

02

03

04

05

main

foo

error

06

07

08


True Distance should be ….

error

01

02

03

04

05

main

foo

error

06

07

08



errorerror

01

02

03

04

05

main

foo

error

06

07

08




errorerror

01

02

03

04

05

main

foo

error

06

07

08




errorerror

01

02

03

04

05

main

foo

error

06

07

08






errorerror

01

02

03

04

05

main

foo

error

06

07

08




errorerror

01

02

03

04

05

main

foo

error

06

07

08




errorerror

01

02

03

04

05

main

foo

error

06

07

08


8 steps



errorerror

01

02

03

04

05

main

foo

error

06

07

08


Solution: Interprocedural CFG

All the nodes in the ICFG that are part of a subroutine will be indexed on two things PC Value Return address to where the subroutine will return

when it encounters a return statement


01(init)



01(init)

02(init)



01(init)

02(init)

06(03)



01(init)

02(init)

06(03)



01(init)

02(init)

03(init)

06(03)

07(03)

08(03)



01(init)

02(init)

03(init)

06(03)

07(03)

08(03)

06(05)

07(05)

08(05)

04(init)

05(init)



01(init)

02(init)

03(init)

06(03)

07(03)

08(03)

06(05)

07(05)

08(05)

04(init)

05(init)



01(init)

02(init)

03(init)

06(03)

07(03)

08(03)

06(05)

07(05)

08(05)

04(init)

05(init)


8 steps


Nested Function Calls x → f → g y → f → g Same problem as before

main:1 call x2 call yerror

f:7 call g8 rts

g:9 xyza rts

x:3 call f4 rts

y:5 call f6 rts

f gx

1:call x(init)

3:call f(2)

7:call g(4)

9(8)

2:call y(init)

5:call g(error)

7:call g(6)

error

mainx f g

y f

a:rts(8)

8:rts(4)

4:rts(2)

8:rts(6)

6:rts(error)


Improved ICFG Algorithm

ICFG_Algorithm(state S)sa0 = icfgState(S)<sa1,sa2,….,san> // are abstracted states in the call stack of Sd = 0for i = 0 to n do srtn = rtn(sai) if FSM(sai, error) < FSM(sai,srtn) then d = d + FSM(sai,error) return d d = d + FSM(sai, srtn) + 1return d





Abstract states from the stack

08

04

02

PC: 09 0a(08)

08(04)

04(02)

02(init)

abstract statesgenerated fromthe stack

sa0

sa1

sa2

sa3











Marking returns statically

foo prologue

beq

epilogue return






Calculating the Heuristic:D = 0FSM ((a,8),error) = 4

f gx

1:call x(init)

3:call f(2)

7:call g(4)

9(8)

2:call y(init)

5:call g(error)

7:call g(6)

error

mainx f g

y f

a:rts(8)

8:rts(4)

4:rts(2)

8:rts(6)

6:rts(error)

08

04

02

PC: 09



08

04

02

PC: 09

Calculating the Heuristic:D = 0FSM ((a,8),error) = 4FSM ((a,8), (rts,8) = 11 < 4D += 1

x f g

1:call x(init)

3:call f(2)

7:call g(4)

9(8)

2:call y(init)

5:call g(error)

7:call g(6)

error

main

y f

a:rts(8)

8:rts(4)

4:rts(2)

8:rts(6)

6:rts(error)





Improved ICFG AlgorithmPC: 0a

D = 11

1:call x(init)

3:call f(2)

7:call g(4)

9(8)

2:call y(init)

5:call g(error)

7:call g(6)

error

main

y f

a:rts(8)

8:rts(4)

4:rts(2)

8:rts(6)

6:rts(error)

x f g


Results: Number of states generated

BFS DFS FSM Improved ICFG

Hyman’s mutex 4528 7006 3648 1560Naïve dining phil (threads) 47,246 8062 152,196 14,140Moody dining phil (threads) 225,269 44,238 555,609 28,565Lazy dining phil (threads) 317,131 56,685 >2.86 mil 50,984Bulls and cows 27,613 28,014 28,014 28,007


Conclusions Small overhead allowed use of more static information The Dynamic call stack with static analysis gave a better

estimate Testing shows an significant improvement in FSM distance The Improved ICFG algorithm can be used on any graph The algorithm is admissible and consistent


QUESTIONS

Improving Error Discovery using Guided Search

Documents

cs dept

shortest distance

rtsedelkamps fsm heuristic

current state

cfg verification

fsm distancethis distance

error discovery

error trailminimal operations