TSC Workshop, May 2008, U Oklahoma 1 Teaching Software Correctness May 13-15, 2008, University of Oklahoma Rex Page, U Oklahomapage@ou.edu Assistants Carl.
Post on 27-Dec-2015
213 Views
Preview:
Transcript
TSC Workshop, May 2008, U Oklahoma 1
Teaching Software CorrectnessMay 13-15, 2008, University of
Oklahoma
Rex Page, U Oklahoma page@ou.edu
AssistantsCarl Eastlund (lead), Northeastern U cce@ccs.neu.eduRyan Ralston, U Oklahomastrawdog@ou.eduZac White, U Oklahomazacwhite@gmail.com
http://www.cs.ou.edu/~rlpage/SEcollab/tsc
1
Collaboration with Matthias Felleisen - NSF/DUE 0633664, 0813529, 0632872
Session 01 — 9:00-10:15, May 13
TSC Workshop, May 2008, U Oklahoma 2
Goals New CS-course content at some schools
Predicate-based testing Mechanical logic to verify software properties
Strate
g
y
Go through material from SE at OU 30% of 3-credit course (4 of 15 lecture-
weeks) Coverage of coherent subsets: 1 to 3 weeks
Lab projects To gain a student-like experience
Hard
part Convincing yourself that it's important
Material is unusual … many students will resist
Instructor zeal transfers to some students Instructor apathy transfers to all students
Devising projects that students can do Instructors must solve assignments
beforehand
Benefits New appreciation meaning of "correctness"
Prep for future programming environments
A few students ready to lead next decade
TSC Workshop, May 2008, U Oklahoma 3
History
Years of rumination about software
education• defect control• equation-based programs
2003 SE course• Programs: Scheme (DrScheme)• Verify correctness: ACL2
2004 SE course• Program/Verify: ACL2• File I/O only
2005 SE course• ACL2, File I/O• ESC paper, FDPE'05
2006 SE course• DrACuLA prog'g environment• File I/O + GUI
2007 SE course• DrACuLa prog's env• with DoubleCheck
2008 plans• DrACuLa + DblChk• + Modules
TSC Workshop, May 2008, U Oklahoma 44
Low defect rate
Process
Design
Quality
Predictable quality Reliable schedule Do it right
not necessarily fast
Primarily for people Think “Strunk and White”
The Three Themes of SE-I
PSP - Humphrey
Equation-Based Programming
ACL2 – J MooreLisp with mechanical logic
Besides a mathematical inclination, an exceptionally good mastery of one's native tongue is the most vital asset of a competent programmer. — Dijkstra
TSC Workshop, May 2008, U Oklahoma 55
Provably Correct Software300 Years in the Making
If we had some exact language … or at least a kind of truly philosophic writing, in which the ideas were reduced to a kind of alphabet of human thought,
then all that follows rationally from what is given could be found by a kind of calculus, just as arithmetical or geometrical problems are solved.
- Gottfried Leibniz (about 1700)
Symbolic logic - Boole, Frege, Peano, … (mid to late 1800s)
Formal logic - Gentzen, Church, Curry, Gödel, … (1930s)
Mechanized logic - Boyer & Moore, Milner, … (late 1900s)
TSC Workshop, May 2008, U Oklahoma 6
Lisp programs Expressed as equations defining software properties Restricted to "true functions"
Every invocation of f(2) delivers the same value (unlike C, C++ …)
Computation model: substitution of equals for equals Programming model
Program specifies properties … interpretation derives processConventional: program specifies process … properties implicit
Derivable properties expressed as logic formulas Turing equivalence
All computations are expressible through equations Equation-based software (Lisp, ML, Haskell …) can
model conventional software (C, C++, Java, …) and vice versa
Lisp versus C Compiled Lisp slower … 10% to order of magnitude C development tools more advanced
… but DrACuLa and ACL2s are moving in the right direction
ACL2 – a mechanized logicACL2 = (ACL)2 = ACLACL
A Computational Logic for Applicative Common Lisp
TSC Workshop, May 2008, U Oklahoma 7
Example Software Propertyappend (here called “cat”) is
associative Defined properites (axioms)(defun cat (xs ys) (if (null xs) ys ; (cat nil ys) = ys
(cons (car xs) (cat (cdr xs) ys)))) ; (cat (list x1 x2 …) ys) = (cons x1 (cat (list x2 …) ys))
Derived property (theorem)(defthm append-is-associative (implies (and (true-listp xs)
(true-listp ys))(equal (cat xs (cat ys zs))
(cat (cat xs ys) zs)))))
axiom 1
axiom 2
theorem
Proof List-induction on xs
Base case: xs = nilInductive case: xs = (cons (car xs) (cdr xs))
ind
uct
ive
defin
itio
n
TSC Workshop, May 2008, U Oklahoma 8
Append is Associativeinductive case
(equal (cat xs (cat ys zs))(cat (cons (car xs) (cdr xs)) (cat ys zs)))
(equal (cat (cons (car xs) (cdr xs)) (cat ys zs)) (cons (car xs) (cat (cdr xs) (cat ys zs))))
(equal (cons (car xs) (cat (cdr xs) (cat ys zs)) ) (cons (car xs) (cat (cat (cdr xs) ys) zs)) )
(equal (cons (car xs) (cat (cat (cdr xs) ys) zs)) (cat (cons (car xs) (cat (cdr xs) ys)
zs))) ) (equal (cat (cons (car xs) (cat (cdr xs) ys) zs) )
(cat (cat (cons (car xs) (cdr xs)) ys) zs) )
(equal (cat (cat (cons (car xs) (cdr xs)) ys) zs) (cat (cat xs ys) zs) )
Induction hypothesis
xs = (cons (car xs) (cdr xs))(defun cat (xs ys) (if (null xs) ys ; = (cat nil ys) (cons (car xs) (cat (cdr xs) ys)))) ; = (cat xs ys)
TSC Workshop, May 2008, U Oklahoma 9
Mechanization Is Necessarywithout it, you get lost in details
Even simple properties lead to big proofs Associativity proof (as presented) lacks many details
Millions of details in proofs about big programsPeople can’t keep track of millions of detailsBut, computers canComputers couldn’t 10 years ago – not enough capacity
People formulate properties … computers push detailsProof organized into lemmas — like software in modulesOn the level of “rigorous” mathematical proof
– Not fully formalSome lemma architectures are better than others
– Just as some modular decompositions of software are superior
Formulation of properties is a big task– Experience and judgment required — as in software
development
TSC Workshop, May 2008, U Oklahoma 10
Things That Have Been Done with ACL2
AMD Athlon (K7) floating point processor (1999) AMD division circuit verified correct
After 1994/97 Pentium division bugs Property
(implies (and (floating-point-numberp p 15 64) (floating-point-numberp d 15 64)
(not (equal d 0)) (rounding-modep mode)) (equal (divide p d mode)
(round (/ p d) mode)))… Other commercial applications
Motorola DSP microcode Rockwell Collins AAMP7 avionics software Numerous NSA applications … “high assurance software” …
TSC Workshop, May 2008, U Oklahoma 11
Testing versus Proving
Testing AMD floating-point
AMD test suite of 80-million cases wasn’t enough– Hardware with bugs passed all 80-million tests
Huge finite number of cases (215+64 215+64 = 2158)– Cannot complete test before sun runs out of fuel
ACL2 proof covers all cases
Proofs cover all cases Associativity proof was for any xs, ys, zs
That is, an infinite number of “test cases”
Testing cannot ensure correctness Testing + logic can ensure that software
satisfies essential properties
TSC Workshop, May 2008, U Oklahoma 12
Applicative Common Lisp
the programming language part of ACL2
TSC Workshop, May 2008, U Oklahoma 13
Data in ACL2 Atoms
Numbers Integer, rational, complex (decimal, binary, octal, hex)
– 5, -12, 17/10, -17/10, #c(0, 1) , #b101 , #o-14 , #x11/A , #x-11/A No floating point numbers (no intrinsic sqrt function)
Characters: #\A, #\b, #\1, #\9, #\Newline, #\Space, … Strings: "Kauffmann, Manolios, & Moore #0-7923-7757-5" Symbols: nil, t, bob, next-term
Conses (dotted pairs, binary trees) (a . b) ((a . left) . (b . right)) (1 . nil), abbreviation: (1) (x .(y .(z . nil))), abbreviation: (x y z) nil, abbreviation: ( ) (x .(y .(z . g))), abbreviation: (x y z . g) ((x . 1) .((y . 2) . ((z . 3) . nil))), abbreviation ((x . 1) (y . 2) (z .
3)) Abbreviation rule
When a dot is followed by left paren, delete the dot, the left paren, and the matching right paren
When a dot is followed by a nil, delete both … or something like that
true lists: nil (element . true-list)
All ACL2 data structures come down to these basics
TSC Workshop, May 2008, U Oklahoma 14
Syntax of ACL2 - FormulasPrefix formulas onlyNo infix, No postfix
Suppressed evaluation Single-quote (apostrophe) prefixing formula suppresses
evaluation Known as a “quoted expression” (synonyms: “formula” =
“expression”) A quoted exp’n denotes the atom or dotted pair following the
quote Example: '((a b c) (d (e f) g) h) or: (quote ((a b c) (d (e f) g)
h)) This is the entire Lisp syntax
Well…okay…There are a few exceptions. Here’s one (others later):
Numbers, chars, and strings automatically denote data No single-quote mark needed for these atoms Nor for pre-defined symbols, such as nil and T
Formula for evaluation (op-symbol opnd1 opnd2 … opndn)
op-symbol must a symbol denoting an operator– Formula denotes value delivered by operator, with given
operands Operands may be formulas, defined symbols, or literals Each operator will require a specific number of operands
TSC Workshop, May 2008, U Oklahoma 15
Basic ACL2 Operators (intrinsics) Building pairs and extracting parts car, cdr, and cons operators – defining equations
(car (cons x ys)) = x(cdr (cons x ys)) = ys(car any-formula-denoting-nil) = nil(cdr any-formula-denoting-nil) = nil
x, ys — any formula
Selecting between formulas “if” operator – defining equations
(if any-formula-not-denoting-nil x y) = x(if any-formula-denoting-nil x y) = y
Detecting equal values “equal” operator
(equal x y) = T if x and y are formulas denoting the same value(equal x y) = nil if x and y are formulas denoting different values
Discriminating atoms, conses, and empty lists endp, null, atom, consp – defining equations
(null x) = (equal x nil)(consp (cons x y)) = T(atom (cons x y)) = nil(atom x) = (null (consp x))(endp xs) = (atom xs) – allowed only on true lists
x, y, xs — any formula
Thes
e op
erat
ors, p
lus a
way to
defi
ne n
ew
oper
ator
s, a
re suffi
cient
to d
escr
ibe
any
com
puta
tion
TSC Workshop, May 2008, U Oklahoma 16
Defining Operators in ACL2
Exercise Define an operator to deliver the 3rd and 4th elements Assume the operand is a list of at least four elements
abbreviation of (car (cdr xs))
defun – a “special form” (defun f (a1 a2 … an) v)
f, a symbol, names the operator being definedai, a symbol, stands for the ith operand (n may be zero)v, a formula, specifies the value the operator will deliver
abbreviation of (cons (car xs) (cons (cadr xs) nil))
After the defun, (f u1 u2 … un) denotes the formula v with each occurrence of ai replaced by the formula ui
defun has effect of attaching a name to an operator defun is a “command” – commands alter the current ACL2 world
– Ordinary formulas do not alter the world defun does not deliver a value
– Ordinary formulas do deliver values Examples
(defun drop-two (xs) (cdr (cdr xs))) (drop-two i’(a b c d e)) = (c d e) (drop-two (drop-two i’(a b c d e))) = (e)
(defun take-two (xs) (list (car xs) (cadr xs))) (take-two i’(a b c d e)) = (a b)
TSC Workshop, May 2008, U Oklahoma 17
Solution 1(defun 3rd&4th (xs) (list (caddr xs) (cadddr xs)))
Solution 2(defun 3rd&4th-another-way (xs) (take-two (drop-two xs)))
Predicate-based testing(defconst *num-tests* 10)(defproperty 3rd&4th=3rd&4th-another-way *num-tests* ((n (random-int-between 0 100) (integerp n)) (x (random-symbol) (symbolp x)) (xs (random-list-of (random-symbol) n) (symbol-listp
xs))) (equal (3rd&4th xs) (3rd&4th-another-way xs)))
Commands defconst defproperty
property name
randomvariable
random value for xs - list of up to n random syms
expected property of xs(predicate)
expected property to test for(predicate)
Solutions
number oftests to run
TSC Workshop, May 2008, U Oklahoma 18
Commands defconst defproperty
property name
randomvariable
random value for xs
expected property of xs(predicate)
expected property to test for(predicate)
Solution 1(defun 3rd&4th (xs) (list (caddr xs) (cadddr xs)))
Solution 2(defun 3rd&4th-another-way (xs) (take-two (drop-two xs)))
Predicate-based testing(defconst *num-tests* 10)(defproperty 3rd&4th=3rd&4th-another-way *num-tests* ((n (random-int-between 0 100) (integerp n)) (x (random-symbol) (symbolp x)) (xs (random-list-of (random-symbol) n) (symbol-listp
xs))) (equal (3rd&4th xs) (3rd&4th-another-way xs)))
Solutions
number oftests to run
Let's run a test
TSC Workshop, May 2008, U Oklahoma 19
Solution 1(defun 3rd&4th (xs) (list (caddr xs) (cadddr xs)))
Solution 2(defun 3rd&4th-another-way (xs) (take-two (drop-two xs)))
Predicate-based testing(defconst *num-tests* 10)(defproperty 3rd&4th=3rd&4th-another-way *num-tests* ((n (random-int-between 0 100) (integerp n)) (x (random-symbol) (symbolp x)) (xs (random-list-of (random-symbol) n) (symbol-listp
xs))) (equal (3rd&4th xs) (3rd&4th-another-way xs)))
Command defthm
Solutions
Theorem(defthm 3rd&4th=3rd&4th-another-way (equal (3rd&4th xs) (3rd&4th-another-way xs)))
May
be it'
s a
theo
rem
TSC Workshop, May 2008, U Oklahoma 20
More ExercisesLogical operators
Define an operator that delivers a value other than nil if neither operand denotes nil, and nil if both operands denote nil(&& formula-denoting-nil any-formula) = nil(&& any-formula formula-denoting-nil) = nil(&& non-nil-formula non-nil-formula) = non-nil-formula
Define an operator that delivers nil if both operands denote nil, and a value other than nil if either operand denotes a non-nil value(|| non-nil-formula any-formula) = non-nil-formula (|| any-formula non-nil-formula) = non-nil-formula (|| formula-denoting-nil formula-denoting-nil) = nil
Define an operator that delivers a value other than nil if its operand denotes nil, and nil if its operand denotes a non-nil value(~ non-nil-formula) = nil(~ formula-denoting-nil) = non-nil-formula
TSC Workshop, May 2008, U Oklahoma 21
SolutionsSolutions
(defun && (x y) (if x y nil))(defun || (x y) (if x x y))(defun ~ (x) (if x nil t))
deMorgan laws as ACL2 theorems(defthm deMorgan-or ; ~(x || y) = (~ x) && (~ y) (equal (~ (|| x y)) (&& (~ x) (~ y))))(defthm deMorgan-and ; ~(x && y) = (~ x) || (~ y) (equal (~ (&& x y)) (|| (~ x) (~ y))))
deMorgan’s laws ~(x || y) = (~ x) && (~ y) ~(x && y) = (~ x) || (~ y)
negation of "or" is "and" of negations
negation of "or" is "and" of negations
deri
ved p
ropert
ies
aka
, “t
heore
ms”
axio
ms
TSC Workshop, May 2008, U Oklahoma 22
More defun Examples(defun rac (xs) ; (rac xs) = last element of xs (if (endp (cdr xs))
(car xs) ; (rac (list x)) = x (rac (cdr xs)))) ; (rac (list x y z …) = (rac (list y z …))
(defun rdc (xs) (if (endp (cdr xs))
nil ; (rdc (list x)) = nil (cons (car xs) (rdc (cdr xs))))) ; (rdc (list x y z …)) =
(cons x (rdc (list y z …))) Define these operators:
Next-to-last element of list (defun rac2 (xs) … ) Drop last two elements (defun rdc2 (xs) …) Insert at end of list (defun snoc (x xs) …)
ind
uct
ive d
efin
itio
ns
How to make an inductive definition State properties correctly Cover all cases Inductive part
closer to non-inductive case
axioms
Exerc
ises
TSC Workshop, May 2008, U Oklahoma 23
non-inductive equation
inductive equation
(defun snoc (x xs) (if (endp xs)
(list x) ; (snoc x nil) = (list x) (cons (car xs) (snoc x (cdr xs))))) ; (snoc x (list x1
x2 …)) = (cons x1 (snoc x (list x2 …)))
Solutions(defun rac2 (xs) (rac (rdc xs)))(defun rdc2 (xs) (rdc (rdc xs)))
Predicate-based test of rdc(defconst *num-tests* 10)(defproperty rdc-snoc-identity *num-tests* ((n (random-int-between 0 100) (integerp n)) (x (random-symbol) (symbolp x)) (xs (random-list-of (random-symbol) n) (symbol-listp
xs))) (equal (rdc (snoc x xs)) xs))
TSC Workshop, May 2008, U Oklahoma 24
non-inductive equation
inductive equation
(defun snoc (x xs) (if (endp xs)
(list x) ; (snoc x nil) = (list x) (cons (car xs) (snoc x (cdr xs))))) ; (snoc x (list x1
x2 …)) = (cons x1 (snoc x (list x2 …)))
Solutions(defun rac2 (xs) (rac (rdc xs)))(defun rdc2 (xs) (rdc (rdc xs)))
Predicate-based test of rdc(defconst *num-tests* 10)(defproperty rdc-snoc-identity *num-tests* ((n (random-int-between 0 100) (integerp n)) (x (random-symbol) (symbolp x)) (xs (random-list-of (random-symbol) n) (symbol-listp
xs))) (equal (rdc (snoc x xs)) xs))
Let’s run this test
TSC Workshop, May 2008, U Oklahoma 25
non-inductive equation
inductive equation
(defun snoc (x xs) (if (endp xs)
(list x) ; (snoc x nil) = (list x) (cons (car xs) (snoc x (cdr xs))))) ; (snoc x (list x1
x2 …)) = (cons x1 (snoc x (list x2 …)))
Solutions(defun rac2 (xs) (rac (rdc xs)))(defun rdc2 (xs) (rdc (rdc xs)))
Predicate-based test of rdc(defconst *num-tests* 10)(defproperty rdc-snoc-identity *num-tests* ((n (random-int-between 0 100) (integerp n)) (x (random-symbol) (symbolp x)) (xs (random-list-of (random-symbol) n) (symbol-listp
xs))) (equal (rdc (snoc x xs)) xs))(defthm rdc-snoc-identity-thm (equal (rdc (snoc x xs)) xs))
Let’s run this test
Must be a theorem, eh?
TSC Workshop, May 2008, U Oklahoma 26
non-inductive equation
inductive equation
(defun snoc (x xs) (if (endp xs)
(list x) ; (snoc x nil) = (list x) (cons (car xs) (snoc x (cdr xs))))) ; (snoc x (list x1
x2 …)) = (cons x1 (snoc x (list x2 …)))
Solutions(defun rac2 (xs) (rac (rdc xs)))(defun rdc2 (xs) (rdc (rdc xs)))
Predicate-based test of rdc(defconst *num-tests* 10)(defproperty rdc-snoc-identity *num-tests* ((n (random-int-between 0 100) (integerp n)) (x (random-symbol) (symbolp x)) (xs (random-list-of (random-symbol) n) (symbol-listp
xs))) (equal (rdc (snoc x xs)) xs))(defthm rdc-snoc-identity-thm (equal (rdc (snoc x xs)) xs))
Let’s run this test
Must be a theorem, eh?Maybe ACL2 can prove it
TSC Workshop, May 2008, U Oklahoma 27
non-inductive equation
inductive equation
(defun snoc (x xs) (if (endp xs)
(list x) ; (snoc x nil) = (list x) (cons (car xs) (snoc x (cdr xs))))) ; (snoc x (list x1
x2 …)) = (cons x1 (snoc x (list x2 …)))
Solutions(defun rac2 (xs) (rac (rdc xs)))(defun rdc2 (xs) (rdc (rdc xs)))
Predicate-based test of rdc(defconst *num-tests* 10)(defproperty rdc-snoc-identity *num-tests* ((n (random-int-between 0 100) (integerp n)) (x (random-symbol) (symbolp x)) (xs (random-list-of (random-symbol) n) (symbol-listp
xs))) (equal (rdc (snoc x xs)) xs))(defthm rdc-snoc-identity-thm (equal (rdc (snoc x xs)) xs))
Let’s run this test
Must be a theorem, eh?May
be n
ot !
What if xs isn’t a true-list?
TSC Workshop, May 2008, U Oklahoma 28
non-inductive equation(defun snoc (x xs) (if (endp xs)
(list x) ; (snoc x nil) = (list x) (cons (car xs) (snoc x (cdr xs))))) ; (snoc x (list x1
x2 …)) = (cons x1 (snoc x (list x2 …)))
Solutions(defun rac2 (xs) (rac (rdc xs)))(defun rdc2 (xs) (rdc (rdc xs)))
(defthm rac-snoc-identity-thm (implies (true-listp xs)
(equal (rac (snoc x xs)) x))))
(defthm rdc-snoc-identity-thm (implies (true-listp xs)
(equal (rdc (snoc x xs)) xs))
Need true-list hypothesis, so state theorem as implication
inductive equation
axio
ms
theore
ms
Properties derivable from axioms
ACL2 intrinsic: true-listp (true-listp xs) iff xs=nil or (true-listp (cdr xs))
TSC Workshop, May 2008, U Oklahoma 29
How to Write Inductive Definitions
aka “recursive functions”
Three rules Cover all cases Specify correct result in each case
One formula for each case Ensure definition is computational
At least one non-inductive caseAll inductive cases make progress towards a non-inductive
case
(defun snoc (x xs) ; cons, backwards
(if (endp xs) (list x)
(cons (car xs) (snoc x (cdr xs)))))
Follo
w these
rules …
writ
e defect-fre
e softw
are
(defun f (x) (if (non-inductive-case x) (non-inductive-formula x) (some-other-op (f closer-to-non-inductive-case))))
formula denoting somethingcloser to non-inductive case
Structural induction (HtDP design recipe): when "obviously" closer to non-inductive case
TSC Workshop, May 2008, U Oklahoma 30
List operators cons, car, cdr list, len, Nth, take, NthCdr, append, reverse equal, true-listp, null, atom, consp , endp
Numeric operators Integer, rational, and complex arithmetic: Integer division: mod, floor, ceiling, round, truncate
All require dividend and divisor: (floor 8 3) = 2Operands may be rationals, but quotients are integers
numeric operands only abs, min, max integer or rational operands only zp, posp natural number operand only Recognizers: natp, integerp, rationalp, acl2-numberp
Character and string operators char-upcase, char-downcase, char-code, code-char character operands char<, char<=, …, char-equal character operands string-append, subseq, string-upcase string operands string<, string<=, …, string-equal string
operands Converters: (coerce list-of-chars ‘String), (coerce string ‘List) Recognizers: characterp, stringp
Some Intrinsic Operatorshttp://www.cs.utexas.edu/users/moore/acl2/v3-3/PROGRAMMING.html
Session 02 Lab: http://www.cs.ou.edu/~rlpage/tsc/
TSC Workshop, May 2008, U Oklahoma 31
The End
top related