Microsoft Research's Leslie Lamport at Build2014 - Thinking for Programmers

Thinking For Programmers

Leslie LamportMicrosoft Research

0

Why Think?

It helps us do most things:

Hunting a sabre-toothed tiger.

Building a house.

Writing a program.

0

Why Think?



Building a house.

Writing a program.

0

Why Think?



Building a house.

Writing a program.

0

Why Think?



Building a house.

Writing a program.

0

Why Think?



Building a house.

Writing a program.

0

When to Think


Before leaving the cave.

Building a house.

Before beginning construction.

Writing a program.

Before writing any code.

1

When to Think



Building a house.


Writing a program.


1

When to Think



Building a house.


Writing a program.


1

When to Think



Building a house.


Writing a program.


1

When to Think



Building a house.


Writing a program.


1

When to Think



Building a house.


Writing a program.


1

When to Think



Building a house.


Writing a program.


1

How to Think

“Writing is nature’s way of letting you know howsloppy your thinking is.”

Guindon

To think, you have to write.

If you’re thinking without writing, you only think you’re thinking.

1

How to Think


Guindon



1

How to Think


Guindon



1

How to Think


Guindon



1

What to Write


Writing not invented, dangerous activity.

Building a house.

Draw blueprints.

Writing a program.

Write a

2

What to Write



Building a house.

Draw blueprints.

Writing a program.

Write a

2

What to Write



Building a house.

Draw blueprints.

Writing a program.

Write a

2

What to Write



Building a house.

Draw blueprints.

Writing a program.

Write a

2

What to Write



Building a house.

Draw blueprints.

Writing a program.

Write a

2

What to Write



Building a house.

Draw blueprints.

Writing a program.

Write a

2

What to Write



Building a house.

Draw blueprints.

Writing a program.

Write a blueprint

2

What to Write



Building a house.

Draw blueprints.

Writing a program.

Write a blueprint specification.

2

Specifications

Don’t Panic!

3

Specifications

Don’t Panic!

3

Specifications

Don’t Panic!

This is a blueprint:

3

Specifications

Don’t Panic!

This is also a blueprint:

3

A spectrum of blueprints

3


3


Very Detailed

3


Very Detailed

3


Very DetailedRough Sketch

3


Very DetailedRough Sketch

3


Very DetailedRough Sketch Ordinary

3

A spectrum of specifications

5


Formal

5


FormalProse

5


FormalProse Mathematical Prose

5


Prose

Most code is really simple.

It can be specified with a couple of lines of prose.

5


Prose


It can be specified with a couple of lines of prose.

5


Mathematical Prose

Some code is subtle.

It requires more thought.

5


Mathematical Prose

Some code is subtle.

It requires more thought.

5


Formal

Some code is complex or very subtle or critical.

5


Formal


Especially in concurrent/distributed systems.

5


Formal


We should use tools to check it.

5

How to Write a Spec

Writing requires thinking.

5

How to Write a Spec


5

How to Think About Programs

Like a Scientist

A very successful way of thinking.

Science makes mathematical models of reality.

Astronomy:

Reality: planets have mountains, oceans, tides, weather, . . .

Model: planet a point mass with position & momentum.

6


Like a Scientist



Astronomy:



6


Like a Scientist



Astronomy:



6


Like a Scientist



Astronomy:



6


Like a Scientist



Astronomy:



6


Like a Scientist



Astronomy:



6


Like a Scientist



Astronomy:



6

Computer Science

Reality: Digital systems

a processor chip

a game console

a computer executing a program...

Models: Turing machines

Partially ordered sets of events...

6

Computer Science


a processor chip

a game console




6

Computer Science


a processor chip

a game console




6

Computer Science


a processor chip

a game console




6

Computer Science


a processor chip

a game console




6

Computer Science


a processor chip

a game console




6

Computer Science


a processor chip

a game console




6

Computer Science


a processor chip

a game console




6

Computer Science


a processor chip

a game console




6

Computer Science


a processor chip

a game console




6

The Two Most Useful Models

Functions

Sequences of States

7


Functions

Sequences of States

7


Functions

Sequences of States

7

Functions

Model a program as a function mapping input(s) to output(s).

In math, a function is a set of ordered pairs.

Example: the square function on natural numbers

{〈0,0〉, 〈1,1〉, 〈2,4〉, 〈3,9〉, . . . }

8

Functions




{〈0,0〉, 〈1,1〉, 〈2,4〉, 〈3,9〉, . . . }

8

Functions




{〈0,0〉, 〈1,1〉, 〈2,4〉, 〈3,9〉, . . . }

8

Functions



Not in TLA+


{〈0,0〉, 〈1,1〉, 〈2,4〉, 〈3,9〉, . . . }

8

Functions




{〈0,0〉, 〈1,1〉, 〈2,4〉, 〈3,9〉, . . . }

8

Functions




{〈0,0〉, 〈1,1〉, 〈2,4〉, 〈3,9〉, . . . }

square(2) = 4

8

Functions




{〈0,0〉, 〈1,1〉, 〈2,4〉, 〈3,9〉, . . . }

Domain of square is {0,1,2,3, . . .} a.k.a. Nat

8

Functions




{〈0,0〉, 〈1,1〉, 〈2,4〉, 〈3,9〉, . . . }

To define a function, specify:

Domain of square = Nat

square(x ) = x2 for all x in its domain.

8

Functions




{〈0,0〉, 〈1,1〉, 〈2,4〉, 〈3,9〉, . . . }

To define a function, specify:

Domain of square = Nat

square(x ) = x2 for all x in its domain.

8

Functions




{〈0,0〉, 〈1,1〉, 〈2,4〉, 〈3,9〉, . . . }

Functions in math 6= functions in programming languages.

8

Functions




{〈0,0〉, 〈1,1〉, 〈2,4〉, 〈3,9〉, . . . }

Functions in math 6= functions in programming languages.

Math is much simpler.

8

Limitations of the Function Model

Specifies what a program does, but not how.

– Quicksort and bubble sort compute the same function.

Some programs don’t just map inputs to outputs.

– Some programs run “forever”.

– Operating systems

9







9







9







9







9







9

The Standard Behavioral Model

A program execution is represented by a behavior.

A behavior is a sequence of states.

A state is an assignment of values to variables.

A program is modeled by a set of behaviors:

the behaviors representing possible executions.

9







9







9







9







9







9

An Example: Euclid’s Algorithm

Computes GCD of M and N by:

– Initialize x to M and y to N .

– Keep subtracting the smaller of x and y from the larger.

– Stop when x = y .

For M = 12 and N = 18, one behavior:

[x = 12, y = 18] → [x = 12, y = 6] → [x = 6, y = 6]

10


An algorithm is an abstract program.


[x = 12, y = 18] → [x = 12, y = 6] → [x = 6, y = 6]

10







[x = 12, y = 18] → [x = 12, y = 6] → [x = 6, y = 6]

10







[x = 12, y = 18] → [x = 12, y = 6] → [x = 6, y = 6]

10







[x = 12, y = 18] → [x = 12, y = 6] → [x = 6, y = 6]

10







[x = 12, y = 18] → [x = 12, y = 6] → [x = 6, y = 6]

10

How to Describe a Set of Behaviors

With two things:

– The set of possible initial states.

– A next-state relation,describing all possible successor states of any state.

11


With two things:



11


With two things:



11


With two things:



11


With two things:



For infinite behaviors, also need fairness.

11


With two things:



For infinite behaviors, also need fairness.

I will ignore this.

11


With two things:



What language should we use?

We can use math.

11


With two things:



What language should we use?

We can use math.

11

The Set of Initial States

Described by a formula.

For Euclid’s Algorithm: (x = M ) ∧ (y = N )

Only possible initial state: [x = M , y = N ]

12





12





12





12

The Next-State Relation


Unprimed variables for current state,Primed variables for next state.

For Euclid’s Algorithm: ( x > y

∧ x ′ = x − y

∧ y ′ = y )

∨ ( y > x

∧ y ′ = y − x

∧ x ′ = x )

13





∧ x ′ = x − y

∧ y ′ = y )

∨ ( y > x

∧ y ′ = y − x

∧ x ′ = x )

13





∧ x ′ = x − y

∧ y ′ = y )

∨ ( y > x

∧ y ′ = y − x

∧ x ′ = x )

13





∧ x ′ = x − y

∧ y ′ = y )

∨ ( y > x

∧ y ′ = y − x

∧ x ′ = x )

13





∧ x ′ = x − y

∧ y ′ = y )

∨ ( y > x

∧ y ′ = y − x

∧ x ′ = x )

13





∧ x ′ = x − y

∧ y ′ = y )

∨ ( y > x

∧ y ′ = y − x

∧ x ′ = x )

13

For Euclid’s Algorithm

Take M = 12, N = 18

Next : ( x > y

∧ x ′ = x − y

∧ y ′ = y )

∨ ( y > x

∧ y ′ = y − x

∧ x ′ = x )

Behavior:

[x = 12, y = 18] → [x = 12, y = 6] → [x = 6, y = 6]

16


Take M = 12, N = 18

Next : ( x > y

∧ x ′ = x − y

∧ y ′ = y )

∨ ( y > x

∧ y ′ = y − x

∧ x ′ = x )

Behavior:

[x = 12, y = 18] → [x = 12, y = 6] → [x = 6, y = 6]

16


Init : (x = M ) ∧ (y = N )

Next : ( x > y

∧ x ′ = x − y

∧ y ′ = y )

∨ ( y > x

∧ y ′ = y − x

∧ x ′ = x )

Behavior:

[x = 12, y = 18] → [x = 12, y = 6] → [x = 6, y = 6]

16


Init : (x = 12) ∧ (y = 18)

Next : ( x > y

∧ x ′ = x − y

∧ y ′ = y )

∨ ( y > x

∧ y ′ = y − x

∧ x ′ = x )

Behavior:

[x = 12, y = 18] → [x = 12, y = 6] → [x = 6, y = 6]

16


Init : (x = M ) ∧ (y = N )

Next : ( x > y

∧ x ′ = x − y

∧ y ′ = y )

∨ ( y > x

∧ y ′ = y − x

∧ x ′ = x )

Behavior:

[x = 12, y = 18] → [x = 12, y = 6] → [x = 6, y = 6]

16


Init : (x = M ) ∧ (y = N )

Next : ( 12 > 18∧ x ′ = 12− 18∧ y ′ = 18 )

∨ ( 18 > 12∧ y ′ = 18− 12∧ x ′ = 12 )

Behavior:

[x = 12, y = 18] → [x = 12, y = 6] → [x = 6, y = 6]

16


Init : (x = M ) ∧ (y = N )

Next : ( FALSE12 > 18∧ x ′ = 12− 18∧ y ′ = 18 )

∨ ( TRUE18 > 12∧ y ′ = 18− 12∧ x ′ = 12 )

Behavior:

[x = 12, y = 18] → [x = 12, y = 6] → [x = 6, y = 6]

16


Init : (x = M ) ∧ (y = N )

Next : ( 12 > 18∧ x ′ = 12− 18��

HHHH

HHH

FALSE∧ y ′ = 18 )

∨ ( 18 > 12∧ y ′ = 18− 12∧ x ′ = 12 )

Behavior:

[x = 12, y = 18] → [x = 12, y = 6] → [x = 6, y = 6]

16


Init : (x = M ) ∧ (y = N )

Next : ( 12 > 18∧ x ′ = 12− 18��

HHHH

HHH

FALSE∧ y ′ = 18 )

∨ ( 18 > 12∧ y ′ = 18− 12∧ x ′ = 12 )

Behavior:

[x = 12, y = 18] → [x = 12, y = 6] → [x = 6, y = 6]

16


Init : (x = M ) ∧ (y = N )

Next : ( 12 > 18∧ x ′ = 12− 18��

HHHH

HHH

FALSE∧ y ′ = 18 )

∨ ( 18 > 12∧ y ′ = 18− 12∧ x ′ = 12 )

Behavior:

[x = 12, y = 18] → [x = 12, y = 6] → [x = 6, y = 6]

16


Init : (x = M ) ∧ (y = N )

Next : ( x > y

∧ x ′ = x − y

∧ y ′ = y )

∨ ( y > x

∧ y ′ = y − x

∧ x ′ = x )

Behavior:

[x = 12, y = 18] → [x = 12, y = 6] → [x = 6, y = 6]

16


Init : (x = M ) ∧ (y = N )

Next : ( 12 > 6∧ x ′ = 12− 6∧ y ′ = 6 )

∨ ( 6 > 12∧ y ′ = 6− 12∧ x ′ = 12 )

Behavior:

[x = 12, y = 18] → [x = 12, y = 6] → [x = 6, y = 6]

16


Init : (x = M ) ∧ (y = N )

Next : ( TRUE12 > 6∧ x ′ = 12− 6∧ y ′ = 6 )

∨ ( FALSE6 > 12∧ y ′ = 6− 12∧ x ′ = 12 )

Behavior:

[x = 12, y = 18] → [x = 12, y = 6] → [x = 6, y = 6]

16


Init : (x = M ) ∧ (y = N )

Next : ( 12 > 6∧ x ′ = 12− 6∧ y ′ = 6 )

∨ ( 6 > 12∧ y ′ = 6− 12��

��

HHHHH

HH

FALSE∧ x ′ = 12 )

Behavior:

[x = 12, y = 18] → [x = 12, y = 6] → [x = 6, y = 6]

16


Init : (x = M ) ∧ (y = N )

Next : ( 12 > 6∧ x ′ = 12− 6∧ y ′ = 6 )

∨ ( 6 > 12∧ y ′ = 6− 12��

��

HHHHH

HH

FALSE∧ x ′ = 12 )

Behavior:

[x = 12, y = 18] → [x = 12, y = 6] → [x = 6, y = 6]

16


Init : (x = M ) ∧ (y = N )

Next : ( x > y

∧ x ′ = x − y

∧ y ′ = y )

∨ ( y > x

∧ y ′ = y − x

∧ x ′ = x )

Behavior:

[x = 12, y = 18] → [x = 12, y = 6] → [x = 6, y = 6]

16


Init : (x = M ) ∧ (y = N )

Next : ( FALSE6 > 6∧ x ′ = 6− 6∧ y ′ = 6 )

∨ ( FALSE6 > 6∧ y ′ = 6− 6∧ x ′ = 6 )

Behavior:

[x = 12, y = 18] → [x = 12, y = 6] → [x = 6, y = 6]

16


Init : (x = M ) ∧ (y = N )

Next : ( 6 > 6∧ x ′ = 6− 6��

HHHH

HHH

FALSE∧ y ′ = 6 )

∨ ( 6 > 6∧ y ′ = 6− 6��

HHHHH

HH

FALSE∧ x ′ = )

Behavior:

[x = 12, y = 18] → [x = 12, y = 6] → [x = 6, y = 6]

16


Init : (x = M ) ∧ (y = N )

Next : ( 6 > 6∧ x ′ = 6− 6��

HHHH

HHH

FALSE∧ y ′ = 6 )

∨ ( 6 > 6∧ y ′ = 6− 6��

HHHHH

HH

FALSE

NO NEXT STATE

∧ x ′ = )

Behavior:

[x = 12, y = 18] → [x = 12, y = 6] → [x = 6, y = 6]

16

What About Formal Specs?

Need them only to apply tools.

Require a formal language.

16




16




16

The Language: TLA+

17

The Language: TLA+

This

Init : (x = M ) ∧ (y = N )

Next : ( x > y

∧ x ′ = x − y

∧ y ′ = y )

∨ ( y > x

∧ y ′ = y − x

∧ x ′ = x )

17

The Language: TLA+

becomes

MODULE EuclidEXTENDS Integers

Init∆= (x = M ) ∧ (y = N )

Next∆= ( x > y

∧ x ′ = x − y

∧ y ′ = y )

∨ ( y > x

∧ y ′ = y − x

∧ x ′ = x )

17

The Language: TLA+

plus some boilerplate.

MODULE EuclidEXTENDS Integers

Init∆= (x = M ) ∧ (y = N )

Next∆= ( x > y

∧ x ′ = x − y

∧ y ′ = y )

∨ ( y > x

∧ y ′ = y − x

∧ x ′ = x )

17

The Language: TLA+

You type

--------------- MODULE Euclid ----------------EXTENDS Integers

Init == (x = M) /\ (y = N)

Next == ( x > y/\ x’ = x - y/\ y’ = y )

\/ ( y > x/\ y’ = y - x/\ x’ = x )

===============================================

17

You can model check TLA+ specs.

You can write formal correctness proofsand check them mechanically.

But math works only for toy examples.

To model real systems, you need a real languagewith types, procedures, objects, etc.

Wrong

18





Wrong

18





Wrong

18





Wrong

18




To model real systems, you need a real language

��

��

��

��

PPPP

PPPP

PPPP

PPPP

PPPP

PP

with types, procedures, objects, etc.

Wrong

18

TLA+ Use at Amazon (incomplete list)

19

Design Bugs

Not just simple coding bugs

Serious, often impossible to find by testing.

Ask Chris Newcombe about TLA+.

I’ll talk about informal specs, starting with an example.

19

Design Bugs





19

Design Bugs





19

Design Bugs





19

Design Bugs





19

TLATEX — the TLA+ pretty-printer

19


The input:

Foo => /\ a = b

/\ ccc = d

20


The input:

Foo => /\ a = b

/\ ccc = d

The naive output:

Foo ⇒ ∧ a = b

∧ ccc = d

20


The input:

Foo => /\ a = b

/\ ccc = d

The naive output:

Foo ⇒ ∧ a = b

∧ ccc = d

The user probably wanted these aligned.

20


The input:

Foo => /\ a = b

/\ ccc = d

The right output:

Foo ⇒ ∧ a = b

∧ ccc = d

The user probably wanted these aligned.

20


The input:

/\ aaa + bb = c

/\ iii = jj * k

21


The input:

/\ aaa + bb = c

/\ iii = jj * k

The naive output:

∧ aaa + bb = c

∧ iii = jj ∗ k

21


The input:

/\ aaa + bb = c

/\ iii = jj * k

The naive output:

∧ aaa + bb = c

∧ iii = jj ∗ k

The user probably didn’t wanted these aligned.

21


The input:

/\ aaa + bb = c

/\ iii = jj * k

The right output:

∧ aaa + bb = c

∧ iii = jj ∗ k

The user probably didn’t wanted these aligned.

21

There is no precise definition of correct alignment.

We can’t specify mathematically what the user wants.

If we can’t specify correctness, specification is useless.Wrong.

Not knowing what a program should do meanswe have to think even harder.

Which means that a spec is even more important.

What did I do?

Let’s look at my Java code.

21






What did I do?


21






What did I do?


21






What did I do?


21






What did I do?


21






What did I do?


21






What did I do?


21






What did I do?


21

/**************************************************************************** CLASS FindAlignments ** ** Contains the one public method ** ** public static void FindAlignments(Token[][] spec) ** ** that sets the aboveAlign, belowAlign, and isAlignmentPoint fields for ** all tokens in the tokenized spec spec--except for NULL and MULTI ** comment tokens, for which the aboveAlign field is set by ** CommentToken.processComments. (This method should be called after ** CommentToken.processComments.) ** ** There are six kinds of alignments, illustrated by the following ** example: ** ** Alignment Type Tokens so aligned ** -------------- ----------------- ** FirstNonLeftComment foo, x, + , /\ , y/comment r, u/comment k ** LeftComment comments j ** CommentInner comments p ** InfixInner ==, => , > ** AfterInfixInner m, c, ** InfixArg a, z/z, y/z ** ** foo == LET x == /\ a => m * n > c ** | | | | | | | | | | ** (* j *) | | | | /\ a => m / q > c (* p *) ** | | | | | | ** (* j *) | | x == z | ** | | | | ** | | + z \* p ** | | | | | ** 22


/**************************************************************************** CLASS FindAlignments ** ** Contains the one public method ** ** public static void FindAlignments(Token[][] spec) ** ** that sets the aboveAlign, belowAlign, and isAlignmentPoint fields for ** all tokens in the tokenized spec spec--except for NULL and MULTI ** comment tokens, for which the aboveAlign field is set by ** CommentToken.processComments. (This method should be called after ** CommentToken.processComments.) ** ** There are six kinds of alignments, illustrated by the following ** example: ** ** Alignment Type Tokens so aligned ** -------------- ----------------- ** FirstNonLeftComment foo, x, + , /\ , y/comment r, u/comment k ** LeftComment comments j ** CommentInner comments p ** InfixInner ==, => , > ** AfterInfixInner m, c, ** InfixArg a, z/z, y/z ** ** foo == LET x == /\ a => m * n > c ** | | | | | | | | | | ** (* j *) | | | | /\ a => m / q > c (* p *) ** | | | | | | ** (* j *) | | x == z | ** | | | | ** | | + z \* p ** | | | | | **

* There are six kinds of alignments, ** illustrated by the following example: *

22


* | | | | (* p *) ** | | | | | ** | | + y (* p *) ** | | | ** | | \* r ** | | ** foo == ** | ** (* k *) ** | ** u + v ** ** For InfixInner and CommentInner alignment, a token’s belowAlign ** field points to the token directly below it with which it is ** aligned. ** ** For all alignment types, if a token t1 is aligned with a token ** t2 above it, then t1.aboveAlign points to: ** ** IF t2 has no aboveAlign pointer THEN t2 ** ELSE t2.abovealign. ** ** Thus, in the example above, the aboveAlign pointer for all the p ** comments point to the first p comment. ** ** To define the types of alignment, we use the following definitions: ** ** - A LEFT-COMMENT is a comment that is the first token on its line but ** not the last token on its line. (The comments (* j *) in the ** example are left-comments.) For all the types of alignment except ** CommentInner, left-comments are treated as if they didn’t exist. ** ** - A RIGHT-COMMENT is a comment that is the last token on its line. * 22

* Any comment other than a left-comment or a right-comment is treated ** like any other non-built-in token. We also define: ** ** - The COVERING TOKEN of a token t is the right-most token ct that ** covers t on the last line (one with biggest line number) containing ** a token that covers t, where a token t1 COVERS a token t2 if t1 ** lies on an earlier line than t2 and t1.column \leq t2.column. ** However, if there is a DASH between ct and t, or ** on the same line as ct, then t has no covering token. ** (This definition has two versions, depending on whether or not ** left-comments are ignored.) ** ** - The BLOCKING TOKEN of a token t is the left-most token s with ** s.column \geq t.column on the last line before t’s line containing ** such a non-left-comment token s. ** ** Here are the descriptions of the different types of alignment. ** ** CommentInner: ** Token t at position pos is CommentInner above-aligned with its ** blocking token bt at position bpos iff: ** /\ t is a right-comment ** /\ bt is a right-comment ** /\ t.column = bt.column ** /\ \/ bt is not above-aligned with anything ** \/ bt is CommentInner aligned with a token above it ** ** FirstNonLeftComment ** If t is the first token on a line that is not a left-comment, and is ** not a right-comment that is CommentInner aligned with a token above ** it, then t is FirstNonLeftComment aligned with its covering token. ** *

22


* Here are the descriptions of the different ** kinds of alignment. *

22


22

* ** LeftComment: ** If t is a left-comment token, then it is LeftComment aligned ** with its covering token (where left-comments are visible). ** ** InfixInner: ** Token t at position pos is InfixInner aligned with its covering ** token ct at position cpos iff t is not FirstNonLeftComment aligned, ** and both t and ct are built-in symbols with the same nonzero alignment ** class. (The name InfixInner is misleading because non-infix symbols ** like ":" get aligned by this mechanism.) ** ** AfterInfixInner: ** Token t is After InfixInner aligned with the token above it iff ** ** LET lt == the token to the left of t ** alt == the token at position lt.aboveAlign ** at == the token to the right of alt ** IN /\ t is not the first token on the line ** /\ lt is InnerAligned with token alt above it ** /\ alt is not the last token on its line. ** /\ at is the covering token of t ** /\ t.column = at.column ** /\ at is not a comment ** (The name AfterInfixInner means "after InfixInner alignment". ** Remember that some non-infix symbols get InfixInner aligned.) ** ** InfixArg: ** Token t is InfixArg aligned with its covering token ct iff: ** LET lt == the token to the left of t ** alt == the token at position lt.aboveAlign ** IN /\ t is not the first token on the line ** /\ lt is the first non-left-comment token *

23


* LeftComment:

* If t is a left-comment token, then it is ** LeftComment aligned with its covering ** token. *

23


* LeftComment:

* If t is a left-comment token, then it is ** LeftComment aligned with its covering ** token. *

23


23


23

* | | | | (* p *) ** | | | | | ** | | + y (* p *) ** | | | ** | | \* r ** | | ** foo == ** | ** (* k *) ** | ** u + v ** ** For InfixInner and CommentInner alignment, a token’s belowAlign ** field points to the token directly below it with which it is ** aligned. ** ** For all alignment types, if a token t1 is aligned with a token ** t2 above it, then t1.aboveAlign points to: ** ** IF t2 has no aboveAlign pointer THEN t2 ** ELSE t2.abovealign. ** ** Thus, in the example above, the aboveAlign pointer for all the p ** comments point to the first p comment. ** ** To define the types of alignment, we use the following definitions: ** ** - A LEFT-COMMENT is a comment that is the first token on its line but ** not the last token on its line. (The comments (* j *) in the ** example are left-comments.) For all the types of alignment except ** CommentInner, left-comments are treated as if they didn’t exist. ** *

23


* - A LEFT-COMMENT is a comment that is the ** first token on its line but not the last ** token on its line. *

23


23

And so on for 215 lines.

This is a mathematical-prose spec of alignment.

It consists of 6 rules plus explanation (comments).

24




24




24

Why did I write this spec?

It was a lot easier to understand and debug 6 rulesthan 850 lines of code.

I did a lot of debugging of the rules, aided by debugging codeto report what rules were being used.

The few bugs in implementing the rules were easy to catch.

Had I just written code, it would have taken me much longerand not produced formatting as good.

24






24






24






24






24






24

About 10 years later I enhanced the program to pretty-printPlusCal code in comments.

A lot easier to modify a 6-rule spec than 850 lines of code.

Change to spec: Added one rule.

Changes to code:Added 2 methods.Changed one old method:

8 calls to an old method→ calls to a new methodAdded 1 call to a new method.

Without spec, probably would have completely rewritten code.

25







25







25







25







25

Why not a formal spec?

Getting it right not that important.

It didn’t have to work on all corner cases.

There were no tools that could help me.

26





26





26





26

What is Typical About This Spec

The spec is at a higher-level than the code.

It could have been implemented in any language.

No method or tool for writing better code would havehelped to write the spec

No method or tool for writing better code would havemade the spec unnecessary.

It says nothing about how to write code.

You write a spec to help you think about the problembefore you think about the code. 27











































What is Not Typical About This Spec

It’s quite subtle.

95% of the code most people write requires less thought;specs that are shorter and simpler suffice.

It’s a set of rules.

A set of rules/requirements/axioms is usually a bad spec.

It’s hard to understand the consequences of a set of rules.

The best way to write a functional spec is to write a function.

You can then check that the function implies any desiredrequirements.

28









28









28









28









28









28









28









28

Specifying How to Compute a Function

Specifying what the pretty-printer should do was hard.

Figuring out how to implement it was easy.

Specifying what a sorting program should do is easy.

Figuring out how to implement it efficiently is hard(if no one has shown you).

It requires thinking, which means writing a specification.

28







28







28







28







28







28

An example: Quicksort

A divide-and-conquer algorithm for sorting an arrayA[0], . . . , A[N − 1].

For simplicity, assume the A[i ] are numbers.

29




29




29

It uses a procedure Partition(lo, hi).

It chooses pivot in lo . . . (hi − 1), permutes A[lo], . . . ,A[hi ]

to make A[lo], . . . ,A[pivot ] ≤ A[pivot + 1], . . . ,A[hi ],and returns pivot .

For this example, we don’t care how this procedureis implemented.

30





30





30

Let’s specify Quicksort in pseudo-code.

procedure Partition(lo, hi) {

Pick pivot in lo . . . (hi − 1);

Permute A[lo], . . . ,A[hi ] to makeA[lo], . . . ,A[pivot ] ≤ A[pivot + 1], . . . ,A[hi ];

return pivot ; }

procedure QS (lo, hi) { if (lo < hi ) { p := Partition(lo, hi);QS (lo, p);QS (p + 1, hi); } }

main { QS (0,N − 1) ; }

But is it really Quicksort?

31




return pivot ; }


main { QS (0,N − 1) ; }


31




return pivot ; }


main { QS (0,N − 1) ; }


31




return pivot ; }


main { QS (0,N − 1) ; }


31




return pivot ; }


main { QS (0,N − 1) ; }


31




return pivot ; }


main { QS (0,N − 1) ; }

Informal: no formal syntax, no declarations, . . .

Easy to understand.

But is it really Quicksort?31




return pivot ; }


main { QS (0,N − 1) ; }

Informal: no formal syntax, no declarations, . . .

Easy to understand.

But is it really Quicksort?31




return pivot ; }


main { QS (0,N − 1) ; }


31

It’s the way Quicksort is almost always described.

But recursion is not a fundamental part of Quicksort.

It’s just one way of implementing divide-and-conquer.

It’s probably not the best way for parallel execution.

31





31





31





31

Problem: Write a non-recursive version of Quicksort.

Almost no one can do it in 10 minutes.

They try to “compile” the recursive version.

32




32




32

Solution:

Maintain a set U of index ranges on whichPartition needs to be called.

Initially, U equals {〈0,N − 1〉}

We could write it in pseudo-code,but it’s better to simply write Init and Next directly.

32

Solution:




32

Solution:




32

Solution:




32

Init : A = any array of numbers of length N

∧ U = {〈0,N − 1〉}

Before writing Next , let’s make a definition:

Partitions(B , pivot , lo, hi)∆=

the set of arrays obtained from B by permutingB [lo], . . . , B [hi ] such that . . .

Next :

A relation between old values of A, Uand new values A′, U ′.

33


∧ U = {〈0,N − 1〉}




Next :


33


∧ U = {〈0,N − 1〉}




Next :


33


∧ U = {〈0,N − 1〉}




Next :


33

Next :U 6= {}

∧ Pick any 〈b, t〉 in U :IF b 6= t

THEN Pick any p in b . . (t−1) :A′ = Any element of Partitions(A, p, b, t)

∧ U ′ = U with 〈b, t〉 removed and〈b, p〉 and 〈p+1, t〉 added

ELSE A′ = A

∧ U ′ = U with 〈b, t〉 removed

35

Next :U 6= {} Stop if U = {}




ELSE A′ = A


35

Next :U 6= {}




ELSE A′ = A


35

Next :U 6= {}




ELSE A′ = A


35

Next :U 6= {}




ELSE A′ = A


35

Next :U 6= {}




ELSE A′ = A


35

Next :U 6= {}




ELSE A′ = A


35

Next :U 6= {}




ELSE A′ = A


35

Next :U 6= {}




ELSE A′ = A


35

Next :U 6= {}




ELSE A′ = A


38

Why can (almost) no one find this version of Quicksort?

Their minds are stuck in code.

They can’t think at a higher level.

38




38




38

Next :U 6= {}




ELSE A′ = A


Easy to write this as a formula.

39

Next :U 6= {}




ELSE A′ = A


Pick an arbitrary value

39

Next :U 6= {}

∧ ∃ 〈b, t〉 ∈ U :

IF b 6= t



ELSE A′ = A


Pick an arbitrary value is existential quantification.

39

Next :U 6= {}

∧ ∃ 〈b, t〉 ∈ U :

IF b 6= t



ELSE A′ = A



39

Next :U 6= {}

∧ ∃ 〈b, t〉 ∈ U :

IF b 6= t

THEN ∃ p ∈ b . . (t−1) :A′ = Any element of Partitions(A, p, b, t)


ELSE A′ = A



39

Next :U 6= {}

∧ ∃ 〈b, t〉 ∈ U :

IF b 6= t

THEN ∃ p ∈ b . . (t−1) :A′ = Any element of Partitions(A, p, b, t)


ELSE A′ = A


Or sometimes

39

Next :U 6= {}

∧ ∃ 〈b, t〉 ∈ U :

IF b 6= t

THEN ∃ p ∈ b . . (t−1) :A′ ∈ Partitions(A, p, b, t)


ELSE A′ = A


Or sometimes even simpler.

39

Next :U 6= {}

∧ ∃ 〈b, t〉 ∈ U :

IF b 6= t


∧ U ′ = (U \ {〈b, t〉}) ∪ {〈b, p〉, 〈p+1, t〉}

ELSE A′ = A

∧ U ′ = U \ {〈b, t〉}

And so on.

39

Next :U 6= {}

∧ ∃ 〈b, t〉 ∈ U :

IF b 6= t


∧ U ′ = (U \ {〈b, t〉}) ∪ {〈b, p〉, 〈p+1, t〉}

ELSE A′ = A

∧ U ′ = U \ {〈b, t〉}

A TLA+ formula.

39

Next :U 6= {}

∧ ∃ 〈b, t〉 ∈ U :

IF b 6= t


∧ U ′ = (U \ {〈b, t〉}) ∪ {〈b, p〉, 〈p+1, t〉}

ELSE A′ = A

∧ U ′ = U \ {〈b, t〉}

If you prefer pseudo-code. . .

39

PlusCal

Like a toy programming language.

Algorithm appears in a comment in a TLA+ module.

An expression can be any TLA+ expression.

Constructs for nondeterminism.

Compiled to an easy to understand TLA+ spec.

Can apply TLA+ tools.

40

PlusCal







40

PlusCal







40

PlusCal







40

PlusCal







40

PlusCal







40

PlusCal







40

Programs that Run Forever

I’ve been talking about programs that compute a function.

Programs that run forever usually involve concurrency:

– Operating systems.

– Distributed systems.

Few people can get them right by just thinking (and writing).

I’m not one of them.

We need tools to check what we do.

Use TLA+/ PlusCal.

41









Use TLA+/ PlusCal.

41









Use TLA+/ PlusCal.

41









Use TLA+/ PlusCal.

41









Use TLA+/ PlusCal.

41









Use TLA+/ PlusCal.

41









Use TLA+/ PlusCal.

41









Use TLA+/ PlusCal.

41









Use TLA+/ PlusCal.

41

The Other 95%

Prose


41

The Other 95%

/************************************************************ CLASS ResourceFileReader ** ** A ResourceFileReader returns an object for reading a ** resource file, which is a file kept in the same ** directory as the tlatex.Token class. The constructor ** takes a file name as argument. The object’s two public ** methods are ** ** getLine() : Returns the next line of the file as a ** string. Returns null after the last line. ** ** close() : Closes the file. ************************************************************/

42

Why did I write that spec?

To be sure I knew what the code should do before writing it.

Without writing a spec, I only thought I knew what it should do.

Later, I didn’t have to read the code to know what it did.

General rule:A spec of what the code does should say everythingthat anyone needs to know to use the code.

43






43






43






43






43





How the code worked was too simple to require a spec.

43

What programmers should know about thinking.

Everyone thinks they think.

If you don’t write down your thoughts,you’re fooling yourself.

43

What everyone should know about thinking.



43




43




43


You should think before you code.

A spec is simply what you write before coding.

43


You should think before you code.


43


You should write before you code.


43


You should write before you code.


43

What code should you specify?

Any piece of code that someone else might want touse or modify.

It could be:

An entire program or system.

A class.

A method.

A tricky piece of code in a method.

44


Any piece of code that someone else might want touse or modify.

It could be:


A class.

A method.


44


Any piece of code that someone elseyou

might want touse or modify in a month.

It could be:


A class.

A method.


44




It could be:


A class.

A method.


44




It could be:


A class.

A method.


44




It could be:


A class.

A method.


44




It could be:


A class.

A method.


44




It could be:


A class.

A method.


44

What should you specify about the code?

What it does.

Everything anyone needs to know to use it.

Maybe: how it does it.

The algorithm / high-level design.

44


What it does.




44


What it does.




44


What it does.




44


What it does.




44

How should you think about / specify the code?

Above the code level.

In terms of states and behaviors.

Mathematically, as rigorously / formally as necessary.

Perhaps with pseudo-code or PlusCal if specifying how.

45






45






45






45






45

How do you connect the spec to the code?

Comments connecting mathematical concepts and theirimplementation.

Example:

Mathematical concept: graph

Implementation: array of node objects & array of link objects

45



Example:



45



Example:



45



Example:



45

What about coding?

Nothing I have said implies anything abouthow you should code.

You still have to think while you code.

What you write while coding is code.

I have nothing to say about how you should code.

Use any programming language you want.

Use any coding methodology you want:test-driven development, agile programming, . . .

46

What about coding?







46

What about coding?







46

What about coding?







46

What about coding?







46

What about coding?







46

What about coding?







46

You’ll still have to test and debug.

Writing specs is an additional step.

It may save time by catching errors early,when they’re easier to correct.

It will improve your programming,so you write better programs.

46





46





46





46

Why are programmers reluctant to write specs?

Writing is hard.

46

Why are programmers reluctant to write specs?

Writing is hard.

46

Why is writing hard?


Thinking is hard.

It’s easier to think your thinking.

Writing is like running.

The less you do it, the slower you are.

You have to strengthen your writing muscles.

It takes practice.

It’s easier to find an excuse not to.

47



Thinking is hard.





It takes practice.


47



Thinking is hard.





It takes practice.


47



Thinking is hard.





It takes practice.


47



Thinking is hard.





It takes practice.


47



Thinking is hard.





It takes practice.


47



Thinking is hard.





It takes practice.


47



Thinking is hard.





It takes practice.


47



Thinking is hard.





It takes practice.


47

What if the spec is wrong?

Maybe you made a mistake.

Maybe the requirements change,or an enhancement is needed.

The code will have to be changed,maybe even before the program is finished.

This eventually happens to all useful programs.

47






47






47






47






47

In an ideal world, a new spec would be writtenand the code completely rewritten.

In the real world, the code is patchedand maybe the spec is updated.

If this is inevitable, why write specs?

48




48




48

Reason 1

Whoever has to modify the code will be grateful forevery word or formula of spec you write.

And “whoever” may be you.

That’s why you should update the spec whenchanging the code.

48

Reason 1




48

Reason 1




48

Reason 1




48

Reason 2

Every time code is patched, it becomes a little uglier,harder to understand, and harder to maintain.

If you don’t start with a spec, every piece of the codeyou write is a patch.

The program starts out ugly, hard to understand,and hard to maintain.

“No battle was ever won according to plan,but no battle was ever won without one.”

Dwight D. Eisenhower

49

Reason 2






49

Reason 2






49

Reason 2






49

Reason 2






49

Reason 2






49

Thinking doesn’t guarantee that you won’t make mistakes.

Not thinking guarantees that you will.

49

Thinking doesn’t guarantee that you won’t make mistakes.

Not thinking guarantees that you will.

49

Microsoft Research's Leslie Lamport at Build2014 - Thinking for Programmers

Software

sabretoothed tiger

dangerous activity

draw blueprints

natures way