© schmiedecke 07 In2-2-Algor.Paradigms 2
Algorithmic Paradigms
[Wikipedia]• Since the late 1960s, the word paradigm has referred to a
thought pattern in any scientific discipline or other epistemological context.
Philosopher of science Thomas Kuhn defined it as• a set of practices that define a scientific discipline• during a particular period of time, governing:
-what is to be observed and scrutinized,-the kind of questions that are supposed to be asked and
probed for answers in relation to this subject,-how these questions are to be structured,-how is an experiment to be conducted, and what equipment is available to conduct the experiment.
-how the results of scientific investigations should be interpreted.
© schmiedecke 07 In2-2-Algor.Paradigms 3
Algorithmic Paradigms
• Paradigm:- Pattern of thought which governs scientific apprehesion
during a certain period of time
• Algorithm:- Description of a general problem solving method using
executable (elementary) steps.
• Algorithmic Paradigm:- what costitutes a step?
© schmiedecke 07 In2-2-Algor.Paradigms 4
First Paradigm: Imperative Algorithms
• Based on a basic abstract machine model- linear execution model- storage- control structures
� von Neumann architecture
// Factorial
public int fac1 (int i) {
int result = 1;
for (int j=i; j>1; j--) result *= j;
return result;
}
What are the domain and co-domain of this
algorithm?
i
result
j
© schmiedecke 07 In2-2-Algor.Paradigms 5
Second Paradigm:Applicative Algorithms
• Based on the principle of inductive definition• Evaluation pattern taken from mathematical function evaluation• Recursion is a basic step
��������
��������������// Factorial recursive
public int fac1 (int i) {
if (i<2) return 1;
return i*fac1(i-1);
}
© schmiedecke 07 In2-2-Algor.Paradigms 6
Further Programming Paradigms
• Artificial Intelligence has come up with some even less machine related programming concepts:
- Logical Programming (� Prolog)- Genetic Programming- Neural Networks
• Descriptive programming• Execution step not strictly specified by programmer, but by
the implementation� Programming paradigms, but strictly speaking not
algorithmic paradigms
© schmiedecke 07 In2-2-Algor.Paradigms 7
The Object Oriented Paradigm
• Also called a programming paradigm • focussing on structure, not on problem solving• different level of detail
• consider it an implementation paradigm.
© schmiedecke 07 In2-2-Algor.Paradigms 8
Applicative Algorithms
• Idea is to apply a function to an argument
• so you need a function definition like
��������
��������������
• and a method for evaluating that function for a given argument:- Replace variable(s) by arguments- Replace left hand side by right hand side- Repeat until termination
© schmiedecke 07 In2-2-Algor.Paradigms 9
More Formally
• An applicative algorithm is a list of functions
�����������������������������
���
� �� ������� ��� �� ������� �
function name function expressionfunction parameters
• A function evaluation for a given set of arguments- replace the parameters in the function expression by the corresponding
arguments- evaluate all basic operations and functions in the function expression
from left to right according to basic priorities
• The semantics of the applicative algorithm is the evaluation of the first function.
© schmiedecke 07 In2-2-Algor.Paradigms 10
Notation
• To make functions conform the algorithm definition, we introduce the conditional operator:
• Instead of: ��������
��������������
• We write: �������� ������ ���� �� ����������
• In Java, the conditional operator is (?:) int f(int n) { return n==1 ? 1 : n*f(n-1); }
© schmiedecke 07 In2-2-Algor.Paradigms 11
Recursion is Natural
• To learn downhill skiing- climb up a few metres and learn to plow a right bend to the
bottom- climb up a few metres and learn to plow a left bend to the
bottom- take the lift all the way up and combine right and left bends until
you reach the bottom
• Let's "learn" to do the Towers of Hanoi...
© schmiedecke 07 In2-2-Algor.Paradigms 12
Towers of Hanoi
� For a stack of 1:move disc from start to dest
� For a stack of 2:move disc from start to auxmove disc from start to destmove disc from aux to dest
Call this: move 2 from start to dest using aux� For a stack of 3:
move 2 form start to aux using destmove disc from start to destmove 2 from aux to dest using start
source:http://www.cut-the-knot.org/recurrence/hanoi.shtml
Puzzle invented by Edouard Lucas in 1883:• Put an ordered stack of discs
from one peg to another, using athird peg,
• such that all stacks are orderedat all times.
© schmiedecke 07 In2-2-Algor.Paradigms 13
Express Hanoi in Java
public class Hanoi extends Applet {private Tower start, aux, dest;private int size; public Hanoi () {
this.size = 8; // replace by reading from ComboBox..start = new Tower(size);aux = new Tower(0);dest = new Tower(0);
}public void start() { move(size, start, dest, aux); }private void move(int number, Tower s, Tower d, Tower a) {
if (number==1) { d.add(s.remove()); repaint(); }else { move(n-1,s,a,d); move(1,s,d,a); move(n-1,a,d,s); }
} public void paint(Graphics g) {
start.display(g, size, 50, 10);aux.display(g, size, 100, 10);dest.display(g, size, 150, 10);
}}
© schmiedecke 07 In2-2-Algor.Paradigms 14
Properties of Hanoi
• Is it correct?Theorem:
if a disc is moved to a stack, it is smaller that ist baseIndiuctive proof: �Any movement of a partial stack ends with one stack
empty (Lemma1)� if a disc is moved which is bigger than the top of a stack,
it is moved to an empty stack (Lemma2)- true for first step.- true for second step.- if a partial stack of n-1 discs has been moved properly, a
bigger disc is moved to an empty stack in the nth step.
© schmiedecke 07 In2-2-Algor.Paradigms 15
Is Hanoi a Function?
• Hanoi is an Applet ( that does all the administrative work, including displaying the results)
• move is a void method.• Mathematically it can be viewed as mapping a set of three
Towers to another set of three Towers.• But computationally, it modifies the three Towers rather
than producing new ones for every step.
• But there are functional languages which actually implement mappings –
• We will soon learn about the LISP language....
© schmiedecke 07 In2-2-Algor.Paradigms 16
Recursion is Powerful
• It can be shown that every algorithm that can be specified can be specified recursively!
• So, recursion has maximum computation potential.
• In most situations, you can replace recursion by iteration.• Can you do so for Hanoi?
• Later we will learn about a certain type of complex recursion which cannot be expressed iteratively (so-calledR Gramars).
© schmiedecke 07 In2-2-Algor.Paradigms 17
Replacing Recursion by Iteration
• Lets consider two "famous" recusive algorithms:• the factorial n!
�������� � ��� �� � � �� ���������
• and the Fibonacci numbers (rabbit generations)
����������������� ���� �� ��������������
© schmiedecke 07 In2-2-Algor.Paradigms 18
Replacing Recursion by Iteration
• the factorial n!
�������� � ��� �� � � �� ���������
int fac(int n) {int result = 1;
for (int i=2; i<=n; i++) result *= i;
return result;
}
Solution is straightforward:use local variable to store intermediate results otherwise returned by recursive calls
© schmiedecke 07 In2-2-Algor.Paradigms 19
Replacing Recursion by Iteration
• and the Fibonacci numbers (rabbit generations)
������� � ���� �� � � �� ��������������
int fib(int n) {int current = 1, prev = 0, temp;for (int i=1; i<n; i++) { temp = current;
current = current + prev;prev = temp;
}return current;
} Same idea, use local variables to store values thatwould result from recursive calls
© schmiedecke 07 In2-2-Algor.Paradigms 20
The Domain Problem• What is the domain of the factorial function?
�������� � ��� �� � � �� ���������
int fac(int n) {int result = 1;for (int i=2; i<=n; i++) result *= i;return result;
}
• Oh dear, the applicative agorithm will run indefinitely.....• Generally, algorithms must be suspected of being partial functions, • i.e they may not terminate on certain arguments.• It may be hard to find out, or even impossible....• There are algorithms for which we do not know whether they terminate
for all valid input data. (There is one in your lab assignments �)
• So, are both algorithms above equivalent?
© schmiedecke 07 In2-2-Algor.Paradigms 21
Tail Recursion
• There is a "nice" type of recursion, called tail recursion:• Such functions have a direct solution for a fixed set of
arguments.• For every recursive call, a monotonic modification is made to the
parameters moving them "towards" the direct solution arguments.
• Tail recursion is easily translated into iteration.• The termination of tail recursive functions can generally be
determined easily (but not always).
• So, in programming, try to use tail recursion and make itobvious.
© schmiedecke 07 In2-2-Algor.Paradigms 22
Recursion and Complexity
• How do you determine the complexity of a recursive function?• Use recursive call as basic step.
• Fac is called n times, so the complexity is O(n):�������� � ��� �� � � �� ���������
• Fib is called 2*n times, so the complexity is O(n) too:�������� � ��� �� � � �� ���������
• But what about this function:������ � �������� � ���� �� ������������
© schmiedecke 07 In2-2-Algor.Paradigms 23
Incredible Growth:The Ackermann Function
• Here is the worst example ever known:
�������� �� �������� ��������� �������� �������
���������������������
• Some function results
����������
����������
����������
����������
�����������
���������������������������������������������������
• As the growth is achieved by adding 1, the time consumption, i.e. the functions comlexity, is just as incredible
© schmiedecke 07 In2-2-Algor.Paradigms 24
Implementing Recursive Calls
• Early imperative programming languages, like Fortran, could not deal with recursion.
• There abstract machine model was directly taken from the von Neumann machine:
mainpgm
subroutine1
subroutine 2
subroutine3
data of main pgm
data of subroutine1
data of subroutine2
data of subroutine3
Code Data
© schmiedecke 07 In2-2-Algor.Paradigms 25
Implementing Recursive Calls
• A dynamic call stack was the answer to the problem:
mainpgm
procedure1
procedure2
procedure3
call of main pgm
1st call of proc1
call of proc2 inside proc 1
call of proc3
recusive callof proc1
call of proc2 inside proc 1
Code Data on call stack....................................................
© schmiedecke 07 In2-2-Algor.Paradigms 26
Mathematics of Applicative Programming:The Lambda Calculus
• Inventend by Church and Kleene in the 1930ies.• Mathematical model of function application (to arguments) • Computability question: Which problems can be solved
algorithmically, and can it be decided whether a problem has a solution.
• Abstract machine for applicative algorithms: If it terminates on a given algorithm, it is said to be computable.
© schmiedecke 07 In2-2-Algor.Paradigms 27
Lamda Calculus Simplified
• Lambda stands for argument binding:
• Function f applied to argument a results r:��������������
• If f is x+2, and a is 7 we get��������������
• With nesting:���������������������������
� �������������� ��
• So, computation is described as rewriting of one list of terms by another
© schmiedecke 07 In2-2-Algor.Paradigms 28
The LISP language
• Describing computation as a set of rewriting rules has many offsprings in artificial intelligence.
• Here is a very early rewriting language, invented in 1958 by John Mc Carthy:
• LISP – List Programming
- The idea is that every program is a list of rewriting rules.- The result of a rewriting is again a list.- Terminal symbols can be specified to be excluded from
rewriting.- Lambda binding is applied to introduce parameters.- Special list operations cons, car and cdr allow list
manipulation and traversion.
• LISP is still in use – implemented in Unix, control language in the EMACS editior.
© schmiedecke 07 In2-2-Algor.Paradigms 29
Some LISP
(DEFUN THROW-DIE () (+ (RANDOM 6) 1) )
(DEFUN THROW-DICE () (LIST (THROW-DIE) (THROW-DIE)) )
(DEFUN BOXCARS-P (A B)
(AND (EQUAL '6 A) (EQUAL '6 B) ) )
(DEFUN SNAKE-EYES-P (A B)
(AND (EQUAL '1 A) (EQUAL '1 B) ) )
© schmiedecke 07 In2-2-Algor.Paradigms 30
Factorial in LISP
(DEFUN FAC (N)
(COND ((EQUAL N 1) 1)
((MULT N (FAC (MINUS N 1)) ) ) )
Evaluate Call of FAC (3) by Rewriting:
�FAC (3)�MULT 3 (FAC (MINUS 3 1))�MULT 3 (FAC (2))�MULT 3 (MULT 2 (FAC (MINUS 2 1))�MULT 3 (MULT 2 (FAC (1)))�MULT 3 (MULT 2 1)�MULT 3 2�6
© schmiedecke 07 In2-2-Algor.Paradigms 31
What's so Charming about LISP?
• Fully declarative – non-operational.• Completely general.• Easy to extend programs by adding definitions.• Results can be added as (redundant) definitions,
such as (DEFUN FAC(3) 6)
• � Supports artifical "learning"
© schmiedecke 07 In2-2-Algor.Paradigms 32
Recursive Fun:Monster Curves
• If you draw a simple shape repeatedly with increasing size and changing orientation, you get a pattern.
• The simplest way to do that is recursion: - draw a pattern- spawning a similar pattern of smaller size and modified
position and orientation- which again spawns another smaller pattern- until the size is below a certain limit
• Such patterns are sometimes called monster curves...• And they are fun doing �
© schmiedecke 07 In2-2-Algor.Paradigms 33
Some Monster Curves• Pythagoras Tree:
- Draw a square starting from ist base line- then, for a given angle, draw a pair of Pythagoras squares opposite to its base line
- continue for each sqare until too small.
• Sierpinski's Triangle:- Take a line between two points- Replace it by three lines of half that length,turning -60°, +60°,+60° left
- Replace each of these lines again by three,turning +60°, -60°, -60° for the first line,
-60°, +60°, +60° for the middle line,+60°, -60°, -60° for the last line.
- Repeat for each line, changing orientation every timeuntil too small
© schmiedecke 07 In2-2-Algor.Paradigms 35
Pythagoras
public void paint (Graphics g) {
g.clearRect(0,0,500,500);
paintTree(g, double x1, double y1, // tree defined double x2, double y2); // by base line
}
// paint tree paints the major square
// determines the base lines of the smaller squares
// and calls itself recursively for the two smaller squares