1 Module 5 • Topics – Proof of the existence of unsolvable problems • Proof Technique – There are more problems/languages than there are programs/algorithms – Countable and uncountable infinities
Dec 20, 2015
1
Module 5
• Topics– Proof of the existence of unsolvable problems
• Proof Technique– There are more problems/languages than there are
programs/algorithms
– Countable and uncountable infinities
2
Overview
• We will show that there are more problems than programs– Actually more problems than programs in any
computational model (programming language)
• Implication– Some problems are not solvable
4
Define set of problems
• We will restrict the set of problems to be the set of language recognition problems over the alphabet {a}.
• That is– Universe: {a}*– Yes Inputs: Some language L subset of {a}*– No Inputs: {a}* - L
5
Set of Problems *• The number of distinct problems is given by the
number of languages L subset of {a}*– 2{a}* is our shorthand for this set of subset languages
• Examples of languages L subset of {a}*– 0 elements: { }
– 1 element: {/\}, {a}, {aa}, {aaa}, {aaaa}, …
– 2 elements: {/\, a}, {/\, aa}, {a, aa}, …
– Infinite # of elements: {an | n is even}, {an | n is prime}, {an | n is a perfect square}
6
Infinity and {a}*
• All strings in {a}* have finite length
• The number of strings in {a}* is infinite
• The number of languages L in 2{a}* is infinite
• The number of strings in a language L in 2{a}* may be finite or infinite
7
Define set of programs
• The set of programs we will consider are the set of legal C++ programs as defined in earlier lectures
• Key Observation– Each C++ program can be thought of as a finite
length string over alphabet P
• P = {a, …, z, A, …, Z, 0, …, 9, white space, punctuation}
8
Example *
int main(int A[], int n){ {26 characters including newline}
int i, max; {13 characters including initial tab}
{1 character: newline}
if (n < 1) {12 characters}
return (“Illegal Input”); {28 characters including 2 tabs}
max = A[0]; {13 characters}
for (i = 1; i < n; i++) {25 characters}
if (A[i] > max) {18 characters}max = A[i]; {15 characters}
return (max); {15 characters}
} {2 characters including newline}
9
Number of programs
• The set of legal C++ programs is clearly infinite
• It is also no more than |P*|
– P = {a, …, z, A, …, Z, 0, …, 9, white space, punctuation}
10
Goal
• Show that the number of languages L in 2{a}* is greater than the number of strings in P*
– P = {a, …, z, A, …, Z, 0, …, 9, white space, punctuation}
• Problem– Both are infinite
12
Bijections
• Two sets have EQUAL size if there exists a bijection between them– bijection is a 1-1 and onto function between
two sets
• Examples– Set {1, 2, 3} and Set {A, B, C}– Positive even numbers and positive integers
14
Proper subset
• Finite sets– S1 proper subset of S2 implies S2 is strictly
bigger than S1• Example
– women proper subset of people
– number of women less than number of people
• Infinite sets– Counterexample
• even numbers and integers
16
Countably infinite set S *
• Definition 1– S is equal in size (bijection) to N
• N is the set of natural numbers {1, 2, 3, …}
• Definition 2 (Key property)– There exists a way to list all the elements of set S (enumerate S) such that the following is true
• Every element appears at a finite position in the infinite list
17
Uncountable infinity *
• Any set which is not countably infinite
• Examples– Set of real numbers– 2{a}*, the set of all languages L which are a
subset of {a}*
• Further gradations within this set, but we ignore them
19
(1) The set of all legal C++ programs is countably infinite
• Every C++ program is a finite string
• Thus, the set of all legal C++ programs is a language LC
• This language LC is a subset of P*
20
For any alphabet , * is countably infinite
• Enumeration ordering– All length 0 strings
• ||0 = 1 string:
– All length 1 strings• || strings
– All length 2 strings• ||2 strings
– …
• Thus, P* is countably infinite
21
Example with alphabet {a,b} *
• Length 0 strings– 0 and
• Length 1 strings– 1 and a, 2 and b
• Length 2 strings– 3 and aa, 4 and ab, 5 and ba, 6 and bb, ...
• Question– write a program that takes a number as input and
computes the corresponding string as output
22
(2) The set of languages in 2{a}* is uncountably infinite
• Diagonalization proof technique – “Algorithmic” proof– Typically presented as a proof by contradiction
23
Algorithm Overview *
• To prove this set is uncountably infinite, we construct an algorithm D that behaves as follows:– Input
• A countably infinite list of languages L[] subset of {a}*
– Output• A language D(L[]) which is a subset of {a}* that is not
on list L[]
– Declaration of algorithm D?
25
Why existence of D implies result
• If the number of languages in 2{a}* is countably infinite, there exists a list L[] s.t.– L[] is complete
• it contains every language in 2{a}*
– L[] is countably infinite
• The existence of algorithm D implies that no list of languages in 2{a}* is both complete and countably infinite– Specifically, the existence of D shows that any
countably infinite list of languages is not complete
26
Visualizing One Possible L[ ] *
L[0]
L[1]
L[2]
L[3]
L[4]...
a aa aaa aaaa ...
IN ININININ
OUTIN IN INOUT
OUT OUT OUT OUT OUT
IN INOUT OUT OUT
ININOUT OUTOUT
•#Rows is countably infinite
•Given
•#Cols is countably infinite
• {a}* is countably infinite
• Consider each string to be a feature– A set contains or does not contain each string
27
Constructing D(L[ ]) *• We construct D(L[]) by using a unique feature
(string) to differentiate D(L[]) from L[i]– Typically use ith string for language L[i]– Thus the name diagonalization
L[0]
L[1]
L[2]
L[3]
L[4]...
a aa aaa aaaa ...
IN ININININ
OUTIN IN INOUT
OUT OUT OUT OUT OUT
IN INOUT OUT OUT
ININOUT OUTOUT
OUT
IN
IN
IN
OUT
D(L[])
28
Questions *
• Do we need to use the diagonal?– Every other column and every row?– Every other row and every column?
• What properties are needed to construct D(L[])?
L[0]
L[1]
L[2]
L[3]
L[4]...
a aa aaa aaaa ...
IN ININININ
OUTIN IN INOUT
OUT OUT OUT OUT OUT
IN INOUT OUT OUT
ININOUT OUTOUT
29
Visualization
Solvable Problems
The set of solvable problems is a proper subset of the set of all problems.
All problems
30
Summary
• Equal size infinite sets: bijections– Countable and uncountable infinities
• More languages than algorithms– Number of algorithms countably infinite– Number of languages uncountably infinite– Diagonalization technique
• Construct D(L[]) using infinite set of features
• The set of solvable problems is a proper subset of the set of all problems