1 R. Rao, CSE 599 Week 2: Theoretical Foundations (Part II) CSE 599 Lecture 2 In the previous lecture, we discussed: What is Computation? History of Computing Theoretical Foundations of Computing Abstract Models of Computation Finite Automata Languages The Chomsky Hierarchy Turing Machines
48
Embed
1 R. Rao, CSE 599 Week 2: Theoretical Foundations (Part II) CSE 599 Lecture 2 F In the previous lecture, we discussed: íWhat is Computation? íHistory of.
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Show that there are more real numbers than natural numbers
Proof: Form a 1-1 mapping from natural numbers to reals, and form a new real number by changing the ith digit of the ith real number: For example, if 1-1 map is given by:
1 0.1000
2 0.0345451
3 0.749399845
4 0.33333333333333….
etc.
New Real Number =0.2404…….(Add 1 to diagonal)This number is different from all the ones listed
Question: Are there problems that no algorithm can solve?
Consider the Halting Problem: Is there a general algorithm that can tell us whether a Turing machine T with tape t will halt, for any given T and input t?
Answer: No!
Proof: By contradiction. Suppose such an algorithm exists. Let D be the corresponding Turing machine. Note that D is just like a UTM except that it is guaranteed to halt: D halts with a “Yes” if T halts on input t D halts with a “No” if T does not halt on input t
We can now make a distinction between two types of computability: Decidable (or recursive) Turing Computable (or partial recursive/recursively enumerable)
A language is decidable if there is a TM that accepts every string in that language and halts, and rejects every string not in the language and halts.
A language is Turing computable if there is a TM that accepts every string in that language (and no strings that are not)
The language 0n1n is decidable: If the tape is empty, ACCEPT Otherwise, if the input does not look like 0* 1*, REJECT Cross off the first 0. Then move right until the first 1 (not crossed
off) Cross off that 1, then move left until the first 0. Repeat the above 2 steps until we run out of 0’s or 1’s
If there are more 0’s than 1’s REJECT If there are more 1’s than 0’s REJECT If there are an equal number, ACCEPT
L = { 1n | n is a composite number } is decidable
All decidable languages are Turing computable. Are there Turing computable languages that are not decidable?
The Halting Problem is Turing Computable: HALT = { <dT,t > | dT is a description of TM T, and T halts on
input t} Proof Sketch: The following UTM H accepts HALT
Simulate T on input t. If T halts, then ACCEPT
Crucial Point: H may not halt in some cases because T doesn’t, but if T does halt, so does H. So L(H) = HALT.
Hilbert’s 10th problem: Given a polynomial equation (e.g. 7x2-5xy-3y2+2x-11=0, or x3+y3=z3), give an algorithm that says whether the equation has at least one integer solution. Try all possible tuples of integers. If one of the tuples is a solution, ACCEPT
Software verification is impossible without restrictions You can’t get around this, but you can reduce the pathological cases Be careful-- these pathological cases won’t go away
There are mathematical facts we can’t prove: Gödels Theorem: Any arithmetic system large enough to contain Q
(a subset of number theory) will contain unprovable statements Based on constructing the statement: “This statement is unprovable” Uses numerical encodings of statements (called Gödel numbers) like
those for a TM
Any sufficiently complex system will have holes in it.
We have seen the limitations of sequential machines that transition from one state to another unique state at each time step.
Consider a new model of computation where at each step, the machine may have a choice of more than one state. For the same input, the machine may follow different computational
paths when run at different times
If there exists any path that leads to an accept state, the machine is said to accept the input
Here’s one example: There is only one nondeterministic transition in this machine What strings does this machine accept? Are NFAs more powerful than DFAs?
DTIME(t(n)) = All languages decided by a deterministic TM in time O(t(n))
P = k1 DTIME (nk)
Importance of P: It corresponds to our notion of the class of problems that can be solved efficiently in time (runs in polynomial number of steps with respect to size of input)
Example: DUP is in P; so is sorting.
P for a TM P in most other models Multitape TM, different alphabet, the RAM model DNA Computing Not necessarily for nondeterministic TMs Not necessarily for Quantum Computers
A verifier for a language L is an algorithm V, where L={ w | V accepts < w,c > for some string c that is “evidence” of w’s membership in L} We measure the time of the verifier in terms of w -- not c, the
evidence The language L is polynomially verifiable if it has a polytime
verifier
NP is the class of languages that have polynomial time verifiers
SAT = { Boolean formulas f | f has a satisfying assignment} Evidence c is an assignment of variables that makes f true
VERTEX-COVER = { < G, k > | G is an undirected graph that has a k-node vertex cover i.e. all edges are “covered” by at least one vertex from a set of k vertices}
TSP = {<C,b> | there is a “tour” of all the cities in C with total length no more than b}
Basic Idea: Use one problem to solve another Problem A is reducible to problem B if you can transform any instance of
problem A to an instance of problem B and solve problem A by solving problem B
Example: Language ACC = { <dT ,t> | T is a TM that accepts input t } HALT = { <dT,t> | T halts on input t} (Let H decide HALT) ACC is reducible to HALT: On input < dT, t >,
Run H on input < dT, t >. If H rejects (T does not halt on t), then REJECT Else, simulate T on input t.
A large number of problems in optimization, engineering, computer science and mathematics are known to be NP complete, including problems in compiler optimization, scheduling, etc.
No one has found an efficient (polynomial time) algorithm for any NP complete problem
If someone finds a polynomial time algorithm for any one NP complete problem, then we can solve all NP complete problems (and all problems in NP) efficiently in polynomial time.
Cook and Levin independently proved that SAT is NP complete
Proof involves constructing a very large Boolean formula that captures the operation of a nondeterministic TM N that runs in polynomial time and solves a problem A in NP
The large formula takes into account: Basic facts such as: N can be in only one state q at any time t, a tape
cell can only contain 1 symbol, read/write head is scanning 1 cell etc. e.g. [S(t,q) ~S(t,q’)] for all q q’ and for t = 0, 1, …, nk
Initial and final conditions after nk steps have been executed N’s program i.e. list of quintuples
Crucial facts: It takes only a polynomial amount of time to generate the Boolean
formula for any NTM The Boolean formula limits the NTM to behaving just as it should Thus, the constructed formula is satisfiable if and only if the NTM
halts in nk time steps and outputs a “Yes” on its tape (which means the original NP problem has the answer “Yes” for the given input)
We have thus shown that any NP problem is polynomial time reducible to SAT i.e. SAT is NP complete
Now, suppose you have a new problem you suspect is NP complete: to show that it is, just reduce SAT to the problem!
VERTEX-COVER = { < G, k > | G is an undirected graph that has a k-node vertex cover} Vertex cover is a subset of nodes such that every edge in the graph
touches at least one node in the cover
Show that VERTEX-COVER is NP complete Proof:
Show that VERTEX-COVER is in NP Show that SAT is polytime reducible to VERTEX-COVER
Does the TM model apply to neural computing? Neurons compute using distributed signals and stochastic pulses Is thinking about symbol processing the wrong way to think about
neural systems? Could some other model (e.g. probabilistic computing) provide us
with a way to describe neural processing?
How useful is the TM model in capturing the abstract computations involved in DNA computing and Quantum computing?
Keep these questions in mind as we explore alternative computing paradigms…
The physical basis is silicon integrated-circuit technology Guest Lecture by Chris Diorio on IC technology (first hour or so)
We will discuss the theory and practice of digital computing, and end by examining their future: Moore’s law and semiconductor scaling in the years to come.