PowerPoint Presentationpeople.csail.mit.edu/rrw/6.045-2019/lec8-color.pdf9 Theorem: If ๐ณhas a streaming algorithm using Q space, then cc(๐๐ณQ4 + 4. Proof Idea: Alice runs streaming
Post on 18-Jan-2020
1 Views
Preview:
Transcript
1
6.045Lecture 8: More on
Communication Complexity,Start up Turing Machines
2
6.045Announcements:
- Dylanโs office hours? Vote!- Midterm: March 19
3
Communication Complexity
A theoretical model of distributed computing
โข Function f : {0,1}* ยฃ {0,1}* ! {0,1}
โ Two inputs, ๐ฅ โ {0,1}* and ๐ฆ โ {0,1}*
โ We assume |๐|=|๐|=๐. Think of ๐ as HUGE
โข Two computers: Alice and Bob
โ Alice only knows ๐ฅ, Bob only knows ๐ฆ
โข Goal: Compute f(๐, ๐) by communicating as few bits as possible between Alice and Bob
We do not count computation cost. We only care about the number of bits communicated.
4
A(x,ฮต) = 0
B(y,0) = 1
A(x,01) = 1
B(y,011) = 0
x y
f(x,y)=0f(x,y)=0
Def. A protocol for a function f is a pair of functionsA, B : {0,1}* ร {0,1}* โ {0, 1, STOP} with the semantics:
On input (๐, ๐), let ๐ := 0, ๐๐ = ฮต. While (๐๐ โ STOP),
๐ + +If ๐ is odd, Alice sends ๐๐ = ๐จ ๐, ๐๐โฏ๐๐โ๐
else Bob sends ๐๐ = ๐ฉ ๐, ๐๐โฏ๐๐โ๐Output ๐๐โ๐. Number of rounds = ๐ โ ๐
5
A(x,ฮต) = 0
B(y,0) = 1
A(x,01) = 1
B(y,011) = 0
x y
f(x,y)=0f(x,y)=0
Def. The cost of a protocol P for f on ๐-bit strings is
๐ฆ๐๐ฑ๐,๐ โ ๐,๐ ๐
[number of rounds in P to compute f(๐, ๐)]
The communication complexity of f on ๐-bit strings, cc(f),is minimum cost over all protocols for f on ๐-bit strings= the minimum number of rounds used by any protocol
computing f(๐, ๐), over all ๐-bit ๐, ๐
6
A(x,ฮต) = 0
B(y,0) = 1
A(x,01) = 1
B(y,011) = 0
x y
f(x,y)=0f(x,y)=0
Example. Let f : {0,1}* ร {0,1}* โ {0,1} be arbitrary
There is always a โtrivialโ protocol:Alice sends the bits of her ๐ in odd roundsBob sends whatever bit he wants in even roundsAfter ๐๐ โ ๐ rounds, Bob knows ๐ and can send ๐(๐, ๐)
Proposition: For every ๐, cc(๐) โค ๐๐
7
x y
f(x,y)=0f(x,y)=0
Example. EQUALS(๐, ๐) = 1 โ ๐ = ๐
Whatโs a good protocol for computing EQUALS?
????
Communication complexity of EQUALS is at most 2๐
8
x y
Examples:๐ณ = { x | x has an odd number of 1s}
โ ๐๐ณ ๐, ๐ = PARITY(x,y) = ฯ๐๐๐ + ฯ๐๐๐ mod 2๐ณ = { x | x has more 1s than 0s}
โ ๐๐ณ ๐, ๐ = MAJORITY(x,y)๐ณ = { xx | x โ {0,1}*}
โ ๐๐ณ ๐, ๐ = EQUALS(x,y)
Connection to Streaming and DFAs
Let ๐ณ โ {0,1}*Def. ๐๐ณ: {0,1}*ร{0,1}*โ {0,1}
for ๐, ๐ with |๐|=|๐| as:๐๐ณ ๐, ๐ = ๐ โ ๐๐ โ ๐ณ
9
Theorem: If ๐ณ has a streaming algorithm using โค ๐ space, then cc(๐๐ณ) โค 4๐ + 4.
Proof Idea: Alice runs streaming algorithm A on ๐, reaches a memory state ๐. She sends ๐ to Bob in 4s+3rounds. Then Bob starts up A from state ๐, runs A on ๐. Gets an output bit, sends bit to Alice.
(โฆwhy 4๐+3 rounds?)
Connection to Streaming and DFAs
x y
Let ๐ณ โ {0,1}*Def. ๐๐ณ: {0,1}*ร{0,1}*โ {0,1}
for ๐, ๐ with |๐|=|๐| as:๐๐ณ ๐, ๐ = ๐ โ ๐๐ โ ๐ณ
10
Theorem: If ๐ณ has a streaming algorithm using โค ๐ space, then cc(๐๐ณ) โค 4๐ + 4.
Corollary: For every regular ๐ณ, cc(๐๐ณ) โค O(1).
Example: cc(PARITY) = 2
Corollary: cc(MAJORITY) โค O(log n),because thereโs a streaming algorithm for {x : x has more 1โs than 0โs} with O(log n) space
What about the Comm. Complexity of EQUALS?
Connection to Streaming and DFAs
Let ๐ณ โ {0,1}* Def. ๐๐ณ ๐, ๐ = ๐ โ ๐๐ โ ๐ณ
11
Theorem: cc(EQUALS) = ๐ฏ(๐).In particular, every communication protocol for EQUALS needs to send โฅ ๐ bits.
No communication protocol can do much better than โsend your whole inputโ!
Corollary: ๐ณ = {xx | x in {0,1}*} is not regular.
Moreover, every streaming algorithm for ๐ณneeds ๐ ๐ bits of memory, for some constant ๐ > 0!
Communication Complexity of EQUALS
12
Theorem: cc(EQUALS) = ๐ฏ(๐). In particular, everyprotocol for EQUALS needs โฅ ๐ bits of communication!
Idea: Consider all possible ways A & B can communicate.
Definition: The communication pattern of a protocol on inputs (๐, ๐) is the sequence of bits Alice & Bob send.
0
1
1
0
Pattern: 0110๐ ๐
Communication Complexity of EQUALS
13
A(xโ,ฮต) = 0
B(yโ,0) = 1
A(xโ,01) = 1
B(yโ,011) = 0
๐โ ๐โ
A(x,ฮต) = 0
B(y,0) = 1
A(x,01) = 1
B(y,011) = 0
๐ ๐
Key Lemma: If (๐, ๐) and (๐โฒ, ๐โฒ) have the same pattern ๐ทin a protocol, then (๐, ๐โฒ) and (๐โฒ, ๐) also have pattern ๐ท
14
A(xโ,ฮต) = 0
B(yโ,0) = 1
A(xโ,01) = 1
B(yโ,011) = 0
๐โ ๐โ
A(x,ฮต) = 0
B(yโ,0) = 1
A(x,01) = 1
B(yโ,011) = 0
๐ ๐
Key Lemma: If (๐, ๐) and (๐โฒ, ๐โฒ) have the same pattern ๐ทin a protocol, then (๐, ๐โฒ) and (๐โฒ, ๐) also have pattern ๐ท
15
Key Lemma: If (๐, ๐) and (๐โฒ, ๐โฒ) have the same pattern ๐ทin a protocol, then (๐, ๐โฒ) and (๐โฒ, ๐) also have pattern ๐ท
A(x,ฮต) = 0
B(yโ,0) = 1
A(x,01) = 1
B(yโ,011) = 0
๐โ ๐โ
A(x,ฮต) = 0
B(y,0) = 1
A(x,01) = 1
B(y,011) = 0
๐ ๐
16
Theorem: The comm. complexity of EQUALS is ๐ฏ(๐).In particular, every protocol for EQUALS needs โฅ ๐ bits of communication.
Proof: By contradiction. Suppose cc(EQUALS) โค ๐ โ ๐.Then there are โค ๐๐ โ ๐ possible communication patterns of that protocol, over all pairs of inputs (๐, ๐) with n bits each.
Claim: There are ๐ โ ๐ such that on (๐, ๐) and on (๐, ๐), the protocol uses the same pattern ๐ท.
By the Key Lemma, (๐, ๐) and ๐, ๐ also use pattern ๐ท
So Alice & Bob output the same bit on (๐, ๐) and (๐, ๐).But EQUALS(๐, ๐) = 0 and EQUALS(๐, ๐) = 1. Contradiction!
Communication Complexity of EQUALS
17
Randomized Protocols Help!
EQUALS needs โฅ ๐ bits of communication, butโฆ
Theorem: For all ๐, ๐ of ๐ bits each, there is a randomized protocol for EQUALS ๐, ๐ using
only O(log ๐) bits of communication, which is correct with probability 99.9%!
18
Turing Machines
Turing Machine (1936)
FINITE
STATE
CONTROL
INFINITE REWRITABLE TAPE
I N P U T
q0q1
A โฆ
19
In each step:- Reads a symbol- Writes a symbol- Changes state- Moves Left or Right
Turing Machine (1936)
INFINITE REWRITABLE TAPE
I N P U TA โฆ
20
21
https://www.cs.utah.edu/~draperg/cartoons/2005/turing.html
Turing Machines versus DFAs
The TM can both write to and read from the tape,and can write symbols that arenโt part of input
The โtape headโ can move right and left
The input is written on an infinite tape
Accept and Reject take immediate effect
with โblankโ symbols after the input
22
23
Deciding the language L = { w#w | w {0,1}* }
STATE
0 1 1 # 0 1 10 #1 0 1X XX
1. If thereโs no # on the tape (or more than one #), reject. 2. While there is a bit to the left of #,
Replace the first bit with X, and check if the first bit bto the right of the # is identical. (If not, reject.) Replace that bit b with an X too.
3. If thereโs a bit to the right of #, then reject else accept
Definition: A Turing Machine is a 7-tuple T = (Q, ฮฃ, ฮ, , q0, qaccept, qreject), where:
Q is a finite set of states
ฮ is the tape alphabet, where ฮ and ฮฃ ฮ
q0 Q is the start state
ฮฃ is the input alphabet, where ฮฃ
: Q ฮ โ Q ฮ {L, R}
qaccept Q is the accept state
qreject Q is the reject state, and qreject qaccept
24
= โblankโ
25
0 โ 0, R
read write move
โ , R
qaccept
qreject
0 โ 0, R
โ , R
This Turing machine decides the language {0}
ฮฃ = {0}
= โblankโ
26
0 โ 0, R
read write move
โ , R
qaccept
0 โ 0, R
โ , R
0 โ 0, R
โ , L
This Turing machine recognizes the language {0}
ฮฃ = {0}
= โblankโ
Turing Machine Configurations
q01101000110 2 (Q [ ฮ)*
corresponds to the configuration:
q0
1 0 0 0 0 01 1 1 1
27
Turing Machine Configurations
0q1101000110 2 (Q [ ฮ)*
corresponds to the configuration:
q1
1 0 0 0 0 00 1 1 1
28
Turing Machine Configurations
0000011110q7 2 (Q [ ฮ)*
corresponds to the configuration:
q7
0 0 0 1 1 00 0 1 1
29
Defining Acceptance and Rejection for TMs
Let C1 and C2 be configurations of a TM MDefinition. C1 yields C2 if M is in configuration C2
after running M in configuration C1 for one step
Example. Suppose (q1, b) = (q2, c, L)Then aq1bb yields q2acbSuppose (q1, a) = (q2, c, R)Then abq1a yields abcq2
Let w ฮฃ* and M be a Turing machine.M accepts w if there are configs C0, C1, ..., Ck, s.t.
โข C0 = q0w [the initial configuration]โข Ci yields Ci+1 for i = 0, ..., k-1, and โข Ck contains the accept state qaccept
30
accepting computation
history of M on x
A TM M recognizes a language L if M accepts exactly those strings in L
A TM M decides a language L if M accepts all strings in L and rejects all strings not in L
A language L is recognizable (a.k.a. recursively enumerable)
if some TM recognizes L
A language L is decidable (a.k.a. recursive)if some TM decides L
31
A Turing machine for deciding { 0 | n โฅ 0 }2n
1. Sweep from left to right, x-out every other 02. If in step 1, the tape had only one 0, accept3. If in step 1, the tape had an odd number of 0โs,
reject4. Move the head left to the first input symbol.5. Go to step 1.
Turing Machine PSEUDOCODE:
Why does this work?
Observation: Every time we return to step 1, the number of 0โs on the tape has been halved.
32
even0โs
Step 1
odd 0โs
0 โ , R
โ , R
qacceptqreject
0 โ x, R
x โ x, Rโ , R
x โ x, R
0 โ 0, Lx โ x, L
x โ x, R
โ , Lโ , R
0 โ x, R0 โ 0, R
โ , Rx โ x, R
q0 q1
q2
q3
q4
{ 0 | n โฅ 0 }2n
Step 2
Step 3
Step 4
33
0 โ , R
โ , R
qacceptqreject
0 โ x, R
x โ x, Rโ , R
x โ x, R
0 โ 0, Lx โ x, L
x โ x, R
โ , Lโ , R
0 โ x, R0 โ 0, R
โ , Rx โ x, R
{ 0 | n โฅ 0 }2n
q0 q1
q2
q3
q4
q00000
q1000
xq300
x0q40
x0xq3
x0q2x
xq20x
q2x0x
q2x0x
โฆ 34
q1x0x
xq10x
Multitape Turing Machines
: Q ฮk โ Q ฮk {L,R}k
FINITE
STATE
CONTROLk
35
FINITE
STATE
CONTROL
0 01
FINITE
STATE
CONTROL 0 01 # # #. . .
Theorem: Every Multitape Turing Machine can be transformed into a single tape Turing Machine
36
FINITE
STATE
CONTROL
0 01
FINITE
STATE
CONTROL 0 01 # # #. . .
Theorem: Every Multitape Turing Machine can be transformed into a single tape Turing Machine
37
FINITE
STATE
CONTROL
0 01
FINITE
STATE
CONTROL 0 01 # # #. . .
Theorem: Every Multitape Turing Machine can be transformed into a single tape Turing Machine
38
FINITE
STATE
CONTROL
0 01
FINITE
STATE
CONTROL 0 01 # # #. .
Theorem: Every Multitape Turing Machine can be transformed into a single tape Turing Machine
39
.
Theorem: Every Multitape Turing Machine can be transformed into a single tape Turing Machine
FINITE
STATE
CONTROL
0 01
FINITE
STATE
CONTROL 0 01 # # #. . .
40
41
Theorem: Every nondeterministic Turing machine Ncan be transformed into a Turing Machine M that accepts precisely the same strings as N.
Nondeterministic Turing Machines
Proof Idea (more details in Sipser p.178-179)Pick a natural ordering on the strings in (Q [ ฮ [ #)*
M(w): For all strings D 2 (Q [ ฮ [ #)* in the ordering,Check if D = C0# #Ck where C0, โฆ,Ck is an
accepting computation history for N on w. If so, accept.
Have multiple transitions for a state, symbol pair
Fact: We can encode Turing Machines as bit strings
0n10m10k10s10t10r10u1 โฆ
n states
m tape symbols
(first k are input
symbols)
start
state
accept
state
reject
state
blank
symbol
( (p, i), (q, j, L) ) = 0p10i10q10j101
( (p, i), (q, j, R) ) = 0p10i10q10j100142
Similarly, we can encode DFAs and NFAs as bit strings, and w 2 ฮฃ* as bit strings
ADFA = { (B, w) | B encodes a DFA over some ฮฃ,and B accepts w 2 ฮฃ* }
ANFA = { (B, w) | B encodes an NFA, B accepts w }
ATM = { (M, w) | M encodes a TM, M accepts w }
For x 2 ฮฃ* define bฮฃ(x) to be its binary encodingFor x, y 2 ฮฃ*, define the pair of x and y as
(x, y) := 0|bฮฃ(x)|1 bฮฃ(x) bฮฃ(y)Then we define the following languages over {0,1}:
43
44
ATM = { (M, w) | M encodes a TM over some ฮฃ,w encodes a string over ฮฃ and M accepts w}
ATM = { z | z decodes to (M, w) and M does not accept w }
Technical Note: Weโll use an decoding of pairs, TMs, and strings so that every binary string decodes to some pair (M, w)
If z โ {0,1}* doesnโt decode to (M, w) in the usual way, then we define that z decodes to the pair (D, ฮต)
where D is a โdummyโ TM that accepts nothing.
Universal Turing Machines
Theorem: There is a Turing machine Uwhich takes as input:- the code of an arbitrary TM M- and an input string wsuch that U accepts (M, w)M accepts w.
This is a fundamental property of TMs:
There is a Turing Machine that can run arbitrary Turing Machine code!
Note that DFAs/NFAs do not have this property.That is, ADFA and ANFA are not regular.
45
46
ADFA = { (D, w) | D is a DFA that accepts string w }
Theorem: ADFA is decidable
Proof: A DFA is a special case of a TM. Run the universal U on (D, w) and output its answer.
ATM = { (M, w) | M is a TM that accepts string w }
Theorem: ATM is recognizable
ANFA = { (N, w) | N is an NFA that accepts string w }
Theorem: ANFA is decidable. (Why?)
The Church-Turing Thesis
Everyoneโs Intuitive Notion
of Algorithms
= Turing Machines
This is not a theorem โit is a falsifiable scientific hypothesis.
And it has been thoroughly tested!
w L ?
accept reject
TM
yes no
w ฮฃ*
L is decidable
(recursive)
w L ?
accept reject or loop
TM
yes no
w ฮฃ*
L is recognizable
(recursively enumerable)
Theorem: L is decidableiff both L and L are recognizable
48
Recall: Given L ยต ฮฃ*, define L := ฮฃ* \ L
How? Any ideas?M1 always accepts x, when x is in L
M2 always accepts x, when x isnโt in L
Theorem: L is decidableiff both L and L are recognizable
Given:a TM M1 that recognizes L anda TM M2 that recognizes L,
want to build a new machine M that decides L
49
Theorem: L is decidableiff both L and L are recognizable
Decider for L:Simulate M1(x) on one tape, while also simulating M2(x) on another tape. Exactly one will accept. If M1 accepts then accept. If M2 accepts then reject
Recall: Given L ยต ฮฃ*, define L := ฮฃ* \ L
Given:a TM M1 that recognizes L anda TM M2 that recognizes L,
want to build a new machine M that decides L
50
top related