The Power of Randomness in Computation 呂及人中研院資訊所.

The Power of The Power of Randomness in Randomness in ComputationComputation

呂及人呂及人中研院資訊所中研院資訊所

PART I:PART I: RandomizationRandomization

Random SamplingRandom Sampling

PollingPolling

With probability >99%% in population = % in sample 5%independent of population size

Population: 20 million, voting yellow or red

Random Sample: 3,000

LessonLesson

A small set of random samples A small set of random samples gives a good picture of the whole gives a good picture of the whole population.population.

Allow Allow sub-linearsub-linear time algorithms! time algorithms! More applications:More applications:

– Volume estimationVolume estimation– ClusteringClustering– Machine learning, ...Machine learning, ...

FingerprintsFingerprints

ProblemProblem

Alice: x nn

x = y ?

Measure: communication complexity

Bob: y nn

First AttemptFirst Attempt

Alice: x nn

x = y ? Bob: y nn

i r{1..n}i, xi xi = yi ?

Works only when ((xx,,yy)) is large

SolutionSolution

Alice: x nn

Bob: y nn

x y: Probi[C(x)i=C(y)i] 0

x = y: Probi[C(x)iC(y)i] = 00

i r{1..m}i, C(x)i C(x)i = C(y)i ?

C: error-correcting code

with 1

xC(x) yC(y)

can repeat several times to reduce error

LessonLesson

Transform the data, before Transform the data, before random sampling!random sampling!

Dimensionality Dimensionality ReductionReduction Raw data Raw data AA {0,1}{0,1}nn, for very large , for very large

nn..– e.g. images, voices, DNA sequences, e.g. images, voices, DNA sequences,

......– ||AA| << 2| << 2nn..

Goal: Goal: – compressing each element of compressing each element of AA, ,

while keeping its “essence”while keeping its “essence”

Proof SystemsProof Systems

Classical Proof Classical Proof SystemsSystems Prover: provides the proof.Prover: provides the proof.

– Hard.Hard. Verifier: verifies the proof.Verifier: verifies the proof.

– Relatively easy!Relatively easy!– Still needs to read through the proof.Still needs to read through the proof.– What if you, the reviewer, receive a papWhat if you, the reviewer, receive a pap

er of 300 pages to verify...er of 300 pages to verify...

Probabilistically Probabilistically Correct Proof (PCP)Correct Proof (PCP)

Verifier: Verifier: flips “some” random coinsflips “some” random coins reads only a “small” parts of the reads only a “small” parts of the

proofproof tolerates a “small” errortolerates a “small” error

Proof?Proof?

A format of arguments agreed upon A format of arguments agreed upon by Prover and Verifier by Prover and Verifier – soundness & completeness.soundness & completeness.

Choosing a good proof formatChoosing a good proof format Fast & simple verification!Fast & simple verification!

Probabilistically Probabilistically Correct Proof (PCP)Correct Proof (PCP)

Prover: Prover: transforms the proof transforms the proof

by encoding it with some error correby encoding it with some error correcting (testing) code! cting (testing) code!

PCP for NPPCP for NP

NP = PCP (NP = PCP (O(logO(log nn), 3), 3).). NP contains SNP contains SATAT, T, TSPSP, ..., and, ..., and

MMATHATH = { = {((SS,1,1tt)) : ZFC : ZFC ||== SS in in tt steps}. steps}.

Graph Non-Graph Non-IsomorphismIsomorphism

Isomorphic?Isomorphic?

GG11 GG22

Isomorphic!Isomorphic!

GG11 GG22

ProblemProblem

Input: two graphs Input: two graphs GG11 and and GG22

Output: yes iff Output: yes iff GG11 and and GG2 2 are are not not isomisomorphic.orphic.

GG11 iso. iso. GG2 2 short proof short proof (GNSIO (GNSIO co-NP) co-NP)

GG11 not iso. not iso. GG2 2 short proof ??? short proof ???

Randomized AlgorithmRandomized Algorithm

Verifier: Verifier: – Picks a random Picks a random ii {1,2} {1,2}– Sends Sends GG, a random permutation of , a random permutation of GGii

Prover:Prover:– Sends Sends jj {1,2} {1,2}

Verifier:Verifier:– Outputs “non-isomorphic” iff Outputs “non-isomorphic” iff i i = = jj..

New FeaturesNew Features

Non-transferable proofsNon-transferable proofs Zero-knowledge proofsZero-knowledge proofs IP=PSACEIP=PSACE

““a lot more can be proved a lot more can be proved efficiently”efficiently”

ReachabilityReachability

ProblemProblem

Input: Input: undirectedundirected graph graph GG and two n and two nodes odes ss, , tt

Output yes iffOutput yes iff s s is connected tois connected to t t inin G G

Complexity: poly(Complexity: poly(nn) time! ) time! Question: O(logQuestion: O(log nn)) space?space?

number of nodes

Randomized AlgorithmRandomized Algorithm

Take a random walk a length poly(Take a random walk a length poly(nn) fro) from m ss..

Output yes iff Output yes iff tt is visited during the walk. is visited during the walk.

Complexity: randomized O(logComplexity: randomized O(log nn) space ) space – only need to remember the current nodeonly need to remember the current node

LessonLesson

Interesting probabilistic Interesting probabilistic phenomenon behind:phenomenon behind:– Mixing rate of Markov chainMixing rate of Markov chain

(related to resistance of electrical (related to resistance of electrical networks)networks)

Primality TestingPrimality Testing

ProblemProblem

Input: a number Input: a number xx Output: yes iff Output: yes iff xx is a prime is a prime Important in cryptography, ...Important in cryptography, ...

Randomized AlgorithmRandomized Algorithm

Generate a random Generate a random r r { {11, ..., , ..., xx}} Output yes iffOutput yes iff

– GCD (GCD (xx, , rr) = 1 & ) = 1 & – [[rr//xx] ] rr((xx-1)/2-1)/2 (mod (mod xx))

Jacobi symbol

PART II:PART II: DerandomizatioDerandomizationn

IssuesIssues

Randomized algorithmRandomized algorithm MM for for AA::– MM has access to perfectly random has access to perfectly random yy xx, Prob, Probyy[ [ MM((xx,,yy) ) AA((xx) ] < 0.000000001) ] < 0.000000001

Issues?Issues?– Small probability of error.Small probability of error.– Need Need perfectly randomperfectly random yy. . How?How?

SolutionsSolutions

Randomness extractorsRandomness extractors Pseudo-random generatorsPseudo-random generators DerandomizationDerandomization

Randomness Randomness ExtractorsExtractors

SettingSetting

slightly random almost random

EXT

short random seed: catalyst

Goal: short seed,short seed, long long outputoutput

ApplicationsApplications

Complexity Complexity CryptographyCryptography Data structuresData structures Distributed computingDistributed computing Error-correcting codesError-correcting codes Combinatorics, graph theoryCombinatorics, graph theory ......

Pseudo-Random Pseudo-Random GeneratorsGenerators

Random?Random?

Are coin tosses really random?Are coin tosses really random? They “look random” to you, They “look random” to you,

because you don’t have enough because you don’t have enough power (computation / power (computation / measurement).measurement).

In many cases, “look random” is In many cases, “look random” is good enough!good enough!

PRGPRG

random seed

pseudo-random

PRG

Goal: short seed,short seed, long long outputoutput

DefinitionDefinition

G: {0,1}{0,1}nn {0,1}{0,1}mm, for n<m, is an -PRG against a complexity class C:

predicate T C, | Probr[T(G(r)) = 1] Proby[T(y) = 1] | < .

PRG exists?PRG exists?

From an “From an “average-caseaverage-case hardhard” func” functiontion

ff: {0,1}: {0,1}nn {0,1}{0,1},,define PRGdefine PRG GG: {0,1}: {0,1}nn {0,1}{0,1}nn+1+1 as as

GG((rr) = ) = rr 。。 ff((rr))

PRG exists?PRG exists?

From an “From an “worst-caseworst-case hardhard” functio” functionn

ff: {0,1}: {0,1}nn {0,1}{0,1},,define PRGdefine PRG GG: {0,1}: {0,1}nn {0,1}{0,1}nn+1+1 as as

GG((rr) = ) = rr 。。 ff((rr)) From a one-way function... From a one-way function...

Pseudo-RandomnessPseudo-Randomness

Foundation of cryptographyFoundation of cryptography– Public-key encryptionPublic-key encryption– zero-knowledge proofs, zero-knowledge proofs, – secure function evaluation, ...secure function evaluation, ...

Secret is there, but it looks randomSecret is there, but it looks random More applications: learning More applications: learning

theory, mathematics, physics, ...theory, mathematics, physics, ...

DerandomizatoinDerandomizatoin

Open ProblemsOpen Problems

Does randomness help poly-time / lDoes randomness help poly-time / log-space / nondet. poly-time compog-space / nondet. poly-time computation?utation?BPP = P?BPP = P? BPL = L?BPL = L? BPNP = NP?BPNP = NP?

Open ProblemsOpen Problems

Is there a PRG with seed length Is there a PRG with seed length O(loO(logg nn)) that fools poly-time / log-space that fools poly-time / log-space / nondet. poly-time computation? / nondet. poly-time computation?

DerandomizationDerandomization

Rand. algorithm M for language A:Rand. algorithm M for language A:ProbProbyy [ M([ M(xx,,yy) = A() = A(xx) ] > 0.99, ) ] > 0.99, xx

Construct PRG Construct PRG GG (fooling M)(fooling M) s.t.s.t.ProbProbrr [ M([ M(xx,,GG((rr))) = A() = A(xx) ] > 0.5, ) ] > 0.5, xx

To determine A(To determine A(xx),), take majority votake majority vote ofte of M(M(xx,,GG((rr))) over all possible ) over all possible rr..

BreakthroughsBreakthroughs

Primality Primality P: P: Agrawal-Kayal-Saxena 2002Agrawal-Kayal-Saxena 2002

Undirected Reachability Undirected Reachability L: L:Reingold 2005Reingold 2005

Still OpenStill Open

Graph non-isomorphism in NP?Graph non-isomorphism in NP?(If two graphs are non-isomorphic, is (If two graphs are non-isomorphic, is there always a short proof for that?)there always a short proof for that?)

ConclusionConclusion

Randomness is usefulRandomness is useful Interesting probabilistic Interesting probabilistic

phenomena behindphenomena behind Randomness is in the eye of the Randomness is in the eye of the

beholderbeholder Exciting area!Exciting area!

AppendixAppendix

PCP for any L PCP for any L NP NP

efficient verifier efficient verifier VV – uses uses O(logO(log nn)) random bits, random bits,– reads 3 bits from the proof.reads 3 bits from the proof.

Correctness:Correctness:– xxL L short proof short proof pp,,

Prob [Prob [V acceptsV accepts pp] > 0.9] > 0.9– xxL L short proof short proof pp,,

Prob [Prob [V acceptsV accepts pp] < 0.5] < 0.5

Definitions (extractor)Definitions (extractor)

Randomness measure: Randomness measure: min-entropymin-entropy..HH((XX) ) kk xx, Prob [ , Prob [ XX==xx ] ] 2 2--kk..

ZZ isis -random-random if if || || ZZ UU || ||11 .. EEXTXT is anis an ( (kk,,)-)-extractorextractor if if

HH((XX) ) kk E EXTXT((XX,,UU) ) isis -random.-random.