Random walks on undirected graphs and a little bit about Markov Chains Guy.

Random walks on undirected Random walks on undirected graphs and a little bit about graphs and a little bit about

Markov ChainsMarkov Chains

GuyGuy

One dimensional Random walkOne dimensional Random walk

A random walk on a line.A random walk on a line. A line is A line is

0 1 2 3 4 5 n

If the walk is on 0 it goes into 1.Else it goes to i+1 or to i-1 with probability 1/2

What is the expected number of steps to go to n?

The expected time functionThe expected time function

T(n)=0T(n)=0 T(i)T(i)=1+(T(i+1)+T(i-1))/2, =1+(T(i+1)+T(i-1))/2, ii00 T(0)=1+T(1)T(0)=1+T(1) Add allAdd all equation givesequation gives T(n-1)=2n-1 T(n-1)=2n-1.. From that we get:From that we get: T(n-2)=4n-4T(n-2)=4n-4 T(i)=2(n-i)n-(n-i)T(i)=2(n-i)n-(n-i)22

T(0)=nT(0)=n22

2-SAT: definition2-SAT: definition

A A 2-CNF2-CNF formula: formula:

(x+ (x+ ¬¬y)y)&& ( (¬¬ x+ x+ ¬¬ z) z) &(x+y) &(z+¬&(x+y) &(z+¬ x) x) (x+y)(x+y) and the others are called CLAUSES and the others are called CLAUSES CNF means that we want to satisfy all of CNF means that we want to satisfy all of

themthem (x+ (x+ ¬y)¬y) is satisfied if is satisfied if X=T or y=FX=T or y=F.. The question: is there a satisfying The question: is there a satisfying

assignment? assignment? 22n n is not a number the is not a number the computer can run even for computer can run even for n=70n=70

Remark: the case of two literals very Remark: the case of two literals very specialspecial

If three literals per clause: NPCIf three literals per clause: NPC Not only that, but can not be approximated Not only that, but can not be approximated

better than better than 7/87/8 Obviously for Obviously for (x+(x+¬y+¬z)¬y+¬z) if we draw a if we draw a

random value the probability that the random value the probability that the clause is satisfied is clause is satisfied is 1-1/8=7/81-1/8=7/8. Best . Best approximation possibleapproximation possible

Also hard Also hard 3-SAT(5)3-SAT(5)

The random algorithm for The random algorithm for 2-SAT2-SAT

2-SAT 2-SAT : Start with an arbitrary assignment: Start with an arbitrary assignment Let Let CC be a non satisfied clause. Choose at random be a non satisfied clause. Choose at random

one of the two literals of one of the two literals of CC and flip its value. and flip its value. We know that if the variables are We know that if the variables are xx11 and and xx22 the the

optimum disagrees with us on optimum disagrees with us on xx11 or on or on xx22. .

Distance to Distance to OPTOPT: with probability : with probability ½½ smaller by smaller by 11 and with probability and with probability ½½ larger by larger by 11 (worst case). (worst case). Thus Thus E(RW)E(RW)≤≤nn22

RP algorithm (can make a mistake)RP algorithm (can make a mistake)

If you do these changes If you do these changes 2n2n22 times the probability times the probability that we do not get a truth assigment isthat we do not get a truth assigment is ½ ½ if one if one existsexists

You can do You can do nn44 and the probability that if there is a and the probability that if there is a truth assignment we don’t find it is truth assignment we don’t find it is 1/n1/n22

What we use here is called the Markov inequality What we use here is called the Markov inequality that states there are at mostthat states there are at most 1/3 1/3 of the numbers in of the numbers in a collection of numbers that are at least a collection of numbers that are at least 3 3 times times the averagethe average

There are several deterministic algorithms for There are several deterministic algorithms for 2-SAT2-SAT

Shuffling cardsShuffling cards

Take the top card and place it with a random Take the top card and place it with a random place (including first place). place (including first place). One of nOne of n..

A simple claim: if a card is in a random A simple claim: if a card is in a random place it will be in a random place after next place it will be in a random place after next move.move.

We know We know Pr(card is in place i)=1/nPr(card is in place i)=1/n For the card to be in For the card to be in ii after next step three after next step three

possibilitiespossibilities

Probability of the card to be in place Probability of the card to be in place ii

One possibility: it is not the first card and it is One possibility: it is not the first card and it is in place in place ii. Chooses one of . Chooses one of i-1i-1 first places. first places.

Second possibility (disjoint events) its at Second possibility (disjoint events) its at place place 11 and exchanged with and exchanged with ii..

Third possibility: it is in place Third possibility: it is in place i+1i+1 and the and the first card is placed in one of the places after first card is placed in one of the places after

i+1i+1 1/n*(i-1)/n+1/n*1/n+1/n*1/(n-i)=1/n1/n*(i-1)/n+1/n*1/n+1/n*1/(n-i)=1/n

Stopping timeStopping time

If all the cards have been upstairs If all the cards have been upstairs its its randomrandom

Check the lowes card. To go up by Check the lowes card. To go up by 11 G(1/n)G(1/n) thus expectation thus expectation nn..

To go from second last to third last To go from second last to third last G(2/n)G(2/n) and expectation and expectation n/2n/2..

This gives This gives n+n/2+n/3+….= n(ln n+n+n/2+n/3+….= n(ln n+ӨӨ(1))(1)) FASTFAST

Random Walks on undirected Random Walks on undirected graphsgraphs

Given a graph choose a neigbor at random Given a graph choose a neigbor at random with probability with probability 1/d(v)1/d(v)

Random WalksRandom Walks

Given a graph choose a vertex at random.Given a graph choose a vertex at random.

Random WalksRandom Walks

Given a graph choose a vertex at random.Given a graph choose a vertex at random.

Random WalksRandom Walks

Given a graph choose a vertex at random.Given a graph choose a vertex at random.

Random WalksRandom Walks

Given a graph choose a vertex at random.Given a graph choose a vertex at random.

Random WalksRandom Walks

Given a graph choose a vertex at random.Given a graph choose a vertex at random.

Random WalksRandom Walks

Given a graph choose a vertex at random.Given a graph choose a vertex at random.

Markov chainsMarkov chains

Generalization of random walks on Generalization of random walks on undirected graph.undirected graph.

The graph is directedThe graph is directed The sum over the values of outgoing edges The sum over the values of outgoing edges

is is 11 but it does not need to be uniform. but it does not need to be uniform. We have a matrix We have a matrix P=(pP=(pijij)) with with ppijij is the is the

probability that on stateprobability that on state i i it will go to it will go to j j.. Say that for Say that for ΠΠ0 0 =(x=(x11,x,x22,….,x,….,xnn))

Changing from state to stateChanging from state to state

Probability(state=i)=Probability(state=i)=j j xxj * j * ppjiji

This is the inner product of the current state and This is the inner product of the current state and column column jj of the matrix. of the matrix.

Therefore if we are in distribution Therefore if we are in distribution ΠΠ00

after one step the distribution is at state: after one step the distribution is at state:

ΠΠ11= = ΠΠ0 0 *P*P And after i stages its in And after i stages its in ΠΠii= = ΠΠ0 0 *P*Pii

What is a steady state?What is a steady state?

Steady stateSteady state

Steady state is Steady state is ΠΠ so that:so that: ΠΠ*P= *P= ΠΠ. For any i and any round the probability that it is . For any i and any round the probability that it is

in i is the same always.in i is the same always. Conditions for convergence:Conditions for convergence: The graph has to be strongly connected. The graph has to be strongly connected. Otherwise there may be many components Otherwise there may be many components that have no edges out. No steady state.that have no edges out. No steady state. hhiiii the time to get back to i from i is finite the time to get back to i from i is finite Non periodic. Slightly complex for Markov chains. For Non periodic. Slightly complex for Markov chains. For

random walks on undirected graphs: not a random walks on undirected graphs: not a bipartite bipartite graphgraph..

The bipartite graph exampleThe bipartite graph example

If the graph is bipartite and If the graph is bipartite and VV11 and and VV22 are its are its

sets then if we start with sets then if we start with VV11 we can not be we can not be

on on VV11 vertex in after odd number of vertex in after odd number of

transitions.transitions. Therefore a steady state is not possible.Therefore a steady state is not possible. So we need for random walks on graphs So we need for random walks on graphs

that the graph is connected and not that the graph is connected and not bipartite. The other property will follow. bipartite. The other property will follow.

Fundamental theorem of Markov Fundamental theorem of Markov chainschains

Theorem: Given an aperiodic MC so that hTheorem: Given an aperiodic MC so that hii ii

is not infinite for any i, and non-reducible is not infinite for any i, and non-reducible Markov chain than:Markov chain than:

1) 1) There is a There is a uniqueunique steady state. Thus to steady state. Thus to

find the steady state just find find the steady state just find ΠΠ*P= *P= ΠΠ

2) 2) hhiiii=1/=1/ΠΠii . . Geometric distribution argument.Geometric distribution argument.

Remark: the Remark: the mixing timemixing time is how fast the chain is how fast the chain gets to (very close to) the steady state.gets to (very close to) the steady state.

Because its unique you just have to Because its unique you just have to find the correct find the correct ΠΠ

For random walks in an undirected graph we For random walks in an undirected graph we claim that the steady state is:claim that the steady state is:

(2d(2d11/m,2d/m,2d22/m,……,2d/m,……,2dmm/m)/m)

It is trivial to show that this is the steady It is trivial to show that this is the steady state. Multiply this vector and the i column.state. Multiply this vector and the i column.

The only important ones are neighbors of i.The only important ones are neighbors of i.

So So sumsum(j,i)(j,i)E E 1/d 1/djj*(2d*(2djj/m)=2d/m)=2dii/m/m

The expected time to visit all the The expected time to visit all the vertices of a graphvertices of a graph

A matrix is doubly stochastic if and only if all A matrix is doubly stochastic if and only if all columns also sum to columns also sum to 11..

Exercise (very simple): doubly stochastic Exercise (very simple): doubly stochastic then then {1/n}{1/n} or uniform is the steady state. or uniform is the steady state.

Define a new Markov chain of edges with Define a new Markov chain of edges with directions which means directions which means 2m2m states. states.

The walk is defined naturally.The walk is defined naturally. Exercise: show that the Matrix is doubly Exercise: show that the Matrix is doubly

stochasticstochastic

The time we spend on every edgeThe time we spend on every edge

By the above, we spend the same time on By the above, we spend the same time on every edge in the two directions over all every edge in the two directions over all edges (of course you have to curve the edges (of course you have to curve the noise. We are talking on a limit here).noise. We are talking on a limit here).

Now, say that I want to bound Now, say that I want to bound hhijij the the

expected time we get from expected time we get from ii to to jj..

Showing Showing hhijij+h+hjiji≤≤ 2m 2m

Assume that we start at Assume that we start at i--->ji--->j By the Markov chain of the edges it will take By the Markov chain of the edges it will take 2m2m

steps until we do this move againsteps until we do this move again Forget about how you got to Forget about how you got to j j (no memory).(no memory). Since we are doing now Since we are doing now i---->ji---->j again we know that: again we know that: a) As it was in a) As it was in jj, it returned to , it returned to ii..

This is half the inequality This is half the inequality hhjiji

b) Now it goes b) Now it goes i----> ji----> j. This takes at most . This takes at most hhijij

c) Since this takes at most c) Since this takes at most 2m2m the claim follows. the claim follows.

Consider a spanning tree and a walk Consider a spanning tree and a walk on the spanning treeon the spanning tree

Choose any paths that traverses the tree so Choose any paths that traverses the tree so that the walk goes from a parent to a child that the walk goes from a parent to a child once and back once.once and back once.

Per parent child we have Per parent child we have hhijij+h+hjiji≤2m≤2m

Thus over the n-1 edges of the graph the Thus over the n-1 edges of the graph the cover time is at most cover time is at most 2m(n-1)<n2m(n-1)<n33

Tight example: the Tight example: the

n/2 vertices

clique

u1 u2 un/2

BridgesBridges

For bridges For bridges hhijij+h+hjiji=2m=2m (follows from proof). (follows from proof).

Say you are the intersection vertex Say you are the intersection vertex uu11 of the of the

clique and of the path. Its harder to go right clique and of the path. Its harder to go right than left.than left.

Thus it takes about Thus it takes about nn22 time to go from time to go from uu11 to to uu22

and the same time to go from and the same time to go from uu22 to to uu3 3 etc.etc.

This gives This gives ΩΩ(n(n33)) Funny name: Lollipop graph.Funny name: Lollipop graph.

Exponential cover time for directed Exponential cover time for directed graphsgraphs

Spectrum of a graphSpectrum of a graph

We can represent an We can represent an n*nn*n graph with a graph with a symmetric matrix symmetric matrix AA. Let the vertices be . Let the vertices be

{1,2,…,n}{1,2,…,n} Put Put AAijij=1=1 if and only if if and only if (I,j)(I,j)EE

Note that the matrix is Note that the matrix is symmetricsymmetric An eigenvalue is a An eigenvalue is a so that so that A*v= A*v= v v for some for some vv Since the matrix is symmetric all eigenvalues Since the matrix is symmetric all eigenvalues

are real numbers.are real numbers.

Relation of graphs and algebraRelation of graphs and algebra

If we have a If we have a dd-regular graph and we count -regular graph and we count all walks of distance exactly all walks of distance exactly kk, we get , we get n*dn*dkk

One average there is a pair One average there is a pair u,vu,v so that so that

walks(u,v)≥walks(u,v)≥ddkk/n/n What happens if the average degree is What happens if the average degree is dd??

We use the symmetric matrix We use the symmetric matrix AA that that represents the graphrepresents the graph G G. And the vertices . And the vertices {1,2….,n}.{1,2….,n}.

The inequality still holds, two slides The inequality still holds, two slides proofproof

Say Say 00 ≥ ≥ 11 ≥ …………..≥ ………….. ≥ ≥ n-1n-1

00= max= max|x|=1|x|=1{x{xTT* A *X}* A *X}

Choose Choose xxii=1/n=1/n1/21/2. Its easy to see that we get. Its easy to see that we get AAijij/n=d/n=d

Thus : Thus : 0 0 ≥≥ AAijij/n=d/n=d..

Known: eigenvalues of Known: eigenvalues of AAk k areare ( (ii))kk

The number ofThe number of i i to to j j walks is thewalks is the ij ij entry inentry in A Akk

The walks proofThe walks proof

By symmetry ABy symmetry Akk(i,j)= A(i,j)= Akk(j,i)(j,i) =W=Wkk(i,j)(i,j) Trace((ATrace((Akk))22)=)= A A2k2k(i,i)= (i,i)= i i jj A Akk(i,j)(i,j) * A* Akk(j,i)(j,i)

= = i i jj W Wkk(i,j)(i,j)22== 2k 2k ≥ ≥ 002k 2k ≥ d≥ d2k2k

By averaging By averaging WWkk(I,j)(I,j)22≥ d≥ d2k2k/n/n2 2

for some for some I,j I,j Taking a sqre root we get Taking a sqre root we get WWkk(I,j)(I,j)≥ d≥ dkk/n/n

QEDQED

ExpandersExpanders

The definition is roughly: for every The definition is roughly: for every SS V V of of size at most size at most n/2n/2 the number of edges the number of edges leaving leaving SS is at least is at least c*|S|c*|S| for some constant for some constant cc..

We are interested in d-regular expanders We are interested in d-regular expanders with d a universal constant.with d a universal constant.

The largest eignevalue is The largest eignevalue is 00 ==dd. A graph is . A graph is an expander iff an expander iff 00 >>>>11. .

At best At best 1 1 is aboutis about d d1/21/2..

Random walks on expandersRandom walks on expanders

The mixing time is very fast.The mixing time is very fast. The diameter is The diameter is O(log n)O(log n) and the mixing time is and the mixing time is

O(log n)O(log n) also. The proof uses the fact that the also. The proof uses the fact that the second eigenvalue is much smaller than the firstsecond eigenvalue is much smaller than the first

Remarkable application: say we want Remarkable application: say we want 1/21/2kk error error probability. Need probability. Need nn bits to get probability bits to get probability 1/21/2

Random walk on expanders allows Random walk on expanders allows n+O(k)n+O(k) bits bits to get to get 1/21/2k k upper bound on the error. upper bound on the error.

Random walks a ‘real’ exampleRandom walks a ‘real’ example

Brownian motionBrownian motion Random drifting of particles suspended in a Random drifting of particles suspended in a

fluid (a liquid or a gas).fluid (a liquid or a gas). Has a mathematical model used to describe Has a mathematical model used to describe

such random movements, which is often such random movements, which is often called a called a particle theoryparticle theory. .

Imagine a stadium full of people and balons Imagine a stadium full of people and balons and the people pushing the balons and the people pushing the balons randomly. randomly.