The First-Order Variable Hierachy on Ordered Graphs Benjamin Rossman MIT.

The First-Order Variable Hierachy on Ordered Graphs

Benjamin Rossman

MIT

Bounded Variable Logics

The variable complexity of a first-order formula is the maximum number of free variables in a subformula of .

FOm = { first-order formulas with variable complexity m }

Example: (in the language {E,<} of ordered graphs)

9x1 9x2 ( x1<x2 Æ Ex1x2 Æ 9x3 ( x2<x3 Æ Ex2x3 Æ

9x4 ( x3<x4 Æ Ex3x4 Æ 9x5 ( x4<x5 Æ Ex4x5 ).

This sentence expresses "there is an increasing path of length 5". Its variable complexity is 2.


The variable complexity of a first-order formula is the maximum number of free variables in a subformula of .

FOm = { first-order formulas with variable complexity

Example:9x 9y ( x<y Æ Exy Æ 9x ( y<x Æ Eyx Æ

9y ( x<y Æ Exy Æ 9x ( y<x Æ Eyx ).

This sentence expresses "there is an increasing path of length 5". Its variable complexity is 2.

Variable complexity 2 means we can rewrite the formula using only 2 distinct variables (quantified multiple times). Hence, FOm is called the m-variable fragment of first-order logic.


The variable hierarchy refers to

FO1 ½ FO2 ½ ½ FOm ½ where FO = m¸1 FOm.

Question: Is the variable hierarchy strict/non-collapsing (in terms of expressive power) on a given class of structures?

The answer is YES on the class of all structures (or all finite structures): "the universe contains at least m elements" is expressible in FOm but not FOm-1.


Some collapse results: FO ´ FO2 on finite linear orders. FO ´ FO3 on finite linear orders with any

number of unary relations [Poizat '82]. Question: What about ordered graphs

(= finite simple graphs with a linear order)? k-CLIQUE is a natural candidate property for

proving the separation FOk-1 < FOk on ordered graphs.


One can show 3-CLIQUE is not definable in FO2 by playing the 2-pebble Ehrenfeucht-Fraisse game on (seq. of) ordered graphs:

It had been an open question (since [Immerman '82]) whether FO ´ FO3 on ordered graphs.

G1

G2


One can show 3-CLIQUE is not definable in FO2 by playing the 2-pebble Ehrenfeucht-Fraisse game on (seq. of) ordered graphs:

It had been an open question (since [Immerman '82]) whether FO ´ FO3 on ordered graphs.

G1

G2

Games on ordered graphs become difficult with ¸ 3 pebbles/variables.

One explanation may be that every finite ordered graph is defined up to isomorphism by a sentence of FO3.


Theorem [R. '08]. k-CLIQUE is not definable in FObk/4c.

Corollary. FObk/4c < FOk on ordered graphs.

Lemma [Immerman '08]. On ordered graphs,

FOk-1 ´ FOk ) FOk ´ FOk+1

(i.e., if the variable hierarchy does not collapse, then it is strict).

Corollary. The variable hierarchy is strict (i.e., FOk < FOk+1) on ordered graphs.








These results hold not only for ordered graphs, but for classes of finite graphs with arbitrary numerical predicates (e.g., arithmetic + and £).








Moreover, we show that k/4 variables cannot distinguish classes {ordered graphs with no k-cliques} and {ordered graphs with a K-clique} for all k < K.

Circuits

Circuits are comprised of AND, OR, NOT gates (with unbounded fan-in).

We consider polynomial-size, constant-depth (i.e. AC0) circuits with input variables (representing potential edges in an n-vertex graph).

Descriptive Complexity: {m-variable, quantifier-depth d formulas} (on structures with arbitrary

numerical predicates, e.g., linear order)

´ {non-uniform AC0 circuits with size O(nm) and depth d}. [Immerman '82]

AC0 Circuit (in normal form)

Æ

Æ Æ

Ç Ç Ç Ç Ç Ç Ç Ç Ç Ç Ç Ç

ÆÆ Æ Æ Æ Æ

Ç Ç Ç.......

.....

....

Ç

Æ

1,2 n-1,n1,2 1,3 1,3 1,4 1,4 i, j i, j n-1,nn-2,n n-2,n......

size nO(1)

depth O(1)

Ç

AC0 Lower Bound for k-Clique

The k-clique problem (on graphs of size n) requires depth-d circuits of what size?

Trivial upper bound: O(nk) (acheived in depth 2)

J. Lynch '86: (np(k/d3))

P. Beame '90: (nk/89d2)



Trivial upper bound: O(nk) (acheived by depth 2)


P. Beame '90: (nk/89d2)

These lower bounds degrade in the exponent of n as the depth d increases, becoming trivial when d > pk.





P. Beame '90: (nk/89d2)

We show: (nk/4)

k-Clique Lower Bound




P. Beame '90: (nk/89d2)

We show: (nk/4)

We eliminate dependence on d in the exponent of n. Result holds up to depth d = O(plog n) and potentially up to clique-size k = o(log n).

Erdos-Renyi Random Graphs

The random graph ErdosRenyi(n,p),

p = p(n) 2 [0,1], has n vertices. Edges are independently included with probability p.

We parameterize p(n) = n– where > 0 (so small ! large p ! denser graph).

= 2/(k-1) is the threshold for the appearance of k-cliques. That is, ER(n,n-) almost surely has k-cliques if < 2/(k-1). ER(n,n-) almost surely has no k-clique if > 2/(k-1).

ErdosRenyi(n,p)

n = 50

p = n–0.68

(p is above the threshold

n-2/3 for the appearance

of 4-cliques)

Erdos-Renyi Random Graphs

A Bit of Notation

Notation 1. For an ordered graph G = hV, E, <i and a set A µ V, let

G[A] = G [ (clique on A) = hV, E [ , <i.

Notation 2. For ordered graphs G and H,

, G and H satisfy the same FOm

sentences of quantifier-depth d

, 9 winning strategy for Duplicator in

the d-round m-pebble game on G,H

HG md

Random Graph + Random Cliques

Let G = Erdos-Renyi(n,1/2) with linear order. Let e = random 2-element subset of {1,...,n}.

Theorem. [Ajtai, Furst-Saxe-Sipser, Hastad, Boppana, ...]

For every (quantifier-depth) d,

almost surely as n ! 1.

NB. This is a disguised way of saying that

AC0 functions have average sensitivity no(1).

G[e]G d


Let G = Erdos-Renyi(n,n –

2/(k

–

1.5)) with lin. order.

NB. With high probability, G has no k-clique

but n(1) many (k–1)-cliques.

X µ {1,...,n} random of size |X| = k–1.

Obs 1. For every (quantifier-depth) d,

asymptotically almost surelyG[X]G d



2/(k

–




X, A µ {1,...,n} random of size |X| = k–1, |A| = k.


asymptotically almost surely

Obs 2. a.a.s.

G[X]G d

G[A]G k/



2/(k

–




X, A µ {1,...,n} random of size |X| = k–1, |A| = k.


asymptotically almost surely

Obs 2. a.a.s.

Main Theorem. For every (quantifier-depth) d,

a.a.s.

G[X]G d

G[A]G k/

G[A]G k/4d

Theorem. G = Erdos-Renyi(n, n–2/(k–1.5)), |A| = k.

Then a.a.s. for every d.G[A]G k/4d

Proof Idea. For each k/4-tuple of vertices v = (v1,...,vk/4), we

identify a small subset B(v) µ A (of size · k/2, but empty for most v).

Key Property: For every B(v) µ C µ A such that |C| · k/2,

hG[B(v)], vi ´d hG[C], vi.







In other words: Once you add the clique on B(v) to the pebbled

structure hG, vi, adding a larger subclique of A up to size · k/2 does

not change the ´d-theory.







Duplicator's Strategy: "Mentally substitute" the pebbled structure hG[B(v)], vi for hG[A], vi.


Then a.a.s. for every d.

Def. Call v = (v1,...,vk/4), w = (w1,...,wk/4) 2 [n]k/4

neighbors if vi wi for exactly one i 2 {1,...,k/4}.

Suppose we could show (with high probability):

There exist sets B(v) µ A for all v 2 [n]k/4 s.t.

1. B(u) = ; for some u,

2. B(v) ¶ A Å {v1,...,vk/4} for all v,

3. hG[B(v)], vi ´d hG[B(w)], vi for all nbrs v, w.

Then the conclusion follows easily!

G[A]G k/4d

G[A]G k/4d

Assume: There exist sets B(v) µ A, v 2 [n]k/4, such that

1. B(u) = ; for some u, 2. B(v) ¶ A Å {v1,...,vk/4} for all v,

3. hG[B(v)], vi ´d hG[B(w)], vi for all neighbors v, w.

Winning strategy for Duplicator in hG, ui hG[A], ui (k=12)

k/4d

u1 u2 u3 u1 u2 u30123456

G G[A]Round


k/4d

u1 u2 u3

x u2 u3

u1 u2 u3

x u2 u3

0123456

G G[A]Round

Duplicator matches Spoilerexactly in Round 1.





k/4d

u1 u2 u3

x u2 u3

u1 u2 u3

x u2 u3

0123456

G G[A]Round hG[B(u1,u2,u3)], x, u2, u3i´d hG[B(x,u2,u3)], x, u2, u3i

We have:

• G = G[B(u1,u2,u3)] by (1)

• ´d above by (3)





k/4d

u1 u2 u3

x u2 u3

x y u3

u1 u2 u3

x u2 u3

0123456

G G[A]Round

Suppose Spoiler movespebble 2 to y in G.

hG[B(u1,u2,u3)], x, u2, u3i´d hG[B(x,u2,u3)], x, u2, u3i





k/4d

u1 u2 u3

x u2 u3

x y u3

u1 u2 u3

x u2 u3

0123456

G G[A]Round hG[B(u1,u2,u3)], x, y, u3i´d-1 hG[B(x,u2,u3)], x, y', u3i




Duplicator plays an auxiliary game to

find vertex y'.


k/4d

u1 u2 u3

x u2 u3

x y u3

u1 u2 u3

x u2 u3

x y' u3

0123456

G G[A]Round hG[B(u1,u2,u3)], x, y, u3i´d-1 hG[B(x,u2,u3)], x, y', u3i

Duplicator replies by movingpebble 2 to y' in G[A].





k/4d

u1 u2 u3

x u2 u3

x y u3

u1 u2 u3

x u2 u3

x y' u3

0123456

G G[A]RoundhG[B(u1,u2,u3)], x,

y, u3i´d-1 hG[B(x,u2,u3)], x, y', u3i´d hG[B(x,y',u3)], x, y', u3i

assumption (3)





k/4d

u1 u2 u3

x u2 u3

x y u3

u1 u2 u3

x u2 u3

x y' u3

x y' z''

0123456

G G[A]RoundhG[B(u1,u2,u3)], x,

y, u3i´d-1 hG[B(x,u2,u3)], x, y', u3i´d hG[B(x,y',u3)], x, y', u3i

Suppose Spoiler movespebble 3 to z'' in G[A].





k/4d

u1 u2 u3

x u2 u3

x y u3

u1 u2 u3

x u2 u3

x y' u3

x y' z''

0123456

G G[A]RoundhG[B(u1,u2,u3)],

x, y, z i´d-2 hG[B(x,u2,u3)], x, y', z' i´d-1 hG[B(x,y',u3)], x, y', z''i




Duplicator plays two auxiliary games to

find vertex z.


k/4d

u1 u2 u3

x u2 u3

x y u3

x y z

u1 u2 u3

x u2 u3

x y' u3

x y' z''

0123456


x, y, z i´d-2 hG[B(x,u2,u3)], x, y', z' i´d-1 hG[B(x,y',u3)], x, y', z'' i

Duplicator replies by movingpebble 1 to z in G.





k/4d

u1 u2 u3

x u2 u3

x y u3

x y z

u1 u2 u3

x u2 u3

x y' u3

x y' z''

0123456


x, y, z i´d-2 hG[B(x,u2,u3)], x, y', z' i´d-1 hG[B(x,y',u3)], x, y', z'' i´d hG[B(x,y',z'')], x, y', z'' i

assumption (3)





k/4d

u1 u2 u3

x u2 u3

x y u3

x y z

t y z

u1 u2 u3

x u2 u3

x y' u3

x y' z''

0123456


x, y, z i´d-2 hG[B(x,u2,u3)], x, y', z' i´d-1 hG[B(x,y',u3)], x, y', z'' i´d hG[B(x,y',z'')], x, y', z'' i

Spoiler





k/4d

u1 u2 u3

x u2 u3

x y u3

x y z

t y z

u1 u2 u3

x u2 u3

x y' u3

x y' z''

t''' y' z''

0123456


t, y, z i´d-3 hG[B(x,u2,u3)], t', y', z' i´d-2 hG[B(x,y',u3)], t'',y', z'' i´d-1 hG[B(x,y',z'')], t''',y',z'' i´d hG[B(t''',y',z'')], t''',y',z'' i

Duplicator





k/4d

u1 u2 u3

x u2 u3

x y u3

x y z

t y z

u1 u2 u3

x u2 u3

x y' u3

x y' z''

t''' y' z''

t''' y' j''''

0123456


t, y, z i´d-3 hG[B(x,u2,u3)], t', y', z' i´d-2 hG[B(x,y',u3)], t'',y', z'' i´d-1 hG[B(x,y',z'')], t''',y',z'' i´d hG[B(t''',y',z'')], t''',y',z'' i

Spoiler





k/4d

u1 u2 u3

x u2 u3

x y u3

x y z

t y z

t y j

u1 u2 u3

x u2 u3

x y' u3

x y' z''

t''' y' z''

t''' y' j''''

0123456


t, y, j i´d-4 hG[B(x,u2,u3)], t', y', j' i´d-3 hG[B(x,y',u3)], t'',y', j'' i´d-2 hG[B(x,y',z'')], t''',y',j''' i´d-1 hG[B(t''',y',z'')], t''',y',j'''' i´d hG[B(t''',y',j'''')], t''',y',j'''' i

Duplicator





k/4d

u1 u2 u3

x u2 u3

x y u3

x y z

t y z

t y j

u1 u2 u3

x u2 u3

x y' u3

x y' z''

t''' y' z''

t''' y' j''''

0123456


t, y, j i´d-4 hG[B(x,u2,u3)], t', y', j' i´d-3 hG[B(x,y',u3)], t'',y', j'' i´d-2 hG[B(x,y',z'')], t''',y',j''' i´d-1 hG[B(t''',y',z'')], t''',y',j'''' i´d hG[B(t''',y',j'''')], t''',y',j'''' i

Auxiliary games on G + various small cliques give strategy for G versus G + large clique





k/4d

u1 u2 u3

x u2 u3

x y u3

x y z

t y z

t y jet cet era

u1 u2 u3

x u2 u3

x y' u3

x y' z''

t''' y' z''

t''' y' j''''

et cet era

0123456

G G[A]Round...and so on, for d rounds.

NB. Assumption (2) implies this is a winning strategy for Duplicator (i.e., quantifier-free types are preserved in each round).







Alas, this assumption is too strong!

We cannot actually find sets B(v) µ A, v 2 [n]k/4, meeting these conditions.




Alas, this assumption is too strong!

We cannot actually find sets B(v) µ A, v 2 [n]k/4, meeting these conditions.

So we weaken this assumption (by simply restricting the notion of neighboring tuples).

The weaker assumption will still imply G[A]G k/4d

What actually works

For a sufficiently small constant > 0, we fix an arbitrary tree T with vertices 1,...,n and degree n and diameter 2/ (plus all self-loops).

Call v = (v1,...,vm), w = (w1,...,wm) 2 [n]m T-neighbors if 9i 2 {1,...,m} such that (vi,wi) is an edge of T and vj = wj for all j i.

T-guarded d-round Ehrenfeucht-Fraisse game on ordered graphs H1,H2 with vertex set [n]:

Spoiler and Duplicator are constrained to move along edges of T. (Notation. )2dT,1 HH

We are able to show:

Lemma. W.h.p. 9 sets B(v) µ A, v 2 [n]k/4, s.t.


2. B(v) ¶ A Å {v1,...,vk/4} for all v,

3. hG[B(v)], vi ´T, 2d/ hG[B(w)], vi for all

T-neighbors v and w.

It follows that (by same argument).

We conclude that since 2/ = Diam(T)

T-guarded rounds simulate one unguarded round.

G[A]G k/42d/ T,

G[A]G k/4d

What actually works

We are able to show:

Lemma. W.h.p. 9 sets B(v) µ A, v 2 [n]k/4, s.t.


2. B(v) ¶ A Å {v1,...,vk/4} for all v,

3. hG[B(v)], vi ´T, 2d/ hG[B(w)], vi for all

T-neighbors v and w.

It follows that (by same argument).

We conclude that since 2/ = Diam(T)

T-guarded rounds simulate one unguarded round.

G[A]G k/42d/ T,

G[A]G k/4d

What actually works

These sets B(v) have a nifty definition!

Main challenge: Proving 8v, |B(v)| · k/2 with high probability.

Arguments involve Hastad's Switching Lemma & new results on randomly restrictly AC0 functions.

Main Theorem (circuit version)

Let > 0 and k 2 N.

Suppose C is a sequence of constant-depth circuits of size O(n(1+)/2) on inputs.

Then almost surely C has the same value on

1. a random graph G = ErdosRenyi(n,n –

),

2. the graph G [ (random k-clique).

Full Sensitivity

A Boolean function

f : n ! is fully sensitive if it depends on all n input bits.

Formally: for every i 2 [n] = {1,...,n}, there exists x 2 {0,1}n such that

f(x) f(x with ith bit flipped).

Random Restrictions

Fix a Boolean function

f : n ! A restriction is a function

: n! ¤.That is, for each input bit of f, the restriction either fixes a value (0 or 1) or leaves the input bit "unassigned" (¤).

We can apply to f to get a function

fd : -1¤ ! .

Random Restrictions

Suppose f : n ! is an AC0 function (computed by poly-size constant-depth circuits).

Let : n! ¤ be a random restriction subject to

1. |-1(¤)| = k (for a fixed constant k)

2. Pr[(i) = 1 | (i) ¤] = 1/2

So restriction fd : k ! has k input bits.

Random Restrictions




2. Pr[(i) = 1 | (i) ¤] = 1/2


Lemma 1.

Pr[fd is fully sensitive] · 1/nk - o(1)

Random Restrictions




2. Pr[(i) = 1 | (i) ¤] = 1/2.


Lemma 1.


Proof uses a standard application of Hastad's Switching Lemma.

In the special case k = 1, lemma says that AC0 functions have average sensitivity no(1).

Lemma 1. Pr[fd is fully sensitive] · 1/nk - o(1)

Proof.

Consider an AC0 circuit computing f.

f

Fix small > 0 and hit f with a random restriction

: [n] ! {0,1,¤} such that Pr[(i) = ¤] = n–

Pr[(i) = 1 | (i) ¤] = 1/2


Proof.

fd

Fix small > 0 and hit f with a random restriction

: [n] ! {0,1,¤} such that Pr[(i) = ¤] = n–

Pr[(i) = 1 | (i) ¤] = 1/2

expected size n1-

¤0 0 ¤¤¤¤¤1 1 0 1 1 0 ¤1 0 ¤¤¤¤¤1 0 1 1 0 0 1 0 1 0 ¤


Proof.

fd

expected size n1-

¤0 0 ¤¤¤¤¤1 1 0 1 1 0 ¤1 0 ¤¤¤¤¤1 0 1 1 0 0 1 0 1 0 ¤

Hastad's Switching Lemma ) there is a constant c such that fd is computed by a decision tree of depth c (and hence depends on at most 2c = O(1) inputs) with high probability (better than 1 - 1/nk).

2c


Proof.

expected size n1-

0 0 ¤¤¤1 1 0 1 1 01 0 1 0 1 1 0 0 1 0 1 0

Pick : [n] ! {0,1,¤} by randomly setting all but k variables in -1(¤) to 0 or 1.

2c

1 1 0 1 1 0 0 0 1 0

fd

Assuming fd has a small decision tree, we have

Random Restrictions

Suppose f : n ! is an AC0 function. Let : n! ¤ be a random restriction

subject to

1. |-1(¤)| = k

2. Pr[(i) = 1 | (i) ¤] = 1/2


Lemma 1.


Random Restrictions

Suppose f : n ! is an AC0 function. Let : n! ¤ be a random restriction

subject to

1. |-1(¤)| = k

2. Pr[(i) = 1 | (i) ¤] = n –


Lemma 2.

Pr[fd is fully sensitive] · 1/nk – k – o(1)

Lemma 2. Pr[fd is fully sensitive] · 1/nk – k – o(1)

Proof.

f

Apply Lemma 1 to this modified circuit, which generates an n- biased distribution on n bits from the uniform distribution on n log(n) bits.

Æ

log(n)

ÆÆ ................. ........

Full Vertex Sensitivity

A Boolean function

f : ! (taking graphs as inputs) is fully vertex sensitive if every "vertex" belongs to a sensitive "edge".

Formally: for every i 2 [n], there exist j 2 [n] - {i} and a graph G with vertex set [n] such that

f(G) f(G © {i,j}).

Random Restrictions

Suppose f : ! is an AC0 function. Let : ! ¤ be a random restriction

subject to

-1(¤) µ is a k-clique

2. Pr[(i) = 1 | (i) ¤] = n –

So restriction fd : ! has inputs.

Lemma 3.

Pr[fd is fully vertex sensitive] · 1/nk – – o(1)

Proof. Same idea as in lemmas of 1 and 2.

Random Restrictions

Suppose f : ! is an AC0 function. Let : ! ¤ be a random restriction

subject to

-1(¤) µ is a k-clique

2. Pr[(i) = 1 | (i) ¤] = n –

So restriction fd : ! has inputs.

Lemma 3.

Pr[fd is fully vertex sensitive] · 1/nk – – o(1)

Obs: nk – is roughly the expected number of k-cliques in G(n,n

– ).

Open Questions

Is k-CLIQUE definable in k-1 variables? (We showed that k/4 are required.)

Does k-CLIQUE require constant-depth circuits of size (nk–

) for every ?

Thank you!

The First-Order Variable Hierachy on Ordered Graphs Benjamin Rossman MIT.

Documents

n pr i

degree n

n input bits

expected size n

n logn bits

n biased distribution

average sensitivity

o1 slide