Impossibility of Distributed Consensus with One Faulty Process

Impossibility of Impossibility of Distributed Consensus Distributed Consensus

with One Faulty with One Faulty ProcessProcess

Michael J. Fischer Michael J. Fischer

Nancy A. LynchNancy A. Lynch

Michael S. PatersonMichael S. Paterson

Presented by: Oren D. Rubin

AgendaAgenda::

MotivationMotivation The Consensus ProblemThe Consensus Problem Goal Goal AssumptionsAssumptionsTerminologyMain

MotivationMotivation

General 1’s army

General 4’s army

General 3’s army

General 2’s army

• 4 allied armies, each one led by a general, besiege a castle.

• To seize castle, all four must attack together, otherwise armies defeats

• Communications by messengers, reliable, but take unbounded time…

• A Generals may get killed !! (and never be replaced)

MotivationMotivation……

Transaction commit – all data Transaction commit – all data managers must make the same managers must make the same decision in order to preserve the decision in order to preserve the consistency of the database. consistency of the database. Can I

commit?

Yes!!

No!!

The Consensus ProblemThe Consensus Problem• There is a set of distributed processes with initial There is a set of distributed processes with initial

values values {0,1} {0,1} – This strengthen the impossibility result and simplifies the This strengthen the impossibility result and simplifies the

discussion. discussion.

• They must all decide on the same value They must all decide on the same value {0,1}, based on {0,1}, based on their initial states. their initial states.

• There must be some initial state of the process set for There must be some initial state of the process set for which the reached decision is 0 and another for which it is which the reached decision is 0 and another for which it is 1. 1. – To avoid trivial consensus protocols (which always result in To avoid trivial consensus protocols (which always result in

the same decision)the same decision)• Some “non-faulty” processes eventually decide on some Some “non-faulty” processes eventually decide on some

value and this decision is irrevocablevalue and this decision is irrevocable

GoalGoal

No completely asynchronous No completely asynchronous consensus protocol can consensus protocol can tolerate even a single tolerate even a single unannounced process death unannounced process death (no Byzantine failures).(no Byzantine failures).

AssumptionsAssumptions

Processing is completely asynchronous Processing is completely asynchronous Reliable, includes “atomic broadcast” Reliable, includes “atomic broadcast”

(virtual synchrony), could be out of order.(virtual synchrony), could be out of order. No assumptions about the relative speeds No assumptions about the relative speeds

of processes.of processes. Unknown delay time in message delivery.Unknown delay time in message delivery. No access to synchronized clocks (no time No access to synchronized clocks (no time

- outs).- outs). No ability to detect the death of a process.No ability to detect the death of a process.

Terminology• System ModelSystem Model - message passing based.- message passing based.

– message is a pair of (p, m) : destination process and message value

• N (>1) processesN (>1) processes

• The message system – Holds a message buffer

• Unbounded.

– Supports operations • Send(p,m) - places (p,m) in message buffer.• Receive(p) – extract a message (p,m) from the

message buffer (m is delivered) or return “null” (finite number of times).

Terminology ...Terminology ...• ProcessProcess – – automaton, finite or infinite states (deterministic).automaton, finite or infinite states (deterministic).

Each process p comprises an Each process p comprises an internal stateinternal state– Input register Input register Xp Xp -- fixed initial value.fixed initial value.– output register output register YpYp - initialed with ‘b’ (blank), fixed fixed after

rewritten.– Internal storageInternal storage - unbounded, fixed initial initial value.

Performs atomic Performs atomic stepssteps (A.K.A. (A.K.A. eventsevents) composed of -) composed of -– Receive a message (could be “null”).– Changes state (depending on message received).– Sends finite set of messages to other processes

• Configuration – system’s global state, comprises all processes’ internal states and the message buffer– Initial configuration: initial states for all processes and

message buffer is empty.– A step takes one configuration to another (completely

determined by (p,m) ).

• Event: (on process p) e = (p,m) : process p performs an atomic step.– Message m delivered to p. – Triggers state transition in p.– Finite number of message sent by p(p, “null”) can always be applied on a configuration

• Event e applicable to configuration C: if e message buffer or e = (p,“null”).

• e(C): resulting configuration after applying event e on configuration C:– Process p has a new internal state (the one resulted

from message being delivered).– All other processes’ states unchanged.– Message buffer changed (e removed, process's

messages added, if any).

Terminology ...Terminology ...

• Schedule (run): finite/infinite sequence of events that can be applied on a configuration C0.– Events are applicable to configuration C0

– S = e1e2e3…ei…– S(C0) is the configuration resulted a finite run.

• Reachable configuration C’ from C: If a finite run S exists such that S(C0) = C’.

• If C0 is an initial configuration then C’ is said to be accessible.

C0 C1 C2 Cie1 e3e2 ei+1ei…

Terminology ...Terminology ...

Terminology ...Terminology ...• Non-faultyNon-faulty process in a run: a process that take infinitely process in a run: a process that take infinitely

number of steps on that run, number of steps on that run, Faulty Faulty otherwise.otherwise.• Admissible runAdmissible run: a run with one faulty member at most and all : a run with one faulty member at most and all

messages to non-faulty members will be delivered eventually.messages to non-faulty members will be delivered eventually.• Decision value of a configuration C: Decision value of a configuration C: a set of all processes’ non-a set of all processes’ non-

blank Yp values (their blank Yp values (their decision statesdecision states).).– Only 4 Decision values possible: {}, {0}, {1}, {0,1}Only 4 Decision values possible: {}, {0}, {1}, {0,1}

• Deciding run: Deciding run: some process reaches a decision states during some process reaches a decision states during the run the run i.e. a process sets his Yp value (to either 0 or 1).i.e. a process sets his Yp value (to either 0 or 1).

• Partially correct Partially correct protocol:protocol:– All accessible configuration don’t have more than one decision All accessible configuration don’t have more than one decision

valuevalue– There exists two accessible configurations G and H S.T. their There exists two accessible configurations G and H S.T. their

decision values are {0} and {1} correspondingly decision values are {0} and {1} correspondingly • Totally correct Totally correct protocol:protocol:

– Partially correct.Partially correct.– Every admissible run is a deciding ones.Every admissible run is a deciding ones.

• C is 0-valent: for every schedule S applicable to C, if process p decides on a value v in S(C) then v=0.

I.e. S(C) Decision values is either {} or {0}Decision values is either {} or {0} C may be 0-valent although no process has decided {0} yet!!

• C is 1-valent: similar definition.

• C is univalent: C is either 0-valent or 1-valentI.e. fate of decision definitive!!

• C is bivalent: exists schedules S0 and S1, applicable to C, such that:– S0(C) is 0-valent – S1(C) is 1-valent

I.e. both decisions are still possible!!

Terminology ...Terminology ... Valence of configuration C

Terminology ... Terminology ... Valence of configuration C

e’5

0-valent Configuration

p7.Yp = 0


p1.Yp = 0

…

…

e’

e’’

e’’’’e’’’

e

bivalent configuration

0-valent configuration





p7.Yp = 1

MainMainEvent Commutatively: Let C be any

configuration and e, e’ be any events applicable to C occurring to different processes. Then e( e’(C) )= e’( e(C) )

C0

C3

C1 C2

e e’

e’ e

MainMain•Schedule Commutatively: Let C be

any configuration and S, S’ be any events applicable to C occurring to different processes. Then

• S( S’(C) )= S’( S(C) )

C0

C3

C1 C2

S S’

S’ S

• Event Commutatively Proof:– Internal states of the process involved are mutual

excluded.– The message buffer is a set.

• Schedule Commutatively Proof:– e1e2e3…ei…en e’1e’2e’3…e’i…e’m

– e1e2e3…ei…e’1 ene’2e’3…e’i…e’m

– e’1e1e2e3…ei…en e’2e’3…e’i…e’m

– e’1e’2e’3…e’i…e’m e1e2e3…ei…en

MainMain

S S’

S’ S

• Lemma 1: Every Totally correct protocol has an initial Lemma 1: Every Totally correct protocol has an initial configuration C that is bivalentconfiguration C that is bivalent

– There is an initial configuration C0 that is 0-valent– There is an initial configuration C1 that is 1-valent

– Let’s assume the contrary, that all configuration are univalent Let’s assume the contrary, that all configuration are univalent (since the protocol is partial correct).(since the protocol is partial correct).

• Adjacent configuration: Adjacent configuration: 2 configurations are adjacent is they differ 2 configurations are adjacent is they differ in only one process’s (process pin only one process’s (process pii) Xp value.) Xp value.

There must exist adjacent configurations CThere must exist adjacent configurations C00, C, C11 S.T. C S.T. C0 0 isis0-valent and C0-valent and C1 1 is 1-valent (next slide)is 1-valent (next slide)..

Take any admissible deciding run (with schedule S) whereTake any admissible deciding run (with schedule S) whereprocess pprocess pii takes no steps (one faulty process allowed). takes no steps (one faulty process allowed).

S can be applied to both CS can be applied to both C00 and C and C1 1 and they both will reach theand they both will reach thesame decision value (since nothing changes except psame decision value (since nothing changes except p ii’s Xp value ’s Xp value

whichwhichis untouched).is untouched).

decision value=1 decision value=1 C C0 0 is bivalent.is bivalent.decision value=0 decision value=0 C C1 1 is bivalent.is bivalent.Contradiction!!!Contradiction!!!

MainMain

MainMain

P1

P0

Pi

Pn

processes

Xp=0

Xp=0

Xp=0

Xp=0

Xp=1

Xp=0

Xp=0

Xp=0

Xp=1

Xp=1

Xp=1

Xp=0

Xp=1

Xp=1

Xp=1

Xp=1

0-valent 1-valent

Xp=1

Xp=1

Xp=0

Xp=0

adjacent

Not necessary

The 1-valent

• Lemma 2:Lemma 2: Let C be any bivalent configuration, and e be any event applicable to C. There exists a finite schedule S applicable to C that does not contain e, such that e( S (C) ) is also bivalent.

F = { S(C) : S finite schedule applicable to C that does not contain e}D = {e(C’) : C’ F}Need to show that D contains a bivalent configuration.

MainMain

e e

e e e

e

e

ee

D configurations

F configurations

Bivalent

Assume the contrary, D doesn’t have a bivalent configuration

•Neighbors configuration: configuration C0 and C1 are neighbors if one resulted from the other in one step e’ = (p’,m’)

There exists neighbors C0, C1 S.T. C1=e’(C0) or

C0=e’(C1) And that D1=e(D0), D0=e(D1) are 1-valent and 0-valent correspondingly (next

slide)

MainMain

• Key: Though each run can be infinite, in finite Key: Though each run can be infinite, in finite number of step the run is decidednumber of step the run is decided

Algorithm to findingAlgorithm to finding C0, C1

a. Start with a bivalent configurationb. If there exists an event e’’ that leads to bivalent configuration then go to b

with e(C).else (must be eventually because protocol is totally correct) all events lead to univalent configuration including e (which lead to a 0-valent or a 1-valent configuration)but there must exist another event e’’’ which leads to the other-valent (since we reached a bivalent configuration)

MainMain

e’5


p7.Yp = 0


p1.Yp = 0

…

…

e’’’’e’’

e’e’’’

e







p7.Yp = 1

C0 C1

• Without loss of generality Without loss of generality C1=e’(C0)

Main … (proof continued)Main … (proof continued)

e’

e

C0

D0

C1

F configurations

D configurations

D1

e

•Case 1: p not equals to p’– By the commutatively property D1 is 0-

valent and 1-valent, Contradiction!!

MainMain

e’

e

C0

D0

C1

F configurations

D configurations

D1

e

e’

• Case 1: p equals to p’– Be S the schedule of a finite deciding run in which process p takes no steps

(S is applicable to D1 and D0 due to commutatively) S(C0)=Aby commutatively e(A)=E0 = S( e(C0) ) which is 0-valent configurationAlso by commutatively e(A)=E1 = S( e’( e(C0) ) ) which is 1-valent configuration

But since S is a deciding run A must be a univalent configuration and applying events on it only lead to univalent configuration Contradiction !!

–

–

MainMain

e’

e

C0

D0

C1

ee’

E0

S

AS

e

eD1

E0S

0-valent

1-valent

• The last 2 contradictions proved that D contains a bivalent The last 2 contradictions proved that D contains a bivalent configuration.configuration.

• The idea: postpone the event that leads to a univalent The idea: postpone the event that leads to a univalent configuration by that delaying the decision.configuration by that delaying the decision.

• The algorithm:The algorithm:a. Execution begins with the bivalent configuration Ca. Execution begins with the bivalent configuration C0 0 which is which is

promised.promised.b. we order the messages in the message buffer, according to the b. we order the messages in the message buffer, according to the

time they were sent, earliest first.time they were sent, earliest first.c. We go over the processes in a round robin fashion (infinitely), for c. We go over the processes in a round robin fashion (infinitely), for

each process:each process:• Let m be the first message in the message buffer destined to the Let m be the first message in the message buffer destined to the

process in the head of the queue or “null”process in the head of the queue or “null”• By lemma 2 there exists a bivalent configuration C’ S.T. C’ is reachable By lemma 2 there exists a bivalent configuration C’ S.T. C’ is reachable

from C by a schedule S in which (p,m) is the last step applied.from C by a schedule S in which (p,m) is the last step applied.• We apply S.We apply S.

since all messages are delivered this infinite run is admissible.since all messages are delivered this infinite run is admissible.

Main… finallyMain… finally

THE ENDTHE END

Impossibility of Distributed Consensus with One Faulty Process

Documents