Database Systems II Concurrency Control

CMPT 454, Simon Fraser University, Fall 2009, Martin Ester 1

Database Systems II

Concurrency Control


IntroductionThe consistency property requires that a transaction transforms a consistent DB state into another consistent DB state.

The isolation property requires that concurrent transactions are executed as if they were executed in isolation.

More specifically, concurrent transactions are executed in a way that is equivalent to executing the same transactions serially in some order.


IntroductionA schedule is a sequence of actions of one or more transactions.

The actions that we consider in this chapter are read and write operations in the buffer (not on disk).

Need to ensure that schedules are serializable.

At the same time, want to execute as many transactions as possible at the same time in order to maximize the throughput of the system and to minimize the response time.


IntroductionExample

T1: Read(A, t) T2: Read(A,s)t t+100 s s2Write(A,t) Write(A,s)Read(B,t) Read(B,s)t t+100 s s2Write(B,t) Write(B,s)

Constraint: A=B


Serial SchedulesA schedule is serial, if actions of different transactions are not interleaved, otherwise it is non-serial.A serial schedule executes one transaction at a time.Serial schedules can be denoted by the sequence of their transactions: e.g., (T1,T2) or (T2,T1).For a serial schedule, isolation is trivially satisfied.But the throughput of the DBS is very low, and the response times are very high.


Serial Schedules

Schedule A (serial)

T1 T2Read(A,t); t t+100Write(A,t);Read(B,t); t t+100;Write(B,t);

Read(A,s); s s2;

Write(A,s);

Read(B,s); s s2;

Write(B,s);

A B25 25

125

125

250

250250 250Constraint: A=B


Serial Schedules

Schedule B (serial)T1 T2

Read(A,s); s s2;

Write(A,s);

Read(B,s); s s2;

Write(B,s);Read(A,t); t t+100Write(A,t);Read(B,t); t t+100;Write(B,t);

A B25 25

50

50

150

150150 150

resulting DB state different from schedule A but both results satisfy A = B


Serializable SchedulesA schedule S is serializable, if there is a serial schedule S’ (of the same actions) such that - for every initial DB state, and- for every semantics of the transactions, the effects of S and S’ are the same.

The order of transactions in the serial schedule is undefined (T1 before T2 or T2 before T1).

A serializable schedule transforms a consistent DB state into another consistent DB state.


Serializable SchedulesSchedule C (non-serial)

T1 T2Read(A,t); t t+100Write(A,t);

Read(A,s); s s2;

Write(A,s);Read(B,t); t t+100;Write(B,t);

Read(B,s); s s2;

Write(B,s);

A B25 25

125

250

125

250250 250

schedule equivalent to serial schedule (T1,T2)schedule is serializable


Serializable SchedulesSchedule D (non-serial)

T1 T2Read(A,t); t t+100Write(A,t);

Read(A,s); s s2;

Write(A,s);

Read(B,s); s s2;

Write(B,s);Read(B,t); t t+100;Write(B,t);

A B25 25

125

250

50

150250 150

resulting DB state inconsistent with A = Bschedule is not serializable


Serializable SchedulesSchedule E (non-serial)

T1 T2’Read(A,t); t t+100Write(A,t);

Read(A,s); s s1;

Write(A,s);

Read(B,s); s s1;

Write(B,s);Read(B,t); t t+100;Write(B,t);

A B25 25

125

125

25

125125 125

same as schedule D, but changed semantics of T2 resulting DB state consistent with A = B


Serializable SchedulesSemantics of a transaction: “function” to be computed, defined by the transaction code.

In general, it is too hard to analyze the semantics of a transaction automatically.

Therefore, the scheduler ignores the semantics of the transactions and considers only the sequence of read and write operations.

We assume the worst case: if there is something that T can do to make the DB state inconsistent, then T will do that.


Serializable SchedulesWe adopt the following notations:

rT(X): transaction T reads database element

X,

wT(X): transaction T writes database

element X.

We use r1(X) or w1(X) as shorthand for rT1(X)

or wT1(X), resp.

An action is of the form rT(X) or wT(X).

A transaction Ti is a sequence of actions

with subscript i.


Serializable SchedulesA schedule S of a set of transactions Trans is a sequence of actions that contains all actions of all transactions T in Trans in the same order in which they appear in the definition of T.

Example

T1=r1(A) w1(A) r1(B) w1(B)

T2=r2(A) w2(A) r2(B) w2(B)

S = r1(A) w1(A) r2(A) w2(A) r1(B) w1(B) r2(B)

w2(B)


Conflict-SerializabilityConflict-serializability is stronger than serializability, but easier to enforce.

Most commercial DBMS enforce conflict-serializability.

It is based on the notion of a conflict.

A pair of consecutive actions in a schedule constitutes a conflict if swapping these actions may change the effect of at least one of the transactions involved.


Conflict-Serializability

Most pairs of actions do not cause a

conflict.

ri (X) and rj (Y) never cause a conflict, even

if

X = Y, since they do not modify the DB

state.

ri(X) and wj(Y) do not cause a conflict if .

wi(X) and rj(Y) do not cause a conflict if .

wi(X) and wj(Y) do not cause a conflict if

.

YX

YX

YX


Conflict-SerializabilityThe following three situations do cause a

conflict:

Actions of the same transaction, i.e. i = j.

Two writes of the same database element

by different transactions, i.e. wi(X) and

wj(X), .

Depending on the schedule, the results of

either wi(X) or wj(X) survive, which may be

different.

A read and a write of the same database

element by different transactions, i.e. ri(X)

and wj(X),

. ri(X) may read a different version of

X.

ji

ji


Conflict-SerializabilityAny two actions of different transactions

may be swapped, unless they involve the

same database element and at least one of

them is a write.

If there is a sequence of non-conflicting

swaps that transforms schedule S into a

serial schedule S’, then S is serializable.

Schedules S1, S2 are conflict equivalent, if

S1 can be transformed into S2 by a series

of swaps on non-conflicting actions.


Conflict-SerializabilityA schedule is conflict serializable if it is

conflict equivalent to some serial schedule.

Example

S=r1(A)w1(A)r2(A)w2(A)r1(B)w1(B)r2(B)w2(B)

is conflict equivalent to the serial schedule

S’=r1(A) w1(A) r1(B)w1(B) r2(A) w2(A) r2(B) w2(B)

operations on critical DB elements are

always

first performed by T1, then by T2


Conflict-SerializabilityIf transactions Ti and Tj contain at least two pairs of conflicting actions, then for each of these pairs the action of Ti has to be performed before that of Tj (or always Tj before Ti).

Given a schedule S, Ti takes precendence

over Tj, denoted by Ti <S Tj, if there are

actions Ai of Ti and Aj of Tj such that- Ai is ahead of Aj in S,- both Ai and Aj involve the same database element, and at least one of them is a write.


Conflict-SerializabilityIf Ti takes precendence over Tj, then a schedule S’ that is conflict equivalent to S must have Ai before Aj.

Precedence graph: directed graph with nodes representing the transactions of S,

i.e. node label i for transaction Ti,edges representing precedence relationships,

i.e. edge from node i to j if Ti <S Tj.

Notation: P(S)


Conflict-SerializabilityExample S = w3(A) w2(C) r1(A) w1(B) r1(C) w2(A) r4(A)

w4(D)

P(S)

3 1 2 4

based on A

based on C


Conflict-SerializabilityLemma 1 S1, S2 conflict equivalent P(S1) = P(S2)

ProofAssume P(S1) P(S2)

Ti, Tj: Ti Tj in P(S1) and not in P(S2)

S1 = …pi(A)... qj(A)… pi, qj

S2 = …qj(A)…pi(A)... in conflict

S1, S2 not conflict equivalent


Conflict-SerializabilityNote P(S1)=P(S2) S1, S2 conflict equivalent

Counter example S1=w1(A) r2(A) w2(B) r1(B)

S2=r2(A) w1(A) r1(B) w2(B)

P(S1)=P(S2)= 1 2

S1 not conflict equivalent to S2, since w1(A) andr2(A) cannot be swapped


Conflict-SerializabilityTheorem 2P(S) acyclic S conflict serializable

Proof (i)

Assume S is conflict serializable. S’: S’ is serial, S conflict equivalent to S’. P(S’) = P(S) according to Lemma 1. P(S’) is acyclic because S’ is serial. P(S) is acyclic.


Conflict-SerializabilityProof (ii)

Assume P(S) is acyclic. Transform S as follows: (1) Take T1 to be transaction with no incoming edges.

T1 exists, since P(S) is acyclic. (2) Move all T1 actions to the front:

S = ……. qj(A)…….p1(A)…..This does not create any conflicts, since there isno Tj with Tj T1.

(3) We now have S’ = < T1 actions ><... rest ...>. (4) Repeat above steps to serialize rest.

T1

T2 T3

T4

P(S)


Conflict-SerializabilityHow to enforce that only conflict-serializable schedules are executed?

There are two alternative approaches:- pessimistic concurrency control Lock data elements to prevent P(S) cycles from occurring.- optimistic concurrency control Detect P(S) cycles and undo participating transactions, if necessary.


Enforcing Serializability by LocksBefore accessing a database element, a transaction requests a lock on that element in order to prevent other transactions from accessing the same database element at the “same” time.

Typically, different types of locks are used for different types of access operations, but we first introduce a simplified lock protocol with only one type of lock.


Enforcing Serializability by LocksWe introduce two new actions:

li (X): lock database element X ui (X): unlock database element X, i.e. release lock.

A locking protocol must guarantee the consistency of transactions: - A transaction can only read or write database X element if it currently holds a lock on X.- A transaction must unlock all database elements that is has locked at some later time.

A consistent transaction is also called well-formed.


Enforcing Serializability by LocksA locking protocol must also guarantee the legality of schedules: At most one transaction can hold a lock on database element X at a given point of time.

If there are actions li (X) followed by lj (X)

in some schedule, then there must be an

action ui(X) somewhere between these two

actions.


Enforcing Serializability by LocksExample

S1 = l1(A)l1(B)r1(A)w1(B)l2(B)u1(A)u1(B)

r2(B)w2(B)u2(B)l3(B)r3(B)u3(B)

S1 illegal, because T2 locks B before T1 has unlocked it

S2 = l1(A)r1(A)w1(B)u1(A)u1(B)

l2(B)r2(B)w2(B)l3(B)r3(B)u3(B)

T1 inconsistent, because T1 writes B before locking it

S3 = l1(A)r1(A)u1(A)l1(B)w1(B)u1(B)

l2(B)r2(B)w2(B)u2(B)l3(B)r3(B)u3(B)

schedule legal and all transactions consistent


Enforcing Serializability by LocksSchedule F

Schedule F is legal, but not serializable.

T1 T2 25 25

l1(A);Read(A)A A+100;Write(A);u1(A) 125

l2(A);Read(A)A Ax2;Write(A);u2(A)

250l2(B);Read(B)B Bx2;Write(B);u2(B)

50l1(B);Read(B)B B+100;Write(B);u1(B)

150 250

150

A B


Two-Phase LockingA legal schedule of consistent transactions is not necessarily conflict-serializable.

However, a legal schedule with the following locking protocol is conflict-serializable.

Two-phase locking (2PL)In every transaction, all lock actions precede all unlock actions.

Growing phase: acquire locks, no unlocks.

Shrink phase: release locks, no locks.


Two-Phase LockingExample

# locksheld byTi

time Growing Shrinking Phase Phase


Two-Phase LockingSchedule G

T1 T2l1(A);Read(A)A A+100;Write(A)l1(B); u1(A)

l2(A);Read(A) A Ax2;Write(A);ll22(B)(B)

Read(B);B B+100Write(B); u1(B)

l2(B); u2(A);Read(B) B Bx2;Write(B);u2(B);

Schedule G is serializable.

delayed

changed order!


Two-Phase LockingIn 2PL, each transaction may be thought of as executing all of its actions when issuing the first unlock action.

Thus, the order according to the first unlock action defines a conflict-equivalent serial schedule.

Theorem 3(1) legality of schedule, and (2) consistency of transactions and (3) 2PL

conflict-serializability.


Two-Phase LockingLemma 4

Ti Tj in S SH(Ti) <S SH(Tj)

where Shrink(Ti) = SH(Ti) = first unlock action of Ti

Proof Ti Tj means thatS = … pi(A) … qj(A) … and pi,qj conflict

According to (1), (2):S = … pi(A) … ui(A) … lj(A) ... qj(A) …

According to (3): Therefore, SH(Ti) <S SH(Tj).

SH(Ti) SH(Tj)


Two-Phase LockingProof of theorem 3

Given a schedule S.

Assume P(S) has cycle

T1 T2 …. Tn T1

By lemma 4: SH(T1) < SH(T2) < ... < SH(T1).

Contradiction, so P(S) acyclic.

By theorem 2, S is conflict serializable.

2PL allows only serializable schedules.


Two-Phase LockingNot all serializable schedules are allowed by 2PL.

Example S1: w1(x) w3(x) w2(y) w1(y)

The lock by T1 for y must occur after w2(y), so the unlock by T1 for x must also occur after w2(y) (according to 2PL).

Because of the schedule legality, w3(x) cannot occur where shown in S1 because T1 holds the x lock at that point.

However, S1 serializable (equivalent to T2, T1, T3).


Two-Phase LockingDeadlocks may happen under 2PL, when two or more transactions have got a lock and are waiting for another lock currently held by one of the other transactions.

Example (T2 reversed) T1: Read(A, t) T2: Read(B,s)

t t+100 s s2Write(A,t) Write(B,s)Read(B,t) Read(A,s)t t+100 s s2Write(B,t) Write(A,s)


Two-Phase LockingPossible schedule

Deadlock cannot be avoided, but can be detected(cycle in wait graph).

At least one of the participating transactions needs to be aborted by the DBMS.

T1 T2l1(A); Read(A) l2(B);Read(B)A A+100;Write(A) B Bx2;Write(B)

ll11(B)(B) l l22(A)(A)

delayed, wait for T1delayed, wait for T2


Two-Phase LockingSo far, we have introduced the simplest possible 2PL protocol and showed that it works.

There are many approaches for improving its performance, i.e. allowing a higher degree of concurrency:

- shared locks,- increment locks,- multiple granularity locks,- tree-based locks.


Shared and Exclusive LocksIn principle, several transactions can read database element A at the same time, as long as none is allowed to write A.

In order to enable more concurrency, we distinguish two different types of locks:

- shared (S) lock: there can be multiple shared locks on X, permission only to read A.

- exclusive (X) lock: there can be only one exclusive lock on A, permission to read and write A.


Shared and Exclusive LocksWe introduce the following lock actions for database element A and transaction i:sl-i(A): lock A in S mode

xl-i(A): lock A in X modeu-i(A): unlock whatever modes Ti has locked A

Modify consistency of transactions as follows:

- A read action ri(A) must be preceded by sl-i(A) or xl-i(A) with no intervening ui(A).

- A write action ri(A) must be preceded by xl-i(A) with no intervening ui(A).


Shared and Exclusive LocksTypically, a transaction does not know its needs for locks in advance.

What if transaction Ti reads and writes the same database element A?

Ti will request both shared and exclusive locks on A at different times.

Example Ti=... sl-1(A) … r1(A) ... xl-1(A) …w1(A) ...u(A)…

If Ti knows lock needs, request X lock right away.


Shared and Exclusive LocksModify legality of schedules as follows:

- If xl-i(A) appears in a schedule, then there cannot follow an xl-j(A) or sl-j(A),without an intervening ui(A).

- If sl-i(A) appears in a schedule, then an xl-j(A) cannot follow without an intervening ui(A).

All other consistency and legality as well as the 2PL requirements remain unchanged.

The proof of Theorem 3 still works.

,ji


Shared and Exclusive LocksA compatibility matrix is a convenient way to specify a locking protocol.

Rows correspond to lock already held by another transaction, columns correspond to a lock being requested by current transaction.

Lock requested

S X

Lock held S Yes No

in mode X No No


Shared and Exclusive LocksIf a transaction first reads A and later writes A, it has to upgrade its S lock to an X lock.

Upgrading is a frequent source of deadlocks.

T1 T2sl-1(A)

sl-2(A)r1(A)

r2(A)xl-1(A)xl-1(A)

xl-2(A)xl-2(A)w1(A)w1(A)


Update LocksIn order to avoid such deadlocks (as far as possible), we introduce another type of lock.

An update lock ul-i(A) gives transaction i the privilege to - read database element A and to- upgrade its lock on A to an X lock.

An update lock is not shared.

Read locks cannot be upgraded.


Update LocksCompatibility matrix

Lock requested S X U

Lock held S Yes No Yes in mode X No No No

U No No NoExample T1 T2

ul-1(A)ul-2(A)ul-2(A)

r1(A)xl-1(A)w1(A)

U is not symmetric!


Locks With Multiple GranularityDatabase elements can be tuples, blocks or entire relations.

At which level of granularity shall we lock?

There is a trade-off: the lower the level of granularity, the more concurrency, but the more locks and the higher the locking overhead.

Best trade-off depends on application: e.g., lock blocks or tuples in bank database, and entire documents in document database.


Locks With Multiple GranularityEven within the same application, there may be a need for locks at multiple levels of granularity.

Database elements are organized in a hierarchy:

relations R1

blocks B1 B2 B3 B4

tuples t1 t2 t3 t4 t5

contained in


Locks With Multiple GranularityThe warning protocol manages locks on a hierarchy of database elements.

We introduce two new types of locks:

- IS: intention to request an S lock and

- IX: intention to request an X lock.

An IS (IX) lock expresses the intention to request an S (X) lock for a subelement further down in the hierarchy.


Locks With Multiple GranularityTo request an S (or X) lock on some database element A, we traverse a path from the root of the hierarchy to element A.

If we have reached A, we request the S (X) lock.

Otherwise, we request an IS (IX) lock.

As soon as we have obtained the requested lock, we proceed to the corresponding child (if necessary).


Locks With Multiple GranularityCompatibility matrix

Requester

IS IX S X

IS Yes Yes Yes No Holder IX Yes Yes No No

S Yes No Yes NoX No No No No

If two transactions intend to read / write a subelement, we can grant both of them an I lock and resolve the potential conflict at a lower level.


Locks With Multiple GranularityAn I lock for a superelement constrains the locks that the same transaction can obtain at a subelement.

If Ti has locked the parent element P in IS, then Ti can lock child element C in IS, S.

If Ti has locked the parent element P in IX, then Ti can lock child element C in IS, S, IX, X.

P

C


Locks With Multiple GranularityExample T2 wants to request an X lock on tuple t3

R1

B1

B2 B3B4T1(IX)

t2 t3 t4 t5

T1(IX)

T1(X)

T2(IX)

T2(IX)

T2(X)


Locks With Multiple GranularityExample T2 wants to request an S lock on block B2

R1

B1

B2 B3B4T1(IX)

t2 t3 t4 t5

T1(IX)

T1(X)

T2(IS)

T2(S) not granted!


Optimistic Concurrency Control

Optimistic approaches to concurrency control assume that unserializable schedules are infrequent.

Unlike in pessimistic approaches (locking), unserializable schedules are not prevented, but detected and some of the transactions aborted.

The two main optimistic approaches are timestamping (not covered in class) and validation (next section).


Concurrency Control by Validation

We allow transactions to proceed without locking.

All DB modifications are made on a local copy.

At the appropriate time, we check whether the transaction schedule is serializable.

If so, the modifications of the local copy are applied to the global DB.

Otherwise, the local modifications are discarded, and the transaction is re-started.



For each transaction T, the scheduler maintains two sets of relevant database elements:

- RS(T), the read set of T: the set of all database elements read by T.

- WS(T), the write set of T: the set of all database elements written by T.

This information is crucial to determine whether some schedule that has already been executed was indeed serializable.



Transaction T is executed in three phases:

1. Read: transaction reads all elements in its read set from DB and is executes all its actions in its local address space.

2. Validate: the serializability of the schedule is checked by comparing RS(T) and WS(T) to the read / write sets of the concurrent transactions.If validation is unsuccessful, skip phase 3.

3. Write: write the new values of the elements in WS(T) back to the DB.



At any time, the scheduler maintains three sets of transactions and some relevant information.

START: set of transactions that have started, but have not yet completed their validation phase. For each element T of START, keep START(T).

VAL: set of transactions that have completed validation, but not yet their write phase. For elements T of VAL, record VAL(T).

FIN: set of transactions that have completed all three phases. For T in FIN, keep FIN(T).



Make validation an atomic operation.

If T1, T2, T3, … is validation order, then the resulting schedule will be conflict equivalent to serial schedule S = T1, T2, T3.

Can think of each transaction that successfully validates as executing entirely at the moment that it validates.



Example

It is possible that T1 wrote database element B after T2 has read it.

Schedule is not conflict-equivalent to T1,T2.

RS(T1)={B} RS(T2)={A,B}WS(T1)={B,D}

WS(T2)={C}

time

T1

start

T1

validated

T2

validatedT2

reads B

=

T2

start T1

writes B



Example

New value of B written by T1 must have been written back to the DB before T2 has read B.

Schedule is conflict-equivalent to T1, T2.

T1 finishphase 3 time

T1

startT1

validated

T2

validatedT2

start

=

T2

start

RS(T1)={B} RS(T2)={A,B}WS(T1)={B,D}

WS(T2)={C}

T2

reads B

T1

writes B



Example

The new value of D written by T1 may be output to the DB later than the new value written by T2.

Schedule is not conflict-equivalent to T1, T2.

RS(T1)={A} RS(T2)={A,B}WS(T1)={D,E} WS(T2)={C,D}

time

T1

validatedT2

validated

T1 finishphase 3

=

T2

output D

T1

output D



Example

The new value of D written by T1 must be output to the DB earlier than the new value of D written by T2.

Schedule is conflict-equivalent to T1, T2.

T1 finishphase 3

time

T1

validatedT2

validated

T1 finishphase 3

T1

output D

T2

output D

RS(T1)={A} RS(T2)={A,B}WS(T1)={D,E} WS(T2)={C,D} =



.)1()2( TWSTRS

The above examples motivate the following two validation rules for a given transaction T2.

We consider all transactions T1 that have validated before T2.

For all T1 with FIN(T1) > START(T2):

For all T1 with FIN(T1) > VAL(T2):

If T2 does successfully validate, if the two validation rules are satisfied for all these T1.

.)1()2( TWSTWS



U validates successfully, since there are no other transactions that have validated before U.

T: RS(T)={A,B} WS(T)={A,C}

V: RS(V)={B} WS(V)={D,E}

U: RS(U)={B} WS(U)={D}

W: RS(W)={A,D} WS(W)={A,C}

startvalidatefinish



T validates successfully, since RS(T) and WS(T) have no intersection with WS(U).





startvalidatefinish



V validates successfully, since RS(V) has no intersection with WS(U) and FIN(U) < VAL(V) and neither RS(V) nor WS(V) have intersection with WS(T).





startvalidatefinish



W validates unsuccessfully, since RS(W) has intersection with WS(V) and FIN(V) > START(W).





startvalidatefinish


Concurrency Control Mechanisms

We conclude by comparing pessimistic and optimistic concurrency control mechanisms.

Locking delays transactions, but avoids rollbacks.

Validation does not delay transactions, but can cause a rollback (and re-start).

Rollbacks may waste a lot of resources.

If interactions between transactions are infrequent, then there will be few rollbacks, and validation will be more efficient.

Database Systems II Concurrency Control

Documents