-
SQL Server: The Transaction Log
As a general database concept, the journal or transaction log is
a core part of the database functionality.
As changes take place in the database, the transaction log is a
core part of the DB architecture that
makes disaster recover possible. Every database modification or
“transactions” are written to the
transaction log in a sequential fashion.
The transaction log is a physical file which is stored on the
disk. The location of the log is defined when a
database is created.
The log file will appear as a file type of “ldf” in the file
path specified. For example, “NewDB_log.ldf”
The log files are set to autogrow, which means that they will
claim more physical hard drive space as
more transaction are processed by the database.
-
What is stored on a transaction log?
The log stores every transaction made to the database for the
most part. A log is needed for the
database to support ACID transactions. ACID stands for
atomicity, consistency isolation and durability
which means:
• Atomicity: An atomic transaction is either fully completed, or
is not begun at all • Consistency: A transaction enforces
consistency in the system state by ensuring that at the end of
any transaction the system is in a valid state • Isolation: When
a transaction runs in isolation, it appears to be the only action
that the system is
carrying out at one time • Durability: A transaction is durable
meaning that once it has been successfully completed, all of
the
changes it made to the system are permanent
In theory, a log file has the following information:
– Transaction ID: In SQL Server, this is the Log Sequence
Number.
– Date and time of each update
– Before image: A snapshot of the data before the modification
took place.
– After image: A snapshot of the data after the modification
took place.
– End of a transaction and start of a transaction
– Successful completion (commit) of a transaction
Many types of operations are recorded in the transaction log.
These operations include:
• The start and end of each transaction.
• Every data modification (insert, update, or delete). This
includes changes by system stored procedures or
data definition language (DDL) statements to any table,
including system tables.
• Every extent and page allocation or deallocation.
• Creating or dropping a table or index.
• Rollback operations. Each transaction reserves space on the
transaction log to make sure that enough log
space exists to support a rollback that is caused by either an
explicit rollback statement or if an error is
encountered.
Logical Architecture
The SQL Server transaction log operates logically as if the
transaction log is a string of log records.
Each log record is identified by a log sequence number (LSN).
Each new log record is written to the logical end of
the log with an LSN that is higher than the LSN of the record
before it.
Log records are stored in a serial sequence as they are created.
Each log record contains the ID of the transaction
that it belongs to.
For each transaction, all log records associated with the
transaction are individually linked in a chain using
backward pointers that speed the rollback of the
transaction.
-
Log records for data modifications record either the logical
operation performed or they record the before and
after images of the modified data.
When the logging reaches the end of the file it starts again
from the beginning, but only if all the
requirements has been met and the inactive parts has been
truncated. The truncation process is necessary to
mark all inactive parts so they can be used again and
overwritten
A log record is no longer needed in the transaction log if all
of the following are true:
• The transaction of which it is part has committed • The
database pages it changed have all been written to disk by a
checkpoint • The log record is not needed for a backup (full,
differential, or log) • The log record is not needed for any
feature that reads the log (such as database mirroring or
replication)
Virtual Log Files
The structure of the file is internally split into “Virtual Log
Files” (VLFs).
Virtual log files have no fixed size, and there is no fixed
number of virtual log files for a physical log file. The
Database Engine chooses the size of the virtual log files
dynamically while it is creating or extending log files.
The transaction log is a wrap-around file. For example, consider
a database with one physical log file divided into
four virtual log files. When the database is created, the
logical log file begins at the start of the physical log file.
New log records are added at the end of the logical log and
expand toward the end of the physical log. Log
truncation frees any virtual logs whose records all appear in
front of the minimum recovery log sequence number
(MinLSN).
The MinLSN is the log sequence number of the oldest log record
that is required for a successful database-wide
rollback. The transaction log in the example database would look
similar to the one in the following illustration.
-
When the end of the logical log reaches the end of the physical
log file, the new log records wrap around to the
start of the physical log file.
This cycle repeats endlessly, as long as the end of the logical
log never reaches the beginning of the logical log. If
the old log records are truncated frequently enough to always
leave sufficient room for all the new log records
created through the next checkpoint, the log never fills.
However, if the end of the logical log does reach the start of
the logical log, one of two things occurs:
• If the FILEGROWTH setting is enabled for the log and space is
available on the disk, the file is extended by
the amount specified in the growth_increment parameter and the
new log records are added to the
extension.
• If the FILEGROWTH setting is not enabled, or the disk that is
holding the log file has less free space than
the amount specified in growth_increment, a 9002 error is
generated
If the log contains multiple physical log files, the logical log
will move through all the physical log files before it
wraps back to the start of the first physical log file.
-
Log Truncation
Log truncation is essential to keep the log from filling. Log
truncation deletes inactive virtual log files from
the logical transaction log of a SQL Server database, freeing
space in the logical log for reuse by the
physical transaction log.
If a transaction log were never truncated, it would eventually
fill all the disk space that is allocated to its
physical log files. However, before the log can be truncated, a
checkpoint operation must occur. A
checkpoint writes the current in-memory modified pages (known as
dirty pages) and transaction log
information from memory to disk.
When the checkpoint is performed, the inactive portion of the
transaction log is marked as reusable.
Thereafter, the inactive portion can be freed by log
truncation.
The following illustrations show a transaction log before and
after truncation. The first illustration shows a
transaction log that has never been truncated. Currently, four
virtual log files are in use by the logical log.
The logical log starts at the front of the first virtual log
file and ends at virtual log 4.
The MinLSN record is in virtual log 3. Virtual log 1 and virtual
log 2 contain only inactive log records.
These records can be truncated. Virtual log 5 is still unused
and is not part of the current logical log.
The second illustration shows how the log appears after being
truncated. Virtual log 1 and virtual log 2
have been freed for reuse. The logical log now starts at the
beginning of virtual log 3. Virtual log 5 is still
unused, and it is not part of the current logical log.
-
If log truncation cannot occur in time, the log file will grow
and can lead to an out of space error over
time.
To truncate the log, use the DBCC SHRINKFILE
Useful Commands for the Transaction Log
DBCC SQLPERF(LOGSPACE)
We can use this monitor the growth of the log files:
DBCC SQLPERF (Logspace)
This will give us the current lo size and how much space is
currently used (in terms of percentage).
BACKUP
Used to backup the transaction log to a disk or backup device.
This is typically done before shrinking the
file.
USE master; GO ALTER DATABASE MyDB SET RECOVERY FULL; GO
BACKUP LOG MyDB
-
TO DISK = 'Z:\SQLServerBackups\MyDBLog.bak' WITH FORMAT; GO
DBCC SHRINKFILE
This command can be used to shrink a log file to a specific
target size. This should be done after your log
file has been backed up.
-- Truncate the log by changing the database recovery model to
SIMPLE. ALTER DATABASE MyDB SET RECOVERY SIMPLE; GO -- Shrink the
truncated log file to 1 MB. DBCC SHRINKFILE (MyDB_Log, 1); GO --
Reset the database recovery model. ALTER DATABASE MYDB SET RECOVERY
FULL; GO
Transaction Log and its role in recovery
The transaction log is a crucial component of the recoveries we
can perform because it maintains the
before and after image of the transactions.
Forward Recovery
Scenario:
Current prod database is damaged and is not available. Assume
that we have a copy of the production
database from a couple of days ago, and the full copy of the
transaction log.
We can use out database backup (a couple of days old) and apply
the transactions of the transaction log
(because they contain the after images).
-
Back Recovery (Rollback)
Assume that our production database has a set of transaction
which are executed so that our database
is in an inconsistent state. We want to essentially do a
rollback of the transactions that were committed.
We can do this because we have access to the before images of
the committed transaction in the
transaction log.