Data Compression in Blackbaud CRM Page 1 Data Compression in Blackbaud CRM Databases Len Wyatt Enterprise Performance Team Executive Summary ................................................................................................................................... 1 Compression in SQL Server ..................................................................................................................... 2 Perform Compression in Blackbaud CRM Databases ......................................................................... 3 Initial Compression................................................................................................................................. 3 Log Files and Compression .................................................................................................................. 6 Prepare for Compression ...................................................................................................................... 7 Maintain Compression ........................................................................................................................... 7 What to Do with the Saved Space ....................................................................................................... 8 Choose Compression Types ................................................................................................................ 8 Benefits of Compression in Blackbaud CRM Databases ..................................................................... 9 Storage Space Reduction and Compression Time........................................................................... 9 Performance Gains Measured in the Lab......................................................................................... 11 Performance Gains Measured by Heifer .......................................................................................... 14 Executive Summary Compression refers to applying mathematical techniques to make objects take less storage space than they normally require. Most people are familiar with using the ZIP utility to compress documents. This is useful to attach documents in email because they take up less space and are more likely to fit in a recipient’s mailbox. SQL Server can apply similar methods to data in its tables and indexes. The data is still in the database, but it’s stored in a way that requires less physical space. SQL Server can apply various techniques to compress data. Compression is probably not applied by default because it requires extra CPU time to compress data when it is stored and to decompress data when it is read. The Microsoft SQL Server team may not have wanted to decide this trade-off for everyone. However, modern servers are rarely limited by CPU resources. The limiting factor in servers is more likely the amount of data that can be sent to and fetched from the disk subsystem (I/O), so making data smaller is usually a good tradeoff. And since compression saves memory on the server by compressing objects in memory as well as on disk, it also reduces the amount of I/O since more information can be retained in memory.
15
Embed
Data Compression in Blackbaud CRM Databases · Data Compression in Blackbaud CRM Databases ... Log Files and Compression ... Most people are familiar with using the ZIP utility to
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Option 4: Daily Backup Cycle. If your site does backups nightly, you may want to perform compression at night to avoid creating load during the day. In this scenario, you can add a call to usp_RunCompression in your daily maintenance scripts before the daily backup. Each night, the system compresses objects as long as log space is available, and then it stops until the next night. Over the course of several days, space becomes available – and performance improves – as the system works through all the objects.
Prepare for Compression
Compression doesn’t change any logic in the database, but it requires a lot of physical work. You should
test compression to make sure you understand the process and to confirm that nothing goes wrong. All
Blackbuad CRM customers have staging systems in addition to their production systems, and we
recommend that you use your staging system for testing.
Set up the staging system with a recent copy of the production database.
Familiarize yourself with the performance of key operations on the staging system. Use a
stopwatch to time operations that are part of your acceptance criteria for upgrades.
Compress the database with the scripts described in this paper. The process should take a few
hours, although this varies based on your machine and the size of the database.
Use the stopwatch again to time key operations. If they aren’t faster, you still saved a bunch of
space. And if they are faster, you are a hero.
Validate the functioning of maintenance processes and other operational tools.
After everything checks out, schedule a time to repeat the process on the production system.
Maintain Compression
Blackbaud CRM is a complex application that dynamically generates new objects in the database over
time. As of version 4.0, none of these objects are compressed. Customizations can also create objects in
your database that are not compressed. To take advantage of compression for these objects, we
recommend that you periodically perform the compression steps. Perhaps on a monthly basis, run
usp_CreateCompressionQueue and usp_RunCompression. After the initial mass conversion to compress
objects, periodic re-checks are quick. We also recommend that you perform the compression steps after
upgrades because until version 4.0, patches and service packs are not aware of the compression option.
Data Compression in Blackbaud CRM Page 8
What to Do with the Saved Space
After you implement compression, it may be tempting to reclaim the saved space on the disks of your
server system. Before you do that, please consider two things:
Databases tend to grow over time, so the saved space is valuable as a buffer against future
growth. When you leave the space in place, you create extra time before you need additional
space. Of course, it is still wise to monitor space usage as suggested in Manage File Growth in
Blackbaud CRM Databases. That article includes a script to monitor the space that files use so
that you can plan for growth.
The way to reclaim the space is to “shrink” the data files. This operation is bad for performance
because it severely fragments the storage. For more information, see Why you should not shrink
your data files by SQL Server guru Paul Randal. If you absolutely must shrink data files, then
follow that action with a process to defragment the indexes. Shrinking and defragmenting are
time-consuming operations. They are also logged operations that carry some risk of expanding
the transaction log file, which is exactly what we took care to avoid in the compression
operations!
Choose Compression Types
The usp_CreateCompressionQueue SP determines the set of objects to compress. It evaluates each
table and index in the database to determine whether they are eligible for compression, and then it
determines the type of compression to use. The evaluation process is controlled by settings that an
experienced DBA can adjust with usp_UpdateCompressionSettings. The Performance Team
recommends page compression because it saves the most storage space and has excellent performance
characteristics. Our tests do not shown a compelling reason to choose the other compression types over
page compression.
In usp_UpdateCompressionSettings, you set CompressionType to “1” to use page compression for all
objects. You set CompressionType to “2” uses row compression for all objects. And if your site wants to
improve performance by focusing on response time rather than maximum space savings, you set
CompressionType to “3” to use a hybrid compression approach that applies heuristics on an object-by-
object basis to seek optimal system performance. Note: Initial tests do not indicate any performance
benefits from this method beyond the benefits from page compression. We left the option in place, but
we cannot recommend it over page compression without additional testing.
When you set CompressionType to “3,” the hybrid approach uses statistics about the usage of individual
objects to determine their compression types. Statistics are collected from the running system when
usp_CreateCompressionQueue is run. To get a complete picture, you should run the SP after the system
runs for a significant time – at least a day. To retain statistics from previous runs, set
OverwriteTableData to “0.” (The default is “1.”)
Hybrid compression determines the compression type to use for each object based on the following