Diagnosing Unbounded Heap Growth in C++Problem Description
• Types of Unbounded Heap Growth– Reference Lost (Leak)
• Reference lost to memory without freeing it• Well studied, wide variety of tools that identify leaks
– Reference Retained• But no longer needed• On shutdown, memory freed, so not reported as leak• If growing, it’s accumulating in a data structure• Few tools exist to deal with this problem in C++
– Valgrind, IBM Rational Purify, Insure++ do not detect this• Generally also referred to as a leak
1
Diagnosing Unbounded Heap Growth in C++Problem Description
• Memory Tumor– Data structure with unbounded growth– Examples:
• Storing unlimited history info in memory• Incorrect removal functions• Simplest example:
2
void main() { std::vector<int> tumorV; while( inputKey != ESC ) tumorV.push_back(1);}
Diagnosing Unbounded Heap Growth in C++Detection Approach
• 3 Aspects of Tumor Detection– Container tracking
• Hold references to all data structures in the system
– Growth tracking• Track size changes per data structure over time• Report those with unbounded growth
– Automated Test• Created by user to exercise all code paths
3
Diagnosing Unbounded Heap Growth in C++ Review
• Container tracking– CAT (Central Aggregate Tracker)
• Maintains references to all aggregates in the system– Create wrappers for each aggregate type in
system• Templated constructors, multiple inheritance• Add to CAT on construction, remove on destruction
– Namespace replacement to enable wrappers• Find and replace to apply new namespace• Wrappers disabled with compile time flag• Example: trak::std::vector<int>
4
Diagnosing Unbounded Heap Growth in C++ Detection Approach
• Growth Tracking– Take periodic samples of the CAT– Exponentially increasing interval sizes
• Reduces false positives & negatives over time– Report growing aggregates at each sample
5
Diagnosing Unbounded Heap Growth in C++Testing
• Automated Test– Detecting growth requires a test that exhibits
the growth• Good complete test design is important
– Need cyclic tests that cover all code paths• May require multiple tests
– Slow growing tumors may grow quickly with different usage patterns• Eg. WebKit MiniBrowser pop-up windows
6
Diagnosing Unbounded Heap Growth in C++ Detection Approach
• Growth Tracking– Heuristic
• Take periodic samples of the CAT• Two Interval Analysis
– 1st interval establishes aggregate age, gives time to stabilize– 2nd interval proves stability, non-tumors shouldn’t grow– 2nd interval becomes the 1st for next more accurate test
7
Diagnosing Unbounded Heap Growth in C++ Detection Approach
• Growth Tracking– Two interval analysis
8
time
memory
1 2 3 4
Diagnosing Unbounded Heap Growth in C++ Detection Approach
• Growth Tracking– Two interval analysis
9
time
memory
1 2 3 4
Reported as tumor(false positive)
Not reported(growth stabilized)
Diagnosing Unbounded Heap Growth in C++ Detection Approach
• Growth Tracking– Heuristic
• Take periodic samples of the CAT• Two Interval Analysis
– 1st interval establishes aggregate age, gives time to stabilize– 2nd interval proves stability, non-tumors shouldn’t grow– 2nd interval becomes the 1st for next more accurate test
• Exponentially increasing interval sizes– Reduces false positives & negatives over time
10
Diagnosing Unbounded Heap Growth in C++ Detection Approach
• Growth Tracking– Exponentially increasing interval size
11
time
memory
1 2 3 4
In this example: constant intervals would not report growth half the time
Diagnosing Unbounded Heap Growth in C++ Detection Approach
• Growth Tracking– Heuristic
• Take periodic samples of the CAT• Two Interval Analysis
– 1st interval establishes aggregate age, gives time to stabilize– 2nd interval proves stability, non-tumors shouldn’t grow– 2nd interval becomes first for next more accurate test
• Exponentially increasing interval sizes– Reduces false positives & negatives over time
• Monitor size maximums– Reduces size fluctuation false positives
12
Diagnosing Unbounded Heap Growth in C++ Detection Approach
• Growth Tracking– Max size variable
13
1 2 3
time
memory
ceiling
4Growth would be reported without
max size
Diagnosing Unbounded Heap Growth in C++ Detection Approach
• Growth Tracking– Heuristic
• Take periodic samples of the CAT• Two Interval Analysis
– 1st interval establishes aggregate age, gives time to stabilize– 2nd interval proves stability, non-tumors shouldn’t grow– 2nd interval becomes the 1st for next more accurate test
• Exponentially increasing interval sizes– Reduces false positives & negatives over time
• Monitor size maximums– Reduces size fluctuation false positives
• At each interval report all aggregates that:– Increased their size maximum– Have existed for two full intervals
• Sort results by size & reporting frequency to prioritize investigation
14
