Diagnosing Unbounded Heap Growth in C++ Problem Description Types of Unbounded Heap Growth –Reference Lost (Leak) Reference lost to memory without freeing.

Diagnosing Unbounded Heap Growth in C++Problem Description

• Types of Unbounded Heap Growth– Reference Lost (Leak)

• Reference lost to memory without freeing it• Well studied, wide variety of tools that identify leaks

– Reference Retained• But no longer needed• On shutdown, memory freed, so not reported as leak• If growing, it’s accumulating in a data structure• Few tools exist to deal with this problem in C++

– Valgrind, IBM Rational Purify, Insure++ do not detect this• Generally also referred to as a leak

1

Diagnosing Unbounded Heap Growth in C++Problem Description

• Memory Tumor– Data structure with unbounded growth– Examples:

• Storing unlimited history info in memory• Incorrect removal functions• Simplest example:

2

void main() { std::vector<int> tumorV; while( inputKey != ESC ) tumorV.push_back(1);}

Diagnosing Unbounded Heap Growth in C++Detection Approach

• 3 Aspects of Tumor Detection– Container tracking

• Hold references to all data structures in the system

– Growth tracking• Track size changes per data structure over time• Report those with unbounded growth

– Automated Test• Created by user to exercise all code paths

3

Diagnosing Unbounded Heap Growth in C++ Review

• Container tracking– CAT (Central Aggregate Tracker)

• Maintains references to all aggregates in the system– Create wrappers for each aggregate type in

system• Templated constructors, multiple inheritance• Add to CAT on construction, remove on destruction

– Namespace replacement to enable wrappers• Find and replace to apply new namespace• Wrappers disabled with compile time flag• Example: trak::std::vector<int>

4

Diagnosing Unbounded Heap Growth in C++ Detection Approach

• Growth Tracking– Take periodic samples of the CAT– Exponentially increasing interval sizes

• Reduces false positives & negatives over time– Report growing aggregates at each sample

5

Diagnosing Unbounded Heap Growth in C++Testing

• Automated Test– Detecting growth requires a test that exhibits

the growth• Good complete test design is important

– Need cyclic tests that cover all code paths• May require multiple tests

– Slow growing tumors may grow quickly with different usage patterns• Eg. WebKit MiniBrowser pop-up windows

6

Diagnosing Unbounded Heap Growth in C++ Detection Approach

• Growth Tracking– Heuristic

• Take periodic samples of the CAT• Two Interval Analysis

– 1st interval establishes aggregate age, gives time to stabilize– 2nd interval proves stability, non-tumors shouldn’t grow– 2nd interval becomes the 1st for next more accurate test

7

Diagnosing Unbounded Heap Growth in C++ Detection Approach

• Growth Tracking– Two interval analysis

8

time

memory

1 2 3 4

Diagnosing Unbounded Heap Growth in C++ Detection Approach

• Growth Tracking– Two interval analysis

9

time

memory

1 2 3 4

Reported as tumor(false positive)

Not reported(growth stabilized)

Diagnosing Unbounded Heap Growth in C++ Detection Approach

• Growth Tracking– Heuristic

• Take periodic samples of the CAT• Two Interval Analysis

– 1st interval establishes aggregate age, gives time to stabilize– 2nd interval proves stability, non-tumors shouldn’t grow– 2nd interval becomes the 1st for next more accurate test

• Exponentially increasing interval sizes– Reduces false positives & negatives over time

10

Diagnosing Unbounded Heap Growth in C++ Detection Approach

• Growth Tracking– Exponentially increasing interval size

11

time

memory

1 2 3 4

In this example: constant intervals would not report growth half the time

Diagnosing Unbounded Heap Growth in C++ Detection Approach

• Growth Tracking– Heuristic

• Take periodic samples of the CAT• Two Interval Analysis

– 1st interval establishes aggregate age, gives time to stabilize– 2nd interval proves stability, non-tumors shouldn’t grow– 2nd interval becomes first for next more accurate test

• Exponentially increasing interval sizes– Reduces false positives & negatives over time

• Monitor size maximums– Reduces size fluctuation false positives

12

Diagnosing Unbounded Heap Growth in C++ Detection Approach

• Growth Tracking– Max size variable

13

1 2 3

time

memory

ceiling

4Growth would be reported without

max size

Diagnosing Unbounded Heap Growth in C++ Detection Approach

• Growth Tracking– Heuristic

• Take periodic samples of the CAT• Two Interval Analysis

– 1st interval establishes aggregate age, gives time to stabilize– 2nd interval proves stability, non-tumors shouldn’t grow– 2nd interval becomes the 1st for next more accurate test

• Exponentially increasing interval sizes– Reduces false positives & negatives over time

• Monitor size maximums– Reduces size fluctuation false positives

• At each interval report all aggregates that:– Increased their size maximum– Have existed for two full intervals

• Sort results by size & reporting frequency to prioritize investigation

14