Memory Usage - Microsoft Azuredebugdiagtraining.azurewebsites.net/slides... · 2016. 4. 14. · Other HEAPs .NET Heap. Virtual Memory Layout Virtual Memory Process 1 Physical Memory

Memory Usage and .NET

Module Overview - Troubleshooting .NET Memory Issues

Memory Management: General Review

GC Heap

Generations: Structure & Benefits

GC Sequence

GC Modes

Maximizing GC Performance

Roots

Finalization

Review

2

Native Memory Usage

Memory Management: General Review

Virtual Memory

32-bit system

- 4 GB (2GB-2GB for user and kernel mode)

- Can be altered with /3GB switch or USERVA in boot.ini

- Important: /LARGEADDRESSAWARE flag

64-bit system

- 16 TB – huge address space

Memory states

- Free, Reserved, Committed

- Perfmon: Process\Virtual Bytes, Process\Private Bytes

4

Virtual Memory Map (2GB-32Bit)

No access region

Private Process Space

Stacks

0x00000000

0x0000FFFF

0x7FFE1000

0x7FFFFFFF

0x7FFE0000

Module images

Heaps

Virtual Allocations

Unused

TEBs

PEB

Shared User Data (Date,Time,Win32 Version,..)

No access region

NT (Windows 32) Heaps

Other HEAPs .NET Heap

Virtual Memory Layout

Virtual Memory

Process 1 Physical Memory

(RAM)

Virtual Memory

Process 2

NTDLL.DLL

Process 2 Data

Proc2.exe

Process 2

Data

Process 1

Data

NTDLL.DLL

Proc2.exe

Process 2 Data

Process 2

Data

Process1 Data

Proc1.exe

NTDLL.DLL

Process1 Data

Proc1.exe

Process 1

Data

0x00000000

0xFFFFFFFF

Use

r M

od

eK

ern

el M

od

e

Memory Limits 32 <-> 64

Memory Limits 32-bit 64-bit

Win 7/2008

64Bit

Win 8.1/2012 R2

Total Virtual Address Space 4 GB 16 TB 256 TB

Virtual Address Space per

32-bit Process

2 GB 2 GB 2 GB


32-bit Process compiled

with

/LARGEADDRESSAWARE

3 GB if

system is

booted with

/3gb switch

and

4GB 4GB


64-bit Process

x64

Not

applicable

=16 TB-64k

no access

region

0-7ffffff0000

=128 TB-64k no

access region

Paged Pool 470 MB 128 GB 384GB

Non-paged Pool 256 MB 128 GB 15.5TB

System Cache 1 GB 1 TB 16TB

Memory Management: General Review #3

Memory Managers

- NT Heap

- GC Heap (mscorwks.dll, clr.dll)

Acquiring and Committing Memory

- VirtualAlloc

- HeapAlloc

- New/malloc

8

.NET Heap

Layered Memory Management in Win32

Win32 Application

Local, Global

Memory API

VC6 <

malloc

NT Heap

Virtual Memory Allocator

Memory Mapped

Files

NT Virtual Memory Manager

Physical

Memory Hard Disk

Win32

SubSystem

.NET Application

Kernel


VirtualAlloc API

- lpAddress – where to allocate

- dwSize – how much

- flAllocationType (MEM_COMMIT, MEM_RESERVE, MEM_RESET

)

- flProtect - protection level

Smallest chunk it can reserve: 64KB

base memory allocator in Windows.

enables a process to reserve a range of its virtual address space

without consuming physical storage until it is needed

Memory managers are taking the role of reserving the memory

10


Memory Managers

- NT Heap

- GC Heap (mscorwks.dll, clr.dll)

Acquiring and Committing Memory

- VirtualAlloc

- HeapAlloc

- New/malloc

11

NT Heap (HeapAlloc,..)

Used from the „most“ Win32 Applications

Users can create their own process heaps using HeapCreate()

or use the default process heap by using the GetProcessHeap() API.

Allocation does not have to be aligned on a page boundary

In Windows 2000 and 2003, applications that frequently use allocations of varying sizes or lifetimes can use the low fragmentation heap (LFH) )(via HeapSetInformation())

Design changed to use LFH up from Vista

X64: Required to return memory on a 16-byte boundary

Process Environment Block

(PEB)Describes a user-mode process.

Created, maintained, and destroyed by the operating system.

Contains information needed by the image loader, heap manager, and other Win32 subsystems.

One and only one PEB for each user process.

This is the user-mode parallel of the kernel’s KPROCESS block.

Accessible using the !PEB debugger within Windbg debugger

!PEB0:010> !peb

PEB at 7ffd7000

InheritedAddressSpace: No

ReadImageFileExecOptions: No

BeingDebugged: Yes

ImageBaseAddress: 00440000

Ldr 00241e90

Ldr.Initialized: Yes

Ldr.InInitializationOrderModuleList: 00241f28 . 002443f0

Ldr.InLoadOrderModuleList: 00241ec0 . 002443e0

Ldr.InMemoryOrderModuleList: 00241ec8 . 002443e8

Base TimeStamp Module

440000 40f6450f Jul 15 10:49:19 2004

\\?\C:\WINDOWS\Microsoft.NET\Framework\v1.1.4322\aspnet_wp.exe

7c900000 411096b4 Aug 04 09:56:36 2004

C:\WINDOWS\system32\ntdll.dll

7c800000 411096b4 Aug 04 09:56:36 2004

Thread Environment Block (TEB)

Per-thread structure that describes a running thread within a process.

The operating system maintains a region of virtual address space that stores TEBs for running threads.

Similar to the kernel’s ETHREAD structure.

Accessible under the debugger using the !TEB command for the current thread context.

contains pointer to ExceptionList, Stack Base, Last Win32 Error, ...

!TEB

0:000> !teb

TEB at 000007fffffde000

ExceptionList: 0000000000000000

StackBase: 0000000000230000

StackLimit: 000000000021c000

SubSystemTib: 0000000000000000

FiberData: 0000000000001e00

ArbitraryUserPointer: 0000000000000000

Self: 000007fffffde000

EnvironmentPointer: 0000000000000000

ClientId: 000000000000139c . 0000000000001280

RpcHandle: 0000000000000000

Tls Storage: 000007fffffde058

PEB Address: 000007fffffd8000

LastErrorValue: 5

LastStatusValue: c0000034

Count Owned Locks: 0

HardErrorMode: 0

Understanding Managed Memory

Managed heaps

Loader heap

Native heaps

Threads

Images (.dll files)

Other (virtual allocations)

Free

0x00000000 0x7FFFFFFF

17

.NET Memory Management

Memory Management

Native

malloc, free, new, delete

You can choose by yourself when to call free / ~destruktor

No „automatic“ way to free memory

Memory Management

.NET

new, a = null

Leaks are not leaks in the classical sense of meaning

Automatic collection of not referenced objects with the

Garbage Collector

time of free / calling ~Destructor is NOT predictable

MyFontObj.Dispose() - C#: using (MyFontObj)

{ .. }

GC Heap

Segments

- Reserved “chunks” of virtual memory

Small Object Heap (SOH) segment(s)

- Objects categorized in generations

- Gen 0, 1 & 2

Large Object Heap (LOH) segments

- “House” large objects

New object created at increasing addresses of the heap

Allocation is very fast

21

Generations: Structure and Benefits

One heap for generations 0, 1, 2 and large objects

- Large Object is > 85,000 bytes

Efficient memory usage

- Locality of reference

Objects close together on heap

Allocated consecutively and stored contiguously

Created at same time and therefore related

- Speed of allocations

Locality of reference

Allocations located by adding value to a pointer

22

GC Heap

The oldest objects are in the oldest heap segment, new objects are created in the youngest heap segment.

The Heap will expand/contract through adding/removing the youngest heap segments

Ephemeral generations (gen0 and gen1) live in the same heap segment

Gen2 lives in its own segments or share the ephemeral (youngest) segment.

oldest youngest

Gen 2 Gen 1 Gen 0 freeGen 2

What can trigger a garbage collection?

Allocation request that cannot be satisfied

Explicit call to GC.Collect()

Under high system memory pressure

24

Display the heap with sos..

!EEHeap

...

Number of GC Heaps: 1

generation 0 starts at 0x0a788fc0

generation 1 starts at 0x0a251000

generation 2 starts at 0x00ab1000

ephemeral segment allocation context: (0x0a7b0c64, 0x0a7b0fcc)

segment begin allocated size

0x00ab0000 0x00ab1000 0x01a64ca0 0x00fb3ca0(16,465,056)

0x0b250000 0x0b251000 0x0bddd068 0x00b8c068(12,107,880)

0x03b40000 0x03b41000 0x04b3fbf4 0x00ffebf4(16,772,084)

0x04b40000 0x04b41000 0x05b3fbdc 0x00ffebdc(16,772,060)

0x05b40000 0x05b41000 0x06b3fbf4 0x00ffebf4(16,772,084)



0x09250000 0x09251000 0x09d17c28 0x00ac6c28(11,299,880)

0x0a250000 0x0a251000 0x0a7b0fcc 0x0055ffcc(5,636,044)

Large object heap starts at 0x01ab1000

segment begin allocated size

0x01ab0000 0x01ab1000 0x01af3830 0x00042830(272,432)

Total Size 0x7ba2ca8(129,641,640)

------------------------------

GC Heap Size 0x7ba2ca8(129,641,640)

0x0a7b0fcc

-0x0a251000

=

0x0055ffcc

size

Syncblk Index

Ptr to MethodTable

Instance Data

Object A

Objects in Managed Memory SyncBlock Table

Syncblk Index

Ptr to MethodTable

Instance data

Object B

MethodTable for ClassA

MethodTable for ClassB

gcinfo

Ptr to EEClass

Virtual pointers

gcinfo

Ptr to EEClass

Virtual pointers

Opt Padding 64Bit

Opt Padding 64Bit

• Locality of reference

• Allocated consecutively and stored contiguously

• Allocations located by adding value to a pointer

Object Layout in Memory

The Gargabe Collection Process

Gen 0Gen 1

MarkPlanRelocate+Compactt

Gen 2

31 2 5 6 7 98 104 11 12 13 14 15 16

GC Sequence in Motion

28

GC Sequence Non-Concurrent

Markgc_heap::mark_phase

Sweepmake_free_lists

Plangc_heap::plan_phase

Relocategc_heap::relocate_phase

Compactgc_heap::compact_phase

Compact?

SuspendEE

RestartEE

No

Yes

29

Non-Concurrent Workstation GC

30

What’s new in 4.0?

Background GC vs Ephemeral GC

Full GC is running in the Background

If required ephemeral collection is possible at the same time

- Background GC is suspended while the allocating thread is doing the

collection

For Concurrent Workstation mode only < .NET 4.5

31

Concurrent Workstation GC

Background GC in action!

32

Server GC

33

Server GC Notification within .NET 4.0

public static void Main(string[] args)

{

try

{

// Register for a set of notifications.

// Parameters require tuning. First is

// for Gen2, second, Large Object Heap

GC.RegisterForFullGCNotification(10, 10);

// Start a thread using WaitForFullGCProc

Thread thWaitForFullGC = new Thread(new

ThreadStart(WaitForFullGCProc));

thWaitForFullGC.Start();

}

catch (InvalidOperationException invalidOp)

{

Console.WriteLine("GC Notifications are

not supported while concurrent GC is

enabled.\n” + invalidOp.Message);

}

}

public static void WaitForFullGCProc()

{

while (true)

{

// Wait for a notification

GCNotificationStatus s =

GC.WaitForFullGCApproach();

if (s == GCNotificationStatus.Succeeded)

{

// This call will direct new traffic

// away from machine; wait for old

// traffic to finish; then call

// GC.Collect()

OnFullGCApproachNotify();

}

// Wait for a notification of completion

s = GC.WaitForFullGCComplete();

if (s == GCNotificationStatus.Succeeded)

{

OnFullGCCompleteEndNotify();

}

}

}

Concurrent workstation GC –(mscorwks::WKS)

Non-concurrent workstation GC (mscorwks::WKS)

Server GC (mscorwks::SVR)

Design goal Balance throughput and responsiveness for client application with GUI

Maximize throughput on single-processor computer

Maximize throughput on MP machine for server apps which creates multiple threads to handle same type of requests

# of heaps 1 1 1 per core

Threads which performs GC

Dedicated thread withtimeout

The thread that does the allocation

Dedicated thread/corewith no timeout

EE Suspension

EE is suspended much shorter, but several times during GC

EE is suspended during GC

EE is suspended during GC (.NET < 4.5)

Config file setting

<gcConcurrentenabled = true/>

<gcConcurrentenabled = false/>

<gcServer enabled = true/>

API GCLatencyMode.Interactive

GCLatencyMode.Batch

GC Modes .NET 4.x

35

.NET 4.5: Background GC For Server

Mode (On, Off)

<configuration>

<runtime>

//default is client/workstation<gcServer enabled="true"/>

//default is background, this disables it<gcConcurrent enabled="false"/>

</runtime>

</configuration>

.NET 4.5: Background GC For Server

Mode Gen0/Gen1 collections can proceed during Gen2 GC

.NET 4.5.1CompactOnce - next time a full GC occurs the LOH will be compacted.

Once done, it reverts back to the default behavior which is non compacting.

GCSettings.LargeObjectHeapCompactionMode = GCLargeObjectHeapCompactionMode.CompactOnce;

// This will cause the LOH to be compacted (once).

GC.Collect();

Maximizing GC Performance

Consider problems caused by the following :

Allocating too many temporary objects

- E.g. “Chunky” ASP.NET page request

Pre-allocating objects

Too many roots (keeping objects alive)

Pointer/Reference rich objects

- E.g. “Kitchen Sink” static/GCHandle cache such as Hashtable

Too many almost-long-lived objects (generational churn)

Pinning (GCHandleType.Pinned, Interop, fixed location)

Too many Weak references to really small objects

Extensive using of the LOH results in fragmentation

39

Roots

Primary Roots:

- Objects on threads still in use

- Objects allocated on GC Handle

Tree of object references (Object Graph) leading to roots not

collectable

GC Handle

- Purpose: Prevent objects from GC’ed

- Type: Strong, Weak & Pinned

!GCRoot to locate roots in Debugger

40

Finalization: Problems

Releases resources before GC

Problems

- Resource-intensive

- Objects live longer

- At least two GCs required

- Only one Finalizer thread

• May get backlogged or blocked

41

Finalization and Garbage Collection - Start

1. GC is unable to fulfil the memory request.

2. GC marks all objects which are „reachable“

3. All objects which are not reachable and have an entry in theFinalisation list move to the „far reachable“ Freachable list

A B C D E FG H I J

C E F I J

Finalization List

Managed Heap

Freachable List

ROOTS

(strong

references)

Global,

Static, Local

Variables+

CPU Registers

Finalization: Programming

Clean up managed or unmanaged resources before Finalize

method is called

Don’t use Finalize method to clean up managed resources

Use Dispose method also in finalization code

Use the Dispose pattern:

Dispose(true) called from user’s code

Dispose(false) called by CLR

GC.SuppressFinalize()

43

Finalization: Tips

Use finalizers sparingly

Use with objects that have few object pointers

Use short finalization code path

C# using statement is a best practice

Monitor performance

- Perfmon counters for finalization

- Dumps and !Finalizequeue SOS command

44

Demo: Finalizers

The instructor will demonstrate how

application behavior differs when

calling GC.SuppressFinalize.

Memory Leaks

Unmanaged

- Symptom: Private Bytes increase while #Bytes In All Heaps

remains flat

- Troubleshooting path: Various tools (DebugDiag, LeakDiag, etc.)

Managed

- Symptom: Private Bytes and #Bytes In All Heaps both increase

- Troubleshooting path: For most cases, one or more dump files

46

Symptoms in ASP.NET

Out Of Memory Exceptions

Memory Recycling

• memoryLimit

• PeriodicRestartMemory

• PeriodicRestartPrivateMemory

System event log: worker process recycles if memory limit set

Log Name: System

Source: Microsoft-Windows-WAS

Date: 7/16/2010 1:15:27 AM

Event ID: 5117

Task Category: None

Level: Information

Keywords: Classic

User: N/A

Computer: <COMPUTER NAME>

Description:

A worker process with process id of '7420' serving application pool

'DefaultAppPool' has requested a recycle because it reached its private

bytes memory limit.

47

Perfmon Counters

General

.NET CLR Loading

• Current AppDomains

• Current Assemblies

• Bytes In LoaderHeap

.NET CLR Memory

• # Bytes In All Heaps

Process

• Handle Count

• Private Bytes

• Virtual Bytes

ASP.NET-specifc

ASP.NET Application

• Cache Total Entries

• Cache API Entries

• OutputCache Total Entries

48

Memory – Troubleshooting

Performance Counter can be used to indentify the

kind of leak

Native and managed leaks

- set complus_GCBreakOnOOM=1

can used to get DebugBreak() on OOM

- Can be use with the adplus –CTCFG option to get a dump

Memory – Troubleshooting sequence

Yes

.NET CLR Memory: Bytes in all Heaps go upProcess: Private Bytes go up

Obtain Perfmon log (started preferably

during process startup) and 2

process dumps 5-7 minutes apart during problem occurrence

Use DebugDiag or Xperf, LeakDiag

Obtain Perfmon log (started preferably

during process startup) and process

dump during problem occurrence

at least 2 hang dumps during

problem occurrence -> with some time gap between to see the memory growthAnalyse with Windbg

or Perfview

memory leak

net issue?

No

.NET CLR Memory: Bytes in all Heaps stay flatProcess: Private Bytes are going up

Tools for Memory Leakage

Native

Applocation Tracing using DebugDiag, PerfView, Windows Performance Toolkit (Wprui/Xperf)

Managed

Dump Analysis with Windbg or Visual Studio 2013 Ultimate or PerfViewAllocation Tracing with PerfView using the Collect Feature with

• .NET Sample Alloc (fast) • .NET Alloc

Demo: managed Memory with Windbg and PerfView

Troubleshooting Memory ASP.NET Issues

Historically for x86, if we get OOM exceptions and private bytes are

>= 800 MB we are either leaking memory or just have high memory

consumption

- Use VMMAP, Perfmon, or a debugger to narrow down where

memory is used

- Use DebugDiag for native leaks or PerfView or WinDbg for

managed memory leaks

53

Troubleshooting Memory ASP.NET Issues

(Cont.)

If private bytes are < 700 MB and we get OOM exceptions we have

likely fragmented the memory space somehow and need to figure

out why our memory is fragmented

- Use VMMAP in conjunction with WinDbg to determine what is

fragmenting the memory space.

54

useful commands for Windbg

Displays the size and locations of the managed heaps!sos.eeheap [-gc|-loader]

Displays statistics of all managed objects still in memory!sos.dumpheap –stat

Displays statistics of all managed objects of a specific type still in memory!sos.dumpheap –mt <MethodTable> or!sos.dumpheap –type <type>

Displays the size of a specific object and all it’s children!sos.objsize <addr>

Displays who is holding on to a specific object!sos.gcroot <addr>

Displays all appdomains and the assemblies that are loaded in each domain!sos.dumpdomain

Write a log to get a for the CLR Profiler

!TraverseHeap

References

Garbage Collection Performance Hints:

- http://msdn.microsoft.com/en-us/library/ms973837.aspx

Large Object Heap:

- http://msdn.microsoft.com/en-us/magazine/cc534993.aspx

VM Hoarding:

- http://blogs.msdn.com/b/maoni/archive/2005/10/03/476750.aspx

56

http://msdn.microsoft.com/en-us/library/ms973837.aspx

http://msdn.microsoft.com/en-us/magazine/cc534993.aspx

http://blogs.msdn.com/b/maoni/archive/2005/10/03/476750.aspx

Memory Usage - Microsoft Azuredebugdiagtraining.azurewebsites.net/slides... · 2016. 4. 14. · Other HEAPs .NET Heap. Virtual Memory Layout Virtual Memory Process 1 Physical Memory

Documents