Memory management

Memory management

Linked Lists

structs and memory layout

list.nextlist.prev

list.nextlist.prev

list.nextlist.prev

fox foxfox

Linked lists in Linux

fox foxfox

list {.next.prev

}

node

list {.next.prev

}

list {.next.prev

}

What about types?

• Calculates a pointer to the containing struct

struct list_head fox_list;struct fox * fox_ptr = list_entry(fox_list->next, struct fox, node);

List access methodsstruct list_head some_list;

list_add(struct list_head * new_entry, struct list_head * list);

list_del(struct list_head * entry_to_remove);

struct type * ptr;

list_for_each_entry(ptr, &some_list, node){…

}

struct type * ptr, * tmp_ptr;

list_for_each_entry_safe(ptr, tmp_ptr, &some_list, node) {list_del(ptr);kfree(ptr);

}

Page Frame Database/* Each physical page in the system has a struct page associated with * it to keep track of whatever it is we are using the page for at the * moment. Note that we have no way to track which tasks are using * a page */

struct page { unsigned long flags; // Atomic flags: locked, referenced, dirty, slab, disk atomic_t _count; // Usage count, atomic_t _mapcount; // Count of ptes mapping in this page

struct { unsigned long private; // Used for managing page used in file I/Ostruct address_space * mapping; // Used to define the data this page is

holding };

pgoff_t index; // Our offset within mapping struct list_head lru; // Linked list node containing LRU ordering of pages void * virtual; // Kernel virtual address};

Memory Zones• Not all memory addresses are the same

– ZONE_DMA: DMA memory (< 16MB) • Really old I/O devices that have constrained addresses

– ZONE_DMA32: 32 bit DMA memory ( < 4GB)• Older I/O devices that only support 32 bit DMA

– ZONE_NORMAL: Generic Kernel memory• Always directly addressable by the kernel

• Linux groups memory into zones– Based on the use cases for memory– Allow allocations to occur in a given zone– How?

Buddy Allocator

• Memory allocations are all backed by physical pages– Kernel allocations are persistent• Cannot be moved or swapped

– Must find contiguous sets of pages• Allocations all come from free lists– Linked list of unallocated resources

• Code example

Allocating pages• Return entry/entries from page list

– Scans various lists for page(s) to allocate

• struct page * alloc_pages(gfp_t flags, int order);• void * page_address(struct page * page)• unsigned long page_to_pfn(struct page * pg);

kmalloc

• kernel version of malloc– manages global heap, accessible by all kernel

threads– Returns kernel virtual addresses

• void * kmalloc(size_t size, gfp_t flags);

gfp_t

• What are these gfp_t flags?– Directions to allocator– Where to get the memory from– What steps allocator can take to find memory

• Some Examples:– Zone

• GFP_DMA, GFP_DMA32, GFP_NORMAL– Behavior

• GFP_ATOMIC, GFP_KERNEL

vmalloc

• Linux limits the number of contiguous pages you can allocate– MAX_ORDER typically is 11 (32MB)

• 2^11 pages

• What if you need to allocate more?– Must do the allocation in virtual memory

• void * vmalloc(unsigned long size);– Allocates a virtually contiguous address region– Backed by physically discontinuous pages

Slab allocator• Optimization for kernel allocations

– Provides a free list (or cache) of unused allocations of a certain type– Don’t have to search for a free region– Allocations become (almost) constant time

• Create special caches for certain types of common allocations– i.e. network packets, inodes, process descriptors– Allocate those types using a special allocator

• Slab subsystem dynamically ensures that enough memory is available– Allocates and frees pages behind the scenes