Skip to main content

Memory Pool Configuration

Fragmentation

  • A heap will fragment, when buffers with different lifetimes get allocated
  • Fragmentation endangers system stability over run time
drawing-4-1678251879.png

 

Heap Areas

  • Permanent Allocation 
    • will never return the allocated block to the backing heap
    • will be managed by the associated bin (bin free list)
  • Transient Allocation
    • will immediately return the allocated block  to the backing heap
  • Two distinct memory areas
  • The permanent area will never fragment
  • The transient area can fragment
drawing-4-1678251999.png

 

Default Memory Pool Example

Let’s have a look on an example:

static const MemoryPoolConfig config = FEATSTD_MEMORYPOOL_CONFIG_INITIALIZER(
    config,                            				 // name of the variable receiving the configuration
    "FeatStdDefaultMemoryPool",         // the name of the MemoryPool
    MemoryPoolFlags::None,              // Flags that control the memory pool behavior
    128,                                			// all bins up to 128 bytes are small bins
    cPoolBinSpecs,								// the bin configurations
    0,                                  			// preset allocations
    DefaultBackingHeap,                 	// type name of the backing heap
    &OnOutOfMemoryDef,                  // optional the out of memory function
    0                                   			// optional OnDestroy function
);

static const BinSpecification cPoolBinSpecs[] = {
    FEATSTD_MEMORYPOOL_BIN_SPECIFICATION(MemoryPoolBinFlags::None, 8), // 732
    FEATSTD_MEMORYPOOL_BIN_SPECIFICATION(MemoryPoolBinFlags::None, 12), // 594
    FEATSTD_MEMORYPOOL_BIN_SPECIFICATION(MemoryPoolBinFlags::None, 16), // 340
    FEATSTD_MEMORYPOOL_BIN_SPECIFICATION(MemoryPoolBinFlags::None, 20), // 143
    FEATSTD_MEMORYPOOL_BIN_SPECIFICATION(MemoryPoolBinFlags::None, 24), // 270
    FEATSTD_MEMORYPOOL_BIN_SPECIFICATION(MemoryPoolBinFlags::None, 28), // 18
    FEATSTD_MEMORYPOOL_BIN_SPECIFICATION(MemoryPoolBinFlags::None, 32), // 156
    FEATSTD_MEMORYPOOL_BIN_SPECIFICATION(MemoryPoolBinFlags::None, 36), // 93
    FEATSTD_MEMORYPOOL_BIN_SPECIFICATION(MemoryPoolBinFlags::None, 40), // 7
    FEATSTD_MEMORYPOOL_BIN_SPECIFICATION(MemoryPoolBinFlags::None, 44), // 128
    FEATSTD_MEMORYPOOL_BIN_SPECIFICATION(MemoryPoolBinFlags::None, 48), // 107
    FEATSTD_MEMORYPOOL_BIN_SPECIFICATION(MemoryPoolBinFlags::None, 52), // 114
    FEATSTD_MEMORYPOOL_BIN_SPECIFICATION(MemoryPoolBinFlags::None, 64), // 30
    FEATSTD_MEMORYPOOL_BIN_SPECIFICATION(MemoryPoolBinFlags::None, 72), // 114
    FEATSTD_MEMORYPOOL_BIN_SPECIFICATION(MemoryPoolBinFlags::None, 76), // 8
    FEATSTD_MEMORYPOOL_BIN_SPECIFICATION(MemoryPoolBinFlags::None, 80), // 9 
    FEATSTD_MEMORYPOOL_BIN_SPECIFICATION(MemoryPoolBinFlags::None, 88), // 5
    FEATSTD_MEMORYPOOL_BIN_SPECIFICATION(MemoryPoolBinFlags::None, 92), // 6
    FEATSTD_MEMORYPOOL_BIN_SPECIFICATION(MemoryPoolBinFlags::None, 96), // 8
    FEATSTD_MEMORYPOOL_BIN_SPECIFICATION(MemoryPoolBinFlags::None, 100), // 4
    FEATSTD_MEMORYPOOL_BIN_SPECIFICATION(MemoryPoolBinFlags::None, 108), // 14
    FEATSTD_MEMORYPOOL_BIN_SPECIFICATION(MemoryPoolBinFlags::None, 112), // 5
    FEATSTD_MEMORYPOOL_BIN_SPECIFICATION(MemoryPoolBinFlags::None, 116), // 16
    FEATSTD_MEMORYPOOL_BIN_SPECIFICATION(MemoryPoolBinFlags::None, 120), // 34
    FEATSTD_MEMORYPOOL_BIN_SPECIFICATION(MemoryPoolBinFlags::None, 124), // 54
    FEATSTD_MEMORYPOOL_BIN_SPECIFICATION(MemoryPoolBinFlags::None, 128), // 81
    FEATSTD_MEMORYPOOL_BIN_SPECIFICATION(MemoryPoolBinFlags::None, 136), // 153
    FEATSTD_MEMORYPOOL_BIN_SPECIFICATION(MemoryPoolBinFlags::None, 144), // 145
    FEATSTD_MEMORYPOOL_BIN_SPECIFICATION(MemoryPoolBinFlags::None, 168), // 15
    FEATSTD_MEMORYPOOL_BIN_SPECIFICATION(MemoryPoolBinFlags::None, 180), // 4
...
    FEATSTD_MEMORYPOOL_BIN_SPECIFICATION(MemoryPoolBinFlags::None, 16384), // 1
    // mandatory final entry
    FEATSTD_MEMORYPOOL_BIN_SPECIFICATION(MemoryPoolBinFlags::None, 0)
};

Memory Pool Example Outcome

  • This configuration leads to fragmentation
  • All flags (bin & pool) are set to None  Allocations are always transient
    • Transient allocations can fragment
    • When all allocations are transient, fragmentation is certain
  • Small Bin Limit is set to 128
    • All allocations up to 128 take the best fit bin size + 4 Byte overhead
    • All allocations > 128 take exactly the requested size + 8 Byte overhead
    • The configured bins > 128 are not considered at all, they just consume their header memory

Interpret MemoryPoolAnalysis.xlsx

When a permanent block with size 12800 Byte is used for 160 Byte allocation…

12800
　C:\CGI-Studio_3.10\cgi_studio_candera\src\Candera\Engine2D\Core\RenderNode.cpp(66)

  • The default behavior of Memory Pool for allocations is:
      1. Determine the bin according the requested buffer size 
      2. Check the free list of the identified bin
      3. If no block is found in the free list, try to allocate a buffer from the backing heap. 
      4. If the allocation from the backing heap fails, try to allocate from the next larger bins (overflow).
  • In the example above, step 4 occurred -> Allocation of a new bin failed!

Proper Memory Pool Analysis

  • A proper configuration for initial analysis…

 

  • has a “small bin limit” of 4 Byte
    • All allocations shall take the requested size
  • defines PermanentOnly flag on the pool
    • Bins shall be reused (freed blocks are returned to bin free list)
  • defines additionally NoOverflow flag on the pool
    • inhibit overflow allocations
    • If specified, allocation requests from a bin will never overflow to a larger bin
    • Note: a test run with this flag might fail (Out of memory…) due to many unused blocks -> in this case remove the flag

Proper Memory Pool Configuration

  • A proper final Memory Pool configuration…

 

  • has a “small bin limit” in the upper area (where allocations are rarely)
    • Keep number of transient allocations to a minimum
  • uses Flags on bins instead of the pool
    • defines PermanentOnly flag on small bins
    • Defines one bin with None flag, uses transient allocations for large blocks, which are rarely used (no reuse intended)
  • uses few bins
    • balance wasted space vs. unused (rarely used) blocks

 

Wasted Space vs. Unused blocks

  • Many permanent bins leads to many unused blocks
    • free blocks in bin free list
  • Few permanent bins leads to memory “waste” in blocks
    • Allocation request smaller than block size
  • The memory “waste” of few bins is smaller than free blocks from too many bins.
    • Free bin blocks are reused more often
drawing-4-1678252820.png

 

Process of Configuration

  • Take dumps with proper configuration for analysis
    • Play more than one use case to get better data
  • If it is not possible to run with flag NoOverflow …
    • Remove the flag
    • Analysis is still valid, most overflows come from low size bins which are anyway “dense”
  • Based on the results
    • Reduce the number of bins drastically
    • Choose a “small bin limit” where allocations are temporary requests Release of temporary memory vs. fragmentation

Reduce Number of Bins

Reduce the number of bins generously
Too many bins -> many free blocks not used
“Wasted Space” < Not used blocks!

drawing-4-1678253010.png

Choose “Small Bin Limit”

  • Remember: Avoid non used blocks!
  • Summarize bins to small bins as far as possible
  • “Small Bin Limit” shall start where allocations are temporary requests and fragmentation is more unlikely 
drawing-4-1678254380.png

Result

drawing-4-1678254563.png

 

Advanced Topics

  • Pre-Allocation
    • Define how many permanent blocks shall be pre-allocated in each bin
  • Combine with OverflowPreferred flag on bins
    • first try to overflow and then allocate from the backing heap
    • Limit overflows by adding flag OverflowBarrier to bin configuration
    • Such a configuration might improve the memory layout
  • A sophisticated out of memory function could trigger the release of unneeded memory
    • Return value controls if the allocation request will be repeated or abandoned

 

 

 

 

 

 

Back to top