Candera Memory Pools Analysis

• Memory Pools Configuration
• Memory Pool Configuration
• “Unexpected” Allocations

Memory Pools Configuration

Introduction

If free memory in backing heap is exhausted, "Out of Memory" occurs. The possible reasons are as the below.

• Fragmentation of allocated blocks, not possible to fit in a new block.
  
• Initial analysis was wrong, configuration does not fit.
  
• “Unexpected” allocations.
  
• Executed use case cannot fit into supplied memory.

Please see the following pages:

• Memory Pool Configuration
  
• “Unexpected” Allocations
  

 

Memory Pool Configuration

Fragmentation

• A heap will fragment, when buffers with different lifetimes get allocated
  
• Fragmentation endangers system stability over run time
  

Heap Areas

• Permanent Allocation 
  
  • will never return the allocated block to the backing heap
    
  • will be managed by the associated bin (bin free list)
• Transient Allocation
  
  • will immediately return the allocated block  to the backing heap
    
• Two distinct memory areas
  
• The permanent area will never fragment
  
• The transient area can fragment

Default Memory Pool Example

Let’s have a look on an example:

 

Memory Pool Example Outcome

• This configuration leads to fragmentation
  
• All flags (bin & pool) are set to None -> Allocations are always transient
  
  • Transient allocations can fragment
    
  • When all allocations are transient, fragmentation is certain
• Small Bin Limit is set to 128
  
  • All allocations up to 128 take the best fit bin size + 4 Byte overhead
    
  • All allocations > 128 take exactly the requested size + 8 Byte overhead
    
  • The configured bins > 128 are not considered at all, they just consume their header memory

Interpret MemoryPoolAnalysis.xlsx

When a permanent block with size 12800 Byte is used for 160 Byte allocation…

12800
　C:\CGI-Studio_3.10\cgi_studio_candera\src\Candera\Engine2D\Core\RenderNode.cpp(66)

• The default behavior of Memory Pool for allocations is:

1. 1. 1. Determine the bin according the requested buffer size 
         
      2. Check the free list of the identified bin
         
      3. If no block is found in the free list, try to allocate a buffer from the backing heap. 
         
      4. If the allocation from the backing heap fails, try to allocate from the next larger bins (overflow).

• In the example above, step 4 occurred -> Allocation of a new bin failed!

Proper Memory Pool Analysis

• A proper configuration for initial analysis…
• has a “small bin limit” of 4 Byte
  
  • All allocations shall take the requested size
    
• defines PermanentOnly flag on the pool
  
  • Bins shall be reused (freed blocks are returned to bin free list)
    
• defines additionally NoOverflow flag on the pool
  
  • inhibit overflow allocations
    
  • If specified, allocation requests from a bin will never overflow to a larger bin
    
  • Note: a test run with this flag might fail (Out of memory…) due to many unused blocks -> in this case remove the flag

Proper Memory Pool Configuration

• A proper final Memory Pool configuration…
• has a “small bin limit” in the upper area (where allocations are rarely)
  
  • Keep number of transient allocations to a minimum
    
• uses Flags on bins instead of the pool
  
  • defines PermanentOnly flag on small bins
    
  • Defines one bin with None flag, uses transient allocations for large blocks, which are rarely used (no reuse intended)
    
• uses few bins
  
  • balance wasted space vs. unused (rarely used) blocks

Wasted Space vs. Unused blocks

• Many permanent bins leads to many unused blocks
  
  • free blocks in bin free list
    
• Few permanent bins leads to memory “waste” in blocks
  
  • Allocation request smaller than block size
    
• The memory “waste” of few bins is smaller than free blocks from too many bins.
  
  • Free bin blocks are reused more often

Process of Configuration

• Take dumps with proper configuration for analysis
  
  • Play more than one use case to get better data
    
• If it is not possible to run with flag NoOverflow …
  
  • Remove the flag
    
  • Analysis is still valid, most overflows come from low size bins which are anyway “dense”
    
• Based on the results
  
  • Reduce the number of bins drastically
    
  • Choose a “small bin limit” where allocations are temporary requests Release of temporary memory vs. fragmentation

Reduce Number of Bins

Reduce the number of bins generously
Too many bins -> many free blocks not used
“Wasted Space” < Not used blocks!

Choose “Small Bin Limit”

• Remember: Avoid non used blocks!
  
• Summarize bins to small bins as far as possible
  
• “Small Bin Limit” shall start where allocations are temporary requests and fragmentation is more unlikely 

Result

Advanced Topics

• Pre-Allocation
  
  • Define how many permanent blocks shall be pre-allocated in each bin
    
• Combine with OverflowPreferred flag on bins
  
  • first try to overflow and then allocate from the backing heap
    
  • Limit overflows by adding flag OverflowBarrier to bin configuration
    
  • Such a configuration might improve the memory layout
    
• A sophisticated out of memory function could trigger the release of unneeded memory
  
  • Return value controls if the allocation request will be repeated or abandoned

“Unexpected” Allocations

VRAM Memory Pool

• [Applicable for platforms Traveo, Traveo II, RH850]
  
• Why in the VRAM 4 buffers are allocated for render buffers of a single Render Target?
  
• What happens:
  
  • During a transition first scene is unloaded, next scene is loaded
    
  • In case of startup, the first scene is the only one existing
    
  • Default Behavior of Courier: 
    
    • when the last View on a Render Target is unloaded, the Render Target is also unloaded
      
  • The VramAllocator does never immediately free the memory, because the GPU might still write to it (Note: GPU runs
  • asynchronously to the CPU)
    
  • -> New 2 buffers are allocated when the second scene is loaded
• How to fix it?
  
• Send on startup a ViewPlaceholderReqMsg
  
  • increases the reference counter on Render target.
    
  • The message has two parameters
    
  • ViewId: this is a "reference" view which is used to determine the render target object
    
  • Load: true means to increase the reference count, false would decrease it (removes this placeholder from the internal list)
• Alternatively, you can also use a dummy scene with just a camera
  
  • This view would be always active and never unloaded.
    

Asset Cache

• From the Memory Pool Analysis Sheet:

16384
C:\CGI-Studio_3.10\cgi_studio_candera\src\CanderaAssetLoader\AssetLoaderBase\AssetCache.cpp(38)

• -> This is the cache used by the Asset Loader when copying images from flash to RAM
  
• Most or all images are directly rendered from flash
  
• The cache size is best practice for many platforms, but not when flash is directly accessible!
  
• -> Proposal: If flash is directly accessible, reduce the cache
  
  • E.g., to 1 kB (you win 15 kB!)
    
• CMake flag CANDERA_ASSET_CACHE_SIZE