# Candera Memory Pools Analysis

# Memory Pools Configuration

#### Introduction

If free memory in backing heap is exhausted, "Out of Memory" occurs. The possible reasons are as the below.

- Fragmentation of allocated blocks, not possible to fit in a new block.
- Initial analysis was wrong, configuration does not fit.
- “Unexpected” allocations.
- Executed use case cannot fit into supplied memory.

Please see the following pages:

- <span style="color: rgb(230, 126, 35);">[Memory Pool Configuration](https://doc316en.candera.eu/books/appendix-WCP/page/memory-pool-configuration)</span>
- <span style="color: rgb(230, 126, 35);">[“Unexpected” Allocations](https://doc316en.candera.eu/books/appendix-WCP/page/unexpected-allocations)</span>

# Memory Pool Configuration

#### Fragmentation

- A heap will fragment, when buffers with <span style="text-decoration: underline;">different lifetimes</span> get allocated
- Fragmentation endangers system stability over run time

<div drawio-diagram="3131"><img src="https://doc316en.candera.eu/uploads/images/drawio/2023-03/drawing-4-1678251879.png" alt=""/></div>

#### Heap Areas

- Permanent Allocation   
    
    - will never return the allocated block to the backing heap
    - will be managed by the associated bin (bin free list)
- Transient Allocation  
    
    - will immediately return the allocated block to the backing heap
- Two distinct memory areas
- The permanent area will <span style="text-decoration: underline;">never fragment</span>
- The transient area <span style="text-decoration: underline;">can fragment</span>

<div drawio-diagram="3132"><img src="https://doc316en.candera.eu/uploads/images/drawio/2023-03/drawing-4-1678251999.png" alt=""/></div>

#### Default Memory Pool Example

Let’s have a look on an example:

<div drawio-diagram="3139"><img src="https://doc316en.candera.eu/uploads/images/drawio/2023-03/drawing-4-1678255213.png" alt=""/></div>

<div drawio-diagram="3140"><img src="https://doc316en.candera.eu/uploads/images/drawio/2023-03/drawing-4-1678255251.png" alt=""/></div>

#### Memory Pool Example Outcome

- This configuration leads to fragmentation
- All flags (bin &amp; pool) are set to *None -&gt;* Allocations are always transient  
    
    - Transient allocations can fragment
    - <span style="text-decoration: underline;">When all allocations are transient, fragmentation is certain</span>
- Small Bin Limit is set to 128  
    
    - All allocations up to 128 take the best fit bin size + 4 Byte overhead
    - All allocations &gt; 128 take exactly the requested size + 8 Byte overhead
    - The configured bins &gt; 128 are not considered at all, they just consume their header memory

#### Interpret MemoryPoolAnalysis.xlsx

When a permanent block with size 12800 Byte is used for 160 Byte allocation…

**12800**  
 C:\\CGI-Studio\_3.10\\cgi\_studio\_candera\\src\\Candera\\Engine2D\\Core\\RenderNode.cpp(66)

- The default behavior of Memory Pool for allocations is:

1. 1. 1. Determine the bin according the requested buffer size
        2. Check the free list of the identified bin
        3. If no block is found in the free list, try to allocate a buffer from the backing heap.
        4. If the allocation from the backing heap fails, try to allocate from the next larger bins (overflow).

- In the example above, step 4 occurred -&gt; Allocation of a new bin failed!

#### Proper Memory Pool Analysis

- A proper configuration for initial analysis…
- has a “small bin limit” of 4 Byte  
    
    - All allocations shall take the requested size
- defines *PermanentOnly* flag on the pool  
    
    - Bins shall be reused (freed blocks are returned to bin free list)
- defines additionally *NoOverflow* flag on the pool  
    
    - inhibit overflow allocations
    - If specified, allocation requests from a bin will never overflow to a larger bin
    - Note: a test run with this flag might fail (Out of memory…) due to many unused blocks -&gt; in this case remove the flag

#### Proper Memory Pool Configuration

- A proper final Memory Pool configuration…
- has a “small bin limit” in the upper area (where allocations are rarely)  
    
    - Keep number of transient allocations to a minimum
- uses Flags on bins instead of the pool  
    
    - defines *PermanentOnly* flag on small bins
    - Defines one bin with None flag, uses transient allocations for large blocks, which are rarely used (no reuse intended)
- uses few bins  
    
    - balance wasted space vs. unused (rarely used) blocks

#### Wasted Space vs. Unused blocks

- Many permanent bins leads to many unused blocks  
    
    - free blocks in bin free list
- Few permanent bins leads to memory “waste” in blocks  
    
    - Allocation request smaller than block size
- The memory “waste” of few bins is smaller than free blocks from too many bins.  
    
    - Free bin blocks are reused more often

<div drawio-diagram="3133"><img src="https://doc316en.candera.eu/uploads/images/drawio/2023-03/drawing-4-1678252820.png" alt=""/></div>

#### Process of Configuration

- Take dumps with proper configuration for analysis  
    
    - Play more than one use case to get better data
- If it is not possible to run with flag *NoOverflow* …  
    
    - Remove the flag
    - Analysis is still valid, most overflows come from low size bins which are anyway “dense”
- Based on the results  
    
    - Reduce the number of bins drastically
    - Choose a “small bin limit” where allocations are temporary requests Release of temporary memory vs. fragmentation

#### Reduce Number of Bins

Reduce the number of bins generously  
Too many bins -&gt; many free blocks not used  
“Wasted Space” &lt; Not used blocks!

<div drawio-diagram="3134"><img src="https://doc316en.candera.eu/uploads/images/drawio/2023-03/drawing-4-1678253010.png" alt=""/></div>

#### Choose “Small Bin Limit”

- Remember: Avoid non used blocks!
- Summarize bins to small bins as far as possible
- “Small Bin Limit” shall start where allocations are temporary requests and fragmentation is more unlikely

<div drawio-diagram="3137"><img src="https://doc316en.candera.eu/uploads/images/drawio/2023-03/drawing-4-1678254380.png" alt=""/></div>

#### Result

<div drawio-diagram="3138"><img src="https://doc316en.candera.eu/uploads/images/drawio/2023-03/drawing-4-1678254563.png" alt=""/></div>

#### Advanced Topics

- Pre-Allocation  
    
    - Define how many permanent blocks shall be pre-allocated in each bin
- Combine with OverflowPreferred flag on bins  
    
    - first try to overflow and then allocate from the backing heap
    - Limit overflows by adding flag OverflowBarrier to bin configuration
    - Such a configuration might improve the memory layout
- A sophisticated out of memory function could trigger the release of unneeded memory  
    
    - Return value controls if the allocation request will be repeated or abandoned

# “Unexpected” Allocations

#### VRAM Memory Pool

- \[Applicable for platforms Traveo, Traveo II, RH850\]
- Why in the VRAM 4 buffers are allocated for render buffers of a single Render Target?
- What happens:  
    
    - During a transition first scene is unloaded, next scene is loaded
    - In case of startup, the first scene is the only one existing
    - Default Behavior of Courier:   
        
        - when the last View on a Render Target is unloaded, the Render Target is also unloaded
    - The VramAllocator does never immediately free the memory, because the GPU might still write to it (Note: GPU runs
    - asynchronously to the CPU)
    - -&gt; New 2 buffers are allocated when the second scene is loaded
- How to fix it?
- Send on startup a ViewPlaceholderReqMsg  
    
    - increases the reference counter on Render target.
    - The message has two parameters
    - ViewId: this is a "reference" view which is used to determine the render target object
    - Load: true means to increase the reference count, false would decrease it (removes this placeholder from the internal list)
- Alternatively, you can also use a dummy scene with just a camera  
    
    - This view would be always active and never unloaded.

#### Asset Cache

- From the Memory Pool Analysis Sheet:

**16384**  
C:\\CGI-Studio\_3.10\\cgi\_studio\_candera\\src\\CanderaAssetLoader\\AssetLoaderBase\\AssetCache.cpp(38)

- -&gt; This is the cache used by the Asset Loader when copying images from flash to RAM
- Most or all images are directly rendered from flash
- The cache size is best practice for many platforms, but not when flash is directly accessible!
- -&gt; Proposal: If flash is directly accessible, reduce the cache  
    
    - E.g., to 1 kB (you win 15 kB!)
- CMake flag CANDERA\_ASSET\_CACHE\_SIZE