Target setup for Traveo2

• Traveo2 BSP
• Traveo2 NoOS BSP: Prerequisites
• Traveo2 AUTOSAR BSP: Prerequisites
• Running the CGI Player Demo Application on Traveo2
• Running the CGI CourierSampleAppLight Demo Application on Traveo2
• Building Own Application for Traveo2
• Useful Hints and Recommendations
• Traveo 2 Sample
• Candera Graphics Programming - Best Practices for Traveo II
• Graphics driver and Hardware related Best-Practices
• CGI Studio related Best-Practices
• JPEG Decoder
• JPEG Bitmap Converter
• JPEG Decoder Behavior
• MJPEG Stream Behavior
• Video Format Conversion
• Various Device Specific Content
• Traveo2BitmapConverter

Traveo2 BSP

Traveo2 NoOS BSP: Prerequisites

Hardware: Infineon Traveo II

CGI Studio has been tested on Infineon Traveo II (CYTVII-C-2D-6M-500-BGA CPU REV-B) board and Infineon Traveo II (CYTVII-C-2D-4M-216 CPU REV-A) board .

CMake

In order to build Candera, the Meta build system CMake needs to be installed with at least version mentioned here.
CMake makes it possible to create the cross compilation makefiles that can be used with Green Hills MULTI compiler (or IAR Embedded Workbench for ARM). 

Generator

To allow cross compilation with makefiles, MinGW 4.2.1 was used for this release.

Toolchain

The build tool chain is not part of the delivery.

Green Hills

Green Hills MULTI IDE, Green Hills compiler and the corresponding Traveo II graphics driver have to be obtained directly from Green Hills and/or Infineon and should be installed according to their installation guide.
CGI Studio was tested with MULTI IDE 7.1.4 (configured as suggested by Infineon - GHS Traveo II Debug Script Environment), Compiler 2017.1.4

The Green Hills probe 5.6.4 for ARM was used for connection/debug on the target.

Embedded Workbench

IAR EW IDE for ARM, compiler and the corresponding Traveo II graphics driver have to be obtained directly from IAR and/or Infineon and should be installed according to their installation guide.

CGI Studio was tested with IAR EW IDE 8.22.2 (configured as suggested by Infineon), Compiler IAR ANSI C/C++ Compiler 8.22.2.15995/W32 for ARM

The IAR Systems I-jet probe 2.5.0.0 was used for connection/debug on the target.

Infineon Graphics Library

The graphics driver is not part of this release. CGI Studio was integrated and tested with the following versions:

• V1.2.0 for IAR EW
• V2.3.0 for GreenHills

Flashing tool

Infineon Auto Flash Utility 1.2 via MiniProg4 was used in order to flash asset file to the external flash (Hyperflash).

Traveo2 AUTOSAR BSP: Prerequisites

Hardware: Infineon Traveo II

CGI Studio has been tested on Infineon Traveo II (custom 4M) board .

CMake

In order to build Candera, the Meta build system CMake needs to be installed with at least version mentioned here.
CMake makes it possible to create the cross compilation makefiles that can be used with Green Hills MULTI compiler.

Generator

To allow cross compilation with makefiles, MinGW 4.2.1 was used for this release.

Toolchain

The build tool chain is not part of the delivery.

Green Hills

Green Hills MULTI IDE, Green Hills compiler and the corresponding Traveo II graphics driver have to be obtained directly from Green Hills and/or Infineon and should be installed according to their installation guide.
CGI Studio was tested with MULTI IDE 7.1.4 (configured as suggested by Infineon - GHS Traveo II Debug Script Environment), Compiler 2017.1.4

The Green Hills probe 5.6.4 for ARM was used for connection/debug on the target.

AUTOSAR

Autosar environment is not part of this release. The Autosar environment is provided by customer.

Infineon Graphics Library

The graphics driver is not part of this release. CGI Studio was integrated and tested with the libraries received from customer (V1e0.0)

BSP External memory initialization limitation

Please ensure that the external peripheral on the target device is supported by the BSP initialization routine for the external memory:  Cy_GfxEnv_FastInit(bUseSmif0, bUseSmif1, xxx)
The support of external memory components can differ between BSP/Graphic driver versions!

In case of unsupported external memory initialization in the specific driver/BSP version compared to the Traveo II evaluation board the value will be set to 0 (disabled) :Cy_GfxEnv_FastInit(0, 0, xxx) (see corresponding initialization in  \cgi_studio_devices\src\Traveo2\CourierPlatformDefs\Target_NoOs_ARM\KernelTargets\main.c) .
In this case, only the internal RAM and FLASH can be used on the target board.

Running the CGI Player Demo Application on Traveo2

Download the Application

To download and debug the application you need a Green Hills probe (or IAR Systems I-jet probe) that is configured accordingly for your Infineon Traveo II board.

Executable Application

Green Hills

IAR Embedded Workbench

Binary directory contains two IAR EW projects under "iar_projects" (for flash only and/or flash and debug).

Loading the asset

The CGI Player demo application uses an asset file which could be added by the linker or loaded from external flash. CGIDEVICE_ASSET_LOCATION holds that information (options: Monolith or Flash).

In case of Monolith option for CGIDEVICE_ASSET_LOCATION, the asset file will be loaded to internal flash together with the application. You have to select the asset file in CMake via the variable ASSET_TARGET_LOCATION before you build the application.

In case of Flash option for CGIDEVICE_ASSET_LOCATION, the asset file will be loaded during startup of the application. A prerequisite is that it must be flashed to the external flash (Hyperflash) on the board before executing the app.

Running the CGI CourierSampleAppLight Demo Application on Traveo2

Download the Application

To download and debug the application you need a Green Hills probe (or IAR Systems I-jet probe) that is configured accordingly for your Infineon Traveo II board.

Executable Application

Green Hills

IAR Embedded Workbench

Binary directory contains two IAR EW projects under "iar_projects" (for flash only and/or flash and debug).

Loading the asset

The CGI CourierSampleAppLight demo application uses an asset file which could be added by the linker or loaded from external flash. CGIDEVICE_ASSET_LOCATION holds that information (options: Monolith or Flash).

In case of Monolith option for CGIDEVICE_ASSET_LOCATION, the asset file will be loaded to internal flash together with the application. You have to copy the asset file in the directory and name it as “AssetLibCff_Traveo2_Target.bin” before you build the application.

In case of Flash option for CGIDEVICE_ASSET_LOCATION, the asset file will be loaded during startup of the application. A prerequisite is that it must be flashed to the external flash (Hyperflash) on the board before executing the app.

Building Own Application for Traveo2

Generate Makefiles with CMake

For general information about CGI Studio Build System and CMake properties available to configure Candera applications, refer to CGI Studio Build System. To configure and generate makefiles for compiling the CGI CourierSampleAppLight, follow the steps below:

Building Application

After generating your cmake files, in Command Prompt simply navigate to the specified build directory and type "mingw32-make" (or "ninja") to start the build.

The output executable file can be found in your build directory.

Load and debug the application as described in the previous chapter.

Useful Hints and Recommendations

Setting for tasks

In CMake you have the possibility to set the size and the location of the memory for the command queues and also the COPSes reserved for that task.
Additionally, you can set the memory location for the frame buffers (in case they exist in the solution) for the layers.
Disable unused tasks by setting the size to 0. (This has to be in correlation with the solution which you want to execute on the target)

EXTRAM support

By default, all the frame buffers for the layers, the instruction buffers, and the bitmaps (if they are not stored in Client memory) are stored in the VRAM.
However, it is possible to configure the memory location to EXTRAM (HyperRAM) instead. The memory location of the frame buffers and instruction buffers is configured via CMake (see Figure 2).
The memory location of the bitmaps is selected via Scene Composer (refer to Figure 3).

For target build, please keep in mind to check (enable) CGIDEVICE_EXTRAM_MEMORYPOOL_ENABLED in CMake in case you want to store images in external RAM.
Without that check it is not possible to allocate memory in the HyperRAM and the VRAM will be used instead.

Limitations

Each OTF layer (window) must use different TaskPrio ID.

OTF window surface height is minimum two slices (16 lines) and maximum (2 * window height - 1).

Maximum width for LBO blit area: 1600 (Maximum width for OTF layer (window) is 1600)

Supported values for OTF lines: 16, 32, 64, 128, 256

Supported CGI Studio samples for this target are:
LightPlayer　　
   Empty
   2DBasic
   2DEffects
   AdvancedCluster
   Layouter
   Transition
CourierSampleAppLight
   Default CSALight’s solution
   New Traveo2Sample (covers few Traveo2 specific features)

 

Traveo 2 Sample

Summary

This solution is a sample of a cluster that can be executed on Traveo II and is meant to showcase some of Traveo II's new functionality.

Purpose

This solution allows you to see how various Traveo II features work in conjunction to form a cluster sample.

Description

The sample uses following features:

Candera Graphics Programming - Best Practices for Traveo II

Graphics driver and Hardware related Best-Practices

Images and Color formats

Best practice for image drawing:

• Always use as small images as possible to save memory consumption and memory bandwidth.
• Cut off transparent area in the bitmap before usage.

Bitmaps that contain transparent areas

<Origin>

<Improved version>

Bitmap size difference: 102312 ⇔ 49728!

Modify bitmaps with large transparent areas ("U" shape)

<Origin>

<Improved version>

Bitmap size difference: 701568 ⇔ 188904 (55296 + 87528 + 46080)!

Modify bitmaps with large transparent areas (Gauge)

<Origin>

<Improved version>

Size difference: 925848 ⇔ 438588 (180940 + 61376 + 196272)!

Single-colored bitmaps

For single-colored bitmaps use BitmapBrushColorBlend and use an alpha only color format e.g. A8 or A4. If further size reduction is required there are also compressed formats but those prohibit scaling and rotation on all types of LBO render targets.

<Origin>

<Improved version>

Visual result:

size reduction: RGBA: 222028 ⇔ 74092!

Combine Bitmaps

Bitmaps can be combined to one bitmap at design time. E.g., tube, scale, and numbers should already be combined if possible.

Reduce number of objects from 5 to 2. Avoid rendering of non-translatable static text. Only one moving object left and maybe the gauge itself can be put into a background image.

<Origin>

<Improved version>

Bitmap size reduction: 1383508 (1000000 + 347508 + 36000) ⇔ 1154000 (1118000 + 36000)!

---

Compressed image formats

Best practice:

• Use RLE/RLA and CLUT compressions as much as possible wherever possible.
• Prepare the images that they will fit to an image-with alignment multiple of 8.
  If you have a bitmap which has transparencies at the left or right side, you can remove some pixels in x direction to get a multiple of 8. If there are no transparencies, you must add some transparent pixels at the right side of the bitmap to have a multiple of 8.

---

Usage of Bitmap profiles

There are many different bitmap formats available for Traveo2. The goal is always to use a format which uses least memory as possible. The drawback is that some formats have restrictions regarding transformation (rotation, scaling, …).
Best practice is to use RLE formats to reduce memory size. AX (A1, A2, …) formats are used to reduce memory further if just recoloring is needed (white or grayish bitmaps can be recolored in CGI Studio).
If a channel (RGBA) has less than 8 bits you will have a lossy compression.
For special use cases there are formats which can dither bitmaps (Nq, Mc, Nq_FsDithered, Mc_FsDithered. Based on the look and feel on the target, usually different formats of such type must be tested to decide what to use.

---

Transformations

Best practice Scaling and Rotation:

• These operations require reverse lookup and therefore need more bandwidth and compute power than a simple BLIT. If not necessary, avoid these.
• It is better to use images with already corrected dimensions instead of scaling down.

The red gauge should be the required size. The green scale is too big and must be scaled down by to 80%.

Needed size of gauge will be 925848

<Origin>

<Improved version>

Bitmap size comparison: 1449642 ⇔ 925848!

Using Layers

In general, there is always a tradeoff between performance (incl. memory band-with) and memory consumption (e.g., VRAM). Traveo II allows, with the new introduced “Layer-modes” IBO, LBO and OTF, to steer in one of these directions.
Traveo II supports (like Cypress Amber) a huge amount of “separated” Layers, up to 5 “Main-Windows”, 3 of the windows can be split up to 8 sub-windows.
Large area screens shall always be split up to several sub screens to save VRAM consumption.

Layer types

• IBO Mode: regular “dual render target / buffer” approach.
  o Best practices:
  
  • • Use IBO layers for dynamic text drawing with Text engine.
    • Use for content which will not change every frame.
    • Use only small areas/content with IBO (For more details, please refer to "Notes for the different modes (IBO/LBO/OTF)" on this page)
• LBO Mode: Line based rendering operation
  • Increased BLIT Engine performance
  • Note the restrictions, following features are NOT supported in LBO Mode:
    • ROP2 or ROP3 ➔ Mask operations
    • Scaling of compressed images
    • Color Lookup Table (CLUT)
  • Note the limited functionality:
    • Rotation: only in steps of 90°
    • Scaling: limited quality for scale factors <0.5
    • Rendering images with compression in LBO mode requires "COPSes" (For more details, please refer to  Layer usage)
• OTF Mode: On-The-Fly mode
  • Display to a window with a small buffer (a few lines only) in memory
  • All BLIT operations use LBO mode
    
    
  • Note the restrictions/limitation:
    • All source images shall be copied to VRAM!
    • Maximum OTF width is limited to 1600!
      
      
  • Best practice:
    • Use as less as possible OTF layers to ensure the targeted performance.
    • Attention to the number of line buffers: There is only a setting of a power of 2 available for the number of line buffers. 16, 32, 64, 128, ... because this will increase the VRAM consumption significantly!
    • Use OTF layers whenever you must draw high frequently changed content (e.g., gauges, pointer, …)
      
      
• Notes for the different modes (IBO/LBO/OTF)
  • If necessary, it is possible to mix the usage of different modes (IBO and OTF) by using several tasks in the command sequencer.
  • Keep in mind that an IBO mode task will not yield the command sequencer until it has completed.
  • A long running IBO task can therefore prevent an LBO or OTF task from completing in time which can lead to visible artifacts.
  • The same can happen if the complexity in one slice is too high to be completed in time.

---

SurfaceFormat optimization

Before starting the project, it must be decided which format could match the expectations of a customer. If less bits per pixel are sufficient for the look and feel it should be chosen to save VRAM. Keep in mind that for overlapping Layers/RenderTargets an Alpha-value is necessary, otherwise no blending of layers is possible.

---

Text

Best practice for drawing text related content:

• Always pre-render / rasterize non-translatable text.
• Use static pre-rendered numbers instead of rendering numbers as text (e.g., for Speedometer, RPM, Odometer, ...) Especially for speed use a behavior to switch prerendered digits instead of rendering a text.
  
  

  Switch prerendered images instead of text

  
  
  

  Numbers are combined in one image

  
  
  

  Each number as TextNode

  
  
• Text cache type can balance RAM and CPU use. Use no cache if less RAM is available or use bitmap cache if you have enough RAM. The default cache type is set to Bitmap:
  
  

  ---

CGI Studio related Best-Practices

General recommendation

Enable snap to device pixel. This makes sure no filters must be applied that slow down the BLIT operation.

Only use bilinear filter if an image is rotated, scaled, or positioned between two pixels.
If none of the above happens then stick with the “nearest” filter. Be aware that bilinear is the default filter.

Memory pools

Always use CGI Studio MemoryPool feature to be able to get some freedom to shift the Text-memory area (used for Text rendering with e.g., Freetype) and default memory area (for CGI Studio) to the right memory location (internal RAM, VRAM, HyperRAM)

Memory pool mechanism:

Memory Pools, Bins, and Backing Heap:
A memory pool represented by class MemoryPool consists of a configurable number of Bins (MemoryPoolBin). Each Bin has a defined buffer size, maintains its own free list (in debug mode also a list of allocated blocks), and thread synchronization. A Bin manages so called blocks. Each block consists of a header (BlockHeader) and the user buffer (the actual buffer returned to the application).
There are two different types of Bins – small and large block Bins. A small block Bin maintains blocks which user buffer matches exactly the size of the defined Bin buffer size. A large buffer Bin maintains blocks that have at least a user buffer size of the Bin buffer size. Small block Bins are used – as the name already indicates – for small memory allocations. A small buffer Bin has less runtime and memory overhead per allocation than a large block Bin.
Whenever the application requests an allocation of a certain number of bytes, MemoryPool will look up the Bin which buffer size matches best the requested number of bytes. If the selected Bin has entries in its free list, the Bin will best fit select one of the free list nodes.

If a Bin free list is empty, the Bin will try to allocate a new block from the backing heap.
Thus, Bins maintain lists of blocks allocated from the backing heap. If an allocation request cannot be fulfilled by the already allocated blocks, the Bin tries to allocate the block in the backing heap.

The above figure shows the layered architecture of FeatStd MemoryPool.

• One or more applications may access in parallel a memory pool.
• Multiple memory pools may exist in the system, each configured with its own MemoryPoolConfiguration object.
• Each MemoryPool object is assigned to a backing heap it will allocate memory from.
• Multiple MemoryPool objects may be assigned to a single backing heap.
  

The whole memory pool configuration is done at compile time. Thus, no runtime overhead is generated.

Sticky vs Dynamic allocation:
A sticky allocation is an allocation that will never return the allocated block to the backing heap. The block will be managed by the associated Bin (Bin free list).
A block allocated dynamically (not sticky) will be immediately returned to the backing heap when the application frees the block.
With the knowledge whether an allocation is sticky or not, the backing heap can execute the allocation from two distinct memory areas – the sticky area which will never fragment and the dynamic area which can fragment.

Backing Heap:
FeatStd MemoryPool implementation does not depend on a specific backing heap. E.g., standard library malloc and free can be used to implement the backing heap. Such a backing heap is not intended for production environment but might be more convenient than a fixed size heap for development.
FeatStd MemoryPool provides an implementation of a backing heap that can be used out of the box.

The above figure shows the memory layout of the backing heap. The sticky heap area grows top down, while the dynamic area grows bottom up.
A memory allocation in the sticky area bears no additional overhead. An allocation in the dynamic area has an overhead of 4 bytes required to coalesce consecutive free areas efficiently. Free blocks in the dynamic area are maintained in a free list. On allocation a free list entry will be selected with best fit strategy.

Memory pool Bin-size optimization

Best practice: Do the MemoryPoolBin-optimization to optimize the consumed memory and to achieve a better memory fragmentation while runtime.
To defragment the memory as much as possible and to avoid upcoming “out-of-memory” scenarios you should always do the memory pool optimization on the final application/solution.
Therefor you should perform following steps:

1. Create your default memory pool configuration
2. Let the application run for a while with the config, preferably run through all use cases.
3. Shut down the application and display the statistics on the memory blocks used (e.g., display to console or create a specific text file).
4. Create an optimized memory config based on the output statistics and start from the beginning (-> 2).

Layer usage

• With CGI Studio the Screen can be partitioned according to the TV-II window/sub-window structure
• Several Render Target Types
  • (Sub-) Window Surface → IBO mode
    • Double Buffered (Surface Size * Byte/px * 2 buffer count)
    • Use images from Hyperflash (Client memory) to save VRAM
  • Image (Sub-) Window Surface
    • Surface from (compressed) Image in Hyperflash (Client memory) to save VRAM
  • LBO (Sub-) Window Surface
    • Double Buffered (Surface Size * Byte/px * 2 + 16 lines * Byte/px)
    • Bandwidth usage improvement versus higher Memory usage
    • Images shall be in VRAM.
  • OTF Window
    • No Frame Buffer (at least 16 lines (or the next power of 2!) * Byte/px for Scanline)
    • Images shall be located VRAM
• Best practice:
  • Set the task priority and task "COPSes" accordingly in the related CMake-settings for the CGI studio build:
    
    
    
    
  • Set the Line-buffers for OTF layers accordingly to the required solution setup. In general: use as less as number of line buffers (line buffer number needs to be a power of 2) because this will increase the VRAM consumption accordingly. Increase only if you face performance issues (e.g., red flicker on screen)
  • Use as less as possible OTF layers. If possible, just use 1 (but keep the limitation with maximum width of 1600 in mind)
  • Try to use an image Layer whenever possible for static background (see following examples)

Effects

Best practice:

• Use Blur-effect only if not avoidable, Multi pass BLITS like the Blur effect have a huge performance impact because the image must be blitted multiple times for a 9x9 filter!
  
  
  
  
• All images for bitmap brush perspective warp blend must be in VRAM.

---

JPEG Decoder

JPEG Bitmap Converter

Feature Description

The Traveo 2 JPEG Bitmap Converter allows the user to import compressed JPEG images directly into the solution using the JPEG bitmap profile. These JPEG images are then decoded by the onboard JPEG decoder at runtime. This means that the decompressed bitmaps need not be included in the assets, reducing the overall asset size.

Supported Platforms

At present, the JPEG Bitmap Converter is exclusively supported on the Infineon Traveo 2 platform.

Limitations

The JPEG Bitmap Converter currently has the following limitations.

• Preloaded Images in the asset only: The image must be part of the asset and cannot be dynamically supplied at runtime.
• Supported pixel formats: The following table shows the supported pixel formats for JPEG images.

  Pixel Format	Sampling Configuration	Description
  YUV444	Y (Luma) Subsampling: H/V = 1/1 Cb (Chroma) Subsampling: H/V = 1/1 Cr (Chroma) Subsampling: H/V = 1/1	No chroma subsampling. Each Y, Cb, and Cr component has the same resolution.
  YUV420	Y (Luma) Subsampling: H/V = 2/2 Cb (Chroma) Subsampling: H/V = 1/1 Cr (Chroma) Subsampling: H/V = 1/1	Cb and Cr components are subsampled by 2 horizontally and vertically. Commonly used in video compression.
  RGB	N/A	Standard RGB format without chroma subsampling.

• JPEG images with YUV420 pixel format must have an even width.
  
• JPEG images with YUV pixel formats (YUV420/YUV444) do not support changing of color using the color property of the BitmapBrushColorBlend (or any similar effects which allow changing of color using this property) on target.
  
  
• Surfaces with compressed formats such as YUV422 and YUV420 only support simple rotation (ie. multiples of 90)

Enabling the JPEG Decoder feature

If the JPEG Bitmap Converter feature is being used, ensure that the JPEG Decoder feature has been enabled in the CMAKE options. While generating the project for target, turn on the CMAKE flag CGIDEVICE_TRAVEO2_JPEG_DECODER_ENABLED or add this flag to the toolchain file.

How to use the JPEG Bitmap Converter

1. Import the JPEG image: Using the import menu, select "Import Resources"
   
   
   
   
2. Choose a JPEG image and click "Open".
   
   

   
   

    

3. In the import dialog box, create a new JPEG bitmap profile by checking the “Set bitmap profile” checkbox and then click the “Browse” button as shown in the image below. This only needs to be done once for a new solution, if a JPEG bitmap profile already exists, skip to Step 7.
   
   
   
   
4. In the Bitmap Profile Palette window that opens, click on the add button as shown in the image below. If the Default Bitmap Profile is not populated on the right-hand side, populate it as shown below.
   
   
   
   
5. In the Add new Bitmap Profile dialog that opens, populate the fields as shown in the image below. Click “Ok” once done.
   
   
   
   
6. On the Bitmap Profile Palette, confirm that the JPEG Bitmap Converter has been added as shown in the image below and press Ok.
   
   
   
   
7. Select the JPEG Bitmap Profile from the dropdown menu if it is not already selected as shown in the image below. Click Ok.
   
   
   
   
8. Add the image to the scene by dragging and dropping it into the scene editor.
9. Select “Generate Asset Library” option in File menu to generate the asset and check if the compressed JPEG images have been put into the asset.
10. The asset generation export report should show the image of the encoded JPEG image. In the image below it is visible that the image is only 33kb which is much lesser than the decompressed bitmap size for the same image.
    
    
    
    

JPEG Decoder Behavior

Feature Description

The JPEG Decoder behavior allows the user to decode JPEG images at runtime using the hardware decoder. This means that the pre-decoded bitmaps need not be included in the assets, reducing the overall asset size. JPEG images are compressed, making them smaller in size, and they can be decoded by supported platforms.

Supported Platforms

At present, the JPEG decoder is exclusively supported on the Infineon Traveo 2 platform.

Limitations

The JPEG Decoder Behavior currently has the following limitations.

• Preloaded Images in the asset only: The image must be part of the asset and cannot be dynamically supplied at runtime.
• Check the JPEG Bitmap Converter documentation for the supported pixel formats.

Enabling the JPEG Decoder feature

If the JPEG Decoder Behavior is being used, ensure that the JPEG Decoder feature has been enabled in the CMAKE options. While generating the project for target, turn on the CMAKE flag CGIDEVICE_TRAVEO2_JPEG_DECODER_ENABLED or add this flag to the toolchain file.

How to use the JPEG Decoder

1. Import the jpeg image as raw resource: Using the import menu select “Import Raw Resources”. This ensures that the image is available in its original format for further processing
   
   
   
   
2. Add the Bitmap node to the scene: Add a bitmap node to a 2D scene.
   
3. Add the Behavior to a Bitmap Node: Place the JPEG Decoder behavior onto a Bitmap Node in the scene.
   
   
   
   
4. Set the Resource Candera Name: Paste the Candera Name of the raw JPEG image into the Resource Candera Name property using the steps below.
   a. Step 1: Copy the Candera Name of the raw JPEG image from the Solution Explorer under the Asset properties of the JPEG resource.
   
   
   
   b. Step 2: Paste the Candera Name into the ResourceCanderaName property of the JPEG Decoder Behavior in the Properties panel.
   
   

MJPEG Stream Behavior

Feature Description

The MJPEG Stream behavior allows the user to decode and play an MJPEG video stream. The MJPEG Stream behavior uses the onboard JPEG Decoder hardware to decode the JPEG frames and display them on a bitmap node.

Refer to the Video Format Conversion documentation to convert any video to MJPEG format.

Supported Platforms

At present, the Motion JPEG decoder is exclusively supported on the Infineon Traveo 2 platform.

Limitations

The Motion JPEG Decoder Behavior currently has the following limitations.

• The video file must be included as part of the asset and cannot be provided to the hardware dynamically through communication interfaces at runtime.
• Check the JPEG Bitmap Converter documentation for the supported pixel formats.
  

Enabling the JPEG Decoder feature

If the MJPEG Decoder Behavior is being used, ensure that the JPEG Decoder feature has been enabled in the CMAKE options. While generating the project for target, turn on the CMAKE flag CGIDEVICE_TRAVEO2_JPEG_DECODER_ENABLED or add this flag to the toolchain file.

How to use the Motion - JPEG Decoder

1. Import the video as raw resource: Using the import menu select “Import Raw Resources”. This ensures that the video is available in its original format for further processing.
   
   
   
   
2. Add the Bitmap node: Add a bitmap node to a 2D scene.
   
3. Add the Behavior to a Bitmap Node: Place the Motion - JPEG Decoder behavior onto a Bitmap Node in the scene. 
   
   
4. Set the Resource Candera Name: Paste the Candera Name of the raw MJPEG video into the ResourceCanderaName property using the steps below.
   Step 1: Copy the Candera Name of the raw MJPEG video from the Solution Explorer under the Asset properties of the JPEG resource.
   
   
   
   Step 2: Paste the Candera Name into the ResourceCanderaName property of the Motion - JPEG Decoder Behavior in the Properties panel.
   
   
   
   

5. Playing in Loop: The video can be configured to play continuously in a loop, if required.

6. Initial State: The initial playback state of the video can be selected from the dropdown menu, as shown below.
   
   
   
   
7. Frame Rate: Ensure that the frame rate is set to match the actual frame rate of the imported MJPEG video.
   
   

The properties for the Motion JPEG Decoder behavior are bindable and therefore can be modified at runtime as well.

Video Format Conversion

Introduction to ffmpeg

ffmpeg is a free and opensource tool that can be used to convert images and videos to different formats. The tool can be used to convert any video file to MJPEG format so that it can be imported and played by the MJPEG decoder behavior.

Installing ffmpeg

• ffmpeg version used in this documentation is 8.0. Download ffmpeg.
• Extract the ffmpeg tool from the downloaded archive to a location on disk and add the path to the bin folder in the environment PATH variables.

Using the ffmpeg tool to convert MP4 videos to MJPEG

The following command can be used to convert an mp4 file into MJPEG format:

ffmpeg -i input.mp4 -c:v mjpeg -q:v 2 -pix_fmt yuvj420p -huffman optimal -vf "scale=400:225" -f mjpeg -t 10 -r 15 output.mjpeg

The breakdown of the arguments in the command is as follow:

• ffmpeg - The name of the tool
• -i - Specifies the input file
• -c:v mjpeg - Encodes all video channels with the mjpeg codec
• -q:v 2 - This is the quality scale for the video stream. Values are from 1=best to 31=worst
• -pix_fmt yuvj420p - Select the pixel format as YUV420
• -huffman optimal - Use optimal huffman coding
• -vf "scale=400:225" - Set the output video resolution
• -f mjpeg - Force output file format as mjpeg
• -t 10 - Restrict length of output video to 10 seconds
• -r 15 - Set the output video frame rate to 15 fps
• output.mjpeg - name of the output file

Various Device Specific Content

Traveo2BitmapConverter

Description

This document provides a complete list of bitmap formats supported by the Traveo2 Target.
The formats are grouped into categories such as Basic, Grayscale, RLE (Run length Encoding), RLA (Run-Length Adaptive), RLAD (Run-Length Adaptive Dithering), CLUT (color lookup table), JPEG, and YUV.

Below are the supported pixel formats with their corresponding type, format code, and a brief description.

These bitmap converters are supported only on the target. Windows simulation uses the default bitmap converter.

Traveo2 Bitmap Format List

Basic Formats

These formats represent the fundamental bitmap types supported by Traveo2, defining basic RGB and alpha channel combinations with different bit depths for color and transparency.

Serial No	Format	Description
1	Traveo2ExtendedBitmapFormatR6G6B6A6	6 bits per channel with alpha.
2	Traveo2ExtendedBitmapFormatR6G6B6	6 bits per RGB channel.
3	Traveo2ExtendedBitmapFormatR3G3B2	Compact 8-bit color format.
4	Traveo2ExtendedBitmapFormatA8	8-bit alpha-only channel.
5	Traveo2ExtendedBitmapFormatA4	4-bit alpha-only channel.
6	Traveo2ExtendedBitmapFormatA2	2-bit alpha-only channel.
7	Traveo2ExtendedBitmapFormatA1	1-bit alpha mask.

Grayscale Formats.

These formats are used for grayscale images, representing luminance information with optional alpha channels for transparency control.

Serial No	Format	Description
1	Traveo2ExtendedBitmapFormatRGB8	8-bit grayscale RGB.
2	Traveo2ExtendedBitmapFormatA8RGB8	8-bit grayscale RGB with alpha.
3	Traveo2ExtendedBitmapFormatRGB4	4-bit grayscale RGB.
4	Traveo2ExtendedBitmapFormatA4RGB4	4-bit grayscale RGB with alpha.

RLE (Run-Length Encoded) Formats.

These formats use Run-Length Encoding (RLE) to compress bitmap data, efficiently storing repeated pixel values while supporting various RGB and alpha channel combinations.

Serial No	Format	Description
1	Traveo2ExtendedBitmapFormatRleR8G8B8A8	RLE-compressed 32-bit color (8 bits per channel).
2	Traveo2ExtendedBitmapFormatRleR8G8B8	RLE-compressed 24-bit color.
3	Traveo2ExtendedBitmapFormatRleR6G6B6A6	RLE-compressed 6-bit per channel format with alpha.
4	Traveo2ExtendedBitmapFormatRleR6G6B6	RLE-compressed 6-bit RGB.
5	Traveo2ExtendedBitmapFormatRleR4G4B4A4	RLE-compressed 16-bit RGBA.
6	Traveo2ExtendedBitmapFormatRleR5G5B5A1	RLE-compressed 16-bit RGBA (5-5-5-1).
7	Traveo2ExtendedBitmapFormatRleR5G6B5	RLE-compressed 16-bit RGB (5-6-5).
8	Traveo2ExtendedBitmapFormatRleA8	RLE-compressed 8-bit alpha mask.
9	Traveo2ExtendedBitmapFormatRleA4	RLE-compressed 4-bit alpha mask.
10	Traveo2ExtendedBitmapFormatRleA2	RLE-compressed 2-bit alpha mask.
11	Traveo2ExtendedBitmapFormatRleA1	RLE-compressed 1-bit alpha mask.

RLE Grayscale Formats

These formats combine grayscale image representation with Run-Length Encoding (RLE) compression, reducing memory usage while maintaining luminance and optional alpha information.

Serial No	Format	Description
1	Traveo2ExtendedBitmapFormatRleRGB8	RLE-compressed 8-bit grayscale RGB.
2	Traveo2ExtendedBitmapFormatRleA8RGB8	RLE-compressed grayscale with 8-bit alpha.
3	Traveo2ExtendedBitmapFormatRleRGB4	RLE-compressed 4-bit grayscale RGB.
4	Traveo2ExtendedBitmapFormatRleA4RGB4	RLE-compressed 4-bit grayscale with alpha.
5	Traveo2ExtendedBitmapFormatRleRGB1	RLE-compressed 1-bit grayscale (monochrome).

RLA (Run-Length Adaptive) Formats.

These formats use Run-Length Adaptive (RLA) compression, which dynamically adjusts compression based on pixel patterns to achieve better efficiency while preserving image quality across various color and alpha depths.

Serial No	Format	Description
1	Traveo2ExtendedBitmapFormatRlaR8G8B8A8	RLA-compressed 32-bit RGBA.
2	Traveo2ExtendedBitmapFormatRlaR8G8B8	RLA-compressed 24-bit RGB.
3	Traveo2ExtendedBitmapFormatRlaR8GB8A8	RLA-compressed 24-bit RGBA (8-4-4-8).
4	Traveo2ExtendedBitmapFormatRlaR6G6B6A6	RLA-compressed 6-bit per channel RGBA.
5	Traveo2ExtendedBitmapFormatRlaR6G6B6	RLA-compressed 6-bit RGB.
6	Traveo2ExtendedBitmapFormatRlaR4G4B4A4	RLA-compressed 16-bit RGBA.
7	Traveo2ExtendedBitmapFormatRlaR5G5B5A1	RLA-compressed 16-bit RGBA (5-5-5-1).
8	Traveo2ExtendedBitmapFormatRlaR1G1B1A1	RLA-compressed 4-bit color with 1-bit alpha.
9	Traveo2ExtendedBitmapFormatRlaR5G6B5	RLA-compressed 16-bit RGB (5-6-5).
10	Traveo2ExtendedBitmapFormatRlaA8	RLA-compressed 8-bit alpha.
11	Traveo2ExtendedBitmapFormatRlaA4	RLA-compressed 4-bit alpha.
12	Traveo2ExtendedBitmapFormatRlaA2	RLA-compressed 2-bit alpha.
13	Traveo2ExtendedBitmapFormatRlaA1	RLA-compressed 1-bit alpha.

RLA Grayscale Formats

These formats apply Run-Length Adaptive (RLA) compression to grayscale images, optimizing storage while maintaining brightness and optional alpha transparency information.

Serial No	Formats	Description
1	Traveo2ExtendedBitmapFormatRlaRGB8	RLA-compressed grayscale RGB (8-bit).
2	Traveo2ExtendedBitmapFormatRlaA8RGB8	RLA-compressed grayscale RGB with 8-bit alpha.
3	Traveo2ExtendedBitmapFormatRlaRGB4	RLA-compressed grayscale RGB (4-bit).
4	Traveo2ExtendedBitmapFormatRlaA4RGB4	RLA-compressed grayscale RGB (4-bit + alpha).
5	Traveo2ExtendedBitmapFormatRlaRGB1	RLA-compressed grayscale (1-bit monochrome).

RLAD (Run-Length Adaptive Dithering) Formats

These formats use Run-Length Adaptive Dithering (RLAD) compression, combining adaptive run-length encoding with dithering techniques to preserve visual quality and smooth gradients in compressed images.

Serial No	Format	Description
1	Traveo2ExtendedBitmapFormatRleR8G8B8A8	RLE-compressed 32-bit color (8 bits per channel).
2	Traveo2ExtendedBitmapFormatRleR8G8B8	RLE-compressed 24-bit color.
3	Traveo2ExtendedBitmapFormatRleR6G6B6A6	RLE-compressed 6-bit per channel format with alpha.
4	Traveo2ExtendedBitmapFormatRleR6G6B6	RLE-compressed 6-bit RGB.
5	Traveo2ExtendedBitmapFormatRleR4G4B4A4	RLE-compressed 16-bit RGBA.
6	Traveo2ExtendedBitmapFormatRleR5G5B5A1	RLE-compressed 16-bit RGBA (5-5-5-1).
7	Traveo2ExtendedBitmapFormatRleR5G6B5	RLE-compressed 16-bit RGB (5-6-5).
8	Traveo2ExtendedBitmapFormatRleA8	RLE-compressed 8-bit alpha mask.
9	Traveo2ExtendedBitmapFormatRleA4	RLE-compressed 4-bit alpha mask.
10	Traveo2ExtendedBitmapFormatRleA2	RLE-compressed 2-bit alpha mask.
11	Traveo2ExtendedBitmapFormatRleA1	RLE-compressed 1-bit alpha mask.

RLAD Grayscale Formats

These formats apply Run-Length Adaptive Dithering (RLAD) compression to grayscale images, maintaining smooth tonal transitions and optional transparency while minimizing memory usage.

Serial No	Format	Description
1	Traveo2ExtendedBitmapFormatRladRGB8	RLAD-compressed grayscale RGB (8-bit).
2	Traveo2ExtendedBitmapFormatRladA8RGB8	RLAD-compressed grayscale RGB with alpha.
3	Traveo2ExtendedBitmapFormatRladRGB4	RLAD-compressed grayscale RGB (4-bit).
4	Traveo2ExtendedBitmapFormatRladA4RGB4	RLAD-compressed grayscale with alpha (4-bit).
5	Traveo2ExtendedBitmapFormatRladRGB1	RLAD-compressed grayscale (1-bit monochrome).

CULT Masking

This format uses a Color Look-Up Table (CULT) to reference predefined color entries, enabling efficient color representation and masking with reduced memory requirements.

Serial No	Format	Description
1	Traveo2ExtendedBitmapFormat_Clut	Uses color lookup table.

JPEG Format

This format supports standard JPEG-compressed images, providing high compression efficiency for photographic content.

• To enable this format on the target, set the corresponding CMake flag in the toolchain file during CMake configuration.
• This feature is supported only on the Traveo-II 6M and 6MDDR target boards.

Serial No	Format	Description
1	Traveo2ExtendedBitmapFormat_Jpeg	JPEG compressed image in any supported color format. For list of supported color formats, please check JPEG Bitmap Converter Documentation

YUV Formats

These formats represent image data using YUV color encoding, supporting various chroma subsampling modes like 4:2:0, 4:2:2, and 4:4:4 for efficient video and image processing.

Serial No	Format	Description
1	Traveo2ExtendedBitmapFormat_YUV420	YUV planar 4:2:0 format.
2	Traveo2ExtendedBitmapFormat_VYUY422	Packed YUV 4:2:2 (VYUY order).
3	Traveo2ExtendedBitmapFormat_YVYU422	Packed YUV 4:2:2 (YVYU order).
4	Traveo2ExtendedBitmapFormat_YUV444	YUV 4:4:4 full color format.