JPEG Decoder Behavior

Feature Description

The JPEG Decoder behavior allows the user to decode JPEG images at runtime using the hardware decoder. This means that the pre-decoded bitmaps need not be included in the assets, reducing the overall asset size. JPEG images are compressed, making them smaller in size, and they can be decoded by supported platforms.

Supported Platforms

At present, the JPEG decoder is exclusively supported on the Infineon Traveo 2 platform.

Limitations

The JPEG Decoder Behavior currently has the following limitations.

Single Image Decoding: Only one image can be decoded at a time. Preloaded Images in the asset only: The image must be part of the asset and cannot be dynamically supplied at runtime. Baseline JPEG with RGB Encoding: The JPEG decoder behavior only supports baseline JPEG images encoded in RGB format. Scene Editor cannot display the decoded image: The Scene Editor currently cannot display the decoded image; it is only viewable on the hardware.

How to use the JPEG Decoder

Import the jpeg image as raw resource: Using the import menu select “Import Raw Resources”. This ensures that the image is available in its original format for further processing

Add the Bitmap node to the scene: Add a bitmap node to a 2D scene. Add the Behavior to a Bitmap Node: Place the JPEG Decoder behavior onto a Bitmap Node in the scene.

Set the Resource Candera Name: Paste the Candera Name of the raw JPEG image into the Resource Candera Name property using the steps below.
a. Step 1: Copy the Candera Name of the raw JPEG image from the Solution Explorer under the Asset properties of the JPEG resource.

b. Step 2: Paste the Candera Name into the ResourceCanderaName property of the JPEG Decoder Behavior in the Properties panel.

Enable/Disable the Decoder: You can toggle the decoder using the enabled property.

Changes required in the toolchain file

The path to the Infineon Traveo 2 JPEG decoder driver must be added to the toolchain file since the JPEG Decoder behavior uses this driver to decode the JPEG images. Add the following line to the cross-compilation toolchain file:
set(CGIDEVICE_TARGET_TRAVEO2_JPEG_DRIVER_PATH "path\to\jpeg\decoder\driver\root" CACHE PATH "Root path to the Traveo2 JPEG decoder driver.")
Ensure the following folders are available in the driver root path:

Bin\arm-none-ghs\production (\debug or \release) Source\jpgdec\include Source\jpgdec\lib_include

Building the JPEG Decoder Driver from source (if available)

In case the JPEG decoder driver binaries are not available, it can be built from source as well, if available, following the steps given below.

Open "TVII_JPEG_Driver\Build\GHS\jpeg_driver\prj\jpeg_driver.gpj" in Multi Project Manager. In the toolbar, click on Build > Build Top Project jpeg_driver.gpj as shown in the image below

The JPEG Decoder driver can only be built in production environment, however, if the base application is being built in release or debug environment, copies of the Bin/arm-none-ghs/production folder can be created and renamed to release or debug as required.

Converting images to supported format

To ensure compatibility, you may need to convert your JPEG images to an RGB-encoded baseline JPEG format. Follow the steps below to install the necessary tools and use the conversion script.

Software Requirements

The conversion script uses djpeg and cjpeg utilities, which are part of the libjpeg-turbo software. You need to install them and make them accessible via your system’s PATH environment variable.

Installation Steps

Download libjpeg-turbo from this link (libjpeg-turbo). Extract the downloaded files, install the exe and locate libjpeg-turbo-gcc64\bin. Since libjpeg-turbo includes the necessary executables, its bin folder path must be added to the system's Environment Variables. Add the folder path to your system’s Environment Variables:
a. Open System Properties > Environment Variables.
b. Under System Variables, find and edit the Path variable.
c. Add the path to libjpeg-turbo-gcc64\bin.

Using the Conversion Script

The provided batch script, encode_jpeg_as_rgb.bat, automates the conversion of YCbCr JPEGs into RGB-encoded JPEGs. You simply drag and drop a JPEG file onto the script, and it processes the image automatically.

Steps for Conversion

Ensure libjpeg-turbo is installed and is accessible in PATH.
Place the batch script in a directory where your images are stored.
Drag and drop a JPEG file onto the script.
The script will generate a new file with _rgb appended to the filename.

Example Usage

Input: sample.jpg (YCbCr format)
Output: sample_rgb.jpg (RGB format)
After running the script, the folder will look like this, with the converted images having '_rgb' appended to their filenames.

Verifying the Converted Images

To check if the image needs conversion, the following method can be used:

Install ffmpeg from the link: Download FFmpeg
Ensure the ffprobe command can be run on terminal by adding the path to the ffmpeg bin folder into the system PATH environment variable.
Open a terminal in the folder where the image is located and run the command “ffprobe image_name.jpg”.
Ensure that the format of the image as displayed by the ffprobe command is (Baseline), gbrp.

Even when run on single images, the ffprobe command will show the type as mjpeg.