JPEG Decoder Behavior

Abbreviations

Abbreviation	Full Form
JPEG	Joint Photographic Experts Group
RGB	Red, Green, Blue
DDR	Double Data Rate
RAM	Random Access Memory
SRAM	Static RAM
VRAM	Video RAM
SOF	Start of Frame

Behavior Overview

Feature Description

The JPEG Decoder behavior allows the user to decode JPEG images at runtime using the hardware decoder. These JPEG images can be directly stored in the asset without decoding, reducing the overall asset size. JPEG images are compressed, making them smaller in size, and they can be decoded by supported platforms.

Supported Platforms

At the time of phase 1 implementation, the JPEG decoder is exclusively supported on the Infineon Traveo 2 platform.

Limitations

The JPEG Decoder Behavior currently has the following limitations under the phase 1 implementation:

Single Image Decoding: Only one image can be decoded at a time.
Preloaded Images in the asset only: The image must be part of the asset and cannot be dynamically supplied at runtime.
Baseline JPEG with RGB Encoding: The JPEG decoder behavior only supports baseline JPEG images encoded in RGB format.
Scene Editor cannot display the decoded image: The Scene Editor currently cannot display the decoded image; it is only viewable on the hardware.
CMake option “CANDERA_2D_ENABLED” must be set true for the behavior to function properly since it only supports 2D surfaces.

Using the JPEG Decoder Behavior

Import the jpeg image as raw resource: Using the import menu select “Import Raw Resources”. This ensures that the image is available in its original format for further processing
Add the Bitmap node to the scene: Add a bitmap node to a 2D scene.
Add the Behavior to a Bitmap Node: Place the JPEG Decoder behavior onto a Bitmap Node in the scene.
Set the Resource Candera Name: Paste the Candera Name of the raw JPEG image into the Resource Candera Name property using the steps below.
a. Step 1: Copy the Candera Name of the raw JPEG image from the Solution Explorer under the Asset properties of the JPEG resource.

b. Step 2: Paste the Candera Name into the ResourceCanderaName property of the JPEG Decoder Behavior in the Properties panel.

Toolchain file for the JPEG Decoder

If the JPEG decoder feature is being used, ensure that the correct toolchain file has been used. The toolchain files for the Traveo 2 platform are available in the directory cgi_studio_devices\src\Traveo2\ToolchainFiles

Use one of the following two toolchain files with the suffix _JPEG if using the JPEG decoder behavior so that the linker file dedicates a section for the JPEG Decoder output.

MULTI-toolchain-Traveo2_C_6M_JPEG.txt
MULTI-toolchain-Traveo2_C_6MDDR_JPEG.txt

The above toolchain files not only enable the creation of necessary memory sections for the JPEG Decoder but also define linker flags and macro definitions required to enable the JPEG decoder functionality in the behavior code.

Modifying the JPEG Decoder buffer memory section

The following two lines present in linker directives file create a JPEG decode buffer section to which the JPEG decoder will write the decoded image.

.jpeg_decode_buffer_base ALIGN(1) : > .
.jpeg_decode_buffer ALIGN(1) PAD(1M) : > . // JPEG decoder output

The size of the section and the location can be changed according to the target for which the application is being built.
For example, in the 6m DDR board, the location of the jpeg_decode_buffer section can be inside the LPDDR4_RAM area, however for the 6m board, it would have to be in either SRAM or VRAM (preferred).

Example for 6m board

Example for 6m DDR board

Converting images to supported format

To ensure compatibility, you may need to convert your JPEG images to an RGB-encoded baseline JPEG format. Follow the steps below to install the necessary tools and use the conversion script.

Software Requirements

The conversion script uses djpeg and cjpeg utilities, which are part of the libjpeg-turbo software. You need to install them and make them accessible via your system’s PATH environment variable.

Installation Steps

Download libjpeg-turbo from this link (libjpeg-turbo).
Extract the downloaded files, install the exe and locate libjpeg-turbo-gcc64\bin.
Since libjpeg-turbo includes the necessary executables, its bin folder path must be added to the system's Environment Variables.
Add the folder path to your system’s Environment Variables:
a. Open System Properties > Environment Variables.
b. Under System Variables, find and edit the Path variable.
c. Add the path to libjpeg-turbo-gcc64\bin.

Using the Conversion Script

The provided batch script, encode_jpeg_as_rgb.bat, automates the conversion of YCbCr JPEGs into RGB-encoded JPEGs. You simply drag and drop a JPEG file onto the script, and it processes the image automatically.

Steps for Conversion

Ensure libjpeg-turbo is installed and is accessible in PATH.
Place the batch script in a directory where your images are stored. (Code for the script is given at the end of this document in Appendix A)
Drag and drop a JPEG file onto the script.
The script will generate a new file with _rgb appended to the filename.

Example Usage

Input: sample.jpg (YCbCr format)
Output: sample_rgb.jpg (RGB format)
After running the script, the folder will look like this, with the converted images having '_rgb' appended to their filenames.

Verifying the Converted Images

To check if the image needs conversion, the following method can be used:

Install ffmpeg from the link: Download FFmpeg
Ensure the ffprobe command can be run on terminal by adding the path to the ffmpeg bin folder into the system PATH environment variable.
Open a terminal in the folder where the image is located and run the command “ffprobe image_name.jpg”.
Ensure that the format of the image as displayed by the ffprobe command is (Baseline), gbrp.

Even when run on single images, the ffprobe command will show the type as mjpeg.

Conversion Script

Paste the following code into a .bat file to create the conversion script.

@echo off
setlocal

:: Check if a file is dragged and dropped
if "%~1"=="" (
echo Usage: Drag and drop a JPEG file onto this script to convert it.
echo.
echo The output file will be saved with "_rgb" appended to the input file name.
goto end
)

:: Set the input file path
set INPUT_FILE=%~1

:: Check if the input file exists
if not exist "%INPUT_FILE%" (
echo Error: Input file "%INPUT_FILE%" not found.
goto end
)

:: Generate the output file name with "_rgb" suffix
set OUTPUT_FILE=%~dpn1_rgb.jpg

:: Temporary PPM file
set TEMP_PPM="%TEMP%\temp.ppm"

:: Step 1: Convert input JPEG to PPM using djpeg
echo Converting input JPEG to PPM...
djpeg -outfile %TEMP_PPM% "%INPUT_FILE%"
if errorlevel 1 (
echo Error: Failed to convert input JPEG to PPM.
goto end
)

:: Step 2: Convert PPM to RGB-encoded JPEG using cjpeg
echo Converting PPM to RGB-encoded JPEG...
cjpeg -rgb -outfile "%OUTPUT_FILE%" %TEMP_PPM%
if errorlevel 1 (
echo Error: Failed to convert PPM to RGB-encoded JPEG.
goto end
)

:: Clean up temporary PPM file
del %TEMP_PPM%

echo Conversion successful!
echo Output saved to "%OUTPUT_FILE%"
goto end

:end
endlocal@echo off
setlocal

:: Set the input file path
set INPUT_FILE=%~1

:: Check if the input file exists
if not exist "%INPUT_FILE%" (
echo Error: Input file "%INPUT_FILE%" not found.
goto end
)

:: Generate the output file name with "_rgb" suffix
set OUTPUT_FILE=%~dpn1_rgb.jpg

:: Temporary PPM file
set TEMP_PPM="%TEMP%\temp.ppm"

:: Clean up temporary PPM file
del %TEMP_PPM%

echo Conversion successful!
echo Output saved to "%OUTPUT_FILE%"
goto end

:end
endlocal