Operator JPEG_Encoder

Operator Library: Compression

The operator performs a JPEG compression of grayscale 8-bit images. It uses the JPEG baseline algorithm. The operator's output is a Huffman stream. Optionally, JPEG headers are included in the output (parametrizable). If the headers are included, the output format is JFIF (JPEG File Interchange Format), version 1.2.

[Important] Availability

To use the JPEG_Encoder operator, you need either a JPEG Compression Library license, or the VisualApplets 4 license.

The compression rate (and thus the output stream size) depends on two factors:

  • on the selected quantization table which is changeable during runtime.

  • on the input image.

Operator JPEG_Encoder is able to process the full input data rate as specified in the link parametrization. You can define the throughput rate via the input parallelism. Please note that higher parallelism entails a higher FPGA resource utilization.

The maximum image height is 65.535 pixels. If the image height is not a multiple of eight, the operator internally adds dummy lines. This behaviour reduces the overall input data rate.

The operator allows you to define the quantization table. You have two options to configure the table:

  • The quantization table can be calculated automatically on the basis of a quality value (in percent). For calculation, the standard luminance quantization table (see below) is used. Use parameter Quality (in percent) to configure automatic calculation. Automatic calculation is the default setting of the operator.

  • Alternatively, you can set each of the quantization table values individually. Use parameter LuminanceQuantization to enter your values. Parameter Quality is automatically disabled in this case by being set to value -1.

The standard luminance quantization table (default setting) looks as follows:

In default mode (i.e. automatic calculation out of the standard lumninance quantization table set by the parameter Quality), the quantization table is calculated by the following equation:

As described above, the operator's output is a Huffman stream of the encoded image data. For Huffman coding, the standard luminance tables for DC and AC coefficients are used. These tables are fixed and taken from literature, namely W. P. Pennbaker and J. L. Mitchell, 'JPEG Still Image Data Compression Standard', Van Nostrand Rheinhold, 1993. The Huffman stream generated by the encoder includes the JPEG header if parameter IncludeHeader is set to YES. The generated data stream consists of parallel outputs of multiple bytes (8-bit blocks) where the parallelism is automatically derived from the throughput requirements.

Operator Restrictions

  • The operator does not support empty images, i.e., images with no pixels.

  • Input images with varying line lengths are not allowed.

  • The operator has a minimum input image width which depends on the input parallelism. The minimum input image width is at least twice the input parallelism. The minimum input image width is calculated as follows:

    1. On the basis of the input parallelism: The first multiple of 8 is identified which is equal to or bigger than the input parallelism.

    2. To this multiple of 8, value 1 is added.

    3. On the basis of the result of step 2: As the input image width has to be a multiple of the input parallelism, the first multiple of the input parallelism that is bigger than the result of step 2 is identified.

    The result of step 3 is the minimum input image width for the operator.

    Formula for calculating the minimum input image width

    Figure 440. Formula for calculating the minimum input image width


    For example, if the input parallelism is 4, the minimum input image width is 12.

    If the input parallelism is 12, the minimum input image width is 24.

[Tip] Optimizing the Operator Throughput

If the input parallelism is greater than 8, the data throughput depends on the size of the input image. The maximum data throughput can be achieved with the following image size (OptimalSize):

OptimalSize = Ceil(FrameSize/(PathCount * IntervalSize))*PathCount * IntervalSize

FrameSize = ceil(ImageWidth/8)*8 * ceil(ImageHeight/8)*8;

PathCount = ceil(Par/8)

IntervalSize: Next value greater than 7, which has no common factor with PathCount.

Internally, the operator always uses a parallelism that is a multiple of 8. Depending on the input parallelism, the internal parallelism conversion can compensate some of the loss of bandwidth. The loss of bandwidth only occurs at the end of a frame.

As the size of the compressed image data is not predictable, the last output data byte of a compressed frame might not occur aligned to the output parallelism. Consequently, the last data vector of a frame can contain random dummy values. To mark the actual end of a frame, the EOI (End of Image) marker as defined in the JPEG standard will be used. The EOI marker consists of two bytes. The second last byte of each frame is 0xFF, and the last byte of each frame is 0xD9.

To optimize its image throughput rate (band width), the operator outputs the header as soon as header generation is activated - even before image data arrive at the operator's input. This way, the transfer of the header data doesn't interrupt the transfer of image data, as the header is transferred in advance. The drawback of this practice is that the operator's output transfer starts earlier than the actual image data transfer. This may cause irritations under specific circumstances:

  • Using operator SourceSelector directly after JPEG_Encoder: Operator SourceSelector registers a partly processed frame as soon as it gets the header data. Therefore, if SourceSelector is switched to getting image data from JPEG_Encoder, SourceSelector cannot be switched to any another source as it always detects an unfinished frame. In addition, when header generation is enabled and SourceSelector switches from another source to the JPEG_Encoder channel, the first image is lost.

  • To measure the latency, use operator FrameEndToSignal (instead of FrameStartToSignal and SignalToDelay).

  • If working on eVA devices, please make sure the output is capable to accept data transfer before the sensor transfer is started.

Get more information and additional examples on VisualApplets' JPEG compression features in the VisualAppletsOnline Community Forum at New High-Speed JPEG Operator and Examples.

Overflow Management with InfiniteSource

In the InfiniteSource mode it is possible for images to be lost or corrupted, because the JPEG_Encoder module or one of the succeeding modules can't handle the bandwidth. If the overflow occurs while a partial image was already accepted by the operator, all further incoming image data is discarded and the truncated image in the operator is truncated, which results in a partial output image. If the operator is in an overflow state while the start of a frame arrives, the entire frame is discarded and the frame is lost. For each truncated or lost frame, a VA event is generated (TruncatedEvent and LostEvent). The JPEG_Encoder events consist of 3 packets with 2 byte each. The first two VA event packets make up the frame-ID, which is just a counter for each frame that arrives at the JPEG_Encoder input. The frame-ID identifies the exact frame that was lost or truncated. The third packet marks the type of error that occurred. Bit 0 marks whether a frame was lost (Bit0 = 1) or truncated (Bit0 = 0). The other 15 bit of the third packet are only used in case events were lost, which can only occur if events occur too fast for the CPU. Bit1 of the third packet marks any occurrence of a TruncatedEvent that was lost and Bit2 of the same packet marks any occurrence of a LostEvent that was lost. Additionally, for LostEvents that were lost, Bit[15:3] form a counter that holds the number of LostEvents that were lost. If the counter value is zero, the lost-LostEvent-counter has overflown. Since a truncated frame results in JPEG_Encoder output data but a lost frame does not, the VA events only count the number of lost LostEvents.

Overflow Event Data

Figure 441. Overflow Event Data


I/O Properties

Property Value
Operator Type M
Input Link I, data input
Output Link O, data output

Supported Link Format

Link Parameter Input Link I Output Link O
Bit Width 8 8
Arithmetic unsigned as I
Parallelism any automatically calculated
Kernel Columns 1 as I
Kernel Rows 1 as I
Img Protocol VALT_IMAGE2D as I
Color Format VAF_GRAY as I
Color Flavor FL_NONE as I
Max. Img Width 2^16 -1 = 65.5351 any2
Max. Img Height 2^16 -1 = 65.535 1

1

The value must not be lower than the minimum image width requirement.

2

The image width at the output link is configurable. If the output image is bigger than the maximal image width configured for the output link, the image is cut off and the remaining image data is discarded. The last two bytes of each image are always containing the End of Image Marker (EOI).

Parameters

Quality
Type static/dynamic write parameter
Default 50.00
Range [1.00 - 100.00 %], {-1}, step size = 0.01%

Using this parameter, the quality of the JPEG compression can be changed. The quantization matrix is determined from the percentage values using the equation given above. The determined quantization values can be read from parameter quantization_matrix. The parameter is dynamic and should only be changed during idle time of the applet.

Quality settings between 1 and 100 can be defined. Writing to this parameter overwrites manual changes of the quantization matrix made by parameter LuminanceQuantization. If -1 is read from Quality, manual change of the quantization matrix has been made.

The quality parameter is the primary source to define the compression rate of the encoder. The luminance table will be auto-computed from the specified quality and can be read back. However it is possible to modify the luminance table directly. If this happens the quality parameter will be auto-changed to -1 to show that the parameter is not valid anymore and manual overwrite mode for the tables is used. The quality and the quantization tables can be set to static mode if the user wants to optimize resource usage. Static versus dynamic change can be performed only on the quality parameter. The quantization table settings will follow the quality type automatically and cannot be overwritten manually. When the quality is set to static, the operator determines the output parallelism from the quality settings. In most cases the output parallelism will be reduced in comparison to the dynamic mode, which can mean a significant FPGA resource reduction at the cost of giving up the flexibility to change the compression rate during the runtime.

LuminanceQuantization
Type follows Quality type (dynamic or static) write parameter
Default none
Range [1 - 255]

auto computed for quality in range [1.00 - 100.00], when manually set, quality is invalidated to -1.

IncludeHeader
Type static write parameter
Default YES
Range [YES,NO]

The JPEG header is per default included in the compression stream. However, you can disable this feature. If you set parameter IncludeHeader to NO, the JPEG parameters ImageHeight and ImageWidth will become deactivated. If you set parameter IncludeHeader to YES, the header is included and you can decide if the header parameters can be static or are required to be dynamic. If set to static, the values for image width and image height will be statically embedded into the header and cannot be changed, regardless of the input image size. If set to dynamic, you can change the values for image width and image height during the runtime. Please note that these header parameters are not automatically updated to the input image size. If you use the operator with images the size of which is dynamically changing during runtime, you will have to patch the produced header afterwards. If you set ImageHeight and ImageWidth both to "static", you will achieve a slight reduction of the FPGA resource usage.

ImageWidth
Type static/dynamic write parameter
Default 1024
Range 1 - 2^16-1

This parameter is only available if parameter IncludeHeader is set to YES.

Parameter ImageWidth is only used for generating the JPEG image header as described in parameter IncludeHeader.

ImageHeight
Type static/dynamic write parameter
Default 1024
Range 1 - 2^16-1

This parameter is only available if parameter IncludeHeader is set to YES.

Parameter ImageHeight is only used for generating the JPEG image header as described in parameter IncludeHeader.

InfiniteSource
Type static write parameter
Default DISABLED
Range {ENABLED, DISABLED}

The operator can be plugged directly after a camera operator. In this case the InfiniteSource parameter must be set to ENABLED. Then the operator will perform active overflow management and report overflow conditions to the software through VA event system. The overflow can occur only in 2 situations: the sink behind the operator will stop/pause the transmission or the input image height is not a multiple of 8 lines. In the latter case the operator has to pad the missing lines to complete the last row JPEG blocks as required in JPEG standard.

See 'Infinite Sources / Connecting Cameras' for more information.

Examples of Use

The use of operator JPEG_Encoder is shown in the following examples:

More Information

The following Huffman DC and AC coefficients are used. 

 typedef char DCHuffTableType[12][17]; // Huffman table for
      luminance DC coefficients DCHuffTableType Lum_DC_HuffmanTable= { "00", "010", "011", "100",
      "101", "110", "1110", "11110", "111110", "1111110", "11111110", "111111110" }; typedef char
      ACHuffTableType[16][11][17]; // Huffman table for luminance AC coefficients ACHuffTableType
      Lum_AC_HuffmanTable= { { //Run == 0 "1010",//EOB "00", "01", "100", "1011", "11010",
      "1111000", "11111000", "1111110110", "1111111110000010", "1111111110000011" }, { //Run == 1
      "1010",//EOB "1100", "11011", "1111001", "111110110", "11111110110", "1111111110000100",
      "1111111110000101", "1111111110000110", "1111111110000111", "1111111110001000" }, { //Run == 2
      "1010",//EOB "11100", "11111001", "1111110111", "111111110100", "1111111110001001",
      "1111111110001010", "1111111110001011", "1111111110001100", "1111111110001101",
      "1111111110001110" }, { //Run == 3 "1010",//EOB "111010", "111110111", "111111110101",
      "1111111110001111", "1111111110010000", "1111111110010001", "1111111110010010",
      "1111111110010011", "1111111110010100", "1111111110010101" }, { //Run == 4 "1010",//EOB
      "111011", "1111111000", "1111111110010110", "1111111110010111", "1111111110011000",
      "1111111110011001", "1111111110011010", "1111111110011011", "1111111110011100",
      "1111111110011101", }, { //Run == 5 "1010",//EOB "1111010", "11111110111", "1111111110011110",
      "1111111110011111", "1111111110100000", "1111111110100001", "1111111110100010",
      "1111111110100011", "1111111110100100", "1111111110100101" }, { //Run == 6 "1010",//EOB
      "1111011", "111111110110", "1111111110100110", "1111111110100111", "1111111110101000",
      "1111111110101001", "1111111110101010", "1111111110101011", "1111111110101100",
      "1111111110101101" }, { //Run == 7 "1010",//EOB "11111010", "111111110111",
      "1111111110101110", "1111111110101111", "1111111110110000", "1111111110110001",
      "1111111110110010", "1111111110110011", "1111111110110100", "1111111110110101", }, { //Run ==
      8 "1010",//EOB "111111000", "111111111000000", "1111111110110110", "1111111110110111",
      "1111111110111000", "1111111110111001", "1111111110111010", "1111111110111011",
      "1111111110111100", "1111111110111101" }, { //Run == 9 "1010",//EOB "111111001",
      "1111111110111110", "1111111110111111", "1111111111000000", "1111111111000001",
      "1111111111000010", "1111111111000011", "1111111111000100", "1111111111000101",
      "1111111111000110" }, { //Run == 0xA "1010",//EOB "111111010", "1111111111000111",
      "1111111111001000", "1111111111001001", "1111111111001010", "1111111111001011",
      "1111111111001100", "1111111111001101", "1111111111001110", "1111111111001111" }, { //Run ==
      0xB "1010",//EOB "1111111001", "1111111111010000", "1111111111010001", "1111111111010010",
      "1111111111010011", "1111111111010100", "1111111111010101", "1111111111010110",
      "1111111111010111", "1111111111011000" }, { //Run == 0xC "1010",//EOB "1111111010",
      "1111111111011001", "1111111111011010", "1111111111011011", "1111111111011100",
      "1111111111011101", "1111111111011110", "1111111111011111", "1111111111100000",
      "1111111111100001" }, { //Run == 0xD "1010",//EOB "11111111000", "1111111111100010",
      "1111111111100011", "1111111111100100", "1111111111100101", "1111111111100110",
      "1111111111100111", "1111111111101000", "1111111111101001", "1111111111101010" }, { //Run ==
      0xE "1010",//EOB "1111111111101011", "1111111111101100", "1111111111101101",
      "1111111111101110", "1111111111101111", "1111111111110000", "1111111111110001",
      "1111111111110010", "1111111111110011", "1111111111110100" }, { //Run == 0xF "11111111001",
      //ZRL "1111111111110101", "1111111111110110", "1111111111110111", "1111111111111000",
      "1111111111111001", "1111111111111010", "1111111111111011", "1111111111111100",
      "1111111111111101", "1111111111111110" } };
  Prev     Level Up     Next  
Content Top Home