Operator FrameBufferMultiRoi (imaFlex)

Operator Library: Memory

Available for Hardware Platforms
imaFlex CXP-12 Penta
imaFlex CXP-12 Quad

This operator buffers input image data in the Frame Grabber RAM (DRAM) and outputs an arbitrary number of regions of interest (ROI) for each buffered image.

One VisualApplets resource of the type RAM is required (see 'Allocation of Device Resources'). Multiple resources of the type RAM use the same physical RAM with the ' Shared Memory Concept '. Documentation for how to use the shared memory is available in the Application Note: Shared Memory.

The ROIs are defined in the operator parameter fields XLength, YLength, XOffset and YOffset. The following image shows an overview of the operator's output for one input frame, a set of parameters and input parallelism of 1.

The parameter MaxNumRoI specifies the maximum number of ROIs that can be read per buffered input frame. Each ROI coordinate parameter field XLength, YLength, XOffset and YOffset has MaxNumRoI entries.

The parameter NumRoI specifies the actual number of ROIs the operator will provide at the output for each buffered image. This number can't exceed MaxNumRoI when set as dynamic. When set as dynamic, you can update NumRoI during image acquisition. Changes to NumRoI take place, when all ROIs have been read from the current frame.

All ROIs are read sequentially and are provided as individual images on the operator output: ROI 0, ROI 1, ROI 2,.. until ROI N-1. Where N is the current number of ROIs, that is read out from each input frame, specified by the parameter NumRoI. The parameter fields XOffset, XLength, YOffset and YLength specify a set of ROIs. The parameter type of these parameters can be set to dynamic or static, which specifies whether the parameters are adjustable or not adjustable during runtime. The parameter type of all ROI parameters as well as MaxFrameWidth, MaxFrameHeight and MaxFrameSizeMode is adapted automatically, when the parameter type of one ROI parameter is set. Read about how to configure field parameters in 'Parameter Editing'.

All possible rectangular regions are supported for ROI definitions as long as the maximum input frame dimensions are not exceeded, i.e. a single pixel, a single line, a single column, a rectangular region, or the complete frame can be defined as an ROI.

Different ROIs do not need to have the same size.

Each ROI can be defined individually.

ROIs can overlap and ROIs can repeat.

The X coordinate parameters XOffset and XLength need to be in step size of the input parallelism.

You can define empty ROIs by setting XLength or YLength to zero. The operator then provides an empty image on its output. An empty image contains no pixels.

If the operator is used with the image protocol VALT_LINE1D, the Y coordinate parameters are disabled. In 1D operation, the operator will read one ROI as one line. The output of the operator will then be an infinite 1D image stream, where each line is represented by one ROI e.g. line 0 = ROI 0, line 1 = ROI 1, line N = ROI N, line N+1 = ROI 0, ...

The operator provides ROI images at its output as soon as the correspondent input frame is completely buffered, i.e. the input frame's EoF (End of Frame) signal is received. If enough space is available in the DRAM, the next input frames can be written to the DRAM while ROIs are read. The RAM bandwidth is shared between writing and reading but reading can only start after the corresponding input frame is written. After reading the last ROI for the input frame, which is defined by the values of XLength, YLength, XOffset and YOffset at field index NumRoI-1, the current frame is discarded and the next ROI (at field index 0) addresses the next input frame. For information about the latency of the operator, see Table 44, 'Individual Latencies of the Operators in Library Memory'.

All ROIs can be updated dynamically, if the ROI parameter types are set to dynamic. However, you can update the ROI only, if no image acquisition is running.

The operator supports two modes for specifying the maximum size of buffered image data frames saved in the DRAM, which you can configure via the parameter MaxFrameSizeMode:

  • For MaxFrameSizeMode = Auto, the DRAM buffers an image that has the size of the maximum image dimension, which is I.MaxImgWidth * I.MaxImgHeight.

    Images that are smaller than the maximum image dimension are buffered without modifications. However, reading out ROIs is always done on a buffered frame with the maximum possible image dimension. In this case, the missing pixels are filled with random data and should be treated as undefined in further processing.

  • For MaxFrameSizeMode = Custom, the maximum dimensions of the buffered input frame are defined by the parameters MaxFrameWidth and MaxFrameHeight. Even if the actual input frame is bigger, the data that exceeds the specified parameter dimensions is discarded (similar to an ROI without offsets). In that case, the DRAM reserves memory for the frame size MaxFrameWidth * MaxFrameHeight. You can use this mode to optimize memory usage and store more frames to DRAM.

In both cases, there is no check whether the defined ROIs are located in the input image. But no ROIs can be addressed that lie outside MaxFrameWidth and MaxFrameHeight. If the ROIs are located outside of the actual input image data, the operator reads random dummy values, but the ROI size is not altered.

If MaxFrameSizeMode, MaxFrameWidth and MaxFrameHeight are set to parameter type static or dynamic, the ROI parameters are also automatically set to the same parameter type and vice versa. Only dynamic parameters can be adjusted during runtime.

The operator supports empty frames, empty lines and varying line lengths on the input link.

[Note] High Number of ROIs and Timing

With an increasing number of maximum ROI definitions, defined by the MaxNumRoI parameter, the operators' BRAM consumption increases. This may lead to timing problems at high clock rates. For example, a MaxNumRoI of 65536 may not meet timing in a full design. If a high number of ROIs is needed, consider to use the 'FrameBufferMultiRoiDyn' operator instead, where ROI definitions are passed via input link and are not stored in BRAM.

Overflow Management with InfiniteSource

In the InfiniteSource mode, the data source of the FrameBufferMultiRoi is not stoppable and images might get lost or become corrupted. This happens either, because the RAM is full and it can't accept any further data, or the input bandwidth is too high for the shared memory interface. As soon as the operator reaches the overflow state, all incoming data is discarded. This leads to lost or corrupted frames. Corrupted images only occur, if the input bandwidth is too high, since the fill level is counted in full entities. A corrupted image occurs when part of the image has already been written to the RAM, but another part of the image is discarded because the input bandwidth is too high and the RAM can't accept the write data fast enough. As a result, reading a corrupted image leads to undefined output data. As soon as there is enough space in the RAM again and the shared memory interface allows data to be written to the RAM again, the operator recovers from the overflow and stops discarding the input data. Corrupted images are automatically terminated and subsequent images are not affected. The Overflow parameter indicates when an overflow occurred. With the OverflowClearMode parameter, you can define whether the Overflow is reset immediately after overflow recovery or whether you reset the overflow manually.

Operator Restrictions

  • The output mode of this operator is equivalent to the FreeRun mode of the ImageBufferMultiRoI operator and doesn't support the additional RoI Output Modes WaitAfterImage, and WaitAfterRoI.

  • The operator supports empty frames on input link I, but the output ROIs for that frame are filled with random dummy values.

  • If the input image is smaller than the frame that is written to DRAM, there is no check whether the ROI coordinates are inside the input image. In this case, the operator reads random dummy values from the memory.

  • The maximum link properties differ depending on the usage of the operator. Refer to the section Supported Link Format below for more information.

Bandwidth Optimization

For optimal performance, the used number of data bits should match the number provided in the RamDataWidth module parameter as closely as possible. The maximum bandwidth going through the operator is reached, if the product of bit width, kernel size and parallelism is equal to the internal RAM port width RamDataWidth. Note that you can increase the internal bit width by the using kernels, but a full kernel will still be addressed as a single pixel.

To enable the operator to handle long bursts of write data that exceed the available maximum bandwidth of the RAM, enable the WritePriority parameter. This is recommended with infinite data sources, such as cameras (InfiniteSource = ENABLED). If the WritePriority parameter is enabled, reading is inhibited when the available bandwidth of the RAM is exceeded due to a write burst. This allows temporarily more frequent write accesses to the RAM. If a camera delivers a burst of write data, it is expected that said burst is followed by a gap in the write data (otherwise the available RAM bandwidth is exceeded), which then can be used to temporarily ramp up the read bandwidth.

I/O Properties

Property Value
Operator Type M
Input Link I, image data input
Output Link O, data output

Supported Link Format

Link Parameter Input Link I Output Link O
Bit Width [1, 64]12 as I
Arithmetic {unsigned, signed} as I
Parallelism [1, RamDataWidth / BitWidth / KernelColumns / KernelRows]2 as I
Kernel Columns [1, RamDataWidth / BitWidth / Parallelism / KernelRows]2 as I
Kernel Rows [1, RamDataWidth / BitWidth / Parallelism / KernelColumns]2 as I
Img Protocol {VALT_IMAGE2D, VALT_LINE1D} as I
Color Format any as I
Color Flavor any as I
Max. Img Width 2^31 - 13 as I
Max. Img Height 2^RamAddressWidth3 as I

1

The range of the input bit width is:

  • For unsigned grey inputs: [1, 64]

  • For signed grey inputs: [2, 64]

  • For unsigned color inputs: [3, 63]

  • For signed color inputs: [6, 63].

The input bit width must not exceed the native RAM data width RamDataWidth.

2

The product of bit width, parallelism and kernel size must not exceed the native RAM data width:

3

The maximum image width and image height of the input link I differ depending on the operator use:

  • For MaxFrameSizeMode = Auto, the product of I.MaxImageWidth and I.MaxImageHeight must fit into the available RAM size:

  • For MaxFrameSizeMode = Custom, both I.MaxImageWidth and I.MaxImageHeight are limited to the maximum RAM size. The product can exceed the total RAM size as only the sub-image selected by the parameters MaxFrameWidth and MaxFrameHeight is written into the RAM:

    As I.Parallelism pixels are written per RAM vector, I.MaxImageWidth can exceed 2^RamAddressWidth but it can never be greater than 2^31 - 1:

    The sum of the bit widths of I.MaxImageWidth and I.MaxImageHeight must be smaller than 48 bit:

Parameters

RamDataWidth
Type static write parameter
Default N/A
Range Integer number

This parameter provides the maximum data width that can be used in the RAM.

RamAddressWidth
Type static write parameter
Default N/A
Range Integer number

This parameter provides the number of address bits that can be used. The number of available RAM slots is defined by 2^RamAddressWidth. The current RamAddressWidth depends on the hardware platform as well as on the current number of memory operators (i.e. operators that use a resource of type RAM) in the design.

FillLevel
Type dynamic read parameter
Default 0
Range [0%, 100%]

This parameter provides the fill level of the DRAM in percent.

MaxFrameCount
Type dynamic read parameter
Default 2^RamAddressWidth / (MaxFrameHeight * (MaxFrameWidth / I.Parallelism))
Range [1, 2^RamAddressWidth]

This parameter provides the maximum number of frames that currently fit into the memory. The maximum number of frames that fit into the memory depends on the parameters RamAddressWidth, MaxFrameHeight, I.Parallelism and MaxFrameWidth.

MaxFrameCount = 2^RamAddressWidth / (MaxFrameHeight * (MaxFrameWidth / I.Parallelism))

FrameCount
Type dynamic read parameter
Default 0
Range [0, MaxFrameCount]

This parameter provides the current number of frames in the memory.

InfiniteSource
Type static write parameter
Default DISABLED
Range {ENABLED, DISABLED}

The operator can be placed directly behind a camera operator in the design. In this case, the InfiniteSource parameter must be set to ENABLED. The operator will then perform active overflow management and make sure the operator can recover properly from overflows. The overflow can occur either when the data sink behind the operator stops or pauses the transmission and the buffer fill level reaches its maximum; or when the input bandwidth is too high so the write data can't be transferred to the external RAM. When InfiniteSource is set to Disabled, an inhibit signal is generated that stops the preceding operator from transferring data, if the buffer fill level or input bandwidth get too high.

The write prioritization is recommended for any operator that is used with the InfiniteSource. Consequently, it is recommended to set the WritePriority parameter to ENABLED, when the InfiniteSource parameter is set to ENABLED.

See 'Infinite Sources / Connecting Cameras' for more information.

WritePriority
Type static write parameter
Default DISABLED
Range {ENABLED, DISABLED}

The ' Shared Memory Concept ' concept usually distributes the bandwidth equally amongst all connected memory operators (i.e. all operators that use a resource of type RAM). If the WritePriority parameter is DISABLED, the FrameBufferMultiRoi operator assigns the same priority to reading and writing. By setting WritePriority to ENABLED, the FrameBufferMultiRoi operator prioritizes writing over reading, but only while the temporary memory data rate is higher than the available bandwidth. The temporary prioritization of write data leads to a temporary slow down of the read process. Consequently, the average bandwidth must not exceed the available bandwidth for the FrameBufferMultiRoi operator. The write prioritization is recommended for any operator that is used with the InfiniteSource parameter set to ENABLED. When using the write prioritization with a stoppable source, make sure that the write bandwidth isn't constantly high, otherwise reading from the FrameBufferMultiRoi operator is stopped until the buffer is full. Since the write prioritization is a configuration for an individual operator, the impact of the write prioritization decreases with each additional memory operator in the design.

Overflow
Type dynamic read parameter
Default 0
Range [0, 3]

This parameter indicates a buffer overflow. It's a 2-bit bitmap, where each bit indicates a different type of overflow. Bit 0 indicates a fill level overflow and bit 1 indicates a write bandwidth overflow. The display time of an Overflow depends on the selected OverflowClearMode.

OverflowClearMode
Type dynamic write parameter
Default AutoClear
Range {AutoClear, ManualClear, ClearAfterRead, ClearWithProcessReset}

OverflowClearMode determines how the Overflow parameter is cleared when the operator has recovered from an overflow. You can only reset the overflow status with this parameter, if the operator is not in overflow state anymore.

Clear modes:

  • AutoClear: When the operator recovers from an overflow, the Overflow parameter is reset automatically.

  • ManualClear: When the operator recovers from an overflow, the Overflow parameter still shows the overflow until it is manually reset by writing ManualClear into the OverflowClearMode parameter. In this mode, a process reset (e.g. acquisition stop) doesn't clear the Overflow parameter, which means the overflow is still visible after the acquisition has stopped.

  • ClearAfterRead: When the operator recovers from an overflow, the Overflow parameter still shows the overflow until the Overflow parameter is read or a process reset occurs (e.g. when the acquisition is stopped).

  • ClearWithProcessReset: When the operator recovers from an overflow, the Overflow parameter still shows the overflow until a reset occurs (e.g. when the acquisition is stopped).

MaxNumRoI
Type static write parameter
Default 1
Range [1, 65536]

This parameter defines the maximum number of ROIs the operator can store. A high number of ROIs can have a negative effect on timing for high clock rates.

NumRoI
Type dynamic/static write parameter
Default 1
Range [1, MaxNumRoI]

This parameter defines the number of ROIs actually used. The operator will read NumRoI ROIs from each input image and provide them on its output. If the parameter is set to static, the range is [1, 65536], and MaxNumRoI is disabled.

XOffset
Type dynamic/static read/write parameter
Default 0
Range [0, MaxFrameWidth - XLength]

This field parameter defines the array of X-coordinates of the ROIs most left column.

The step size is the parallelism.

Learn how to configure field parameters in 'Parameter Editing'.

XLength
Type dynamic/static read/write parameter
Default 1024
Range [0, MaxFrameWidth - XOffset]

This field parameter defines the array of ROI widths.

The step size is the parallelism.

Learn how to configure field parameters in 'Parameter Editing'.

YOffset
Type dynamic/static read/write parameter
Default 0
Range [0, MaxFrameHeight - YLength]

This field parameter defines the array of Y-coordinates of the ROIs first row.

Learn how to configure field parameters in 'Parameter Editing'.

YLength
Type dynamic/static read/write parameter
Default 1024
Range [0, MaxFrameHeight - YOffset]

This field parameter defines the array of ROI heights.

Learn how to configure field parameters in 'Parameter Editing'.

MaxFrameSizeMode
Type dynamic/static write parameter
Default Auto
Range {Auto, Custom}

This parameter defines whether the parameters MaxFrameWidth and MaxFrameHeight should automatically follow the maximum image dimension at input I (mode=Auto) or whether these parameters can be adjusted (mode=Custom). Custom mode enables setting a maximum frame dimension, which is customized for the application and may be much smaller than the limits defined by the input link. This way, more frames can be stored in the RAM. In mode Auto, the link properties (Max.Img Width and Max.Img Height) of I, must not define a maximum image that is greater than the number of available slots in the memory (2^RamAddressWidth). If the link properties Max.Img Width and Max.Img Height of I define a maximum image that is bigger than the available memory space and you are in mode Custom, the MaxFrameSizeMode parameter can't be edited anymore until you decrease the maximum image on the input link. This parameter can't be written when the acquisition is running. MaxFrameSizeMode has the same parameter type as MaxFrameWidth, MaxFrameHeight and the ROI parameters.

MaxFrameWidth
Type dynamic/static write parameter
Default 1024
Range [1, Max. Image Width at I]

This parameter sets the maximum image width for the current image processing configuration. The lines of input frames that exceed this limit are cut to MaxFrameWidth. Reducing this number below the maximum image width saves memory space and allows storing more frames. This parameter can only be edited when MaxFrameSizeMode is set to Custom. The product of MaxFrameHeight and MaxFrameWidth must not be greater than the number of available memory slots 2^RamAddressWidth multiplied with the input parallelism. This parameter can't be edited when the acquisition is running. MaxFrameWidth has the same parameter type as MaxFrameSizeMode, MaxFrameHeight and the ROI parameters.

MaxFrameHeight
Type dynamic/static write parameter
Default 1024
Range [1, Max. Image Height at I]

This parameter sets the maximum image height for the current image processing configuration. Input frames exceeding this height limit are cut to MaxFrameHeight. Reducing this number below the maximum image height saves memory space and allows to store more frames. This parameter can only be edited when MaxFrameSizeMode is set to Custom. The product of MaxFrameHeight and MaxFrameWidth must not be greater than the number of available memory slots 2^RamAddressWidth multiplied with the input parallelism. This parameter can't be edited when the acquisition is running. MaxFrameHeight has the same parameter type as MaxFrameSizeMode, MaxFrameWidth and the ROI parameters.

  Prev     Level Up     Next  
Content Top Home