Operator Library: Memory
This operator buffers input image data in the frame grabber RAM (DRAM) and reads out multiple dynamic regions of interest (ROI) for each buffered image. The coordinates of the ROIs to be read out are defined by using additional input links. The following image shows an overview of the operator's behavior for one input frame and four ROIs.
One VisualApplets resource of type RAM is required (see 'Allocation of Device Resources'). Multiple resources of type RAM use the same physical RAM with the ' Shared Memory Concept '. Documentation for how to use the shared memory is available in the Application Note: Shared Memory.
All ROIs are read sequentially as individual images: ROI 0, ROI 1, ROI 2,.. until ROI N-1. N is the number of ROIs to be read out for one input frame, that is the image size on the coordinate inputs. All possible rectangular regions are supported as long as the maximum input frame dimensions are not exceeded, i.e. a single pixel, a single line, a single column, a rectangular region, or the complete frame can be defined as an ROI.
Different ROIs do not need to have the same size.
Each ROI can be defined individually.
ROIs can overlap each other.
If the ROI image is empty, i.e. it doesn't contain any pixels, the operator provides an empty image on its output. An empty image contains no pixels.
A negative ROI width (XTopLeft > XBottomRight) or ROI height (YTopLeft > YBottomRight) also leads to an empty ROI on the output. The maximum amount of supported ROIs is determined by the maximum number of pixels per image on the XTopLeft input link: XTopLeft.MaxImageWidth * XTopLeft.MaxImageHeight .
ROI coordinates are provided on four separate coordinate input links as images containing the ROI coordinates: XTopLeft, YTopLeft, XBottomRight, YBottomRight. Each pixel in these images is treated by the operator as a valid ROI coordinate set. Thus, the size of the ROI input images define the number of output ROIs. EoLs (End of Lines) and varying line lengths can be used but are ignored as only valid pixels and EoFs (End of Frames) are evaluated for the ROI inputs of the operator. For each input image at input link I, a new ROI coordinate image set is required. The ROIs are defined by the X,Y coordinates for the top left corner and the X,Y coordinates for the bottom right corner.
Note that ROI X-coordinates are transformed to meet the X-granularity of the input link I defined by its parallelism, i.e., the operator FrameBufferMultiRoiDyn can only cut lines with the granularity of the parallelism. When addressing a pixel that isn't the first pixel of a parallel component for XTopLeft, or a pixel that isn't the last pixel of a parallel component for XBottomRight, the coordinates are rounded up or down and the resulting ROI still contains all pixels of the parallel component.
The following section describes an example to explain the operator's behavior:
In this example, the parallelism = 4 at the input link I.
Therefore, XTopLeft is rounded down to the first pixel of a parallel component: 0, 4, 8, 12, ... and XBottomRight is rounded up to the last pixel of a parallel component 3, 7, 11, 15, ...
For example, with the ROI X-coordinates 3 and 7, the first column of the ROI is column number 3 and the last column is column number 7 of the input image. The operator rounds down XTopLeft to 0; XBottomRight remains 7. Thus, the width becomes 8. The ROI height is not dependent on the parallelism and thus can be of any legal value.
![[Note]](../common/images/admon/note.png) |
Formula: |
|---|---|
|
XTopLeft* = floor(XTopLeft / Parallelism) * Parallelism XBottomRight* = ceil((XBottomRight + 1) / Parallelism) * Parallelism - 1 |
|
Examples: |
|---|---|
|
With a parallelism of 4 at the input link I:
|
All 4 ROI inputs are synchronous to each other, i.e., the images on these links must be of the same size and provided at the same time synchronously. This is always the case when they are sourced by the same M type module through an arbitrary network of O-type modules. For more information, see the section 'M-type Operators with Multiple Inputs'.
The ROI coordinates are asynchronous to the input link I. You can use the CreateBlankImage operator to generate ROI coordinates.
The operator provides ROI images at its output as soon as the correspondent input frame is completely buffered, i.e. the input frame's EoF (End of Frame) signal is received, and the first pixel on each of the ROI image definition input links XTopLeft, YTopLeft, XBottomRight, YBottomRight is received. If enough space is available in the DRAM, the next input frames can be written to the DRAM while ROIs are read. The RAM bandwidth is shared between writing and reading but reading can only start after the corresponding input frame is written. After reading the last ROI for the input frame, which is defined by the last pixel on each of the ROI image definition input links (i.e. the ROI images EoF (End of Frame) is received), the current frame is discarded and the next ROI images address the next input frame. For information about the latency of the operator, see Table 43, 'Individual Latencies of the Operators in Library Memory'.
The operator supports two modes for buffering image data to the DRAM, which you can configure via the parameter MaxFrameSizeMode:
-
For MaxFrameSizeMode = AUTO, the DRAM buffers an image that has the size of the maximum image dimension, which is I.MaxImgWidth * I.MaxImgHeight.
Images that are smaller than the maximum image dimension are buffered without modifications. However, reading out ROIs is always done on a buffered frame with the maximum possible image dimension. In this case, the missing frame pixels are filled with random data and should be treated in further processing as undefined.
-
For MaxFrameSizeMode = CUSTOM, the maximum dimensions of the buffered input frame are defined by the parameters MaxFrameWidth and MaxFrameHeight. Even if the actual input frame is bigger, the data that exceeds the specified parameter dimensions is discarded (similar to an ROI without offsets). In that case, the DRAM reserves memory for the frame size MaxFrameWidth * MaxFrameHeight.
In both cases, there is no check whether the input link ROI coordinates are located in the input image. As the coordinates can be set to any pixel of the maximum input frame. If they are located outside of the actual input image data, the operator reads random dummy values, but the ROI size is not altered.
The operator supports empty frames, empty lines and varying line lengths on all input links.
In the InfiniteSource mode, images might get lost or become corrupted. This happens either, because the RAM is full and it can't accept any further data, or the input bandwidth is too high for the shared memory interface. As soon as the operator reaches the overflow state, all incoming data is discarded. This leads to lost or corrupted frames. Corrupted images only occur, if the input bandwidth is too high, since the fill level is counted in full entities. A corrupted image occurs when part of the image has already been written to the RAM, but part of the image is discarded because the input bandwidth is too high and the RAM can't accept the write data fast enough. As a result, reading a corrupted image leads to undefined output data. As soon as there is enough space in the RAM again and the shared memory interface allows data to be written to the RAM again, the operator recovers from the overflow and stops discarding the input data. The Overflow parameter indicates when an overflow occurred. With the OverflowClearMode parameter, you can define whether the Overflow is reset immediately after overflow recovery or whether you reset the overflow manually.
-
The parallelism of the input link must be a power of 2.
-
The operator supports empty frames on input link I but the output ROIs for that frame are filled with random dummy values.
-
If the input image is smaller than the frame that is written to DRAM, there is no check whether the ROI coordinates are inside the input image. In this case, the operator reads random dummy values from the memory.
-
The maximum link properties differ depending on the usage of the operator. Refer to the section Supported Link Format below for more information.
For optimal performance, the used number of data bits should match as closely as possible the number provided in the module parameter RamDataWidth. The maximum bandwidth going through the operator is reached, if the product of bit width, kernel size and parallelism is equal to the internal RAM port width RamDataWidth. Note that the internal bit width can be increased by the usage of kernels, but a full kernel will still be addressed as a single pixel.
| Property | Value |
|---|---|
| Operator Type | M |
| Input Links | I, image data input XTopLeft, coordinate data input YTopLeft, coordinate data input XBottomRight, coordinate data input YBottomRight, coordinate data input |
| Output Link | O, data input |
Synchronous and Asynchronous Inputs
- The 4 ROI coordinate inputs XTopLeft, YTopLeft, XBottomRight and YBottomRight are synchronous to each other.
- The ROI coordinate inputs XTopLeft, YTopLeft, XBottomRight and YBottomRight are asynchronous to input link I.
|
|
The range of the input bit width is:
The input bit width must not exceed the native RAM data width RamDataWidth. |
|
|
The product of bit width, parallelism and kernel size must not exceed the native RAM data width. 
|
|
|
The bit width of the X-coordinate inputs is:  |
|
|
The bit width of the Y-coordinate inputs is:  |
|
|
The maximum image width and image height of the input link I differ depending on the operator use:
|
| RamDataWidth | |
|---|---|
| Type | static write parameter |
| Default | 0 |
| Range | Integer number |
|
This parameter provides the maximum data width that can be used in the RAM. |
|
| RamAddressWidth | |
|---|---|
| Type | static write parameter |
| Default | 0 |
| Range | Integer number |
|
This parameter provides the number of address bits that can be used. The number of available RAM slots is defined by 2^RamAddressWidth. The current RamAddressWidth depends on the hardware platform as well as on the current number of memory operators (i.e. operators that use a resource of type RAM) in the design. |
|
| MaxFrameSizeMode | |
|---|---|
| Type | dynamic write parameter |
| Default | Auto |
| Range | {Auto, Custom} |
|
This parameter defines whether the parameters MaxFrameWidth and MaxFrameHeight should automatically follow the max. image dimension at input I (mode=Auto) or whether these parameters can be adjusted (mode=Custom). Custom mode enables setting a maximum frame dimension, which is customized for the application and may be much smaller than the limits defined by the input link. This way, more frames can be stored in the RAM. In mode Auto, the link properties (Max.Img Width and Max.Img Height) of I, must not define a maximum image that is greater than the number of available slots in the memory (2^RamAddressWidth). If the link properties Max.Img Width and Max.Img Height of I define a maximum image that is bigger than the available memory space and you are in mode Custom, the MaxFrameSizeMode parameter can't be edited anymore until the maximum image on the input link is defined smaller. This parameter can't be written when the acquisition is running. |
|
| MaxFrameWidth | |
|---|---|
| Type | dynamic write parameter |
| Default | 1024 |
| Range | [1, Max. Image Width at I] |
|
This parameter sets the max. image width for the current image processing configuration. The lines of input frames that exceed this limit are cut to MaxFrameWidth. Reducing this number below the max. image width saves memory space and allows storing more frames. This parameter can only be edited when MaxFrameSizeMode is set to Custom. The product of MaxFrameHeight and MaxFrameWidth must not be greater than the number of available memory slots 2^RamAddressWidth multiplied with the input parallelism. This parameter can't be edited when the acquisition is running. |
|
| MaxFrameHeight | |
|---|---|
| Type | dynamic write parameter |
| Default | 1024 |
| Range | [1, Max. Image Height at I] |
|
This parameter sets the max. image height for the current image processing configuration. Input frames exceeding this height limit are cut to MaxFrameHeight. Reducing this number below the max. image height saves memory space and allows to store more frames. This parameter can only be edited when MaxFrameSizeMode is set to Custom. The product of MaxFrameHeight and MaxFrameWidth must not be greater than the number of available memory slots 2^RamAddressWidth multiplied with the input parallelism. This parameter cannot be edited when the acquisition is running. |
|
| FillLevel | |
|---|---|
| Type | dynamic read parameter |
| Default | 0 |
| Range | [0%, 100%] |
|
This parameter provides the fill level of the DRAM in percent. |
|
| MaxFrameCount | |
|---|---|
| Type | dynamic read parameter |
| Default | 2^RamAddressWidth / (MaxFrameHeight * (MaxFrameWidth / I.Parallelism)) |
| Range | [1, 2^RamAddressWidth] |
|
This parameter provides the maximum number of frames that currently fit into the memory. The maximum number of frames that fit into the memory depends on the parameters RamAddressWidth, MaxFrameHeight, I.Parallelism and MaxFrameWidth. MaxFrameCount = 2^RamAddressWidth / (MaxFrameHeight * (MaxFrameWidth / I.Parallelism)) |
|
| FrameCount | |
|---|---|
| Type | dynamic read parameter |
| Default | 0 |
| Range | [0, MaxFrameCount] |
|
This parameter provides the current number of frames in the memory. |
|
| InfiniteSource | |
|---|---|
| Type | static write parameter |
| Default | DISABLED |
| Range | {ENABLED, DISABLED} |
|
The operator can be placed directly behind a camera operator in the design. In this case, the InfiniteSource parameter must be set to ENABLED. The operator will then perform active overflow management and make sure the operator can properly recover from overflows. The overflow can occur either when the data sink behind the operator stops or pauses the transmission and the buffer fill level reaches its maximum or when the input bandwidth is too high so the write data can't be transferred to the external RAM. When InfiniteSource is set to DISABLED, an inhibit signal is generated that stops the proceeding operator from transferring data, if the buffer fill level or input bandwidth get too high. See 'Infinite Sources / Connecting Cameras' for more information. |
|
| Overflow | |
|---|---|
| Type | dynamic read parameter |
| Default | 0 |
| Range | [0, 3] |
|
This parameter indicates a buffer overflow. It's a 2-bit bitmap, where each bit indicates a different type of overflow. Bit 0 indicates a fill level overflow and bit 1 indicates a write bandwidth overflow. The display time of an Overflow depends on the selected OverflowClearMode. |
|
| OverflowClearMode | |
|---|---|
| Type | dynamic write parameter |
| Default | AutoClear |
| Range | {AutoClear, ManualClear, ClearAfterRead, ClearWithProcessReset} |
|
OverflowClearMode determines how the Overflow parameter is cleared when the operator has recovered from an overflow. You can only reset the overflow status with this parameter, if the operator is not in overflow state anymore. Clear modes:
|
|
Prev
