Calc Color Image for Highest Layer

Function

This Procedure defines the image area of the highest layer in the scene based on the depth map and then outputs the color image and the mask of the area.

  • Input: depth map, color image

  • Output: the color image of the highest layer, the merged mask of the highest layer

Note: the highest layer may include many segments and the masks of those segments will be merged into one before outputting.

The graphical programming of this Procedure is shown in Figure 1.

../../../_images/calc_color_for_highest_layer_01.png

Figure 1. Graphical programming of Calc Color Image for Highest Layer

This Procedure consists of three sub-workflows:

  1. Get the mask of the highest layer.

  2. Morphological transformations.

  3. Get the color image segment under the mask.

Functions of Involved Steps

The functions of the Steps involved in this Procedure are as follows (please click on the links for detailed descriptions of the Steps):

  1. Invalidate Depth Pixels outside 3D ROI invalidates the pixels out of the ROI in a depth map and outputs the depth map within the ROI.

  2. Segment Depth Map segments the depth map within the ROI based on each pixel’s depth and generates the corresponding mask(s).

  3. Get Highest Layer Regions in Depth Map obtains the highest area(s) in a depth map based on the masks and masks’ depth information output from Segment Depth Map.

  4. Merge Mask Images merges the masks input from Get Highest Layer Regions in Depth Map if the input contains more than one mask, to facilitate the morphological transformations later.

  5. Morphological Transformations dilates the mask(s) to facilitate recognition or picking later.

  6. Apply Masks to Image obtains the color image segment(s) under the transformed mask(s).

  7. Unpack Data obtains the color image of the first index in the list of image segments under masks.

Sample Scenario

This Procedure can extract the color image segment of the highest layer for deep learning recognition.

  • The parameter tuning is shown in Figure 2.

../../../_images/calc_color_for_highest_layer_03.png

Figure 2. Parameter tuning

  1. Click icon_click to pop up the window of setting the point cloud ROI.

  2. Click icon_click to pop up the window of setting the depth map ROI.

  3. Set parameters to appropriate values to obtain the highest layer in the depth map.

  4. Set parameters to appropriate values to obtain the mask of the highest layer.

Note

Please see the corresponding descriptions in :ref: steps_guide for details about each Step.

  • Figure 3 is a comparison of the input and the output:

../../../_images/calc_color_for_highest_layer_02.png

Figure 3. Comparison of the input and the output

  1. The input depth map.

  2. The input color image.

  3. The obtained color image of the highest layer.

  4. The merged mask of the highest layer.

Comparison of Similar Procedures