Calc Color Image for Highest Layer
Function
This Procedure defines the image area of the highest layer in the scene based on the depth map and then outputs the color image and the mask of the area. The components of the Procedure are shown below.
Usage Scenario
This Procedure can extract the color image segment of the highest layer for deep learning recognition.
Processing Workflow and Parameter Tuning
The workflow of this Procedure is shown below:
Get Mask for Highest Layer
-
Point clouds that exceed the limit should be filtered out for higher processing efficiency of subsequent Steps. Therefore, only needed areas of the depth map are retained, which means that ROI selection is required in the original depth map.
Double-click to enter this Procedure. Click the Set 3D ROI button in the Step Parameters panel of Invalidate Depth Pixels Outside 3D ROI and select 3D ROI in the pop-up Set ROI window.
The following show the point clouds before and after 3D ROI selection. The image on the left is the original depth map, containing many irrelevant point clouds; the image on the right is the depth map after 3D ROI selection, which no longer contains point clouds of regions outside the selected ROI.
-
Segment Depth Map segments the depth map within the ROI based on each pixel’s depth and generates the corresponding mask(s).
In the Step Parameters Panel of Segment Depth Map, click the Set 2D ROI button and select 2D ROI in the pop-up Set ROI window.
Then, set Max Height Diff between Neighbouring Pixels to determine whether the neighboring pixels belong to the same segment; set Min Segment Area to determine the valid segment.
As shown below, the image on the left is the depth map before segmentation, and that on the right is the depth image after segmentation.
-
Get Highest Layer Regions in Depth Map obtains the highest area(s) in a depth map based on the masks and masks’ depth information output from Segment Depth Map and generates the mask areas of the closest valid region to the camera under proper Layer Height. There could be one or multiple areas that satisfy this condition.
Morphological Transformations
-
Merge Mask Images merges the masks input from Get Highest Layer Regions in Depth Map if the input contains more than one mask to facilitate the morphological transformations later. The following image shows the mask of the highest layer after merging.
-
Morphological Transformations dilates the mask(s) to facilitate recognition or picking later (DilateOperator is selected).
Get Color image Segment under the Mask
-
Apply Masks to Image obtains the color image segment(s) under the transformed mask(s), as shown below.
-
Unpack Data obtains the color image of the first index in the list of image segments under masks.
