Calc Color Image for Highest Layer

You are viewing an old version of the documentation. You can switch to the documentation of the latest version by clicking the top-right corner of the page.

Function

This Procedure defines the image area of the highest layer in the scene based on the depth map and then outputs the color image and the mask of the area. The components of the Procedure are shown below.

calc color for highest layer calc color for highest layer

Usage Scenario

This Procedure can extract the color image segment of the highest layer for deep learning recognition.

Processing Workflow and Parameter Tuning

The workflow of this Procedure is shown below:

calc color for highest layer calc color for highest layer frame

Get Mask for Highest Layer

Point clouds that exceed the limit should be filtered out for higher processing efficiency of subsequent Steps. Therefore, only needed areas of the depth map are retained, which means that ROI selection is required in the original depth map.

Double-click to enter this Procedure. Click the Set 3D ROI button in the Step Parameters panel of Invalidate Depth Pixels Outside 3D ROI and select 3D ROI in the pop-up Set ROI window.

The following show the point clouds before and after 3D ROI selection. The image on the left is the original depth map, containing many irrelevant point clouds; the image on the right is the depth map after 3D ROI selection, which no longer contains point clouds of regions outside the selected ROI.
Segment Depth Map segments the depth map within the ROI based on each pixel’s depth and generates the corresponding mask(s).

In the Step Parameters Panel of Segment Depth Map, click the Set 2D ROI button and select 2D ROI in the pop-up Set ROI window.

Then, set Max Height Diff between Neighbouring Pixels to determine whether the neighboring pixels belong to the same segment; set Min Segment Area to determine the valid segment.

As shown below, the image on the left is the depth map before segmentation, and that on the right is the depth image after segmentation.
Get Highest Layer Regions in Depth Map obtains the highest area(s) in a depth map based on the masks and masks’ depth information output from Segment Depth Map and generates the mask areas of the closest valid region to the camera under proper Layer Height. There could be one or multiple areas that satisfy this condition.

Morphological Transformations

Merge Mask Images merges the masks input from Get Highest Layer Regions in Depth Map if the input contains more than one mask to facilitate the morphological transformations later. The following image shows the mask of the highest layer after merging.
Morphological Transformations dilates the mask(s) to facilitate recognition or picking later (DilateOperator is selected).

Get Color image Segment under the Mask

Apply Masks to Image obtains the color image segment(s) under the transformed mask(s), as shown below.
Unpack Data obtains the color image of the first index in the list of image segments under masks.