Calc Color Image for Highest Layer

You are viewing an old version of the documentation. You can switch to the documentation of the latest version by clicking the top-right corner of the page.

Function

This Procedure defines the image area of the highest layer in the scene based on the depth map and then outputs the color image and the mask of the area. The components of the Procedure are shown below.

calc color for highest layer calc color for highest layer

Usage Scenario

This Procedure can extract the color image segment of the highest layer for deep learning recognition.

Processing Workflow and Parameter Tuning

The workflow of this Procedure is shown below:

calc color for highest layer calc color for highest layer frame

Get Mask for Highest Layer

  1. Point clouds that exceed the limit should be filtered out for higher processing efficiency of subsequent Steps. Therefore, only needed areas of the depth map are retained, which means that ROI selection is required in the original depth map.

    Double-click to enter this Procedure. Click the Set 3D ROI button in the Step Parameters panel of Invalidate Depth Pixels Outside 3D ROI and select 3D ROI in the pop-up Set ROI window.

    The following show the point clouds before and after 3D ROI selection. The image on the left is the original depth map, containing many irrelevant point clouds; the image on the right is the depth map after 3D ROI selection, which no longer contains point clouds of regions outside the selected ROI.

    calc color for highest layer invalidate depth pixels outside 3d roi
  2. Segment Depth Map segments the depth map within the ROI based on each pixel’s depth and generates the corresponding mask(s).

    In the Step Parameters Panel of Segment Depth Map, click the Set 2D ROI button and select 2D ROI in the pop-up Set ROI window.

    Then, set Max Height Diff between Neighbouring Pixels to determine whether the neighboring pixels belong to the same segment; set Min Segment Area to determine the valid segment.

    As shown below, the image on the left is the depth map before segmentation, and that on the right is the depth image after segmentation.

    calc color for highest layer segment depth image
  3. Get Highest Layer Regions in Depth Map obtains the highest area(s) in a depth map based on the masks and masks’ depth information output from Segment Depth Map and generates the mask areas of the closest valid region to the camera under proper Layer Height. There could be one or multiple areas that satisfy this condition.

    calc color for highest layer get highest area in depth image

Morphological Transformations

  1. Merge Mask Images merges the masks input from Get Highest Layer Regions in Depth Map if the input contains more than one mask to facilitate the morphological transformations later. The following image shows the mask of the highest layer after merging.

    calc color for highest layer merge mask images
  2. Morphological Transformations dilates the mask(s) to facilitate recognition or picking later (DilateOperator is selected).

    calc color for highest layer morphological transformations

Get Color image Segment under the Mask

  1. Apply Masks to Image obtains the color image segment(s) under the transformed mask(s), as shown below.

    calc color for highest layer apply masks to image
  2. Unpack Data obtains the color image of the first index in the list of image segments under masks.

    calc color for highest layer unpack data

We Value Your Privacy

We use cookies to provide you with the best possible experience on our website. By continuing to use the site, you acknowledge that you agree to the use of cookies. If you decline, a single cookie will be used to ensure you're not tracked or remembered when you visit this website.