Procedure of Instance Segmentation

Detecting the location of target objects in an image by instance segmentation provides segmentation masks and target object classes for point cloud processing and other operations.

../../../_images/Int_seg_step_com.png

Figure 1 Combination Step of Instance Segmentation

Process Description

As shown in Figure 1, the flow of the Instance Segmentation procedure is as follows.

  1. Scale Image in 2D ROI step extracts and scales the ROI region of the image.

  2. Instance Segmentation step performs the recognition of 2D images and obtains the recognition result (category + mask).

  3. The Recover Scaled Images in 2D ROI step scales the result of the Instance Segmentation step back to the original input image size to get the result of the instance segmentation on the original image.

Example implementation

Input

../../../_images/Ins_seg_origin.png

Figure 2 RGB Original Image

Identification Results

../../../_images/Ins_seg_result.png

Figure 3 Image of Recognition Results

Precautions

  1. The result of instance segmentation is largely affected by the ROI area and zoom ratio. If the user wants to obtain a more ideal instance segmentation result, the user must adjust the ROI and zoom ratio. For specific adjustment methods, refer to Deep Learning Deployment ROI Settings.

  2. The maximum number of recognition of Instance segmentation: its modified value is too large (that is, the actual detection amount is large) . There may be insufficient GPU resources, and GPUs with higher computing power can be replaced according to specific project requirements.

    ../../../_images/Gpu_error.png

    Figure4 Mech-Vision GPU Resource is Insufficient and Error Image is Reported

Deep Learning Deployment ROI Settings

ROI area and zoom ratio settings are divided into two situations. When the collected image is the same as the collected scene environment, the user can use the color image ROI file or parameter settings. When it is different from the scene of the collection environment, the user needs to calculate the zoom ratio by himself.

Consistent with the collection site environment

The following two methods are recommended to use the first one. When using the color image ROI file to set and zoom the ROI area, please refer to the method of obtaining the color image ROI file.

Use color image ROI file

../../../_images/Roi_set_file.png

Figure5 Image ROI Zoom File Settings

  1. Select ROIByFile in Color Map ROI Settings .

  2. The obtained color ROI file.

  3. Check Auto zoom .

Use parameters

../../../_images/Roi_set_pro.png

Figure6 Image ROI Scaling Parameter Settings

  1. Select ROIByParam in Color Map ROI Settings .

  2. Adjust the value of startX and startY to determine the starting point in the upper left corner.

  3. When changing the size of the ROI area, try to ensure that the aspect ratio of the ROI is consistent with that during training. For example, during training, width is 1000, height is 1200, Then the new width is 1000 * some ratio , and the new height is 1200 * some ratio .

  4. Check Auto zoom .

Inconsistent with the collection site environment

../../../_images/Excmple_calculate.png

Figure7 Ratio Calculation

  1. Use the Sample image in the model folder and the images collected on site to select the same feature of the object to be detected, such as Figure 8.In the calculation example, the selected feature is the AB edge.

  2. Measure the pixel of this feature, and get the pixel length in the Sample image as 200, and the pixel length in the collected image as 500 .

    ../../../_images/Roi_set.png

    Figure8 Image ROI Zoom Setting

  3. Uncheck Auto zoom .

  4. Calculate the ratio of the two, that is, 200/500, and fill in the value into the custom zoom factor.