Instance Segmentation

Function

Recognize target objects in an image and locate them at the pixel scale through deep learning.

Sample Scenario

You can use this Step to infer with model files from Mech-DL Kit and Mech-DLK. The results of inference are closely related to the data on which the model was trained. If you find that the inference results are not as expected, try the following methods in sequence:

  1. Please check after setting the region of interest and scaling whether the sizes of the target objects in the image are similar to those in the training data (if you are using supermodel and this method does not solve the problem, please contact the support team).

  2. Please check whether the objects, ambient light, background, etc. in the input color image are similar to those of the training data.

  3. Adjust the confidence threshold and use the first output port to check the visualized recognition results.

Input and Output

../../../../../_images/input_output_39.png

Parameters

Server & Model

Server Address
Default Value: 127.0.0.1:50052
Instruction: This attribute is to set the IP and port of the deep learning server, generally use the default value. If multiple instance segmentation steps are enabled, multiple corresponding deep learning servers need to be opened, and the port setting needs to correspond to the port of the deep learning server one-to-one.
Model File
Instruction: The absolute path or relative path of the deep learning model file is stored. The deep learning model file for instance segmentation is mainly a .pth format file, which can be obtained through the Mech-DLK training model.
Configuration File
Instruction: The absolute path or relative path of the deep learning configuration file is stored. The deep learning configuration file of the instance segmentation step is mainly a .py format file, which can be obtained through the Mech-DLK training model.

Preload Setting

Preload Model on Project Open
Default Value: False
List of Values: True,False
Instruction: When checked as True, the existing deep learning model will be automatically loaded after the project is started, which can greatly improve the speed of the first deep learning run after starting the project. It is recommended to enable the editing of this parameter after the project is debugged.

Max Detected Objects

Default Value: 100
Instruction： This attribute is used to set the maximum number of instances that can be displayed in the image, and the range is 0~500, which can be adjusted according to actual needs.

Acceptable Mean for Gray-scaled Input

This attribute judges whether the average gray value of the input image is in the set gray average value interval, and filters overexposed or over dark images to avoid incorrect instance segmentation, as shown in Figure 1 .
../../../../../_images/example_dim.png

Figure1 Filter Out Dark Images

Min
Default Value: 0
Instruction: If the average gray value of the picture is less than the value of this parameter, the picture will not be processed. The parameter can set the value range: 0~255.
Max
Default Value:255
Instruction: If the average gray value of the picture is greater than the value of this parameter, the picture will not be processed. The parameter can be set in the range of 0~255.

Confidence Settings

Confidence Threshold
Default Value: 0.7
Instruction: After calculation and processing through deep learning, instances with confidence greater than the threshold will be retained as the output result.
Example: The confidence threshold can be adjusted by first enabling the attribute Draw Instance on image and using the visualization output to obtain the confidence of each instance. Then adjust the confidence threshold accordingly, as shown in Figure 2 .
../../../../../_images/visualized_confidence.png

Figure2 Visualize Confidence Threshold

If the number of instances obtained by running at the current threshold is small or the result is not good enough, as shown in Figure 3 , the red data in the figure are instances with confidence lower than the threshold , which will not be output result. At this point, you can try to lower the threshold, as shown in Figure 4. After lowering the threshold, the number of instances as output results has increased significantly.
../../../../../_images/confidence09_result.png

Figure3 Instance Segmentation Results under High Threshold

../../../../../_images/confidence05_result.png

Figure4 Instance Segmentation Results under Low Threshold

If the result does not change much after adjusting the confidence threshold, the user can try to change the deep learning model, or check whether the object for instance segmentation is within the selected ROI.

Visualization Settings

Draw Instance on Image
Default Value: False
List of Values: False, True
Instruction: When set as True , the user can see the instance segmentation results in the visual output, but it takes longer to run. This parameter is recommended to be turned on during debugging, and turned off during actual operation.
Method to Visualize Instances
Default Value: Instance
List of Values: Threshold, Instances, Classes
Instruction: By selecting different parameters, the instance can be filled in different ways. Threshold: The filled color depends on the confidence of the instance. Instances with a confidence greater than the confidence threshold will be filled with green, otherwise they will be filled with red. Instances: Fill each instance with different colors. Classes: Fill the instances of the same label with the same color.
Show Bounding Boxes for Instances
Default Value: True
List of Values: True, False
Instruction: When set as True , the border of the divided instance will be displayed. This parameter needs to be enabled in the parameter Draw Instances on Image before it can take effect.

Font Settings

Customized Font Size
Default Value: False
List of Values: True,False
Instruction: When set as True , the font size displayed in the visual output can be customized to adjust. This parameter needs to be enabled in the parameter Draw Instances on Image before it can take effect.
Font Size
Default Value: 3.000
Instruction: This parameter needs to be set after the parameter Customized Font Size is enabled. Set according to actual needs. The range is 0-10. When it is set to 0, the font is not visible; when it is set to 10, the font is the largest.

Communication Settings

Notify Mech-Viz about Num of Detected Objects
Default Value: True
List of Values: True, False
Instruction: When set as True, the number of split instances will be sent to Mech-viz.

Filtering Settings

Filter Objects Near Image Border
Default Value: False
List of Values: False, True
Instruction: When set as True , the edge workpieces will be filtered out.
Width of Border
Default Value: 10 pixel
Instruction: This parameter is the allowable minimum distance between the left and right width of the workpiece from the edge, and the workpiece which is smaller than the set parameter value will be filtered out.
Height of Border
Default Value: 10 pixel
Instruction: This parameter is the minimum distance between the allowable workpiece in the upper and lower width from the edge, the workpiece which is smaller than the set parameter value will be filtered out.
Display Predefined ROI
Default Value: False
List of Values: True, False
Instruction: When set as True, the image edge size set by the attribute Width of Border and the attribute Height of Border will be displayed during the visual output.

Common Procedures

Procedure of Instance Segmentation