Use the Model

Using Instance Segmentation Model in Mech-Vision

Below is a typical piece of programming involving instance segmentation in Mech-Vision.

Figure 1. Programming involving instance segmentation in Mech-Vision

Step 1: Capture images from the camera.

Step 2: Scale the images within the 2D ROI to keep the ROI of the actual data consistent with that of the training data. Normally, please set Auto Scale under Color ROI Scaling in the parameter panel to True, as shown in Figure 2.

Figure 2. Auto scale

Step 3: Instance Segmentation outputs the instance segmentation results. Please determine whether the results meet the project requirements, whether the masks are complete, and whether there are omissions or mistakes.

After ensuring the paths of the Model File and the Configuration File are correct, please check the settings of the parameters marked with red boxes in Figure 3 based on the following rules.

Figure 3. Important parameters of the Instance Segmentation Step

Preload Settings -> Max Detected Objects (0 to 2000)

Please increase the value when the model needs to detect many objects at a time. If the number of objects that the model needs to detect at a time is small, and the project requires a short cycle time, please decrease the value of this parameter to optimize the cycle time.

Preload Settings -> Confidence Threshold (0 to 1.0)

Normally, the confidence threshold can be kept at the default value 0.7. When a detected object’s confidence is lower than this value, this object is considered not meeting the requirements for picking and its confidence will be displayed in red. Please tune this parameter based on actual project needs. For instance, when objects with lower confidences still have comparatively complete masks and meet the requirements for picking, the confidence threshold can be lowered to include more objects as candidates for picking.

Font Settings -> Customized Font Size

Normally the default value works well. When the objects to recognize are small, please decrease the font size to better display the results.

Visualization Settings -> Draw Instances on Image

When set to True, the recognized objects’ masks are displayed to show the effect of instance segmentation. Setting it to False during the actual project run can reduce the cycle time of the project.

Visualization Settings -> Method to Visualize Instances

• When set to Classes: objects’ masks are displayed in different colors based on object labels.

• When set to Instances: the mask of each object is displayed in a different color.

• When set to Threshold: the masks of objects whose confidences are above the threshold are displayed in green, otherwise, red.

Tip

If a project requires two instance segmentation models, please add two Instance Segmentation Steps, and set the model files and configuration files separately. In addition, please set the Server IP of the two Steps to 127.0.0.1:50052 and 127.0.0.1:50053, respectively; otherwise, port conflicts will occur.

Step 4: Recovery Scaled Image in 2D ROI restores the image to its original size.