Predict Pick Points V2

Function

This Step recognizes the pickable objects based on the 2D images and depth maps and outputs the corresponding pick points.

Note

  • Before using this Step, please wait for the deep learning server to start. If the deep learning server is started successfully, a message saying that Deep learning server started successfully at xxx will appear in the log panel, and then you can run the Step.

  • When you run the Step for the first time, you should load a Picking Configuration File. It takes some time to load the file. Please wait with patience.

Usage Scenario

This Step is designed for piece picking in logistics, supermarket, and cables industry. This Step follows the Scale Image in 2D ROI Step to obtain the information of the scaled depth map, point cloud, and ROI.

Input and Output

../../../../../_images/input_and_output16.png

Parameter Description

Server

Server IP
Description: This parameter is used to set the IP address of the deep learning server.
Default value: 127.0.0.1
Tuning recommendation: Please set the parameter according to the actual requirement.
Server Port (1–65535)
Description: This parameter is used to set the port number of the deep learning server.
Default value: 60054
Value range: 60000–65535
Tuning recommendation: Please set the parameter according to the actual requirement.

Picking Configuration

Picking Configuration Folder Path
Tuning recommendation: Before you run the project, please load the Picking Configuration Folder first. Please contact Mech-Mind Technical Support to request the model files you need first.
Picking Configuration Folder:

  • We provide five types of picking configuration files used for logistics (semantic segmentation), logistics (object detection), supermarket, cables, and pharmaceutical industry, as shown in the table below.

    Usage Scenario

    Picking Configuration Folder Name

    Logistics (semantic segmentation)

    Logistics_Seg_RGBSuction

    Logistics (object detection)

    Logistics_OD_RGBSuction

    Supermarket

    Supermarket_Seg_RGBSuction

    Cables

    Cable_Seg_RGBGrasp

    Medicine Boxes

    MedicineBox_Instance_3DSize_RGBSuction

  • There are two JSON files and one model folder in the picking configuration folder. The deep learning model is stored in the model folder. The folder path should NOT contain the model folder, or else this Step cannot function properly. For example, a correct path can be D:/ConfigurationFiles/Cable_Seg_RGBGrasp.

Hint

If you are not sure about which type of deep learning model you should use, you can consult our Technical Support for some advice.

Logistics (semantic segmentation)
Please refer to Parameter Adjustment in the Logistics (Semantic Segmentation) Scenario.
Logistics (object detection)
Please refer to Parameter Adjustment in the Logistics (Object Detection) Scenario.
Supermarket
Please refer to Parameter Adjustment in the Supermarket Scenario.
Cables
Please refer to Parameter Adjustment in the Cables Scenario.
Medicine Boxes
Please refer to Parameter Adjustment in the Medicine Boxes Scenario.

Attention

It is recommended to use a GeForce GTX 10 Series graphics card with a memory of at least or above 4G when you use the model for the above scenarios. When you run this Step for the first time, the deep learning model will be optimized according to the hardware type and the one-time optimization process takes about 15 to 35 minutes.