Predict Object Poses (Sim2Pick)
Function
When the point cloud of the single object type is input, this Step will directly recognize the object from the input point cloud and output its pose.
Usage Scenario
This Step is used to recognize and locate objects of a single type. This Step is used to recognize and locate objects of a single type. The Step Extract 3D Points in 3D ROI is usually used before this Step to provide the pre-processed point cloud. The Step Show Point Clouds and Poses usually follows this Step to further process the outputs poses of objects in this Step.
|
For a high accuracy of the output object poses, the 3D ROI set in the preceding Extract 3D Points in 3D ROI Step should only include the target object and exclude the frame and bottom of the bin as much as possible. |
Prerequisites
The prerequisites for using this Step are as follows.
Requirement of Graphics Card
To execute this Step successfully, it is recommended to use a graphics card between NVIDIA GTX 1650 Ti and RTX 3060 Ti (including GTX 1650 Ti and RTX 3060 Ti).
Requirement of Driver
To execute this Step, the minimum version requirement for the graphics driver is 472.50.
Obtain the Deep Learning Model File
Use Mech-DLK Sim2Pick to train deep learning model. For more specific operation, please refer to Use Mech-DLK Sim2Pick to Train Model.
After obtaining deep learning model, please refer to Use Deep Learning Model for the model using “Predict Object Poses (Sim2Pick)” Step.
Parameters
Server
- Server IP
-
Description: This parameter is used to specify the IP address of the deep learning server. Keep the default parameter settings.
Default value: 127.0.0.1
- Server Port (1–65535)
-
Description: This parameter is used to specify the port number of the deep learning server.
Default value: 60101
Tuning recommendation: All port numbers are valid, with a maximum value of 65535.
|
After opening the project, please wait for the deep learning server to start. If the deep learning server is started successfully, a message saying that Deep learning server started successfully at xxx will appear in the log panel, and then you can run the project. |
Inference Setting
- Inference Mode
-
Description: This parameter is used to select the inference mode for deep learning. Value list: GPU, CPU
-
GPU: Use GPU for deep learning model inference after optimizing the model. The inference speed is relatively fast. It takes about 10 to 30 minutes to optimize the model for the first time.
-
CPU: Use CPU for deep learning model inference, which will increase the inference time and reduce the recognition accuracy compared with GPU.
Default value: GPU
Tuning recommendation: GPU inference is faster than CPU. Please restart the deep learning server after switching the inference mode.
Model Configuration
-
- Deep Learning Model File
-
Description: This parameter is used to specify the path (vis_xxx\resource\models) where the deep learning model file is stored. The model file is in the ONNX format.
|
Pose Optimization
This Step provides two methods to optimize poses:
-
Adjust poses based on Geometric Center File: Based on the selected geometric center file, adjust the object poses output by deep learning to more pickable poses.
-
Pose Correction: Based on the selected object point cloud file, further optimize the object poses for precise correction.
- Geometric Center File
-
Description: This parameter is used to specify the path where the geometric center file is stored.
- Pose Correction
-
Description: This parameter is used to correct the initially calculated poses to more accurate poses.
Default value: Unselected.
Tuning recommendation: Once this parameter is selected, you need set the path for the Object Ply File. Since the point count of the cloud will affect the pose correction, it is recommended to input a PLY file with fewer points. When there is no “3D Matching” related Steps in the subsequent Steps, it is recommended to enable this feature.
- Object Ply File
-
Description: This parameter is used to specify the path where the object point cloud file is stored. Point cloud files in PLY or STL format are supported.
Object Clustering
Three cluster algorithms, MeanShift, RegionGrowing, and Dbscan are provided. The speeds for each algorithm are listed in the table below.
Algorithm |
Speed |
MeanShift |
Low |
RegionGrowing |
High |
Dbscan |
Medium |
|
When the Inference Mode is set to CPU, RegionGrowing cannot be used. |
- Cluster Method
-
Description: This parameter is used to select a cluster algorithm.
Value list: MeanShift, RegionGrowing, Dbscan
When the cluster method is MeanShift, you can adjust the following parameters.
- Cluster Bandwidth
-
Description: This parameter is used to define the maximum distance between two points that can be grouped into the same cluster. The larger the value, the more distant points can be grouped into one cluster. When this parameter is relatively large, points that belong to two or more objects may fall into the same cluster; while this parameter is relatively small, points belong to the same object may be divided into several clusters.
Default value: 10
Value range: 1–100
Minimum value: 2
Tuning recommendation: Please adjust the parameter according to the actual situation.
- Min Point Count per Cluster
-
Description: This parameter is used to set the lower limit of points in a cluster. When the number of points in a cluster is less than the lower limit, the cluster will be ignored.
Value range: 1–50
Default value: 30
Tuning recommendation: Please adjust the parameter according to the actual situation.
When the cluster method is RegionGrowing, you can adjust the following parameters.
- Min Point Count per Cluster
-
Description: This parameter is used to set the lower limit of points in a cluster. When the number of points in a cluster is less than the lower limit, the cluster will be ignored.
Default value: 30
Value range: 1–50
Tuning recommendation: Please adjust the parameter according to the actual situation.
- Cluster Radius
-
Description: This parameter is used to define the maximum distance between two points that can be grouped into the same cluster. The larger the value, the more distant points can be grouped into one cluster. When this parameter is relatively large, points that belong to two or more objects may fall into the same cluster; while this parameter is relatively small, points belong to the same object may be divided into several clusters.
Default value: 0.5 mm
Value range: 0.1–20
Default value: 0.4 mm
Tuning recommendation: Please adjust the parameter according to the actual situation.
- Number of Neighbours
-
Description: This parameter is used to determine the number of neighboring points around a specified point within the Cluster Radius, and the neighboring points and that point will be grouped into the same cluster.
Default value: 30
Value range: 1–200
Minimum value: 10
Tuning recommendation: Please adjust the parameter according to the actual situation.
When the cluster method is Dbscan, you can adjust the following parameters.
- Cluster Bandwidth
-
Description: This parameter is used to define the maximum distance between two points that can be grouped into the same cluster. The larger the value, the more distant points can be grouped into one cluster. When this parameter is relatively large, points that belong to two or more objects may fall into the same cluster; while this parameter is relatively small, points belong to the same object may be divided into several clusters.
Default value: 10
Value range: 1–100
Minimum value: 2
Tuning recommendation: Please adjust the parameter according to the actual situation.
- Min Point Count per Cluster
-
Description: This parameter is used to set the lower limit of points in a cluster. When the number of points in a cluster is less than the lower limit, the cluster will be ignored.
Value range: 1–50
Default value: 30
Tuning recommendation: Please adjust the parameter according to the actual situation.
Visualization
- Enable
-
Description: This parameter facilitates the visualization of object instances, with each distinct object being represented by a unique color.
Default value: Unselected.
Tuning recommendation: Please set the parameter according to the actual requirement.