Predict Object Poses (Sim2Pick)
Function
When the point cloud of the single object type is input, this Step will directly recognize the object from the input point cloud and output its pose.
Usage Scenario
This Step is used to recognize and locate objects of a single type. This Step is used to recognize and locate objects of a single type. The Step Extract 3D Points in 3D ROI is usually used before this Step to provide the pre-processed point cloud. The Step Show Point Clouds and Poses usually follows this Step to further process the outputs poses of objects in this Step.
|
For a high accuracy of the output object poses, the 3D ROI set in the preceding Extract 3D Points in 3D ROI Step should only include the target object and exclude the frame and bottom of the bin as much as possible. |
Prerequisites
The prerequisites for using this Step are as follows.
Requirement of Graphics Card
To execute this Step successfully, it is recommended to use a graphics card between NVIDIA GTX 1660 Ti and RTX 3060 Ti (including GTX 1660 Ti and RTX 3060 Ti).
Parameter Description
Server
- Server IP
-
Description: This parameter is used to specify the IP address of the deep learning server. Keep the default parameter settings.
Default value: 127.0.0.1
- Server Port (1–65535)
-
Description: This parameter is used to specify the port number of the deep learning server.
Default value: 60101
Tuning recommendation: All port numbers are valid, with a maximum value of 65535.
|
After opening the project, please wait for the deep learning server to start. If the deep learning server is started successfully, a message saying that Deep learning server started successfully at xxx will appear in the log panel, and then you can run the project. |
Inference Setting
- Inference Mode
-
Description: This parameter is used to select the inference mode for deep learning. Value list: GPU, CPU
-
GPU: Use GPU for deep learning model inference after optimizing the model. The inference speed is relatively fast. It takes about 10 to 30 minutes to optimize the model for the first time.
-
CPU: Use CPU for deep learning model inference, which will increase the inference time and reduce the recognition accuracy compared with GPU.
Default value: GPU
Tuning recommendation: GPU inference is faster than CPU. Please restart the deep learning server after switching the inference mode.
Model Configuration
-
- Deep Learning Model File
-
Description: This parameter is used to specify the path (vis_xxx\resource\models\model.onnx) where the deep learning model file is stored. The model file is in the ONNX format.
|
- Select Model
-
Description: This parameter is used to select the point cloud model for matching.
Pose Optimization
- Pose Correction
-
Description: This parameter is used to correct the initially calculated poses to more accurate poses.
Default value: Unselected.
Instruction: When there is no “3D Matching” related Steps in the subsequent Steps, it is recommended to enable this feature.
Object Clustering
Three cluster algorithms, MeanShift, RegionGrowing, and Dbscan are provided. The speeds for each algorithm are listed in the table below.
Algorithm |
Speed |
MeanShift |
Low |
RegionGrowing |
High |
Dbscan |
Medium |
|
When the Inference Mode is set to CPU, RegionGrowing cannot be used. |
-
When the cluster method is MeanShift, you can adjust the following parameters.
- Cluster Radius
-
Description: This parameter is used to define the distance between two points that can be grouped into the same cluster. The larger the value, the more distant points can be grouped into one cluster. When this parameter is relatively large, points that belong to two or more objects may fall into the same cluster; while this parameter is relatively small, points belong to the same object may be divided into several clusters.
Default value: 10
Value range: 1–150
Minimum value: 2
Tuning recommendation: Please adjust the parameter according to the actual situation.
- Min Point Cloud per Cluster
-
Description: This parameter is used to set the lower limit of points in a cluster. When the number of points in a cluster is less than the lower limit, the cluster will be ignored.
Default value: 30
Value range: 1–50
Tuning recommendation: Please adjust the parameter according to the actual situation.
-
When the cluster method is RegionGrowing, you can adjust the following parameters.
- Min Point Cloud per Cluster
-
Description: This parameter is used to set the lower limit of points in a cluster. When the number of points in a cluster is less than the lower limit, the cluster will be ignored.
Default value: 30
Value range: 1–150
Tuning recommendation: Please adjust the parameter according to the actual situation.
- Cluster Radius
-
Description: This parameter is used to define the maximum distance between two points that can be grouped into the same cluster. The larger the value, the more distant points can be grouped into one cluster. When this parameter is relatively large, points that belong to two or more objects may fall into the same cluster; while this parameter is relatively small, points belong to the same object may be divided into several clusters.
Default value: 0.5 mm
Value range: 0.1–100
Default value: 0.4 mm
Tuning recommendation: Please adjust the parameter according to the actual situation.
- Number of Neighbours
-
Description: This parameter is used to determine the number of neighboring points around a specified point within the Cluster Radius, and the neighboring points and that point will be grouped into the same cluster.
Default value: 30
Value range: 1–500
Minimum value: 10
Tuning recommendation: Please adjust the parameter according to the actual situation.
-
When the cluster method is Dbscan, you can adjust the following parameters.
- Cluster Radius
-
Description: This parameter is used to define the distance between two points that can be grouped into the same cluster. The larger the value, the more distant points can be grouped into one cluster. When this parameter is relatively large, points that belong to two or more objects may fall into the same cluster; while this parameter is relatively small, points belong to the same object may be divided into several clusters.
Default value: 10
Value range: 1–150
Minimum value: 2
Tuning recommendation: Please adjust the parameter according to the actual situation.
- Min Point Cloud per Cluster
-
Description: This parameter is used to set the lower limit of points in a cluster. When the number of points in a cluster is less than the lower limit, the cluster will be ignored.
Value range: 1–50
Default value: 30
Tuning recommendation: Please adjust the parameter according to the actual situation.
Visualization
- Enable
-
Description: This parameter facilitates the visualization of object instances, with each distinct object being represented by a unique color.
Default value: Unselected.
Tuning recommendation: Please set the parameter according to the actual requirement.