Predict Object Poses (Sim2Pick)

Function

When the point cloud of the single object type is input, this Step will directly recognize the object from the input point cloud and output its pose.

Usage Scenario

This Step is used to recognize and locate objects of a single type. This Step is used to recognize and locate objects of a single type. The Step Extract 3D Points in 3D ROI is usually used before this Step to provide the pre-processed point cloud. The Step Show Point Clouds and Poses usually follows this Step to further process the outputs poses of objects in this Step.

For a high accuracy of the output object poses, the 3D ROI set in the preceding Extract 3D Points in 3D ROI Step should only include the target object and exclude the frame and bottom of the bin as much as possible.

Input and Output

predict object poses sim2pick input and output

Prerequisites

The prerequisites for using this Step are as follows.

Requirement of Graphics Card

To execute this Step successfully, it is recommended to use a graphics card between NVIDIA GTX 1660 Ti and RTX 3060 Ti (including GTX 1660 Ti and RTX 3060 Ti).

Requirement of Driver

To execute this Step, the minimum version requirement for the graphics driver is 472.50.

Obtain the Deep Learning Model File

Please contact the technical support of the Mech-Mind to obtain the deep learning model file.

Parameter Description

Server

Server IP

Description: This parameter is used to specify the IP address of the deep learning server. Keep the default parameter settings.

Default value: 127.0.0.1

Server Port (1–65535)

Description: This parameter is used to specify the port number of the deep learning server.

Default value: 60101

Tuning recommendation: All port numbers are valid, with a maximum value of 65535.

After opening the project, please wait for the deep learning server to start. If the deep learning server is started successfully, a message saying that Deep learning server started successfully at xxx will appear in the log panel, and then you can run the project.

Inference Setting

Inference Mode

Description: This parameter is used to select the inference mode for deep learning. Value list: GPU, CPU

  • GPU: Use GPU for deep learning model inference after optimizing the model. The inference speed is relatively fast. It takes about 10 to 30 minutes to optimize the model for the first time.

  • CPU: Use CPU for deep learning model inference, which will increase the inference time and reduce the recognition accuracy compared with GPU.

Default value: GPU

Tuning recommendation: GPU inference is faster than CPU. Please restart the deep learning server after switching the inference mode.

Model Configuration

Deep Learning Model File

Description: This parameter is used to specify the path (vis_xxx\resource\models\model.onnx) where the deep learning model file is stored. The model file is in the ONNX format.

  • After modifying the Deep Learning Model File, you need to restart the deep learning server to make the parameter modification take effect.

  • When you execute this Step for the first time, it takes some time to load the model, and the time depends on the performance of the GPU of your PC. Please wait for a while.

Select Model

Description: This parameter is used to select the point cloud model for matching.

Pose Optimization

Pose Correction

Description: This parameter is used to correct the initially calculated poses to more accurate poses.

Default value: Unselected.

Instruction: When there is no “3D Matching” related Steps in the subsequent Steps, it is recommended to enable this feature.

Object Clustering

Three cluster algorithms, MeanShift, RegionGrowing, and Dbscan are provided. The speeds for each algorithm are listed in the table below.

Algorithm

Speed

MeanShift

Low

RegionGrowing

High

Dbscan

Medium

When the Inference Mode is set to CPU, RegionGrowing cannot be used.

  • When the cluster method is MeanShift, you can adjust the following parameters.

    Cluster Radius

    Description: This parameter is used to define the distance between two points that can be grouped into the same cluster. The larger the value, the more distant points can be grouped into one cluster. When this parameter is relatively large, points that belong to two or more objects may fall into the same cluster; while this parameter is relatively small, points belong to the same object may be divided into several clusters.

    Default value: 10

    Value range: 1–150

    Minimum value: 2

    Tuning recommendation: Please adjust the parameter according to the actual situation.

    Min Point Cloud per Cluster

    Description: This parameter is used to set the lower limit of points in a cluster. When the number of points in a cluster is less than the lower limit, the cluster will be ignored.

    Default value: 30

    Value range: 1–50

    Tuning recommendation: Please adjust the parameter according to the actual situation.

  • When the cluster method is RegionGrowing, you can adjust the following parameters.

    Min Point Cloud per Cluster

    Description: This parameter is used to set the lower limit of points in a cluster. When the number of points in a cluster is less than the lower limit, the cluster will be ignored.

    Default value: 30

    Value range: 1–150

    Tuning recommendation: Please adjust the parameter according to the actual situation.

    Cluster Radius

    Description: This parameter is used to define the maximum distance between two points that can be grouped into the same cluster. The larger the value, the more distant points can be grouped into one cluster. When this parameter is relatively large, points that belong to two or more objects may fall into the same cluster; while this parameter is relatively small, points belong to the same object may be divided into several clusters.

    Default value: 0.5 mm

    Value range: 0.1–100

    Default value: 0.4 mm

    Tuning recommendation: Please adjust the parameter according to the actual situation.

    Number of Neighbours

    Description: This parameter is used to determine the number of neighboring points around a specified point within the Cluster Radius, and the neighboring points and that point will be grouped into the same cluster.

    Default value: 30

    Value range: 1–500

    Minimum value: 10

    Tuning recommendation: Please adjust the parameter according to the actual situation.

  • When the cluster method is Dbscan, you can adjust the following parameters.

    Cluster Radius

    Description: This parameter is used to define the distance between two points that can be grouped into the same cluster. The larger the value, the more distant points can be grouped into one cluster. When this parameter is relatively large, points that belong to two or more objects may fall into the same cluster; while this parameter is relatively small, points belong to the same object may be divided into several clusters.

    Default value: 10

    Value range: 1–150

    Minimum value: 2

    Tuning recommendation: Please adjust the parameter according to the actual situation.

    Min Point Cloud per Cluster

    Description: This parameter is used to set the lower limit of points in a cluster. When the number of points in a cluster is less than the lower limit, the cluster will be ignored.

    Value range: 1–50

    Default value: 30

    Tuning recommendation: Please adjust the parameter according to the actual situation.

Visualization

Enable

Description: This parameter facilitates the visualization of object instances, with each distinct object being represented by a unique color.

Default value: Unselected.

Tuning recommendation: Please set the parameter according to the actual requirement.

We Value Your Privacy

We use cookies to provide you with the best possible experience on our website. By continuing to use the site, you acknowledge that you agree to the use of cookies. If you decline, a single cookie will be used to ensure you're not tracked or remembered when you visit this website.