3D Fine Matching

You are currently viewing the documentation for the latest version (2.2.0). To access a different version, click the "Switch version" button located in the upper-right corner of the page.

■ If you are not sure which version of the product you are currently using, please feel free to contact Mech-Mind Technical Support.

Function

This Step can accurately match the point cloud model with the original point cloud and output more accurate poses of target objects.

3d fine matching function description

Usage Scenario

This Step performs fine matching based on the matching results of 3D Coarse Matching V2 and outputs more accurate object poses that can be used as pick points.

Input and Output

Input

Input port Data type Description

Point Clouds (with normals)

PointCloud/XYZNormal[]

List of point clouds to match (usually the same as the point cloud list used in the Step "3D Coarse Matching V2").

Initial Poses

Pose[][]

Candidate pose list (usually obtained from Step "3D Coarse Matching V2").

Object Labels

String[]

The target object labels (optional).

Object Confidences

Number[]

The target object confidence (optional).

Output

Output port Data type Description

Object Center Points

Pose[]

List of object center points.

Object Point Clouds (with normals)

PointCloud/XYZNormal[]

The model point cloud list generated by the matching results.

Model Transformations

Pose[]

Model-to-scene transformation reference pose list.

Object Labels

String[]

Labels of the target objects.

Object Confidences

Number[]

Confidence values of the target objects.

Pose Confidence Values

Number[]

List of confidence values of the corresponding object poses.

Parameter Description

Parameter Description

Parameter Tuning Level

Parameter description: This parameter is used to select the Parameter Tuning Level for the Step.
Value list: Basic and Advanced.
Default value: Basic

Basic Parameter Tuning Level

This section introduces the parameters in the Basic parameter tuning level, and these parameters are also available in the Advanced Parameter Tuning Level.

Model Settings

Parameter Description

Select Target Object

Instructions: Click the drop-down button to select and apply a point cloud template and its corresponding geometric centers.

Matching Method Settings

Parameter Description

Matching Mode

Parameter description: This parameter is used to select between Edge Matching and Surface Matching modes.

Value list: Surface matching, Edge matching.

Default value: Surface matching.

Tuning recommendation: When the surface of the object has obvious recognizable features (such as crankshafts, rotors, and steel rods), it is recommended to use surface matching. You should create a point cloud model that represents the surface features of the object. When the object is relatively flat and shows clear and regular edge features under the camera (such as panels, track pads, connecting rods, and brake discs), it is recommended to use edge matching. You should create a point cloud model that represents the edge features of the object.

The value of Match Mode must be consistent with the model type (Surface model or Edge model) specified by the Select Target Object parameter. For example, if the matching mode is surface matching, the model specified by the Select Target Object parameter must also be a surface model.

Matching Method

Parameter Description Illustration

Matching Method

Description: This parameter is used to select the method for 3D fine matching.
Default value: GMM
Value list: GMM and nearest-neighbor.
Instruction: This parameter is used to select the local matching method to match the point cloud model with the scene point cloud. The point cloud model will move and rotate according to the set range to match the point cloud in the scene obtained in the previous step match.
Example: In most scenarios, GMM has better anti-interference ability and higher matching speed, so it is generally recommended; in rare cases when GMM cannot meet the demand, nearest-neighbor can be selected.

3d fine matching nearest neighbor
Settings under GMM
Parameter Description Illustration

Matching Mode

Default value: Standard

Value list: HighSpeed, Standard, and HighPrecision.

  • HighSpeed: high speed but with less matching accuracy.

  • Standard: a relatively stable mode.

  • HighPrecision: high matching accuracy with low speed.

Number of Iterations

Default value: 30
Instruction: Iteration refers to iteratively running a specific instruction until the conditions are met. The number of iterations refers to the number of repetitions of instruction running in this process. The parameter set here is the upper limit of the number of iterations. The larger the value is, the greater the number of times of matching calculations will be, which means a longer running time and higher accuracy.

Standard Deviation

Default value: 3.000 mm
Instruction: The value here needs to match with the initial pose deviation. When the initial pose deviation is large after the processing of the previous 3D Coarse Matching V2 Step, i.e., the deviation between the scene point cloud and the point cloud model is great, a larger Standard Deviation needs to be set accordingly, which will result in a faster processing speed.
Example: When the set value is larger than the initial pose deviation, there will be a great result deviation. As shown in the figure below, the Standard Deviation on the right is 40.000 mm, far greater than the initial pose deviation. You can see that the matching result on the right with an overly large standard deviation is less accurate. Normally, this value will be no greater than 10 mm.
CAUTION: For smaller workpieces, it is recommended to adjust the standard deviation to a smaller value for more accurate matching results.

3d fine matching standard deviation gmm

Standard Deviation Update Step Number

Default value: 3
Instruction: During the matching process, the Standard Deviation gradually decreases until it reaches the value of Minimum Standard Deviation. The number of times that the standard deviation decreases in this process is the value of this parameter. For some cases with large standard deviation, the more times of the decrease, the more stable the entire decrease process. Under normal circumstances, this parameter does not need to be adjusted.

Speed Up on Large Object Quantities

Default value: Disabled.
Instruction: It is recommended to enable this option if there is a large number of objects in the scene. If this option is enabled, the Step will perform faster when there is a large number of objects. However, if the number of objects is small, the processing speed of the Step may be slowed down.
Settings under Nearest-Neighbor
Iteration Settings
Parameter Description

Nearest Point Search Radius

Default value: 10.000 mm
Value range: [1.000, +∞)
Instruction: This parameter is used to adjust the search radius of the nearest point (in mm), and the value setting should correspond to the input initial pose deviation. When the input initial pose deviation is large (the deviation between the scene point cloud and the point cloud model is large), this parameter needs to be increased; when the initial pose deviation is very small, this parameter can be adjusted to a small value. The minimum value that can be set for this parameter is 1.000 mm. If the radius is less than this lower limit, some sparsely distributed points may not find the closest point, which could affect the output result.

MSE Threshold

Default value: 0.001
Instruction: This parameter (Mean Square Error Threshold) is used to adjust the upper limit of the mean square error. This value is recorded after each iteration and used for comparison in subsequent iteration steps. When the mean square error of an iteration is less than the set value, it is considered that the iteration effect has met the requirements and the iteration will be terminated.

Window Size

Default value: 10
Value range: [3, +∞)
Instruction: This parameter refers to the number of consecutive iterations with small error fluctuations in the iterative optimization process. When this parameter is too small, a local optimal solution may appear, which affects the final matching accuracy. Example: If the window only contains the error of the results of three consecutive iterations, so the error trend can only be obtained by comparing the errors of the three iterations, which may lead to a local optimal solution (the error of the second iteration is less than the error of the first iteration and the first iteration) instead of a global optimal solution. According to the recommended value of the number of iterations which is 30, subsequent iterations with more significant error reduction may occur.

Show Corresponse

Default value: Unselected.
Instruction: If it is selected, the corresponding points will be displayed in each iteration.

Complex Object

Default value: Unselected.
Instruction: Selected if the target object is a complex-shaped object (not a simple geometric shape) or if the Automatic Weight function is required.

Automatic Weight

Default value: Unselected.
Instruction: If it is selected, the weight of iteration points will be calculated automatically. It is recommended to use this function for objects with poor coarse matching results.
Point Pair Rejection
Parameter Description

Reject Pair

Default value: Unselected.
Instruction: This parameter decides whether to remove repeated point pairs. There are multiple point pairs in the process of searching for the nearest point, and sometimes the unwanted point pairs need to be filtered out. If this option is selected, this function will be used to remove them through configurating the following N Sigma Threshold and Point Pair Angle Diff Thre parameters; if not checked, these two parameters will not take effect during the matching process.

N Sigma Threshold

Default value: 1.0000
Instruction: This parameter is the multiplier of the standard deviation threshold of the distance between each points. The point pair whose distance is outside N sigma will be eliminated.

Point Pair Angle Diff Thre

Default value: 45°
Instruction: This parameter is the upper limit of the angle difference (in °) between the normals of two points in every matching point pair. If the angle difference is greater than this value, this point pair will be eliminated.

Apply Weights on Pairs

Default value: Unselected.
Instruction: If it is selected, the weight of the model will be used in the iteration of the algorithm, so it is more likely to match better.

Remove Repeat Correspond

Default value: Unselected.
Instruction: If it is selected, the function of removing repeating corresponding point pairs will be used, and the settings of Remove Repeat Correspond Type will take effect. It is recommended to use this function when the point cloud model has more points than that of a single object in the scene.

Remove Repeat Correspond Type

Default value: MinDis

List of values: MinDis and GlobalMin

Instruction: This parameter is used to select the method of removing repeating corresponding point pairs. There are two methods in total:

  • MinDis: Calculate the distance between the repetitive point pairs. Then, keep the point pair with the smallest distance, and delete the remaining repetitive point pairs. More repeating corresponding point pairs will be eliminated in this method than the other.

  • GlobalMin: Take the average value of the distance between the repetitive point pairs as a reference to find out the point pair whose distance is closest to the average value. Then, retain this point pair and eliminate others. Less repeating corresponding point pairs will be eliminated in this method than the other.

Sample Settings

Parameter Description Illustration

Sampling Interval

Default value: 5.000 mm
Instruction: This parameter is used for the downsampling of the model and scene point cloud. The bigger the interval, the less the point number of the sampled point cloud, and the lower the accuracy of model estimation.
Example: The figure below shows the output results when Sampling Interval is set to 5.000 mm and 25.000 mm, respectively. The white parts are the input object point clouds, and the colored lines are the matching results. It can be seen from the comparison of the two pictures that the smaller the sampling interval is, the more accurate the output result will be.

3d fine matching output onetofour

Validation Settings

Parameter Description Illustration

Confidence Threshold

Default value: 0.500

Value range: [0, +∞)

Instruction: The confidence threshold is used to determine whether the matching result is valid. Matching results with confidences higher than the threshold are valid and those lower are not. A higher threshold leads to fewer valid matches, while the output results are more accurate. A lower threshold leads to more valid matches, while the output results are less accurate.

Example: In the figure below, the confidence threshold of (a) is 0.500, and that of (b) is 0.920. The match of (a) is more accurate than that of (b). It can be known that when the larger Confidence Threshold is, the more likely it is that results with higher matching accuracy are retained.

Even when a higher threshold is set, there may be no valid matching result to output. It may be because that different parts of the target object have the same shape.

3d fine matching threshold finematching

Search Radius for Validation

Description: When the distance between the scene point cloud and the point cloud model is less than this value, it is considered that the scene point cloud and point cloud model coincide. The more points that coincide, the higher the validation score for the matching result.

Default value: 10.000 mm

Instruction: The value of this parameter will affect the matching score. The higher the value of this parameter, the looser the criterion for evaluating the matching result will be, which means that even if the pose matching result is of poor quality, the matching score will be high; the smaller the value of this parameter, the more strict the criterion for evaluating the pose matching result will be. Please set the value according to the actual situation of the point cloud. If the points in the point cloud are sparse, the parameter value should be set larger.

During parameter tuning, you can set an original Search Radius for Validation first and run the Step. Then you can check the matching result in the Debug Output window and compare the result with the pose matching score. If the matching result does not correspond with the matching score, please adjust the Search Radius for Validation according to the actual situation.

Output Settings

Parameter Description Illustration

Maximum Number of Detected Poses in Each Point Cloud

Default value: 1
The larger the value, the more matches are generated.
Instruction: The larger the value is, the more matching pairs will be output. The figure below shows the output results when this parameter is set to 1 (left) and 4 (right).

3d fine matching default setting finematching
Result Visualization
Parameter Description

Show Matching Results

Default Value: Selected.
Instruction: If this option is selected, the output model and scene point cloud will be displayed.

Advanced Parameter Tuning Level

This chapter only introduces the parameters that Basic Mode does not contain. If there are some parameters that are the same as that in the Basic Mode, please refer to Basic.

Symmetry Settings

When a part of the workpiece to be recognized is symmetrical, you will need to adjust the symmetry settings to escape a local optimum of the matching result.

An axis of the object center point as its origin will be specified as the rotation axis, and the point cloud model will rotate around the specified axis according to the symmetry angle and therefore the optimal matching result can be obtained.

Parameter Description

Symmetry Axis

Default value: ROTATE_BY_Z
List of values: ROTATE_BY_X, ROTATE_BY_Y, and ROTATE_BY_Z
Instruction: This parameter is used to select the symmetry rotation axis around which the initial pose is changed.

Angle Step

Default value: 360°
Instruction: This parameter specifies the angle of rotational symmetry.
Example: If an object looks the same before and after rotating 60° around an axis, its angle of rotational symmetry is 60°.

Min Rotation Angle

Default value: −180°
Instruction: This parameter is used to adjust the minimum rotation angle.

Max Rotation Angle

Default value: 180°
Instruction: This parameter is used to adjust the maximum rotation angle.

Example

As shown in the figure below, the main part of the crankshaft, i.e., the part in the frame, has 180° rotational symmetry along the Y-axis.

3d fine matching 3d fine matching add example1
3d fine matching 3d fine matching add example2

If you do not adjust the symmetry settings, there can be mismatches since the shapes of parts of the point cloud model and the crankshaft are similar, as shown below.

3d fine matching 3d fine matching add example3

Only the middle part of the point cloud model and the crankshaft match with each other. However, if the point cloud model rotates 180°, it will match with the entire crankshaft, and the corresponding matching confidence will be higher.

After ROTATE_BY_Y (the Y-axis of the object center point) is set as the Symmetry Axis, and the Angle Step is set to 180°, the crankshaft match with each other correctly, as shown below.

3d fine matching 3d fine matching add example4

Pose Filtering Settings

Parameter Description

Filter Pose by Model Rotation Angle

Instruction: In the Edge Matching mode, the point cloud model may need to be rotated by a certain angle to match with the scene point cloud, and the posed will be filtered by the point cloud model’s rotation angle. When the model’s rotation angle exceeds the Angle Difference Upper Threshold, the pose will be filtered out.
Default value: Disabled.

Angle Difference Upper Threshold

Instruction: See the tooltip of Filter Pose by Model Rotation Angle.
Default value: 135.000°

Model Weight in Validation

When the workpieces to be recognized have the similar main body, while their specific parts differ greatly, the weight of the “parts” should be higher to output a correctly matched result.

Parameter Description

File of Model Fragments with High Weight

Instruction: This file (PLY format) is a fragment of Model File. Based on the actual workpiece, determine which part is selected as the high weight file. The points in this file have a high weight set by the Weight of Each Point.

Weight of Each Point

Default value: 2.0000
Instruction: This parameter is to set the weight value for the parts with salient features in the point cloud model. The initial weight of all points is 1. After setting this parameter for the target cloud, the weight coefficient of the target cloud will become 1Xthisparameter, so as to achieve the purpose of emphasis in the subsequent matching steps.

Search Radius When Set Weight

Default value: 3.000 mm
Instruction: This parameter is used to set the radius for setting weights. The unit is millimeters. The weight setting is an operation performed on the original point cloud model. The point cloud model will be down-sampled before it is used in the matching process, which causes some points to be shifted or lost. In the process of setting the weight of the points near the missing point, the search radius needs to be set, and the search radius is set by this parameter.

Validation Settings

Parameter Description

Consider Error of Normal Angles

Description: When validating the matching results, consider the angle error between the normals of the points in the scene point cloud and their counterparts in the point cloud model. Once this parameter is selected, the number of output matching results will be fewer, but the accuracy of the matching results will be enhanced.
Default value: Unselected.
Instruction: If it is selected, the angle difference of the normal vector of the corresponding point pair will be considered in the pose validation process. In this case, although the output is often reduced, it could be more accurate.

Multiply Corresponding Points Ratio in Scene

Default value: Unselected.
Instruction: If it is selected, the matching score will be multiplied by the proportional coefficient of the corresponding point to the scene point. It is not suitable for the case of estimating poses of multiple objects in one point cloud.

Result Visualization

Parameter Description

Show Sampled Model Cloud

Default value: Unselected.
Instruction: If it is selected, the sampled point cloud model will be visualized in the output.

Show Sampled Scene Cloud

Default value: Unselected.
Instruction: If it is selected, the sampled scene point cloud will be visualized in the output.

Show Validation Point Correspondences

Default value: Unselected.
Instruction: If it is selected, the corresponding relationship between the model and the scene point cloud will be visualized in the output.

Is this page helpful?

You can give a feedback in any of the following ways:

We Value Your Privacy

We use cookies to provide you with the best possible experience on our website. By continuing to use the site, you acknowledge that you agree to the use of cookies. If you decline, a single cookie will be used to ensure you're not tracked or remembered when you visit this website.

  • Preference
  • I accept
  • I don't accept