3D Fine Matching (Multiple Models)

You are viewing an old version of the documentation. You can switch to the documentation of the latest version by clicking the top-right corner of the page.

Function

Use multiple models to accurately match objects in the scene and output accurate pick points of the target objects.

Usage Scenario

This Step performs more accurate matching based on the initial candidate poses output by the Step 3D Coarse Matching (Multiple Models) and outputs accurate poses of the target object.

Input and Output

3d fine matching multiple models input and output

Parameter Description

Parameter Tuning Level

Default value: Basic

List of values: Basic, Advanced

Instruction: Two different parameter tuning levels can be set:

  • Basic: basic adjustment, with a small amount of adjustable parameters

  • Advanced: advanced adjustment, with more adjustable parameters

Basic Mode

This chapter introduces the adjustable parameters in Basic Mode. These parameters can also be configured in Advanced Mode.

Model Settings

Model File (Required)

Default value: model.ply

Instruction: The path of 3D model file. The construction process can refer to the complete point cloud model stitching document. You can enter multiple file paths. Please use semicolons to divide different file paths.

Geo Center Point File (Required)

Instruction: The geometric center file in JSON format. You can enter multiple file paths. Please use semicolons to divide different file paths.

Model Label File (Optional)

Instruction: The model label file in JSON format. You can enter multiple file paths. Please use semicolons to divide different file paths.

Example: Ensure that the order of the file paths is consistent, i.e., if the file path of an object with front side facing up is entered first in the Model Label File, the corresponding front side object file path should also be entered before the back side one in both Model File and Geometric Center File, as shown below.

3d fine matching multiple models model labelfile

After inputting file paths correctly, the output point cloud image is shown as below. Both the front and back point cloud models of the object can match the scene point cloud well.

3d fine matching multiple models multiple template 3dmatching

Cloud Orientation Calculation

Point Orientation Calc Mode

Default value: Origin

Instruction: This parameter is used to specify the method to calculate the orientations of points in the point cloud. There are four methods as shown below. Please select one according to the actual situation of the project.

List of options Instruction

Origin

Directly use the original normal direction of the input point cloud.

StandardMode

Use the CPU to recalculate the normal direction of the input point cloud. It is recommended to use this mode when the model file doesn’t have normal direction: Search for the k points closest to the target, and use principal component analysis (PCA) to obtain the minimum eigenvector as the point’s normal direction.

EdgeTangent

The tangent direction of the input edge point cloud is calculated as the normal direction. This mode can be used when matching the edge point cloud of a flat object. There are two usage scenarios as follows:

Scenario 1: The two sides of the object need to be distinguished. The 3D Fine Matching (Multiple Models) Step can be used and there should be two model point cloud models corresponding to the two sides of the object respectively.

Scenario 2: The two sides of the object need NOT to be distinguished. In this case, the 3D Fine Matching Step with one point cloud model is applicable.

EdgeNormal

Calculate the normal at the edge of the point cloud model. This method is recommended when edge matching is performed on relatively flat objects.

When using the EdgeTangent or EdgeNormal methods, ensure that each edge point cloud does not contain multiple objects; in other words, each object point cloud is separated.

Number of Searching Points

Default value: 10

Instruction: This parameter is used to set the number of points in the vicinity of the target point to search for. The minimum value is 1.

Correspondence Settings

Default value: GMM

List of values: GMM, nearest-neighbor

Instruction: This parameter is used to select the local matching method to match the point cloud model with the scene point cloud. The point cloud model will move and rotate according to the set range to match the point cloud in the scene obtained in the previous step match.

Example: In most scenarios, GMM has better anti-interference ability and faster matching speed, so it is recommended to use this mode; in rare cases when GMM cannot meet the demand, the nearest point (nearest-neighbor) mode is worth trying.

GMM Mode

Matching Mode

Default value: Standard

List of values: HighSpeed, Standard, HighPrecision

Instruction: There are three matching modes. Select the appropriate one according to the project requirements:

HighSpeed: the fastest mode, but with lower precision. Standard: a relatively stable mode. HighPrecision: the highest precision mode, but with slower speed.

Number of Iterations

Default value: 30

Instruction: Iteration is the process of cyclically running for a specific instruction until the conditions are met. Number of Iterations is the number of repeated instructions in this process. The parameter set here is the maximum number of iterations. The larger this value, the more matching calculation times and the longer running time. However, larger Number of Iterations also gives better matching accuracy.

Standard Deviation

Default value: 3.000 mm

Instruction: The value here needs to match with the initial pose deviation. When the input initial pose deviation after the previous step (3D Coarse Matching) is greater, a larger Standard Deviation is required. In this case, the matching speed is faster.

Example: The standard deviation should correspond to the deviation of the initial pose. If the set value is greater than the deviation of the initial pose, the accuracy of the output result will be greatly reduced. Normally, the value of this parameter setting will not exceed 0.01 m.

Standard Deviation Update Step Number

Default value: 3

Instruction: During the matching process, the Standard Deviation gradually decreases until it reaches the value of Minimum Standard Deviation. The number of times that the standard deviation decreases in this process is the value of this parameter. For some cases with large standard deviation, the more times of the decrease, the more stable the entire decrease process. Under normal circumstances, this parameter does not need to be adjusted.

Minimum Standard Deviation

Default value: 0.001

Instruction: the minimum Standard Deviation, namely, the reached minimum value after the decrease process.

Apply Weights on Pairs

Default value: Unselected.

Instruction: If it is selected, the weight of the model will be used in the iteration of the algorithm, so it is more likely to match better.

Nearest-Neighbor Mode

Iteration Settings

Number of Iterations

Default value: 30

Instruction: Iteration is the process of cyclically running for a specific instruction until the conditions are met. Number of Iterations is the number of repeated instructions in this process. In the Nearest-Neighbor Mode, the iteration will stop when the calculation can satisfy the requirement of algorithm accuracy. The parameter set here is the maximum number of iterations. The larger this value, the more matching calculation times and the longer running time. However, larger Number of Iterations also gives better matching accuracy.

Nearest Points Search Radius

Default value: 10.000 mm

Instruction: This parameter is used to adjust the search radius of the nearest point (in mm), and the value setting should correspond to the input initial pose deviation. When the input initial pose deviation is large (the deviation between the scene point cloud and the point cloud model is large), this parameter needs to be increased; when the initial pose deviation is very small, this parameter can be adjusted to a small value. The minimum value that can be set for this parameter is 0.001. If the radius is less than this lower limit, some sparsely distributed points may not find the closest point, which could affect the output result.

Example: When the initial pose deviation is very large, and this parameter is set as the minimum value 0.001, the matching result will be incomplete, as shown by the (b) in the figure below. When this parameter is increased to 0.01, the search radius will become larger, and a complete matching result can be obtained, as shown by (a) in the figure below.

3d fine matching multiple models nearestneighbor smallradius
MSE Threshold

Default value: 0.001

Instruction: This parameter (Mean Square Error Threshold) is used to adjust the upper limit of the mean square error. This value is recorded after each iteration and used for comparison in subsequent iteration steps. When the mean square error of an iteration is less than the set value, it is considered that the iteration effect has met the requirements and the iteration will be terminated.

Window Size

Default value: 10

Instruction: This parameter refers to the number of consecutive iterations with small error fluctuations in the iterative optimization process, which can take a minimum value of 3. When this parameter is too small, a local optimal solution may appear, which affects the final matching accuracy. Example: If the window only contains the error of the results of three consecutive iterations, so the error trend can only be obtained by comparing the errors of the three iterations, which may lead to a local optimal solution (the error of the second iteration is less than the error of the first iteration and the first iteration) instead of a global optimal solution. According to the recommended value of the number of iterations which is 30, subsequent iterations with more significant error reduction may occur.

Show Corresponse

Default value: Unselected.

Instruction: If it is selected, the corresponding points will be displayed in each iteration.

Complex Object

Default value: Unselected.

Instruction: Selected if the target object is a complex-shaped object (not a simple geometric shape) or if the Automatic Weight function is required.

Automatic Weight

Default value: Unselected.

Instruction: If it is selected, the weight of iteration points will be calculated automatically. It is recommended to use this function for objects with poor coarse matching results.

Point Pair Rejection

Reject Pair

Default value: Unselected.

Instruction: This parameter decides whether to remove repeated point pairs. There are multiple point pairs in the process of searching for the nearest point, and sometimes the unwanted point pairs need to be filtered out. If this option is selected, this function will be used to remove them through configurating the following N Sigma Threshold and Point Pair Angle Diff Thre parameters; if not checked, these two parameters will not take effect during the matching process.

N Sigma Threshold

Default value: 1.0000

Instruction: This parameter is the multiplier of the standard deviation threshold of the distance between each points. The point pair whose distance is outside N sigma will be eliminated.

Point Pair Angle Diff Thre

Default value: 45°

Instruction: This parameter is the upper limit of the angle difference (in °) between the normals of two points in every matching point pair. If the angle difference is greater than this value, this point pair will be eliminated.

Apply Weights on Pairs

Default value: Unselected.

Instruction: If it is selected, the weight of the model will be used in the iteration of the algorithm, so it is more likely to match better.

Remove Repeat Correspond

Default value: Unselected.

Instruction: If it is selected, the function of removing repeating corresponding point pairs will be used, and the Remove Repeat Correspond Type parameter will be available and could take effect during the matching process. It is recommended to use this function when the point cloud model has more points than that of a single object in the scene.

Remove Repeat Correspond Type

Default value: MinDis

List of values: MinDis and GlobalMin

Instruction: This parameter is used to select the method of removing repeating corresponding point pairs. There are two methods in total:

MinDis: Calculate the distance between the repetitive point pairs. Then, keep the point pair with the smallest distance, and delete the remaining repetitive point pairs. More repeating corresponding point pairs will be eliminated in this method than the other. GlobalMin: Take the average value of the distance between the repetitive point pairs as a reference to find out the point pair whose distance is closest to the average value. Then, retain this point pair and eliminate others. Less repeating corresponding point pairs will be eliminated in this method than the other.

Sample Settings

Sampling Interval

Default value: 5.000 mm

Instruction: This parameter is used for the downsampling of the model and scene point cloud. The unit is millimeters. The bigger the interval, the less the point number of the sampled point cloud, and the lower the accuracy of model estimation.

Example: The left and right figures below are the output results when this parameter is set as 0.0128 and 0.01, respectively. The white one is the input scene point cloud. It can be seen from the comparison of the two pictures that the smaller the sampling interval is, the more accurate the output result will be.

3d fine matching multiple models 3d matching sampling
3d fine matching multiple models multiple template 3dfinematching
Search Points Number

Default value: 20

Instruction: When Parameter Tuning Level is Advanced, this parameter can be adjusted. This parameter is reserved for compatibility with old versions and is used to adjust the number of search points in the sampling process.

Pose Filtering Settings

Filter Pose by Model Rotation Angle

Instruction: In the Edge Matching mode, the point cloud model may need to be rotated by a certain angle to match with the scene point cloud, and the posed will be filtered by the point cloud model’s rotation angle. When the model’s rotation angle exceeds the Angle Difference Upper Threshold, the pose will be filtered out.

Default value: Disabled.

Angle Difference Upper Threshold

Instruction: See the tooltip of Filter Pose by Model Rotation Angle.

Default value: 135.000°

Validation Settings

Confidence Threshold

Default value: 0.5

Value range: [0, +∞)

Instruction: If the validation score of the result is larger than this threshold (which cannot be negative), the result is regarded as valid. The higher the value is, the more accurate results are retained.

Search Radius

Default value: 10.000 mm

Value range: [0, +∞)

Instruction: This parameter is the search radius when calculating the confidence. The unit is millimeters. Please set the value according to the actual situation of the point cloud. If the points in the point cloud are sparse, the Search Radius should be set slightly larger.

Output Settings

Maximum Number of Detected Poses in Each Point Cloud

Default value: 1

The larger the value is, the more matches are generated.

The figure below shows the output result of the parameter when the value is set to 1.

3d fine matching multiple models multiple singleoutput

The figure below shows the output when the parameter is adjusted to 4.

3d fine matching multiple models multiple fouroutput

Results Visualization

Show Matching Results

Default value: Selected

Instruction: If this option is selected, the output model and scene point cloud will be displayed.

Advanced Mode

This chapter only introduces the parameters that Basic Mode does not contain. If there are some parameters that are the same as that in the Basic Mode, please refer to Basic Mode.

Symmetry Settings

When a part of the workpiece to be recognized is symmetrical, you will need to adjust the symmetry settings to escape a local optimum of the matching result. An axis of the reference frame with the geometric center as its origin will be specified as the rotation axis, and the point cloud model will rotate around the specified axis according to the symmetry angle step and therefore the optimal matching result can be obtained.

Symmetry Axis

Default value: ROTATE_BY_Z

List of values: ROTATE_BY_X, ROTATE_BY_Y, and ROTATE_BY_Z

Instruction: This parameter is used to select the symmetry rotation axis around which the initial pose is changed.

Angle Step

Default value: 360°

Instruction: This parameter is used for changing the initial orientation from Min Rotation Angle to Max Rotation Angle.

Example: If an object looks the same before and after rotating 60° around an axis, its angle of rotational symmetry is 60°.

Min Rotation Angle

Default value: −180°

Instruction: This parameter is used to adjust the minimum rotation angle.

Max Rotation Angle

Default value: 180°

Instruction: This parameter is used to adjust the maximum rotation angle.

Example

As shown in the figure below, the main part of the crankshaft, i.e., the part in the frame, has 180° rotational symmetry along the Y-axis.

3d fine matching multiple models 3d fine matching add example1
3d fine matching multiple models 3d fine matching add example2

If you do not adjust the symmetry settings, there can be mismatches since the shapes of parts of the point cloud model and the crankshaft are similar, as shown below.

3d fine matching multiple models 3d fine matching add example3

Only the middle part of the point cloud model and the crankshaft match with each other. However, if the point cloud model rotates 180°, it will match with the entire crankshaft, and the corresponding matching confidence will be higher.

After ROTATE_BY_Y (the Y-axis of the reference frame with the geometric center as its origin) is set as the Symmetry Axis, and the Angle Step is set to 180°, the point cloud model and the crankshaft match with each other correctly, as shown below.

3d fine matching multiple models 3d fine matching add example4

Model Weight in Validation

When the workpieces to be recognized have the similar main body, while their specific parts differ greatly, the weight of the “parts” should be higher to output a correctly matched result.

File of Model Fragments with High Weight

Instruction: This file (PLY format) is a fragment of Model File. Based on the actual workpiece, determine which part is selected as the high weight file. The points in this file have a high weight set by the Weight of Each Point.

Weight of Each Point

Default value: 2.0

Instruction: This parameter is used to set the weight value for each point in the weight file, and there is no unit. The higher the value is, the better the matching result of the model fragments tend to be. Assuming that the weight coefficient of all initial point clouds is 1, after setting this parameter for the target cloud, the weight coefficient of the target cloud will become 1Xthisparameter, so as to achieve the purpose of emphasis in the subsequent matching steps.

Search Radius When Set Weight

Default value: 3.000 mm

Instruction: This parameter is used to set the radius for setting weights. The unit is millimeters. The weight setting is an operation performed on the original point cloud model. The point cloud model will be down-sampled before it is used in the matching process, which causes some points to be shifted or lost. In the process of setting the weight of the points near the missing point, the search radius needs to be set, and the search radius is set by this parameter.

Example

This example is a continuation of the example in Symmetry Settings.

As the crankshafts have the similar main body, and their specific parts differ greatly, the weight of the “parts” should be higher to output a correctly matched result.

Validation Settings

Apply Angle Deviation in Validation

Default value: Unselected.

Instruction: If it is selected, the angle difference of the normal vector of the corresponding point pair will be considered in the pose validation process. In this case, although the output is often reduced, it could be more accurate.

Multiply Corresponding Points Ratio in Scene

Default value: Unselected.

Instruction: If it is selected, the matching score will be multiplied by the proportional coefficient of the corresponding point to the scene point. It is not suitable for the case of estimating poses of multiple objects in one point cloud.

Results Visualization

Show Sampled Model Cloud

Default value: Unselected.

Instruction: If it is selected, the sampled point cloud model will be visualized in the output.

Show Sampled Scene Cloud

Default value: Unselected.

Instruction: If it is selected, the sampled scene point cloud will be visualized in the output.

Show Validation Point Correspondences

Default value: Unselected.

Instruction: If it is selected, the corresponding relationship between the model and the scene point cloud will be visualized in the output.

We Value Your Privacy

We use cookies to provide you with the best possible experience on our website. By continuing to use the site, you acknowledge that you agree to the use of cookies. If you decline, a single cookie will be used to ensure you're not tracked or remembered when you visit this website.