Summary of Mech-Vision Project Construction of Conventional Matching Scheme

Conventional mactching schems are mainly divided into two categories: Pure matching scheme, deep learning + matching scheme. The difference between the two lies in the way the point cloud area is selected before matching.

  • Pure matching scheme’s point cloud area selection depends on point cloud process.

  • Deep learning + mactching scheme’s point cloud area selection depends on Mask for deep learning recognition.

../../../_images/app_common_vision_project.png

Figure1 Mech-Vision Engineering Structure of Commonly Used Matching Schemes

As shown in Figure1. Mech-Vision Engineering Structure of Commonly Used Matching Schemes consists of the following parts.

  • Capture Images from Camera : Get camera data. Includes color image and depth image. Choose according to the actual situation or fill in the corresponding camera parameters.

  • Point Cloud Preprocessing : Make depth images into point cloud and calculate point cloud’s normal vector. Extract filtered point cloud in ROI.

  • Send Point Cloud to External Service : The point cloud (XYZ-RGB) converted into the depth image is transferred to the Mech-Viz software for visualization.

  • Acquire RGB Images Based on Depth Images : Acquire RGB images according to the highest layer in the ROI of the depth image, and remove interference from other regions for subsequent deep learning recognition.

  • Deep learning recognition gives Mask: detects and recognizes the target object in the input image, and gives the precise position of the target object in the image (Mask), refer to Procedure of Instance Segmentation.

  • Filtering Operations, label processing: Filtering Operations performs on-demand filtering of point clouds, poses, labels, etc. Label processing includes design process label usage, mapping, etc.

  • Point cloud processing: processing and optimizing the point cloud, so that the point cloud can meet the matching requirements.

  • Matching: Contains four methods: single-template edge template matching, single-template full-template matching, multiple-template edge-template matching, and multiple-template full-template matching, which can be used flexibly for different application scenarios.

  • Edit pose in Pose Editor : The matched output result cannot be directly sent to Mech-Viz for operation, and some processing is required. Such as coordinate system transformation, pose adjustment and pose sorting, etc. Through Pose Editor The user can edit the pose quickly.

  • Procedure Out : Export the results of Mech-Vision to Mech-Viz.

Attention

The Acquire RGB Images Based on Depth Images part should be used according to the actual situation. For example, the model data is trained with cutouts. The surrounding environment interferes with the detection target; in other cases, it can be compared with the deep learning recognition effect without adding Acquire RGB Images Based on Depth Images.