Small Non-Planar Workpiece Loading¶

Application Scenarios¶

The project is suitable for picking a large number of small workpieces. Because the workpieces come in large quantities, they may easily overlap, which affects feature recognizing.

In addition, the front and back faces of non-planar workpieces need to be distinguished, which is different from picking tasks of flat workpieces whose front and back faces are the same and do not need to be distinguished.

Mech-Vision Project Workflow¶

Because the small workpieces come in large numbers and may easily overlap, deep learning and 3D matching are needed. The workflow for a Typical Project for Small Non-Planar Workpieces is shown in Figure 1.

Flow chart for Small Non-Planar Workpieces project

Figure 1. The workflow of a Typical Project for Small Non-Planar Workpieces¶

Figure 2is a screenshot of the graphical programming of the project.

Small Non-Planar Workpieces project in Mech-Vision

Figure 2. The graphical programming of a Typical Project for Small Non-Planar Workpiece¶

Steps and Procedures¶

A Procedure is a functional program block that consists of more than one Step.

Point Cloud Preprocessing¶

This Procedure facilitates and shortens the processing time for the subsequent calculations. Point Cloud Preprocessing generates a raw point cloud from the depth map and the color image, deletes the outliers, calculates the normals for the point cloud, and in the end extracts the part of the point cloud within the ROI.

For details about this Procedure, please see Point Cloud Preprocessing.

A sample result of Point Cloud Preprocessing is shown in Figure 3.

../../../_images/small_curved_workpiece_03.png

Figure 3. A sample result of Point Cloud Preprocessing¶

Instance Segmentation (Color Image)¶

In the Typical Project for Small Non-Planar Workpieces, instance segmentation by deep learning is used to find the planar projection of the workpieces and generate masks, thus forming the basis for the subsequent task of generating the workpiece point clouds. A sample result is shown in Figure 4.

Please see Instance Segmentation (Colored Image) for details about this Procedure.

../../../_images/small_curved_workpiece_04.png

Figure 4. A sample result of Instance Segmentation (Color Image)¶

Apply Masks to Point Cloud¶

This Step extracts the part of the point cloud covered by a mask based on the masks and the point cloud of workpieces. A sample result is shown in Figure 5.

Please see Apply Masks to Point Cloud for detailed information about this Step.

../../../_images/small_curved_workpiece_05.png

Figure 5. A sample result of Apply Masks to Point Cloud¶

Filter Out Point Clouds That Exceed the Limit¶

After the point clouds of individual workpieces are generated, this Procedure filters out those that affect 3D matching to improve the accuracy of matching.

A sample result is shown in Figure 6.

Please see Filter Out Point Clouds That Exceed The Limit for details about this Procedure.

Result for filtering out point clouds that exceed the limit

Figure 6. A sample result of Filter Out Point Clouds That Exceed the Limit¶

3D Matching¶

3D Matching is performed on the workpiece point clouds to obtain pick points after the point clouds are filtered.

A sample result of 3D Matching is shown in Figure 7. The colored parts of the point clouds match the model point cloud.

Please see 3D Matching for details about this Procedure.

Figure 7. A sample result of 3D Matching¶

Pose Adjustment Collection¶

This Step is for adjusting the poses.

Please see Pose Editor for instructions on adjusting poses.

Map to Multi Pick Points¶

This Step takes a sorted pose list and maps them to a workpiece to provide different potential picking poses for the robot.

Please see Map to Multi Pick Points for details about this Step.