Details of the Project¶

Application Scenarios¶

The project is suitable for picking a large number of small workpieces. Because the workpieces come in large quantities, they may easily overlap, which affects feature recognizing.

In addition, the front and back faces of non-planar workpieces need to be distinguished, which is different from picking tasks of flat workpieces whose front and back faces are the same and do not need to be distinguished.

Project Workflow¶

Since there are a large number of workpieces and they may overlap with each other, deep learning will be used in instance segmentation and 3D matching. The workflow of a typical project for small non-planar workpieces is shown below.

../../../../../_images/small_curved_workpiece_01.png

Explanation of the Steps¶

The project screenshot is shown below.

../../../../../_images/small_curved_workpiece_02.png

Point Cloud Preprocessing¶

This Procedure generates raw point clouds from the depth map and the color image, and then calclulate the normals of the point cloud, deletes the outliers, and then set an ROI. It facilitates the 3D matching in subsequent Steps.

Please refer to Point Cloud Preprocessing for detailed information about this Procedure.

../../../../../_images/small_curved_workpiece_03.png

Instance Segmentation (Color Image)¶

In the Typical Project for Small Non-Planar Workpieces, instance segmentation by deep learning is used to find the planar projection of the workpieces and generate masks, thus forming the basis for the subsequent task of generating the workpiece point clouds. A sample result is shown in Figure 4.

Please see Instance Segmentation (Colored Image) for details about this Procedure.

../../../../../_images/small_curved_workpiece_04.png

Figure 4. A sample result of Instance Segmentation (Color Image)¶

Apply Masks to Point Cloud¶

This Step extracts the part of the point cloud covered by a mask based on the masks and the point cloud of workpieces. A sample result is shown in Figure 5.

Please see Apply Masks to Point Cloud for detailed information about this Step.

../../../../../_images/small_curved_workpiece_05.png

Figure 5. A sample result of Apply Masks to Point Cloud¶

Filter Out Point Clouds That Exceed the Limit¶

After the point clouds of individual workpieces are generated, this Procedure filters out those that affect 3D matching to improve the accuracy of matching.

A sample result is shown in Figure 6.

Please see Filter Out Point Clouds That Exceed The Limit for details about this Procedure.

Result for filtering out point clouds that exceed the limit

Figure 6. A sample result of Filter Out Point Clouds That Exceed the Limit¶

3D Matching¶

3D Matching is performed on the workpiece point clouds to obtain pick points after the point clouds are filtered.

A sample result of 3D Matching is shown in Figure 7. The colored parts of the point clouds match the point cloud model.

Please see 3D Matching for details about this Procedure.

Figure 7. A sample result of 3D Matching¶

Adjust Poses¶

This Step is for adjusting the poses.

Please see Pose Editor for instructions on adjusting poses.

Map to Multiple Pick Points¶

This Step takes a sorted pose list and maps them to a workpiece to provide different potential picking poses for the robot.

Please see Map to Multiple Pick Points for details about this Step.