Refine The Pick Points By Point Clouds

In the process of grabbing any object, the position and the Symmetry of the object both need to considered. Therefore, this procedure calculate the relatively precise grabbing position and the direction of the long axis, after adjustment and screening, the pick points which meet the requirements in the region of interest are obtained. The program of this procedure in Mech-Vision is shown in Figure 1.

The program of refining the pick points by point clouds in Mech-Vision

Figure 1 The program of refining the pick points by point clouds in Mech-Vision

The introduction of each step in the program is shown in as follows :

  1. Allocator : Allocate the inputs to the next few steps.

  2. Calculate The Position Of The Pick Points : Input the point clouds and the planar pick points list in the region of interest in the scene, calculate the pick points in the space and obtain a relatively precise grabbing position.

  3. Calculate the direction of the X axis : Input the point clouds and the mask of the objects in the region of interest in the scene, calculate the pick points in the space. The direction of the long axis of the pick points can be determined, since the mask includes the shape information of the object.

  4. Point Axes of Poses to Given Direction : Input the pick points list obtained in the above two steps, adjust the posture according to the settings, determine the pick points position and the direction of the long axis, then output the pick points list.

  5. Transform Poses : Input the result of the previous step and obtain the representation fo the pick points in the robot coordinate system.

  6. Validate Poses by Included Angles to Reference Direction : Input the result of the previous step, then calculate the angle between the Z axis of the pick points and the reference direction, after that compare with the threshold in settings, if it is less then “True” otherwise “False”, finally output the list of Boolean values.

  7. Filter : Input the list of Boolean values, the pick points in the robot coordinate system and the label and the size of the gripper correspond to the pick points, output the result corresponding to “True”.

  8. Collect Poses in 3D ROI : Input the result of the previous step, obtain the pick points, their tags and their confidence level in the region of interest according to the file. This step must run under the robot coordinate system.

Calculate The Position Of The Pick Points

The program of this procedure is shown in Figure 2 。

The program of calculating the position of the Pcik Points in Mech-Vision

Figure 2 The program of calculating the position of the Pcik Points in Mech-Vision

The introduction of each step in the program is as follows :

  1. Extract 3D Points in Cylinder : Input the point clouds and the planar pick points list in the region of interest in the scene, extract the point clouds in the set cylinder.

  2. Validate Point Clouds : Input the result of the previous step, compare the number of points of each point clouds with the set threshold. If the result is within the threshold range it is “True”, otherwise it is “False”. Output the list of Boolean values.

  3. Filter : Input the list of Boolean values, the label corresponding to the pick points and the size of the gripper, output the result corresponding to “True”. This step can filter out the label and the size of gripper corresponding to the empty point clouds.

  4. Filter : Input the list of Boolean values, the point clouds in the cylinder and the mask of the object, output the result corresponding to “True”. This step can filter out the the empty point clouds and the mask of the object corresponding to the empty point clouds.

  5. Calc Poses and Dimensions from Planar Point Clouds : Input the point clouds with the required number of points, calculate the spatial Pick Point corresponding to each point cloud and output it.

  6. Flip Poses’ Axes : Input the list of pick points, flip it according to the direction of the specified coordinate axis to make the direction as the same, which is convenient for subsequent grabbing, output the list of pick points after flipping.

  7. Move Poses to Point Cloud Surfaces along Z-Axis : Input the point clouds in the ROI and the list of pick points after flipping, calculate the distance between the Pick Point and the point clouds area corresponding to it, move the Pick Point to the object surface according to the rules settings.

Calculate the direction of the X axis

The program of this procedure is shown in Figure 3.

The program of calculating the direction of the X axis in Mech-Vision

Figure 3 The program of calculating the direction of the X axis in Mech-Vision

The introduction of each step in the program is as follows :

  1. Apply Masks to Point Cloud : Input the mask of the scene and the point clouds in the region of interest, calculate the point clouds corresponding to the mask and output it.

  2. Validate Point Clouds : Input the result of the previous step, compare the number of points of each point cloud with the set threshold. If the result is within the threshold range it is “True”, otherwise it is “False”. Output the list of Boolean values.

  3. Filter : Input the list of Boolean values, the label and the size of the gripper corresponding to the pick points (with accurate position), output the result corresponding to “True”. This step can filter out the label and the size of gripper corresponding to the empty mask.

  4. Filter : Input the list of Boolean values and the point clouds corresponding to the mask, output the result of “True”. This step can filter out the empty mask.

  5. Calc Poses and Dimensions from Planar Point Clouds : Input the result of the previous step, calculate the pick points corresponding to each point cloud.