Troubleshooting

Problem 1: Failure to Locate the Cartons, Resulting in No Output from the Mech-Vision Project

Symptom

The cartons are not located successfully, and there is no output from the Mech-Vision project, resulting in unpicked cartons on site.

single case cartons common problem 1 problem effect

Possible Causes

  • Due to incorrect setting of 3D ROI, the carton(s) is beyond the 3D ROI. Hence, no result can be output from the Mech-Vision project.

    single case cartons common problem 1 reason effect 1
  • In the Point Cloud Preprocessing & Get the Mask of the Highest Layer Procedure, the carton dimensions entered in the Step Parameters panel differ significantly from the recognized carton dimensions, making dimension validation impossible and resulting in failure to locate the carton.

    single case cartons common problem 1 reason effect 2
  • Deep learning failed to correctly segment masks of individual cartons, making dimension validation impossible in the subsequent processing and resulting in failure to locate the carton.

    single case cartons common problem 1 reason effect 3

Solutions

  • Reset 3D ROI based on workstation layout and pallet pattern dimensions, so that the cartons will be within the ROI.

    single case cartons common problem 1 solve effect 1
  • Measure the carton dimensions and compare to the entered dimensions. If the difference is huge, you can adjust the parameters under Read Object Dimensions in the Step Parameters panel of the Point Cloud Preprocessing & Get the Mask of the Highest Layer Procedure to reset the carton dimensions. If the difference is small, you can appropriately increase the value of the Box Dimension Error Tolerance parameter in the Calculate Carton Poses Procedure.

    single case cartons common problem 1 solve effect 2
  • Double-click the Segment Masks of Individual Cartons Using Deep Learning Procedure, check the visualized output of the Deep Learning Model Package Inference Step, and ensure that the visualized output is normal, as shown in the figure below. If the visual output results are abnormal, please refer to Problem 3.

    single case cartons common problem 1 solve effect 3

Problem 2: Poor Quality of Carton Point Cloud

Symptom

The obtained carton point cloud was in poor quality. As shown below, the image on the left is the 2D image captured by the camera, and the image on the right is the point cloud of the carton in poor quality.

single case cartons common problem 2 problem effect

Possible Causes

  • The ambient light on site was too strong.

  • The setting of camera 3D exposure parameter was not proper.

Solutions

  • When the ambient light is too strong, it is recommended to use longer exposure time to enhance the image quality. When the ambient light is too strong and it is not effective to increase the exposure time, you can consider shading the station so that the ambient light intensity at the station remains constant.

  • As the captured image tends to be too bright when the carton is white, it is recommended to use shorter exposure time; as the captured image tends to be too dark when the carton is black, it is recommended to use longer exposure time.

  • When there are cartons with different colors, it is recommended to capture images using the method of combining shorter exposure time and longer exposure time.

Problem 3: Poor Quality of Recognition Result from Deep Learning

Symptom

The performance of deep learning recognition was poor, resulting in no output from the “Segment Masks of Individual Cartons Using Deep Learning” Procedure.

single case cartons common problem 3 problem effect

Possible Causes

  • When capturing 2D images, the camera exposure time and white balance settings were not proper. As shown below, the image on the left is captured when the camera exposure time is normal, and the image on the right is captured when that is abnormal.

    single case cartons common problem 3 reason effect 1
  • The deep learning recognition region (2D ROI) was not set.

  • Masks of individual cartons obtained by using deep learning are abnormal. In this case, you need to iterate your deep learning model.

    single case cartons common problem 3 reason effect 2

Solutions

  • First, you need to confirm whether the 2D image has problems such as overexposure, underexposure, shadows, abnormal white balance, etc. If the above problems exist, you need to adjust the camera exposure time and white balance to ensure that the captured 2D image is normal. After ensuring that the 2D image is normal, if the recognition performance is still poor, please refer to the next part.

  • Reset 2D ROI.

  • Acquire carton image data to iterate your deep learning model.

Problem 4: Dresspack Wrapping and Excessive Rotation During Path Planning

Symptom

When using Mech-Viz for path planning, the robot encounters problems such as dresspack wrapping and excessive rotation.

single case cartons common problem 4 problem effect

Possible Cause

The orientation of the carton poses output by Mech-Vision project was not proper.

Solutions

  • Adjust the Z-axis of the carton poses to a specific orientation in the Mech-Vision project.

    In the Custom mode of the Pose Adjustment Tool, add the Flip pose and minimize the angle between the flip axis and target direction option. This allows flipping the X or Y-axis of the pose to the target direction, so as to control the rotation angle of the gripper within the range of 0 to 90°.

    single case cartons common problem 4 solve effect 1
  • Set soft limits.

    Click Set soft limits in the Robot Functional Panel of Mech-Viz.

    single case cartons common problem 4 solve effect 2

    Manually rotate the sixth axis of the real robot, test the limit rotation angle of the sixth axis in the positive and negative directions, and then set the appropriate soft limits with reference to the limit rotation angle of the sixth axis.

    single case cartons common problem 4 solve effect 3

After finishing the above operations, the picking path for the robot is shown as below.

single case cartons common problem 4 solve effect 4

Problem 5: Too Much Point Cloud Noises in Mech-Viz or Presence of Scene Point Cloud Unrelated to Picking

Symptom

When using Mech-Viz for path planning, there are too much point cloud noises or unnecessary scene point clouds in the 3D simulation area of Mech-Viz.

single case cartons common problem 5 problem effect

Possible Cause

Point cloud preprocessing was not applied in the point clouds sent to Mech-Viz.

Solution

In the From Cloud (XYZ-Normal) to Cloud (XYZ-RGB) Step, the prepossessed point cloud is input into the first input port. Then connect the Send Point Cloud to External Service Step to send the prepossessed point cloud to Mech-Viz.

single case cartons common problem 5 add step

After the point cloud preprocessing, the point cloud in the Mech-Viz 3D simulation area is shown in the figure below. The scene point cloud that is unrelated to picking is removed.

single case cartons common problem 5 solve effect

Problem 6: Path Planning Failed Due to Abnormal Waypoints

Symptom

When using Mech-Viz for path planning, the waypoints (carton poses) were abnormal, and the path planning failed.

Take the following figure as an example. The pose numbers in deep red indicate that the waypoints are abnormal, leading to the path planning failure.

single case cartons common problem 6 problem effect

Possible Causes

It could result from a collision, an unreachable waypoint, or other factors.

If all the poses turn red at the same time, it is most likely due to a point cloud collision.

Solutions

  • In the Vision Move Step, make sure to select the box of Remove Point Cloud of Target Workobject in the Remove Point Cloud of Target Workobject parameter group.

    single case cartons common problem 5 solve effect 1
  • In the Vision Move Step, check whether the parameter values of XY-Plane Point Cloud Removal Range Expansion and Z-Direction Point Cloud Removal Range Expansion are too small in the Remove Point Cloud of Target Workobject parameter group. If the parameter values are too small, increase the parameter values appropriately.

    single case cartons common problem 5 solve effect 2

Problem 7: Failure to Plan the Picking Path for the Robot

Symptom

When using Mech-Viz for path planning, it fails to plan the picking path for the robot.

Take the following figure as an example. The carton pose numbers in grey indicate that the waypoints have not been planned in the motion path.

single case cartons common problem 7 problem effect

Possible Cause

It may be caused by the incorrect setting of the Vision Move Step. If the Box Surface Coverage Lower Limit parameter is set incorrectly, that is, the vacuum gripper surface is smaller than that of the carton, it cannot cover the entire upper surface of the carton.

Solution

Based on the ratio between the vacuum gripper surface area to carton surface area on-site, adjust the Box Surface Coverage Lower Limit parameter in the Vision Move Step.

single case cartons common problem 7 solve effect