Risk Assessment and Fixes

Risk: The special colors (pure black) of carton surfaces, distinctive patterns (where adjacent patterns on two cartons can be combined to form a new complete carton), interference from tapes and zip ties, and other characteristics may result in recognition errors.

Solution: At present, deep learning faces challenges in consistently recognizing this type of cartons. It is best to have a clear expectation of the capability boundaries of deep learning when assessing projects.

Risk: If the difference between the length and width of the upper surface of the carton is too small (difference ≤ 20 mm), i.e., the upper surface of the carton is close to the square, the orientation of the carton may be misjudged.

Solution: If you need to identify the orientations of cartons with nearly square upper surfaces, you can mount a 2D camera at an intermediate location to recognize the orientations of the held cartons before they are placed on the pallet.

Risk: When the camera is mounted at a tilt angle greater than 45°, the recognized carton depth information will be incorrect.

Solution: Try to avoid installing the camera at a tilt angle as much as possible.

Risk: Environmental factors such as on-site lighting, architectural shadows, and others may result in carton surfaces being overly bright or dark, leading to unstable recognition. Common environmental factors include:

Solution: Provide supplemental light and shading according to the actual situation of the workstation. Please refer to shading solution for details.

Risk: When a sliding device is used for the camera bracket, the low positioning accuracy of the sliding rail can result in inaccurate robot picking.

Solution: It is recommended to use a steel frame structure for the camera stand. If a sliding device needs to be mounted, use a servo motor with a slide rail positioning accuracy less than 0.1 mm.

Risk: The carton stack is beyond the camera’s field of view or the stack height exceeds the working distance, resulting in incomplete imaging or poor image quality.

Solution: Mount the camera according to its working distance or adjust the carton stack.

Risk: If the upper surface of the carton is not sealed, the vacuum gripper cannot directly absorb the upper surface of the carton.

Solution: The cartons need to be suctioned from the sides and the top together, and careful consideration needs to be given to the order in which the cartons are picked up and to collision avoidance.

Risk: When the flatness of the upper surface of the carton is poor, the vacuum gripper cannot pick up the carton securely, and therefore the carton can be easily dropped due to inertia when the robot moves.

Solution: The flatness of the incoming cartons should be strictly checked. Usually, the flatness of the incoming cartons should be less than 10 mm. It is recommended to remove the cartons that do not meet the flatness requirement.

Risk: When the corrugated paper is thick, the part of the carton surface in contact with the suction cup is not airtight, and therefore the carton may not be stably picked up.

Solution: Generally, a corrugated paper thickness of at least 3–5 layers is required (depending on the product weight). Cartons that cannot meet this requirement should be removed.

Risk: When the center of gravity of the carton is offset, there is a high requirement on suction cups.

Solution: Conduct stability tests on this type of carton in the early stages of the project.

Risk: Dark tape has a significant impact on vision recognition. Transparent tapes may reveal gaps between carton seals, easily resulting in recognizing one carton as two.

Solution: Choose tapes with colors close to the carton and reduce the gap between carton seals.

Risk: The surface plastic wrap on cartons is prone to aging and leakage, resulting in poor load-bearing capacity. When using vacuum grippers for handling, there is a risk of dropping cartons. Additionally, the plastic film directly beneath the camera may cause overexposure, affecting the recognition performance of 3D vision.

Response: Test the suction stability of the carton surface with the plastic film and adjust the camera parameters or image capturing angle according to the actual situation to avoid overexposure of the carton surface.

Risk: If bellows suction cups are used to pick cartons with packing straps, the suction cup should avoid contact with the packing strap. In addition, reflective packing strap tends to lead to carton recognition failure.

Solution: The suction cup should avoid contact with the packing strap, and it is recommended to increase the safety factor to enhance the effective force capacity of the vacuum gripper. It is also necessary to avoid using reflective packing straps. If inevitable, it is recommended to train deep learning models beforehand.

Risk: When there is liquid, dust, oil, etc., on the suction cup contact surface of the carton, the air tightness and force on the suction cup contact surface may decrease.

Solution: Ensure consistency standards for incoming cartons and ensure that the suction cup contact surfaces of the cartons are dry and clean.

Risk: Plastic film or wrapping film may be applied on the surface of the carton stack or the wrapping film may not be completely removed, which may affect the robot depalletizing.

Solution: Ensure consistency standards for incoming cartons. Make sure all films are removed in advance.

Risk: The incoming pallet may exceed the effective vision recognition area.

Solution: Design limit mechanisms for the infeed system to ensure that the pallet position does not exceed the set area.

Risk: The carton stack on the pallet exceeds the dimensions of the pallet, causing cartons to be outside the camera’s field of view.

Solution: Ensure consistency standards for incoming cartons. Design the pallet pattern based on the pallet size to prevent cartons from protruding beyond the camera’s field of view.

Risk: The stacked cartons may be tilted and squeezed, resulting in height differences between cartons on the same layer and ultimately affecting the robot’s picking stability.

Solution: Ensure consistency standards for incoming cartons. When picking multiple cartons at the same time, the maximum height difference between cartons on the same layer should be less than 15 mm.

Risk: Using one IPC to communicate with multiple robots simultaneously leads to communication challenges.

Solution: Equip each robot with a separate IPC to ensure that one IPC communicates with only one robot.

Risk: When the requirement of vision cycle time is high, the vision system may not provide the vision result within the specified time frame.

Response: The time from receiving commands to responding with vision information is typically less than 4 seconds. If you need to speed up the cycle time, you can mount the camera in Eye to Hand mode to incorporate the vision recognition time into the robot’s motion cycle.