Vision Continue Palletizing
Function
Recognize the existing boxes in an unfinished stack by vision, and then match the recognized boxes with those in a known pallet pattern to find the empty spaces for palletizing.
Usage Scenario
The Vision Continue Palletizing Step is usually used in scenarios where production needs to be continued after a production pause, or in scenarios that require high space utilization (such as the three-dimensional warehouse), where multiple unfilled pallets need to be merged.
| Mech-Viz provides a relevant example project to the “Vision Continue Palletizing” Step to help you learn more about the process of continuing the palletizing through vision. For the example project, you can go to File > New > Example projects > Box palletizing > Vision Continue Palletizing in the software. |
Parameter Description
Move-Type Step Common Parameters
Send Waypoint
Selected by default, i.e., send the current waypoint to the receiver, such as the robot. Once deselected, the current waypoint will not be sent. However, the waypoint will remain in the planned path.
Try Continuously Running through Succeeding Non-Moves
Unselected by default. When non-move Steps, such as Visual Recognition, Set DO, Check DI, etc., are connected between move-type Steps, the sending of the waypoints will be interrupted, and the real robot will take a short pause, reducing the smoothness of running.
When this parameter is selected, the project will continue to run without waiting for the current move-type Step to complete execution, and therefore the robot can move in a smooth way without pauses. However, selecting this parameter may cause the execution of the Step to end prematurely.
Why will this feature cause the execution of the Step to end prematurely?
Mech-Viz will send multiple waypoints simultaneously to the robot when the project is running. When the currently returned JPs of the robot correspond to the last waypoint sent by Mech-Viz, Mech-Viz will assume that the robot has moved to the last waypoint.
For example, there are 10 move-type Steps in a path, and the pose of the 5th move-type Step is the same as that of the last move-type Step. When the robot moves at a low speed, the current JPs will be sent to Mech-Viz after the robot moves to the 5th waypoint. Since the poses of the 5th move-type Step and the last move-type Step are the same, Mech-Viz may mistakenly determine that the robot has reached all waypoints and prematurely ends the command.
Do Not Check Collision with Placed Target Object
Once Detect collisions on target objects is enabled in the Collisions panel, selecting this parameter will disable the collision detection between the robot, robot tool, and placed target objects. Typically, this parameter is selected in the move-type Step following the Step whose Pick or place is set to Place to avoid false collision detections.
Application Example:
The TCP of a depalletizing vacuum gripper is usually set inside the model rather than on the surface of the vacuum gripper. As a result, when picking a box, the vacuum gripper model may overlap with the box model. However, the software does not detect collisions between the end tool and the picked target object, so no collision alarm will be triggered during picking. Once the robot places the box down, the picked box model becomes a scene model, and the software will start to detect the collision between the end tool and the box’s scene model, triggering a collision alarm and preventing the completion of the palletizing task.
Once this parameter is selected, no collision between the robot, end tool, and the model of the placed target object will be detected, and the above issue will be resolved.
Point Cloud Collision Detection Mode
Usually, Auto can be selected, i.e., directly apply the Point cloud collision detection settings in the Collisions panel. For the Steps between picking and placing, Check collision can typically be selected.
Auto |
Default setting. Once Detect collisions on target objects is enabled in the Collisions panel, only point cloud collisions of the “Vision Move” Step and the “Relative Move” Steps that depend on the “Vision Move” Step will be detected, while other move-type Steps will not be detected. |
Do not check collision |
Point cloud collisions of all move-type Steps will not be detected. |
Check collision |
Point cloud collisions of all move-type Steps will be detected. |
Ignore Target Object Symmetry
This parameter is only visible when the Waypoint type of the move-type Step is set to Target object pose.
The target object symmetry here refers to the Rotational symmetry of held target object predefined in the target object editor during collision model setup.
None |
Default setting, i.e., do not ignore symmetry on any axis. |
Around target object frame Z axis |
Only ignore symmetry around the Z-axis. |
Around target object frame X&Y axes |
Ignore symmetry around the X-axis and Y-axis. |
Around all axes |
Once the symmetry around all axes is ignored, the robot will place the object strictly according to the target object pose. |
| When the move-type Steps are used to place the target objects, the consistency of the placing poses of the target objects cannot be guaranteed once the rotational symmetry is applied. If you want all target objects to be placed strictly according to a specific rule, ignore the symmetry of the target object around all axes. |
Plan Failure Out Port
Once this parameter is selected, a Plan failure exit port will be added to the Step.
If the path planning of the current Step succeeds, the workflow will continue along the Success exit port. If the path planning of the current Step fails, the workflow will proceed along the Plan failure exit port. If multiple move-type Steps with “Plan failure” exit ports display in the same plan history entry, the workflow will proceed along the “Plan failure” exit port of the first move-type Step.
Held Target Object Collision Detection Settings
Before configuring this parameter group, please go to the Collisions panel and enable Detect collisions on target objects.
| Disabling collision detection will increase the collision risks. Please select the following parameters with caution. |
Do Not Check Collision with Scene Objects
Once this parameter is selected, collisions between the held target object and the scene model will not be detected, reducing the computational load of collision detection in the software, speeding up path planning, and optimizing the overall cycle time.
Do Not Check Collision with Robot
Once this parameter is selected, collisions between the held target object and the robot will not be detected, reducing the computational load of collision detection in the software, speeding up path planning, and optimizing the overall cycle time.
Do Not Check Collision with Point Cloud
Once Point cloud collision detection is enabled in the Collisions panel, selecting this parameter will stop detecting collisions between the held target object and the point cloud, further reducing the software’s computational load, shortening path planning time, and enhancing the overall cycle time.
Visual Match
This parameter group is used to determine whether the boxes recognized by the vision service match those in the known pallet pattern. The pallet pattern is obtained from the Palletizing Step specified in the Select Palletizing Step parameter.
If a box in a known pallet pattern recognized by the vision service has a distance deviation from the box in the known pallet pattern in the X and Y directions, a distance deviation in the Z direction, or a rotation deviation around the Z-axis that exceeds the following threshold, the vision matching of the box fails. If the “Match Symmetry” parameter is set to Auto, Symmetry180Deg or Symmetry90Deg, the box’s symmetry will be considered during matching.
Max XY Plane Distance Deviation
The maximum distance deviation (mm) between the box recognized by the vision service and the box in the known pallet pattern in the X and Y directions.
Height Threshold for Pallet Pattern Matching
This parameter only takes effect when the Check Placed Boxes Order parameter is selected. It is used to confirm whether the boxes palletized in sequence are correct. If the height difference between the box recognized by the vision service and the box in the known pallet pattern exceeds this threshold, the box matching fails, indicating that the box palletized in sequence is incorrect.
Max Z-Direction Distance Deviation
The maximum distance deviation (mm) between the box recognized by the vision service and the box in the known pallet pattern in the Z-direction.
Max Rotational Deviation around Z-Axis
The maximum rotation deviation (°) between the box recognized by the vision service and the box in the known pallet pattern around the Z-axis.
Max Z-Direction Deviation Angle
The maximum angular deviation (°) between the Z-axis of the box recognized by the vision service and the Z-axis of the robot base reference frame.
Match Symmetry
When the orientations of the boxes recognized by the vision service did not match those in the known pallet pattern, the matching failed. Applying the symmetry can solve this problem during matching.
| Option | Description |
|---|---|
Auto |
When the difference between the length and width of the upper surface of the box is less than 2 cm, the upper surface of the box will be recognized as a square and Symmetry90Deg will be applied automatically. When the difference between the length and width of the upper surface of the box is more than 2 cm, the upper surface of the box will be recognized as a rectangular and Symmetry180Deg will be applied automatically. |
NoSymmetry |
Do not apply symmetry. |
Symmetry180Deg |
Select this option when the upper surface of the box is rectangular. |
Symmetry90Deg |
Select this option when the upper surface of the box is square. |
Pose Correction
Correction Type
If the matching between the vision result and the known pallet pattern is successful, this parameter corrects the known pallet pattern obtained from the Select Palletizing Step parameter according to the average deviation of all matched boxes.
| Option | Description |
|---|---|
Total |
Correct both the translation error and rotation error. |
TranslationOnly |
Only correct the translation in the X-, Y- and Z-direction. |
RotationOnly |
Only correct the rotation around Z-axis. |
Judgment Criteria
This parameter group is used to determine whether the vision result matches the known pallet pattern successfully. By default, when all boxes recognized by the vision service match those on the known pallet pattern, the vision result matches the known pallet pattern successfully.
Box Label of Interest
This parameter specifies the labels of the boxes involved in vision matching. Once this parameter is set, only the boxes with the specified label will be involved in the vision matching. If no object label is set, all boxes recognized by vision projects will be used for visual matching.
Check Placed Boxes Order
Select to check the order of the palletized boxes in the project. This option is disabled by default.
The “Vision Continue Palletizing” Step requires that the box serial number must be continuous, and that serial number 1 cannot be empty. If these requirements are not met, an error will occur.
Take the following pallet pattern for example: There are 10 boxes on one layer, and the palletizing order will be calculated automatically.
For the following situations:
-
If position indicated by 1 is not occupied with a box, and position 2 and 3 are occupied with boxes, an error will occur.
-
If positions indicated by 1, 2, 3, 4, and 6 are occupied, while position 5 is not occupied, an error will occur.
-
If the order number of the palletized box starts from 1 and the subsequent numbers are continuous, the project runs normally.
Select Palletizing Step
This Step does not generate pallet patterns; instead, it requires selecting other palletizing Steps that can configure pallet patterns to obtain actual pallet pattern information.
Vision Service Name
This parameter specifies the vision project that is used to recognize and locate the boxes on the pallet.
| The vision result usually includes the poses of recognized boxes and recognized pallet. The pallet pose is not mandatory data. When the pose of the pallet is not provided in the vision result, the pallet pose in the specified pallet pattern will be used. When the boxes are not centered on the pallet, the relative positions of the box and the pallet in the simulation area may differ greatly from the actual situation. In this case, it is recommended to use the vision result with the pallet pose to improve the matching accuracy. |
Basic Move Settings
Some of the trajectories during box palletization are shown in the figure below.
|
|
|
|
-
P0: reference point; P1: intermediate point; P2: entry point; P3: adjustment point; P4: placement point.
-
P0-P1: intermediate point segment; P1-P2: entry segment; P2-P3: adjustment segment; P3-P4: placement segment.
Motion type
Joint move |
Joint motion, which guides the robot to move in a curved path. It is less likely to reach singularities in the path for joint motion.
|
Linear move |
Linear motion, which guides the robot to move linearly.
|
Singularity Avoidance
When the motion type is Linear move, enabling this function can simulate linear move by joint move with multiple segments, thus reducing singularity problems to a certain extent.
Parameters Setting
| Limit to Motion Segments | Specific Number | No Limit |
|---|---|---|
Feature |
Simulate linear move using joint move with a user-specified number of segments. |
The software calculates the number of segments needed to simulate linear move. |
Advantages |
|
|
Disadvantages |
|
|
| Parameter | Description |
|---|---|
Number of Segments |
The number of joint move segments specified by the user when the Limit to Motion Segments is set to Specific Number. |
Max Position Deviation |
The maximum allowable deviation of the new multi-segment joint motion path from the original linear motion path. The greater the max position deviation, the higher the success rate of singularity avoidance, and the lower the similarity between the actual trajectory and the straight line. |
Max Angle Deviation |
The maximum allowable angular deviation of the new multi-segment joint motion path from the original linear motion path. The greater the max angle deviation, the higher the success rate of singularity avoidance, and the lower the similarity between the actual trajectory and the straight line. |
Velocity & Acceleration
Velocity and acceleration determine how fast the robot can move. Usually, the set acceleration should be lower than the velocity. When the set acceleration is higher than the velocity, the robot will move in a choppy way.
| The velocities of Vision Move and its prior and subsequent Steps should be relatively low to ensure that the objects can be picked steadily. |
Blend radius
Usually, the default setting can be used.
-
The blend radius refers to the distance between the target point and the point where the robot starts to turn. The larger the blend radius, the more smoother the robot motion transitions are. If the robot moves in a relatively small space, please set the blend radius to a smaller value.
-
If the robot moves in a relatively large space without obstacles and the distance between two consecutive path segments is long, please set the blend radius to a larger value.