Box Depalletizing—Multiple at Once
When the Picking Method is set to Box depalletizing, and the Depalletizing Mode is set to Multiple at once, you can configure the following parameters.
In addition, you should refer to Configure End Tool to configure the corresponding end tool.
Move-Type Step Common Parameters
Send Waypoint
Selected by default, i.e., send the current waypoint to the receiver, such as the robot. Once deselected, the current waypoint will not be sent. However, the waypoint will remain in the planned path.
Try Continuously Running through Succeeding Non-Moves
Unselected by default. When non-move Steps, such as Visual Recognition, Set DO, Check DI, etc., are connected between move-type Steps, the sending of the waypoints will be interrupted, and the real robot will take a short pause, reducing the smoothness of running.
When this parameter is selected, the project will continue to run without waiting for the current move-type Step to complete execution, and therefore the robot can move in a smooth way without pauses. However, selecting this parameter may cause the execution of the Step to end prematurely.
Why will this feature cause the execution of the Step to end prematurely?
Mech-Viz will send multiple waypoints simultaneously to the robot when the project is running. When the currently returned JPs of the robot correspond to the last waypoint sent by Mech-Viz, Mech-Viz will assume that the robot has moved to the last waypoint.
For example, there are 10 move-type Steps in a path, and the pose of the 5th move-type Step is the same as that of the last move-type Step. When the robot moves at a low speed, the current JPs will be sent to Mech-Viz after the robot moves to the 5th waypoint. Since the poses of the 5th move-type Step and the last move-type Step are the same, Mech-Viz may mistakenly determine that the robot has reached all waypoints and prematurely ends the command.
Do Not Check Collision with Placed Target Object
Once Detect collisions on target objects is enabled in the Collisions panel, selecting this parameter will disable the collision detection between the robot, robot tool, and placed target objects. Typically, this parameter is selected in the move-type Step following the Step whose Pick or place is set to Place to avoid false collision detections.
Application Example:
The TCP of a depalletizing vacuum gripper is usually set inside the model rather than on the surface of the vacuum gripper. As a result, when picking a box, the vacuum gripper model may overlap with the box model. However, the software does not detect collisions between the end tool and the picked target object, so no collision alarm will be triggered during picking. Once the robot places the box down, the picked box model becomes a scene model, and the software will start to detect the collision between the end tool and the box’s scene model, triggering a collision alarm and preventing the completion of the palletizing task.
Once this parameter is selected, no collision between the robot, end tool, and the model of the placed target object will be detected, and the above issue will be resolved.
Point Cloud Collision Detection Mode
Usually, Auto can be selected, i.e., directly apply the Point cloud collision detection settings in the Collisions panel. For the Steps between picking and placing, Check collision can typically be selected.
Auto |
Default setting. Once Detect collisions on target objects is enabled in the Collisions panel, only point cloud collisions of the “Vision Move” Step and the “Relative Move” Steps that depend on the “Vision Move” Step will be detected, while other move-type Steps will not be detected. |
Do not check collision |
Point cloud collisions of all move-type Steps will not be detected. |
Check collision |
Point cloud collisions of all move-type Steps will be detected. |
Ignore Target Object Symmetry
This parameter is only visible when the Waypoint type of the move-type Step is set to Target object pose.
The target object symmetry here refers to the Rotational symmetry of held target object predefined in the target object editor during collision model setup.
None |
Default setting, i.e., do not ignore symmetry on any axis. |
Around target object frame Z axis |
Only ignore symmetry around the Z-axis. |
Around target object frame X&Y axes |
Ignore symmetry around the X-axis and Y-axis. |
Around all axes |
Once the symmetry around all axes is ignored, the robot will place the object strictly according to the target object pose. |
| When the move-type Steps are used to place the target objects, the consistency of the placing poses of the target objects cannot be guaranteed once the rotational symmetry is applied. If you want all target objects to be placed strictly according to a specific rule, ignore the symmetry of the target object around all axes. |
Plan Failure Out Port
Once this parameter is selected, a Plan failure exit port will be added to the Step.
If the path planning of the current Step succeeds, the workflow will continue along the Success exit port. If the path planning of the current Step fails, the workflow will proceed along the Plan failure exit port. If multiple move-type Steps with “Plan failure” exit ports display in the same plan history entry, the workflow will proceed along the “Plan failure” exit port of the first move-type Step.
Held Target Object Collision Detection Settings
Before configuring this parameter group, please go to the Collisions panel and enable Detect collisions on target objects.
| Disabling collision detection will increase the collision risks. Please select the following parameters with caution. |
Do Not Check Collision with Scene Objects
Once this parameter is selected, collisions between the held target object and the scene model will not be detected, reducing the computational load of collision detection in the software, speeding up path planning, and optimizing the overall cycle time.
Do Not Check Collision with Robot
Once this parameter is selected, collisions between the held target object and the robot will not be detected, reducing the computational load of collision detection in the software, speeding up path planning, and optimizing the overall cycle time.
Do Not Check Collision with Point Cloud
Once Point cloud collision detection is enabled in the Collisions panel, selecting this parameter will stop detecting collisions between the held target object and the point cloud, further reducing the software’s computational load, shortening path planning time, and enhancing the overall cycle time.
Vision Result Global Usage
All Vision Poses in One Move
Usually, Vision Move will only use one pose of the corresponding vision result. However, once this parameter is selected, the robot will move through all waypoints corresponding to the vision poses at one time.
This parameter is usually used when the robot moves in a fixed path where no DO signals will be sent, such as the gluing application.
Reuse Vision Result
When this feature is not enabled, each time “Vision Move” plans a robot picking path successfully, the rest unused vision results will be discarded. If this feature is enabled, the vision result that leads to a planning failure and the rest unused vision results will be retained and used for the next planning. This feature should be used with the “Is Vision Result Used Up” Step.
When there are multiple pickable objects in one vision result and it can be guaranteed that picking one object will not affect the poses of other objects, this option can be selected. The vision result will be reused and no new image will be captured until all vision results are used up.
Application Example
Share Vision Result
This feature enables multiple “Vision Move” Steps in which the same vision service is selected to share the vision result.
When one “Vision Move” Step plans successfully, the corresponding vision result will be used, and the unused vision result will be used by the next “Vision Move” Step with this feature enabled. After all “Vision Move” Steps that share the vision result complete planning, the rest results will be discarded.
In addition, this feature can be used in conjunction with the “Reuse Vision Result” feature. When “Reuse Vision Result” is selected, the rest vision results will not be discarded even after all “Vision Move” Steps have completed one round of planning.
Supplementary Features
Re-Locate Bin with Vision
Once the Update Bin Pose with Vision parameter is selected, the bin will be recognized and located simultaneously while the camera captures the image for target object recognition. Therefore, the bin pose in the simulation area can be updated dynamically, which facilitates the collision detection algorithm to effectively prevent the robot from colliding with the bin.
This feature will provide three fields, scene_object_names, scene_object_poses, and scene_object_sizes, in the vision result to define the name, pose, and dimensions of the scene object to be updated.
Filter Vision Result
-
Result Must Be in Specified Bin
This feature limits that the vision result received by the current “Vision Move” Step must be in a specified bin. The vision result out of the effective bin range will not be used.
Each bin’s effective range for vision result can be configured separately in the corresponding configuration window. Only bins with “Set effective range for pose” enabled can be selected in the drop-down menu.
Application Example
Add “Bin1”, and enable the “Set effective range for pose” parameter for this bin.
Select “Bin1” in the drop-down menu.
-
Filter Poses
Use all
Do not filter poses. All poses in the vision result will be used for path planning.
Filter by target object info
Once this option is selected, the poses can be filtered by the names of the target object and pick point. After selecting a target object, which is configured in the target object editor, in the drop-down list of the Use Specified Target Object parameter, you can Use Specified Pick Point. If a target object is specified but no pick point is specified, all pick points of the specified target object will be used for path planning.
Filter by pick point label
Once this option is selected, the poses can be filtered by the pick point label. After entering the label in the parameter field of Use Specified Pick Point Label, only pick points with the specified label will be used for path planning. Deep learning labels and numeric labels set in the target object editor are supported.
Avoid Picking on Same Target Object
This parameter group is mainly used to avoid picking on the same pick point in scenarios where the object cannot be picked successfully.
Once the Filter Out Unlikely Pick Point parameter is selected, you can adjust the following parameters.
-
Filter Subject
-
Pick point: Only pick points with high failure rates will be deprioritized in the picking and discarded in some rounds.
-
Target object: if any one pick point on the target object is considered to have a high failure rate, all pick points on this object will be deprioritized in the picking and discarded in some rounds.
-
-
Priority Downgrade Radius
A sphere with this parameter value as its radius and the pick point successfully planned in the last round as its center will be introduced. If a pick point in the latest vision result falls within this sphere, the pick point will be put off for picking with a decreased picking priority.
-
Discard Attempt Radius
A sphere with this parameter value as its radius and the pick point successfully planned in the last round as its center will be introduced. If a pick point in the latest vision result falls within this sphere, the pick point will be discarded in this round of planning.
For example, if the robot only moves the crankshaft but fails to pick it on the first attempt, there is a possibility of successful picking on the next attempt. Therefore, Priority Downgrade Radius can be set to downgrade the priority of the pick point, while the pick point will not be discarded. When the robot fails to move the crankshaft at all in the first attempt, it is highly unlikely that the crankshaft can be picked successfully in the next attempt, and therefore Discard Attempt Radius can be set to discard the pose directly.
Picking Sequence
Sorting Strategy
This parameter is used to specify the sorting strategy when picking target objects.
-
Sort by the target object count in the combination
The target object combinations will be sorted by the number of target objects they contain, and those with more target objects will be picked first.
If there are three combinations, 1-2-3, 4-5, and 6, the picking sequence will be 1-2-3, 4-5, and 6.
-
In the strict order of vision result
Pick in the order of the result sent by the vision service. If the path planning fails by using a vision result, the current result will not be skipped and the path planning will fail.
For example, as shown in the figure below, the picking sequence is 1, 2-3, 4, and 5. If the planning of the box group 2-3 fails, only box 1 can be picked successfully.
-
In the partial order of vision result
Pick in the order of the result sent by the vision service. If the path planning fails by using a vision result, the current result will be skipped. Then the next vision result will be used for the subsequent path planning.
For example, as shown in the figure below, the picking sequence is 1, 2-3, 4, and 5. If the planning of the box group 2-3 fails, the planning of box 4 and 5 can be continued.
Box grouping
There are three grouping strategies in total, and each strategy contains different parameters.
Default strategy
Group boxes along both the X-axis and Y-axis of the box pose.
-
Grouping Restrictions
-
Spacing Upper Limit
This parameter limits the maximum distance between two adjacent boxes in the grouping direction. When the distance is smaller than this value, the boxes can be considered to be in a group.
Please note that the value of this parameter cannot be greater than the box width in the vision result. If the set value is greater than the box width, the value will be automatically modified to the box width in the actual execution.
The range indicated by the purple arrow below is the Spacing Upper Limit. When the actual distance is smaller than this value, the boxes can be considered to be in the same group.
-
Deviation Upper Limit
This parameter limits the maximum deviation distance of a box in the direction perpendicular to the grouping direction. When the deviation distance is smaller than this value, the box can be considered to be in the group.
Please note that the value of this parameter cannot be greater than the box width in the vision result. If the set value is greater than the box width, the value will be automatically modified to the box width in the actual execution.
The range indicated by the blue arrow below is the Deviation Upper Limit. When the actual distance is smaller than this value, the box can be considered to be in the group.
-
Angle Deviation Upper Limit
This parameter limits the angle of rotational deviation of a box relative to the grouping direction. When the rotation angle is smaller than this value, the box can be considered to be in the group.
-
Group Row Count Upper Limit
The allowable maximum number of rows in one box group for multi-pick depalletizing.
-
Group along box frame axis
One box will be used as the reference, and other adjacent boxes along the X-axis or Y-axis of this box’s pose will be grouped.
-
Grouping Direction
Specify the axis of the reference pose for box grouping.
-
Grouping Restrictions
-
Spacing Upper Limit
This parameter limits the maximum distance between two adjacent boxes in the grouping direction. When the distance is smaller than this value, the boxes can be considered to be in a group.
Please note that the value of this parameter cannot be greater than the box width in the vision result. If the set value is greater than the box width, the value will be automatically modified to the box width in the actual execution.
The range indicated by the purple arrow below is the Spacing Upper Limit. When the actual distance is smaller than this value, the boxes can be considered to be in the same group.
-
Deviation Upper Limit
This parameter limits the maximum deviation distance of a box in the direction perpendicular to the grouping direction. When the deviation distance is smaller than this value, the box can be considered to be in the group.
Please note that the value of this parameter cannot be greater than the box width in the vision result. If the set value is greater than the box width, the value will be automatically modified to the box width in the actual execution.
The range indicated by the blue arrow below is the Deviation Upper Limit. When the actual distance is smaller than this value, the box can be considered to be in the group.
-
Angle Deviation Upper Limit
This parameter limits the angle of rotational deviation of a box relative to the grouping direction. When the rotation angle is smaller than this value, the box can be considered to be in the group.
-
Group Row Count Upper Limit
The allowable maximum number of rows in one box group for multi-pick depalletizing.
-
Only Group Complete Row
In actual applications, a complete row of boxes might be neglected because there are part of other boxes at ends in the grouping direction. Setting an Obstruction Tolerance Distance can preclude the intruding box from affecting the grouping result.
-
Obstruction Tolerance Distance
For a box group, if any other box intrudes in the direction perpendicular to the grouping direction, and the intrusion distance is smaller than this threshold, the intruding box will be ignored, and the box group will be considered a complete row. Otherwise, the box group will be discarded.
-
The set value of Obstruction Tolerance Distance cannot exceed the edge length of the boxes.
Group along custom frame axis
As shown in the figure below, if the 7 boxes are grouped along the axis in box reference frame:
-
The possible groups along X-axis of the box are: 1-2, 3-4, 5, 6, 7.
-
The possible groups along Y-axis of the box are:1-3, 2-4, 5-6-7.
If you want the boxes to be grouped into 1-2, 3-4, 5-6-7, all boxes should be grouped along the direction indicated by the red arrow at the bottom of the figure. The red arrow is an axis of a custom reference frame.
-
Grouping Approach
-
Grouping Direction
Specify the axis of the reference pose for box grouping.
-
X/Y/Z Coord. of Custom Frame Origin
Specify three coordinates to define the origin of the custom reference frame.
Oa indicates the robot reference frame, and Oc indicates the custom reference frame.
-
Custom Frame Rotation Angle
Specify the angle that the custom reference frame rotates around its Z-axis.
-
-
Grouping Restrictions
-
Spacing Upper Limit
This parameter limits the maximum distance between two adjacent boxes in the grouping direction. When the distance is smaller than this value, the boxes can be considered to be in a group.
Please note that the value of this parameter cannot be greater than the box width in the vision result. If the set value is greater than the box width, the value will be automatically modified to the box width in the actual execution.
The range indicated by the purple arrow below is the Spacing Upper Limit. When the actual distance is smaller than this value, the boxes can be considered to be in the same group.
-
Deviation Upper Limit
This parameter limits the maximum deviation distance of a box in the direction perpendicular to the grouping direction. When the deviation distance is smaller than this value, the box can be considered to be in the group.
Please note that the value of this parameter cannot be greater than the box width in the vision result. If the set value is greater than the box width, the value will be automatically modified to the box width in the actual execution.
The range indicated by the blue arrow below is the Deviation Upper Limit. When the actual distance is smaller than this value, the box can be considered to be in the group.
-
Angle Deviation Upper Limit
This parameter limits the angle of rotational deviation of a box relative to the grouping direction. When the rotation angle is smaller than this value, the box can be considered to be in the group.
-
Group Row Count Upper Limit
The allowable maximum number of rows in one box group for multi-pick depalletizing.
-
Only Group Complete Row
In actual applications, a complete row of boxes might be neglected because there are part of other boxes at ends in the grouping direction. Setting an Obstruction Tolerance Distance can preclude the intruding box from affecting the grouping result.
-
Obstruction Tolerance Distance
For a box group, if any other box intrudes in the direction perpendicular to the grouping direction, and the intrusion distance is smaller than this threshold, the intruding box will be ignored, and the box group will be considered a complete row. Otherwise, the box group will be discarded.
-
The set value of Obstruction Tolerance Distance cannot exceed the edge length of the boxes.
-
Box-Frame Angle Threshold
Before the boxes are grouped, the angle between the box reference frame and the custom reference frame will be calculated. If the angle exceeds this threshold, the corresponding box will not be grouped.
Suction Cup Offset
During the depalletizing process of the boxes, the fixed and non-replaceable suction cups are usually used. Due to various sizes of the boxes, the suction cups need to handle different boxes of various sizes. To avoid picking non-target boxes, the robot may not pick the target box with the center of the vacuum gripper. Instead, the corner or other parts of the vacuum gripper will be used. By setting the Offset Approach and configuring theSuction Cup Orientation, the problem of picking the wrong box can be effectively avoided, ensuring stable picking of boxes of various dimensions.
There are four offset strategies in total, and each strategy contains different parameters.
Default strategy
The software sequentially applies three strategies—Center to center, Edge midpoint to edge midpoint, and Corner to corner—to attempt the vacuum gripper offset.
Custom strategy priority
The software adheres to user-defined offset priorities to attempt the vacuum gripper offset.
There are three custom strategies in total, which are Center to center, Edge midpoint to edge midpoint, and Corner to corner. You can specify the attempt sequence for each strategy by configuring the Offset Priority parameter group.
-
Center to center: The center of the activated section of the vacuum gripper matches that of the box.
-
Edge midpoint to edge midpoint: The edge midpoint of the activated section of the vacuum gripper matches that of the box.
-
Corner to corner: The corner of the activated section of the vacuum gripper matches that of the box.
In the figure below, the orange rectangle represents the box, and the gray rectangle represents the vacuum gripper. The green part of the vacuum gripper indicates that the section is activated, while the red part indicates that the section is deactivated.
-
When a single section is activated
The figures below demonstrate the effects of using Center to center, Edge midpoint to edge midpoint, and Corner to corner from left to right.
-
When both sections are activated
The figures below demonstrate the effects of using Center to center, Edge midpoint to edge midpoint, and Corner to corner from left to right.
-
The effect is similar when there are more than two activated sections of the vacuum gripper.
By edge/corner ID sequence
The software attempts the offset according to the user-specified Edge/corner ID sequence (e.g., 11, 21, 31, 41).
The edge/corner IDs are generated automatically in the vacuum gripper configurator. The two-digit numbers at the edge of each section are edge/corner IDs, as shown below.
By distance to reference point
You need to set a reference point to enable this feature. The offset strategies will be sorted according to the distance between the TCP and the reference point, and the offset strategy with a TCP closer to the reference point has a higher priority.
-
Reference Point X/Y
Specify the position of the reference point. The reference point will appear in the 3D simulation area.
-
Keep Only High Priority Offset
Only offset solutions with a shorter distance to the reference point will be kept.
-
Max Plan Result Count
The maximum number of offset solutions that will be kept.
If there are 4 possible solutions for the vacuum gripper offset in total, and the value of this parameter is set to 2, the 2 solutions in which the vacuum gripper TCPs are closer to the reference point will be kept.
If there are 2 possible solutions for the vacuum gripper offset in total, and the value of this parameter is set to 4, only 2 solutions will be kept.
-
Application Example
The software will attempt the offsets in the order shown below.
Box Surface Coverage Lower Limit
This parameter specifies the minimum proportion of a box’s upper surface covered by the vacuum gripper when the vacuum gripper attempts to pick a box. If the coverage is below the threshold, it is considered that the vacuum gripper cannot attach to the box firmly and the picking is not allowed.
In addition, when the upper surface of a single box is larger than the entire vacuum gripper, and all sections of the vacuum gripper are activated, the coverage can reach 100%.
Supposing that this parameter is set to 50%, the picking feasibilities are shown as below.
Suction Cup Orientation
This parameter specifies the orientation of the vacuum gripper relative to the box group during picking.
-
Free: The orientation of the suction cup is not specified.
-
Custom: You can set the Suction Cup X-Axis Direction and Orientation Reference, and there are four relative positional relationships in total:
-
X-axes of objects are parallel to that of the vacuum gripper.
-
X-axes of objects are perpendicular to that of the vacuum gripper.
-
The X-axis of the vacuum gripper is parallel to the direction along which the boxes are grouped.
-
The X-axis of the vacuum gripper is perpendicular to the direction along which the boxes are grouped.
The figure on the left shows the effect when the X-axis of the vacuum gripper is parallel to the long edge of the box group, and the figure on the right shows the effect when the X-axis of the vacuum gripper is perpendicular to the long edge of the box group.
-
Box Drop Detection
In actual box depalletizing projects, DI check points will be added to the working surface of the suction cups to monitor whether the boxes are dropped during picking and placing by comparing the sensor signal changes.
After the DI check points are added to the vacuum gripper configurator, the software will determine which sensors on the working surface of the vacuum gripper are covered by the picked boxes, and therefore the DI signals that should be detected can be confirmed.
Remove Edge DI
In actual depalletizing process, when the sensors attached to the vacuum gripper near the edge of the box, the software may mistakenly detect that the box is dropped due to large gap between boxes, loose suction at the edge, or other reasons.
To prevent these issues, you can select this parameter and set Box Edge Removal Distance.
Box Edge Removal Distance
DI sensors that are within the removal range will not be taken into account in the box drop detection.
The width of the red frame shown in the figure below represents the Box Edge Removal Distance. For actual application, please adjust this parameter according to the on-site requirement.
Count of Picked Target Objects
Limit Total Count
Select Limit Total Count and you can set the maximum Planned Count and other parameters.
Planned Count |
The maximum number of target objects allowed to be picked during the project execution. |
Number of Picked Target Objects |
The total number of target objects picked so far. |
Count of Picked This Time |
The number of target objects picked this time. When the picking method is set to Regular picking or Box depalletizing with the Depalletizing Mode of One by one, this value is fixed at 1 if the picking proceeds normally. |
Limit Count per Picking
No limit |
There is no limit to the number of target objects that can be picked at a time. |
Maximum value |
The maximum target objects that can be picked at a time. |
Specified value |
Set a fixed limit on the number of target objects to be picked at a time. |
Picking Weight Limit
Once Limit Picking Weight is selected, you can specify Max Weight per Target Object and Max Total Weight. For example, when Max Weight per Target Object is set to 1 kg, and Max Total Weight is set to 2 kg, up to 2 boxes can be picked at once.
If the box quantity threshold defined by this parameter group differs from the threshold defined by Limit Count per Picking, the lower threshold will take effect.