Configuration Workflow

You are currently viewing the documentation for a pre-release version (2.2.0). To access documentation for other versions, click the "Switch Version" button located in the upper-right corner of the page.

■ If you're unsure about the version of the product you are using, please contact Mech-Mind Technical Support for assistance.

This section introduces the deployment prerequisites and the specific configuration workflow of the Picking Simulation tool, as shown in the figure below.

specific configuration workflow

The following sections describe each step in detail.

Deployment Prerequisites

Before using the Picking Simulation tool, you need to first build a basic workflow in Mech-Viz to ensure the simulation process runs smoothly.

basic project example

Before starting the simulation verification, make sure the current workflow meets the specifications supported by the picking simulation. If the workflow does not meet the requirements, the system will display an error message. The main items to check include:

  1. Exit restrictions: The Set DO and Message Branch Steps only support fixed exits, and valid exits must be configured.

  2. Prohibited Steps: The workflow must not contain the following unsupported Steps:

    • Classify

    • Update Scene Objects

    • Update Held Target Object

    • Update Pallet Pose

    • Check Tool

    • Counter

    • Check Completion

  3. Structural restrictions: The workflow must not contain loop structures.

  4. Length restrictions: The number of Steps in the workflow must not exceed 10,000.

  5. Vision-type Step restrictions: The workflow must contain one and only one Vision Move Step, and the Step does not support reusing vision results or sharing vision results.

Before each picking simulation, make sure Keep Execution Status in the toolbar is disabled to avoid abnormal historical states of Steps such as Vision Move caused by multiple rounds of simulation, ensuring the accuracy of the simulation process.

Layout Simulation

Before performing picking simulation, you need to first build a virtual scene that includes carriers and target objects. By properly setting the placement mode and number of target objects, the simulation environment should approximate the actual working conditions as closely as possible, providing a foundation for subsequent picking strategy verification.

Select Carrier

First, specify the carrier model to be used in the simulation scene. You can select an existing carrier from an existing solution, or create a new carrier and configure its parameters.

  • If an existing carrier meets the requirements, you can select it directly.

  • If you need a new carrier, click New carrier to enter the carrier configuration interface, where you can set the carrier name, scene model, carrier dimensions, and other parameters. The scene model is the carrier type setting, supporting bins, cuboids, and cylinders. When cuboid or cylinder is selected, the Use as carrier parameter is selected by default and cannot be deselected.

carrier configuration
  • To modify an existing carrier configuration, click the Edit button.

For the specific configuration workflow of creating or editing a carrier, please refer to Scene Object Configuration.

If you configure a carrier through Resources  Scene Objects, the Use as carrier option is not selected by default. Please select it manually.

Configure Target Object

After the carrier is determined, you need to select the target objects for simulation and configure their display color. When selecting target objects:

  • If an existing target object in the solution meets the requirements, you can select it directly.

  • If you need a new target object, click New target object to go to the Target Object Editor to create a target object. You can create a target object template from an STL/OBJ file or a basic geometric shape.

  • To modify an existing target object configuration, click the Edit button to enter the Target Object Editor.

After the target object template is configured, click the color swatch next to Target object color to set the display color of the target object in the simulation.

Configure Placement Mode

Based on actual production requirements, select the placement mode for target objects, including randomly stacked and neatly arranged.

Randomly Stacked

Target objects are randomly scattered in the carrier, simulating the randomly stacked situation in actual production. The following table describes the specific parameters and their configuration.

Parameter

Description

Placement pose

This parameter is used to set the target object orientation. The target object coordinate system is aligned with the carrier coordinate system by default, meaning the X/Y/Z axes of the target object point in the same direction as the X/Y/Z axes of the carrier.

  • Click Set pose. In the pop-up window, select viewpoints for pose generation. After selecting viewpoints, the system will generate target object models with different orientations based on the selected viewpoints, which will be used to construct the target object placement in subsequent simulations. Both default viewpoints and custom viewpoints are supported. Please select based on actual requirements.

    • Default viewpoints: Typical target object orientations preset by the system.

    • Custom viewpoint: Adjust the target object to the desired angle in the visualization area, then click Acquire to generate a custom viewpoint.

  • All selected viewpoints will participate in sorting together. The sequence numbers are arranged in the order of selection.

  • After completing the viewpoint selection, you can further set the rotation angle range around the X/Y/Z axes to control the range of pose variation.

  • After the configuration is complete, click OK. You can click Preview to view the generated target object placement poses.

Drop zone

This parameter is used to define the random drop range of target objects within the area. If a target object falls outside the area, it will be automatically removed and dropped again until the drop point is within the specified area.

  • Click Add region.

  • In the 3D simulation space, hold Ctrl + left mouse button to adjust the anchor points of the region and select the region of interest. For specific adjustment methods, please refer to Selection Tool Instructions. Alternatively, you can manually enter coordinates to define the drop boundary. Multiple independent regions can be added.

Number of target objects

This parameter is used to set the total number of target objects for the simulation. Please set a reasonable number of target objects based on actual requirements. The more target objects, the higher the simulation complexity, and the lower the running speed.

Neatly Arranged

Target objects are placed in fixed positions and sequence, simulating neatly arranged situations. The following table describes the specific parameters and their configuration.

Parameter

Description

Placement pose

This parameter is used to set the target object orientation. Please refer to the pose setting method under Randomly Stacked, and select the appropriate viewpoints and rotation angles to ensure the target object poses match the actual scenario.

Arrangement direction

This parameter is used to specify the relative arrangement order of target objects in the carrier, including carrier positive X-direction, carrier negative X-direction, carrier positive Y-direction, and carrier negative Y-direction.

The system will cyclically arrange the selected poses along the chosen direction. For example, when "Carrier positive X-direction" is selected, target objects with different poses will be arranged row by row along the carrier’s X-axis; when "Carrier positive Y-direction" is selected, target objects with different poses will be arranged column by column along the carrier’s Y-axis. The specific effects are shown in the figure below:

pose orientation
Select the carrier name under Resources  Scene Objects to display the carrier coordinate axes in the 3D simulation space, making it easier to determine the arrangement direction.

Placement settings

This parameter is used to set the number of rows, columns, and layers of target objects to determine the overall arrangement structure.

Placement spacing

This parameter is used to set the spacing between adjacent target objects, which can be configured separately for the X, Y, and Z directions. If the spacing is too small, collisions may occur during picking. It is recommended to leave a safe distance.

Placement offset

This parameter is used to adjust the overall position of the target object arrangement in space. You can set offset values in the X, Y, and Z directions based on the carrier center to achieve overall translation of the target objects.

When setting the above parameters, it is recommended to combine actual production and picking requirements and debug the target object placement through the visualization display in the 3D simulation space.

Pick Settings

After building the target object scene, you need to configure the picking strategy to more realistically reflect the actual picking logic and evaluate the impact of picking stability and failure on the overall picking effectiveness.

Filter Poses by Angle

By setting a pose filter angle threshold, you can filter out pick points with excessive tilt angles. When the angle between a pick point’s Z-axis and the world coordinate system’s Z-axis exceeds this threshold, the pick point will be filtered out and will not participate in subsequent path planning.

If a large number of pick failures occur during the simulation, you can go to the Target Object Editor to adjust the pick point settings for the target object, so that more pick points can participate in planning.

Set Sorting Strategy

The sorting strategy determines the priority order of picking. You can select different sorting strategies based on actual requirements. Multiple strategies are available, including descending by Z height, ascending by overlap ratio, ascending by Z-axis angle, etc. The details are as follows.

Sorting strategy

Illustration

Description

Descending by Z height

descending by Z height

Sort target objects by the Z-coordinates of their pick points in descending order.

Ascending by distance to carrier center

ascending by distance to carrier center

Sort target objects by the distance from their pick points to the carrier center in ascending order.

Ascending by overlap ratio

ascending by overlap ratio

Sort target objects by their overlap ratio with other objects in ascending order.

Ascending by overlap ratio > Ascending by Z-axis angle

ascending by overlap ratio Z axis angle

Sort target objects first by overlap ratio with other objects in ascending order. When overlap ratios are the same or similar, further sort by Z-axis angle in ascending order.

Ascending by overlap ratio > Ascending by distance to carrier center

ascending by overlap ratio distance to carrier center

Sort target objects first by overlap ratio with other objects in ascending order. When overlap ratios are the same or similar, further sort by distance to carrier center in ascending order.

Ascending by Z-axis angle

ascending by Z axis angle

Sort target objects by the angle between their pick point’s Z-axis and the world coordinate system’s Z-axis in ascending order.

Set Simulation Count

Simulation count is used to control the number of times picking is repeatedly executed. Two modes are supported: Number of consecutive picks and Number of bins.

  • Number of consecutive picks: Picking is continuously executed within the same bin until the specified count is reached or there are no more target objects to pick in the bin. After each pick, the system records the result. This mode is suitable for verifying picking strategies in single-bin scenarios.

  • Number of bins: Picking is continuously executed across multiple bins. After each bin is completed, the next bin is automatically generated until the specified number of bins is reached. This mode is suitable for evaluating the stability of picking strategies in batch operations.

Continue on Pick Failure

This parameter specifies the handling method when picking fails.

  • When enabled, if pick planning for a target object fails, the system will remove the object with the highest Z-axis position and continue attempting to pick the remaining objects until the bin is empty. This mode is suitable for verifying the pick success rate of target objects under ideal conditions, ignoring the impact of interfering objects.

  • When disabled, if pick planning for a target object fails, all subsequent picking simulation will stop immediately. This mode is suitable for verifying the bin clearance rate in real scenarios, more closely approximating actual results.

Continue on pick failure can be used in combination with Simulation count, and it affects the simulation picking process.
  • When Continue on pick failure is enabled:

    • In Number of consecutive picks mode, only the specified number of picks is executed within a single bin, and the simulation ends when completed.

    • In Number of bins mode, after each bin is completed, the next bin is automatically generated, and picking continues until the specified number of bins is reached.

  • When Continue on pick failure is disabled, regardless of the simulation mode selected, the simulation is terminated immediately once pick planning fails.

Camera Deployment (Optional)

If you need to perform camera collision detection or verify the camera’s field of view coverage during picking simulation, or need to save image data captured by the camera during simulation for subsequent use in the vision recognition workflow, please enable camera deployment and perform the relevant configuration. This step is optional. If you do not have the above requirements, you can skip it without affecting the picking simulation process.

Currently, only Eye to Hand camera installation is supported.

Select Camera Type

First, enable Camera settings. Click Select camera, and in the drop-down list, you can select an existing camera or create a new one.

  • If an existing camera meets the requirements, you can select it directly.

  • If you need a new camera, click New camera to enter the camera configuration window.

    1. Enter a custom camera name in the Object name text box.

    2. The scene model defaults to Camera and cannot be changed.

    3. In the Object Settings interface, select the camera model, submodel (if applicable), collision model, and display model based on actual conditions.

    4. It is recommended to enable Camera field of view during initial configuration to facilitate subsequent adjustment of the camera model pose.

    5. The newly added camera model is positioned at the origin of the robot base coordinate system by default. Click Object Pose to switch the configuration interface, where you can modify various parameters to adjust the camera model’s pose.

camera configuration
  • To modify existing camera settings, click the Edit button.

Under Resources  Scene Objects, right-click the camera name and select Show/hide camera field of view. This setting is linked with the Camera field of view toggle in the configuration window, meaning the visibility states of the two controls are always synchronized.

Simulate Real Point Cloud

The system enables simulated point cloud noise by default to replicate the actual camera capture effect.

When enabled, you can set the point cloud filter angle to remove point clouds with angles greater than the set value, simulating the field-of-view limitation when a real camera acquires point clouds.

Check Camera Field of View

Click Acquire data to check the coverage of the camera’s field of view, ensuring that the capture area matches the actual requirements and that target objects are always within the camera’s field of view.

Save Image Data

After enabling Save image data, you can set the maximum storage size and the image save path. The saved images can be used for the virtual mode in Mech-Vision.

Images are saved by default in the Data/PickSim folder under the solution folder.

Simulation Verification

After all configurations are complete, you can proceed to the simulation verification phase. The system will simulate the robot picking process and collect real-time statistics on picking results and key data, facilitating comprehensive evaluation of the feasibility and risk points of the picking solution.

Start Simulation

After completing the above configurations, you can proceed to the picking simulation verification phase. Click Start simulation, and the system will execute picking based on the current configuration and update Pick attempts and Successful picks in real time.

After the simulation is automatically completed, one of the following prompts will appear depending on whether all target objects were picked:

all picked
some failure
no pick point

To view the picking dynamics, you can switch between bins, use the slider and frame controls to flexibly view the picking effects of different bins, and play the entire picking process frame by frame.

picking dynamics

Pick Statistics

Pick statistics is used to view the detailed results and failure reasons of previous simulations, supporting quantitative analysis of picking performance.

Click the Pick statistics button to open the statistics results window. The failure reasons can be used as a basis for optimizing picking strategies. It is recommended to pay close attention to frequently occurring failure types.

statistics

For further locating specific planning failure nodes or target objects, you can perform detailed analysis through the plan history function or by viewing logs.

Only target objects with planning failures will be recorded in plan history. Target objects with successful planning will not be recorded. You can follow the failure prompt to navigate to Plan History to view the corresponding planning content and results.
reason

In addition, by double-clicking the pick point information on a target object, you can view the vision point details. In the Vision Result Details window, click Pick point and Target object under Path planning result to view the planning status and navigate to Plan History for troubleshooting.

vision point
vision point detail

Is this page helpful?

You can give a feedback in any of the following ways:

We Value Your Privacy

We use cookies to provide you with the best possible experience on our website. By continuing to use the site, you acknowledge that you agree to the use of cookies. If you decline, a single cookie will be used to ensure you're not tracked or remembered when you visit this website.