Mech-Vision 1.6 Release Notes

This chapter introduces the new features, improvements, and resolved issues of Mech-Vision 1.6.

Mech-Vision 1.6.2 Release Note

New Feature

New Step - Predict Pick Points (Single Object Type)

A new Predict Pick Points (Single Object Type) Step was added in Mech-Vision 1.6.2, which is used to detect pickable objects in 2D images and depth maps and then output corresponding pick points. This Step is usually used for picking randomly stacked objects of the same type.

You do not need to install the Mech-Mind Software Environment to run this Step. You should use a port number of 60000 or above and import a deep learning model. Different types of objects require different models. Please contact Mech-Mind Technical Support for the model.

Please refer to Notes for “Predict Pick Points” Steps for detailed upgrade notes.

Improvements

“Pick Anything” Typical Application Projects Improved

The Pick Anything Typical Application projects include Pick Anything and Pick Anything (Without Bin). In Mech-Vision 1.6.2, you can use the improved “Pick Anything” typical application projects and do not need to download a special version.

Updated Step Name

Before Mech-Vision 1.6.2

In Mech-Vision 1.6.2

Grasp Pose Estimation

Predict Pick Points (Any Objects)

Resolved Issue

Issue with “Capture Images from Camera” Step

  • Resolved the issue that the setting of play back mode could not take effect for the Virtual Mode in the Capture Images from Camera Step.

Mech-Vision 1.6.1 Release Notes

New Features

Added Save All Option to File Menu

A Save All option was added to the File menu in the menu bar of Mech-Vision 1.6.1. You can use it to save all the opened projects in one click.

New Step - Convert Data Type

The newly added Convert Data Type Step can be used to convert one data type to another.

The currently supported data types include BoolList, DoubleList, String, StringList, Variant, VariantList, etc.

New Step - Convert Lengths Pixel-Wise to Physical

The newly added Convert Lengths Pixel-Wise to Physical Step can be used to calculate the actual length of a specified line segment in 2D images. This Step is applicable to scenarios where 2D images are used for measuring dimensions of objects with relatively planar surfaces.

New Step - Detect and Measure Oblong Hole

The newly added Detect and Measure Oblong Hole Step can used to detect the pixel-wise positions and sizes of oblong holes in images to facilitate subsequent calculation of physical dimensions in measurement scenarios.

New Step - Deep Learning Model Package CPU Inference

The newly added Deep Learning Model Package CPU Inference Step can be used for classification, instance segmentation, and object detection.

  • You can edit the ROI directly in this Step without depending on Scale Image in 2D ROI and Recover Scaled Images in 2D ROI Steps.

  • This Step is only compatible with DLKPACKC models exported by Mech-DLK 2.2.1 or higher versions.

  • It is recommended to use this Step for model inference when the requirement on inference speed is not high, and it is also recommended to deploy CPU models on computers with 12th Gen Intel Core i5 processors or above.

Added Features in Deep Learning Model Package Inference (DLK 2.2.0+)

  • Added the ROI Settings parameter for editing the ROI.

  • Added the Font Settings parameter for customizing font size in the visualized output.

  • When this Step is used with instance segmentation models, results below the confidence threshold are also displayed in the visualized output. The results above the threshold will be displayed in green and the results below the threshold will be displayed in red.

  • Added support for non-Latin characters in the model package path.

Added Rectify to Depth Map Parameter in Capture Images from Camera

The Camera Model and Rectify to Depth Map parameters were added in the Capture Images from Camera Step to make the pixels of the color image and depth map have one-to-one correspondences when cameras of DEEP V4 series and LSR V4 series are used.

Improvements

Example Projects Improved

Added a Brake Discs machine tending project in example projects.

Typical Applications - Pick Anything Projects Improved

The Piece Picking project is renamed as Pick Anything project in typical application projects.

The Pick Anything typical application projects are divided into Pick Anything projects and Pick Anything (Without Bin) projects. You do not need to deploy a deep learning model file any more. The project can be run after you calibrate the camera and configure camera parameters.

In Mech-Vision 1.6.1, the Predict Pick Points (Any Object) Step used in the project was updated as well.

Attention

  • You have to use a special version of Mech-Vision 1.6.1 to open the Pick Anything typical application project, and the Piece Picking typical projects of older versions cannot be executed in this special version software. Please contact Mech-Mind Technical Support to obtain the installation package of the special version of Mech-Vision 1.6.1.

  • Please use a server port number of 5000 or below while running the Pick Anything typical application project.

  • The new Pick Anything typical application projects do not support preload models.

Typical Applications - Large Non-Planar Workpieces Projects Improved

Deployment guidence for large non-planar workpieces machine tending projects was added. The deployment guidance includes camera configuration, recongnition, and deployment, which can help users to construct the project more conveniently.

Matching Model and Pick Point Editor Improved

The improvements of the Matching Model and Pick Point Editor are as follows:

  • Added Show normals option to show normals of the point cloud.

  • Added an eye icon in the upper right corner of the Model files area for displaying/hiding all point cloud models and pick points.

  • Added a Pose manipulator settings button in the lower left corner of the interface for configuring the display settings of the pose manipulator.

3D Fine Matching Step Improved

The improvements of the 3D Fine Matching Step are as follows:

  • Added the Speed Up on Large Object Quantities option in Correspondence Settings. It is recommended to enable this option when there are a large number of objects in the scene.

  • Added the Pose Filtering Settings parameter for filtering overlapping objects.

  • Deleted the Minimum Standard Deviation parameter.

Template Matching Step Improved

The improvements of the Template Matching Step are as follows:

  • Added the ROI parameter.

  • Added the Tile Gray Scale Upper Threshold parameter.

If you cannot find the above parameters, please right-click on the blank area in the Step Parameters panel and select Show all parameters in the context menu.

Blob Analysis Step Improved

The algorithm processing speed of this Step was improved.

Added Convert Types in the Convert Color Space Step

The three newly added convert types in the Convert Color Space Step are as follows:

  • RGB to HSI

  • RGB to HSV

  • RGB to YUV

Measurement Mode Improved

The improvements of the measurement mode are as follows:

  • The pixel coordinates of the cursor’s position is now displayed in the sketchpad.

  • Added Show coordinates option in the sketchpad settings for displaying/hiding the coordinates in the sketchpad.

Output Results from Check Pose Repeatability by Statistics Improved

  • When the input type is PoseListInput, the default unit of the output statistics is mm, and the numbers will be rounded up to 3 decimal places to meet the requirement for high-precision repetitive positional statistics.

  • When the input type is PoseListInput, the value of the Acceptable Position Coordinate Deviation parameter is allowed to be less than 1 mm to meet the requirement for high-precision repetitive positional statistics.

  • The upper limit of Threshold for Acceptable Position Coordinate Deviations was raised to 1000mm, and the upper limit of Threshold for Acceptable Euler Angle Deviations was raised to 360°.

Safety Alert Improved

When virtual data is used in a Mech-Vision project, warning alerts will pop up in Mech-Center and Mech-Viz. You will need to select a proper option in the pop-up windows to ensure production safety.

Support DEEP V4 and LSR V4 series in Depalletizing Typical Application Projects

In Mech-Vision 1.6.1, the DEEP V4 and LSR V4 series of Mech-Eye Industrial 3D Camera are supported in depalletizing typical application projects.

Pop-Up Windows Improved

When there are multiple missing Steps in the project, all of these Steps will be listed in one pop-up window instead of multiple separate pop-up windows.

Resolved Issues

3D Fine Matching Step

  • Resolved the issue that the data of color point cloud could not be loaded properly.

  • Resolved the issue that small objects might be incorrectly matched.

Camera Calibration

  • Resolved the issue that the Save button was unavailable for calibration on 4-axis and 5-axis robots.

  • Resolved the issue that the calculation result could still be incorrect when a wrong camera had been disconnected and a right one was reconnected properly in ETE calibration mode.

  • Resolved the issue that 2D camera could not be calibrated properly.

Mech-Vision 1.6.0 Release Notes

New Features

Data Storage Restructured

The Data Storage feature is restructured for better performance in troubleshooting and regression testing on-site. The detailed changes are as follows:

  • Data Storage features no longer require the Procedure Save Images and Step Parameters to be used.

  • Abnormal data are now stored in the new error_data folder in the data folder.

  • Data from multiple cameras can be saved altogether.

  • The naming conventions of the file and folder names in the data folder are changed.

Added Example Projects

Example Projects of typical applications in various industries are introduced in Mech-Vision V1.6.0 to provide a general picture of what Mech-Vision can do and how Steps work. These projects are highly user-friendly for beginners as they can be run with one click.

New 2D Matching Steps

The following 2D Matching Steps are added:

These Steps can be used to obtain the positions of target objects in an image and align the target objects/ROIs in different images to the same position/orientation.

New Step - Deep Learning Model Package Inference

Mech-Vision V1.6.0 provides built-in deep learning inference models. With the Deep Learning Model Package Inference (Mech-DLK 2.2.0+) Step, deep learning inference can be enabled without installing Mech-Mind Software Environment.

New Step - 3D Coarse Matching V2

A newer version of the 3D coarse matching Step, 3D Coarse Matching V2, is added. This Step is used to coarsely match the point cloud model with the original point cloud and output the coarsely calculated candidate poses of the target objects in the scene.

New Step - Calc Results by Python

Mech-Vision V1.6.0 now has the Python 3.6.8 environment integrated, which allows you to run a Python script directly in Mech-Vision with the Calc Results by Python Step.

Note

If extra Python libraries are required when you use this Step, these libraries must be installed to the python folder under the installation directory of Mech-Vision.

Tip

A Python library can be installed as follows:

  1. Open the Command Prompt or PowerShell program.

  2. Switch to the “python” directory of Mech-Vision, such as C:\Mech-Mind\Mech-Vision-1.6.x\python.

  3. Execute the following command: ./python -m pip install library_name.

Debug Output Improved

The function of the Debug Output panel is improved to better facilitate you in building and analyzing your Mech-Vision project.

Improvements include:

  • When Debug Output mode is disabled, output results (shown by running a Step with Visualize Output or by clicking on the connection between two Steps) are also displayed in the Debug Output panel.

  • Display output in a separate window: click on the icon in the upper right of the Debug Output panel to open a separate maximized window.

    When Debug Output mode is enabled, the separate window also allows for the following usage:

    • With a separate window opened, the result of the same Step is updated every time you run the Step/project and displayed in the same window.

    • Viewpoint adjustment of a 3D result in a separate window is memorized, and when you run the Step/project again, the result of the same Step is displayed with the same viewpoint, so that you can compare results from multiple executions easily.

    • You can open the results from multiple Steps in separate windows simultaneously for easy comparison.

Matching Model and Pick Point Editor Improved

  • Add point cloud from camera:

    • Adaptation for applications with target obejcts that are heavy and/or not easily movable: now you can directly generate a point cloud of the entire scene and then edit the point cloud, separate depth map acquisition of the background and target object not mandatory.

    • When acquiring depth maps of the background and target object separately, background depth map is acquired first to better fit actual usage.

  • New features for point cloud editing:

    • Remove selected points button added

    • Invert selection: select the points you want to keep first, and then invert the selection to quickly delete the unwanted points.

    • Generate edge point cloud: you can now generate a point cloud model of the object edges in the editor.

Improvements

User Interface Improved

The UI of Mech-Vision V1.6.0 is improved in the following aspects for better user experience:

  • Layout and color scheme redesigned

  • Default Layout option added to the View menu: automatically adjust the interface layout with a single click

  • Appearance and user interaction of the Step Parameters tab improved

  • Save, Undo, and Redo buttons added to the project toolbar at the top of the graphical programming workspace

  • Appearance of the search box in Step Library improved

  • Interface of Camera Viewer improved

  • Layout and text in Camera Calibration improved

Edit Procedure Parameter Feature Improved

In Mech-Vision V1.6.0, the Edit Procedure Parameter feature is improved in the following aspects:

Model Selection Parameter Added to the 3D Coarse Matching and 3D Fine Matching Steps

In Mech-Vision V1.6.0, the 3D Coarse Matching and 3D Fine Matching Steps now have a Model Settings parameter category for easy model selection from the model gallery.

The Check Pose Repeatability by Statistics Step Improved

The Check Pose Repeatability by Statistics Step is improved in the following aspects for better usability:

  • Name updated: from Poses Repeatability Statistics to Check Pose Repeatability by Statistics

  • Threshold for Acceptable Position Coordinate Deviations and Threshold for Acceptable Euler Angle Deviations parameters added for filtering out errors

  • Output orientation data now in Euler angles

  • Number of decimal places of the output data customizable

  • Layout of output XLSX table improved

  • Inputting multiple pose data simultaneously and saving results of different poses to different table sheets now supported

  • Restart option now automatically unchecked after the Step is run once with the option checked

  • Step Quick Info and Parameter tooltips updated

Compatibility for V4 and UHP Cameras

Mech-Vision V1.6.0 now supports image capturing and hand-eye calibration with V4 and UHP cameras.

Performance of the Instance Segmentation Step Improved

In Mech-Vision V1.6.0, instance segmentation conducted with the new Deep Learning Model Package Inference (Mech-DLK 2.2.0+) Step has better performance compared to the old Instance Segmentation Step.

Updated Step Names

The names of the following Steps are updated for easier understanding of the functions. When you open a project in Mech-Vision V1.6.0, the name of the Steps in your project are automatically updated.

Before V1.6.0

V1.6.0

Map to Multi Pick Points

Map to Multiple Pick Points

Calc Included Angle Between Specified Axis of Poses

Calc Included Angles between Specified Axes of Poses

Validate Box Masks

Validate Box Object Masks

Segment Depth Image

Segment Depth Map

Get Highest Areas in Depth Image

Get Highest Layer Regions in Depth Map

Pose Adjustment Collection

Adjust Poses

Rotate Poses to Goal Direction

Rotate Poses’ Axes to Specified Directions

Classification by Point Clouds’ Sizes

Classify Point Clouds by Dimensions

Sort and Output Index List

Sort List and Output Index List

Grasp Pose Estimation

Estimate Pick Points

Set Pose Quaternion

Set Pose Quaternions

Inverse Poses

Invert Poses

Validate Poses by Included Angle to Reference Direction

Validate Poses by Included Angles to Reference Direction

Binarize Image

Image Thresholding

CloudXYZ To CloudNormal

From CloudXYZ to CloudNormal

Cloud Num Limit

Trim Point Cloud List

From Poses to Euler Angles

Convert Quaternions to Euler Angles in Poses

Merge Cloud Vector

Merge Point Cloud Lists

Find 2D Contour at Specified Hierarchical Level

Find 2D Contour at Specified Inner-Outer Level

Calc Length Along Axis

Calc Point Cloud Spans along Axes

Poses Repeatability Statistics

Check Pose Repeatability by Statistics

Calc Minimum Bounding Rectangles of Masks

Calc Minimum Circumscribed Rectangles of Masks

Calc Distance to Reference Pose

Calc Distances between Poses

Smooth Trajectory

Smooth Path

Make Poses Point to Reference Place

Point Poses to Reference Positions

From Disparity Image to Depth Image

Convert Disparity Image to Depth Map

Compose Pose From Quaternion and Translation

Compose Poses from Quaternions and Translation Vectors

Merge LineSegment Vector

Merge LineSegment Lists

Merge Depth Images

Merge Depth Maps

Calc Angle Between Vector3D

Calc Angles between Vector3Ds

Mask Cluster

Mask Clustering

Cloud Smooth And Normal Estimation

Smooth Point Cloud and Estimate Normals

Compose Vector3D From Numbers

Compose Vector3Ds from Numbers

Compare Two Depth Image

Compare Two Depth Maps

Generate Point Cloud of Ring

Generate Ring Point Cloud

Adjust Poses to Obtain Acurate Trajectory

Adjust Targets to Get Correct Path

Load Poses in Trajectory and Apply Affine Transform

Load Targets in Path and Apply Affine Transform

Calc Poses from Heat Map of Graspability

Calc Poses from Heat Map of Pickability

Reverse

Reverse List

Cloud Filter By Model And Pose

Filter Point Cloud by Model and Poses

Merge Point Clouds with Similar Height

Merge Point Clouds with Similar Heights

Calc Mask’s Span on Given Line

Calc Mask Spans on Given Lines

Save Local Areas Around Poses as 3D ROI

Save Regions around Poses as 3D ROIs

Adjust Trajectory Circular Motion

Change Circular Motion Direction of Path

Detect Bin (Inscribed Rect Sides)

Detect Bin (Max Inscribed Rect)

Detect Bin (Largest Inscribed Rect)

Detect Bin (Max Inscribed Rect) V2

Filter Poses Outside Bin

Remove Poses outside Bin

Pixel-wise Graspability Evaluation

Pixel-Wise Pickability Evaluation

Mask Filter

Filter Masks

2D Poses To 3D Poses Base Orthographic Projection

Convert Poses 2D to 3D According to Orthographic Projection

Get Valid Ring Clouds

Filter Ring Point Cloud List

Project A Point onto A Plane

Project Points onto Plane

Compose Quaternion From Two Axis (Right-Hand)

Compose Quaternions from Two Axes (Right Hand Rule)

Calc Diagonal Length

Calc Diagonal Lengths

Generate Rect Traj

Generate Rect Path

Generate Traj By Contour

Generate Path from Contour

Is Z Value of Input Greater than Threshold

Compare Z Values of Poses with Threshold

Rectify Ring Pose

Rectify Ring Object Poses

Calc Center Point of Non-zero Areas

Calc Center Points of Non-Zero Regions

Calc Rect 2D Pose

Calc 2D Poses of Rectangles

Calc Pixel Size at Specified Height

Calc Pixel Sizes at Specified Heights

Coherent Line Drawing

Draw Coherent Lines

Image Transform

Adjust Image

Determine Pixel Size

Determine Pixel Sizes

Extract Empty Areas in Depth Image within 3D ROI

Extract Empty Regions in Depth Map within 3D ROI

Rotate Images By Provided Poses

Rotate Images by Specified Poses

Perspective Transform

Perspective Transformation

Clouds In 3d Box

Extract Point Cloud in 3D Box

Move Cloud Along Set Dir

Move Point Cloud along Specified Direction

Cloud Distortion Correction

Correct Point Cloud Distortion

Detect Obscured Objects

Detect Occluded Objects

Transform Plane Cloud To Align Direction

Align Plane Point Clouds

Cloud Scale

Scale Point Cloud

Calc Point Cloud Curvature

Calc Point Cloud Curvatures

Calc Edge Points Normal

Calc Edge Point Cloud Normals

Pose Transformed by Quaternion in Object Coordinate

Rotate Poses by Quaternion Vectors in Object Frames

Calc Length of Vector3D

Calc Lengths of Vector3Ds

Pose Transformed by Pose2 in Object Coordinate

Transform Poses by Matrix in Object Frames

Calc Cross Product of Vector3D

Calc Vector3D Cross Products

Calc Normalized Vector3D

Calc Normalized Vector3Ds

Compose Quaternion From Axis and Angle

Compose Quaternions from Axes and Angles

Rotate Poses to Directions With Symmetry Constraint

Rotate Poses’ Axes to Specified Directions under Symmetry Constraints

Easy Point to Reference Place

Easy Point Poses to Reference Position

Easy Coordinate Transform

Easy Frame Transformation

Inverse Quaternions

Get Inverses of Quaternions

Calc Dot Product of Vector3D

Calc Vector3D Dot Products

Calc Distance from 3D Points to Plane

Calc Distances from 3D Points to Plane

Measure Circle

Measure Circles

Calc Distance from 3D Points to Intersection of Two Planes

Calc Distances from 3D Points to Intersection of Two Planes

Replace Element In Vector

Replace Elements in List

Generate Test Cloud

Generate Test Point Cloud

Save Trajectory Points

Save Path Targets

Evaluate Variation of Depth Image

Evaluate Depth Map Fluctuation

Calculate Calib-board Pose

Calc Calibration Board Poses

Insert End Points And Send Motion Params

Insert End Target and Send Motion Params

Load 2d Trajectory

Load 2D Path

Generate Spiral Traj

Generate Spiral Path

Generate Trajectory Given Depth Image

Generate Path Given Depth Map

Generate Zigzag Traj

Generate Zigzag Path

Auto Trajectory

Extract 2D Path

Merge Label List

Merge Label Lists

Adjust Poses by Obstacles

Adjust Targets by Obstacles

Trajectory Points Matching

Path Target Matching

Adjust Poses by Obstacles V2

Adjust Targets by Obstacles V2

Validate 2D Poses Within Mask

Validate 2D Poses by Mask

Calc Mask Distance

Calc Mask Distances

Varying Normal Area

Extract Regions of Large Normal Deviations

Map Mask Non-zero Area

Extract Image Regions by Mask

Detect Graspable Rectangles

Detect Pickable Rectangles

Depth Cluster along Scan Lines

Depth Clustering along Scan Lines

Divide Cloud into Smaller Parts Evenly

Divide Point Cloud into Smaller Parts Evenly

Deep Learning Inference

Deep Learning Inference (DLK 2.1.0/2.0.0)