Mech-Vision 1.8 Release Notes

This document introduces the new features, improvements, and resolved issues of Mech-Vision 1.8.

Mech-Vision 1.8.0 Release Notes

This document introduces the new features, improvements, and resolved issues of Mech-Vision 1.8.0.

New Features

New “Production Interface” Feature

Mech-Vision 1.8.0 has introduced a new production interface. Engineers can develop a production interface through several configuration steps, and on-site operators can execute the production stably through a graphical user interface. The production interface mainly provides the following functions:

Get quick knowledge of production status: The production interface provides visual information panel to help people get quick knowledge of production status and project execution status, which can help Operator identify and judge production abnormalities.
Quickly view the production results: The production interface provides various running views, including Station live, Recognition result, Deep learning result, and Picking result. Operator can quickly view the production results.
Add new workpieces and switch workpiece types: The production interface provides various ways to create workpiece models for different types and shapes of workpieces, and set production recipes to add new workpieces and switch workpiece types.
Facilitate maintenance and troubleshooting: The production interface provides the role-based access control to effectively reduce the risk of data leakage. It also provides the information of troubleshooting, production logs, and alert records to help the operator quickly troubleshoot and resolve faults to achieve stable production.

After Mech-Vision is upgraded to 1.8.0, pay attention to the following notes when you use the production interface feature.

The production interface that you have deployed with a typical application in Mech-Vision 1.7.0 or earlier cannot be used anymore.
The production interface that you have customized for your application in Mech-Vision 1.8.0 or earlier can still be used. However, the production interface may fail to be opened due to the change in several files. To solve these issues, please contact Mech-Mind Technical Support.

New Pose Adjustment Tool

Mech-Vision 1.8.0 has introduced the “Adjust Poses V2” Step, which provides a new “pose adjustment tool”. This tool improves the usability of pose adjustment, enables the visualization of poses while adjusting, and reduces the learning cost of pose adjustment. You can achieve pose adjustment, pose sorting, pose filtering, and other pose manipulation operations just through three steps, namely pose adjustment, processing rules, and general settings.

Adapter Generator

Mech-Vision 1.8.0 introduced the Adapter generator. By configuring TCP/IP commands for communication between an external device and the vision system, you can efficiently generate an Adapter program. This program seamlessly controls Mech-Vision and Mech-Viz, facilitating smooth data acquisition. As a result, the development time of the Adapter program is reduced and the overall project delivery efficiency is enhanced.

The Adapter generator provides the following functions:

Strong control function: Four commands can be generated, covering basic operations on Mech-Vision and Mech-Viz.
- Run the Mech-Vision or Mech-Viz project.
- Switch the parameter recipe of the Mech-Vision project.
- Set the exit port for the “Branch by Msg” Step in the Mech-Viz project.
- Set Step parameters in the Mech-Vision or Mech-Viz project.
- Eye to hand (ETH) and eye in hand (EIH) mounting modes are supported.
Flexible configuration: The command format can be customized, enabling you to decide the parameters to be returned.
- Optional parameters for vision point data: pose type, number of poses sent, label, custom output, etc.
- Optional parameters for robot path data: pose type, number of poses sent, motion type, label, etc.
Quick program generation: The Adapter program can be generated quickly and conveniently, and is open to secondary development.
- By clicking a graphical interface, you can easily generate the Adapter program according to your needs.
- The generated Adapter program is open to custom development, greatly improving programming efficiency.

For details on the Adapter generator, please refer to Adapter Generator Guide.

TCP/IP Command Reference for Standard Interface

Mech-Vision 1.8.0 has provided the TCP/IP command reference for Standard Interface on the interface, which provides a convenient way for on-site engineers to check the TCP/IP command format, returned data and status codes. The examples may be used to quickly demonstrate the sent commands and returned data via Standard Interface communication.

For details on TCP/IP commands, refer to TCP/IP Interface Commands.

New Error Analysis Tool

Mech-Vision 1.8.0 has introduced the error analysis tool. When an error occurs during recognition and locating, this tool can be used for troubleshooting. Based on the on-site situation you described, the tool will recommend specific items to check, including camera accuracy, robot absolute accuracy and repeatability, and the result of extrinsic parameters after calibration. This tool cannot be used to check the project logic or analyze the matching accuracy.

Text Detection, Text Recognition, and Unsupervised Segmentation Are Supported in Step “Deep Learning Model Package Inference”

In Mech-Vision 1.8.0, the Deep Learning Model Package Inference Step has added the following functions:

Enabled the “text detection” function: can locate and label text in the input images.
Enabled the “text recognition” function: can recognize and output the text in the text area that has been located.
Enabled the “unsupervised segmentation” function: can segment defect areas in images by training models solely on defect-free images.
Enabled the defect determination rules: can segment defects based on the configured defect determination rules.

Added the “Predict Object Poses (Sim2Pick)” Step

Mech-Vision 1.8.0 has added the “Predict Object Poses (Sim2Pick)” Step. When the point cloud of the single object type is input, this Step will directly recognize the object from the input point cloud and output its pose. This Step is generally used in the following scenarios:

Workobjects are randomly stacked and there is a reasonable number.
Workobjects are highly reflective and neatly arranged.
Workobjects are neatly arranged and closely placed.

Other New Steps

Mech-Vision 1.8.0 has added the following Steps:

No.	Step	Description
1	3D Matching and Classification (Multiple Models)	This Step can match the objects in the scene by using multiple point cloud models and output the poses and class labels of the objects.
2	Remove Overlapped Objects (Lite)	This Step is used to remove overlapped objects, i.e., when multiple vision results exist at the same position of a workpiece, ineligible vision results can be removed by adjusting parameters.
3	Remove Coinciding Objects	This Step is used to remove objects that are completely covered or covered by other objects according to the pose confidence values.
4	Sort 3D Poses V2	This Step is used to sort the 3D poses according to specified rules. Compared with the “Sort 3D Poses” Step, this Step included two new sorting orders: “Sort by S shape on plane” and “Sort by Z shape on plane.”
5	Calc Calibration Board Pose and Check Camera Intrinsic Parameters	This Step can detect the calibration circle centers on the 2D image, and then compute the pose of the calibration board and the poses of calibration circles according to the camera intrinsic parameters and the input depth map. In addition, this Step can be used to estimate the accuracy of the camera’s intrinsic parameters based on the calibration circle poses.

No.

Step

Description

3D Matching and Classification (Multiple Models)

This Step can match the objects in the scene by using multiple point cloud models and output the poses and class labels of the objects.

Remove Overlapped Objects (Lite)

This Step is used to remove overlapped objects, i.e., when multiple vision results exist at the same position of a workpiece, ineligible vision results can be removed by adjusting parameters.

Remove Coinciding Objects

This Step is used to remove objects that are completely covered or covered by other objects according to the pose confidence values.

Sort 3D Poses V2

This Step is used to sort the 3D poses according to specified rules. Compared with the “Sort 3D Poses” Step, this Step included two new sorting orders: “Sort by S shape on plane” and “Sort by Z shape on plane.”

Calc Calibration Board Pose and Check Camera Intrinsic Parameters

This Step can detect the calibration circle centers on the 2D image, and then compute the pose of the calibration board and the poses of calibration circles according to the camera intrinsic parameters and the input depth map. In addition, this Step can be used to estimate the accuracy of the camera’s intrinsic parameters based on the calibration circle poses.

Improvements

Improved Software Interface and User Interaction

Mech-Vision 1.8.0 improved the software interface and user interfaces in the following ways.

The Help menu on the menu bar provides access points for the online community. Through them, you can access the online community, visit the download center, view video tutorials, and feedback comments.
Simplified the layout of the toolbar buttons for more convenient operation.
Improved the context menu in the graphical programming workspace, making the functions and their arrangement more logical.
Added a quick entry for Steps under “Advanced Components.” You can open the tool for these Steps by clicking the Open the editor button.
Added an option in the context menu of the graphical programming workspace for one-click closure of all Step visualizations.

Improved Solution Library

To ensure the quality and performance of the projects and solutions in the solution library, some projects and solutions have been temporarily removed from the solution library in Mech-Vision 1.8.0. These projects and solutions will be improved and re-uploaded in subsequent versions. See the following for details.

Improved the “Capture Images from Camera” Step

Mech-Vision 1.8.0 improved the “Capture Images from Camera” in the following ways.

Added an entry to open Mech-Eye Viewer in the Step Parameters panel and the “Choose the camera and the calibration parameter group to use” interface, which allows you to open Mech-Eye Viewer quickly if needed.
Added the “Obtain Flange Pose from Parameters” parameter. In the EIH setup, the robot flange pose at image capturing can be directly input.
From Mech-Vision 1.8.0, Mech-Eye TOF cameras, MindVision 2D cameras, and SmartRay 3D laser profilers are no longer supported in the “Capture Images from Camera” Step.

Improved the “Path Planning” Step

Mech-Vision 1.8.0 improved the Path Planning Step. An Enable option was added under the Update Scene Object Settings. Once this option is switched on, Scene Object Names, Scene Object Dimensions, and Scene Object Poses input ports will be added to the Step to update the poses and dimensions of the scene objects.

The path planning tool has been improved as well. See the following for details.

Added a Global Configuration step, which provides the options “Plan all vision results”, “Update bin pose with vision”, and “Box depalletizing.”
The 3D simulation area now includes an orbit control, as well as buttons for panning the view, zooming views, screen adaptation, switching between perspective/orthographic views, and displaying key buttons.
Modified the interface for adding and selecting the workflow.

Improved Matching Model and Pick Point Editor

Mech-Vision 1.8.0 improved the Matching Model and Pick Point Editor. See the following for details.

No.	Improvement	Description
1	Enabled to “Create geometric model”	Allowing to quickly generate the point cloud model using the created geometric model.
2	Improved tool interface	Improved the 3D visualization space and interactive operations. Demo GIFs are also provided to introduce the operations in the 3D visualization space.
3	Enhanced display settings	Allowing adjustment of the manipulator thickness in Edit > Display Settings.
4	Improved the operation for selecting multiple point cloud models	The shortcut keys were changed from “Ctrl+Left mouse button” to “Shift+Left mouse button”.

No.

Improvement

Description

Enabled to “Create geometric model”

Allowing to quickly generate the point cloud model using the created geometric model.

Improved tool interface

Improved the 3D visualization space and interactive operations. Demo GIFs are also provided to introduce the operations in the 3D visualization space.

Enhanced display settings

Allowing adjustment of the manipulator thickness in Edit > Display Settings.

Improved the operation for selecting multiple point cloud models

The shortcut keys were changed from “Ctrl+Left mouse button” to “Shift+Left mouse button”.

Improved Camera Calibration Tool

Mech-Vision 1.8.0 improved the camera calibration tool. See the following for details.

No.	Improvement	Description
1	Improved the calibration procedure	Streamlined redundant operations and deprecated functions within the calibration tool.
2	Add more important tips	Supplemented illustrations and informative prompts for key steps and parameters within the calibration procedure.
3	Enabled to open the Error Analysis Tool	After calculating the extrinsic parameters, you can verify the robot absolute accuracy, robot repeatability, and extrinsic parameter accuracy with the Error Analysis Tool.
4	Facilitated the analysis of hand-eye calibration result	Enabled to analyze the extrinsic parameters and robot accuracy according to the calibration data, troubleshoot rapidly, and provide relevant improvement suggestions. This feature is only available for calibration on 6-axis robots (excluding gantry robots) and Mech-Eye cameras, including automatic calibration and manual calibration using “Multiple random calibration board poses” to collect data.
5	Streamlined the calibration process for gantry robots and enhance the guidance	Provided the access link for the gantry robot calibration manual and important prompts.
6	Unable to view waypoints in Mech-Viz anymore	During camera calibration, the waypoints can only be viewed in the Scene viewer in the calibration interface instead of Mech-Viz.

No.

Improvement

Description

Improved the calibration procedure

Streamlined redundant operations and deprecated functions within the calibration tool.

Add more important tips

Supplemented illustrations and informative prompts for key steps and parameters within the calibration procedure.

Enabled to open the Error Analysis Tool

After calculating the extrinsic parameters, you can verify the robot absolute accuracy, robot repeatability, and extrinsic parameter accuracy with the Error Analysis Tool.

Facilitated the analysis of hand-eye calibration result

Enabled to analyze the extrinsic parameters and robot accuracy according to the calibration data, troubleshoot rapidly, and provide relevant improvement suggestions. This feature is only available for calibration on 6-axis robots (excluding gantry robots) and Mech-Eye cameras, including automatic calibration and manual calibration using “Multiple random calibration board poses” to collect data.

Streamlined the calibration process for gantry robots and enhance the guidance

Provided the access link for the gantry robot calibration manual and important prompts.

Unable to view waypoints in Mech-Viz anymore

During camera calibration, the waypoints can only be viewed in the Scene viewer in the calibration interface instead of Mech-Viz.

Improved Step Library

Mech-Vision 1.8.0 improved the Step Library. Some of the Steps were removed, and the Step categories of some Steps were changed. See the following for details.

Removed Step	Steps moved to “Legacy”
Calc Disparity Image, Calc 2D Poses of Rectangles, Calc Poses and Dimensions of Rectangles, 2D Matching, Count Boxes, Deep Learning Inference (Mech-DLK 2.1.0/2.0.0), Defect Detection, Detect Fiducial Markers, Detect Edges (Deep Learning), Image Classification, Generate Rectangle Candidates, Instance Segmentation, Load 2D Templates, Object Detection, Pallet Information Recognition, Validate and Calc Parameter Compensation, Evaluate 2D Poses, Generate Rectangular Edge Templates of Specified Sizes, Remove Overlapped Polygons, Remove Polygons outside Mask, Calc Calibration Board Poses	Remove Overlapped Objects, Remove Overlapped Objects V2, 3D Coarse Matching, 3D Fine Matching (Multiple Models), 3D Coarse Matching (Multiple Models), Adjust Poses, Sort 3D Poses

Removed Step

Steps moved to “Legacy”

Calc Disparity Image, Calc 2D Poses of Rectangles, Calc Poses and Dimensions of Rectangles, 2D Matching, Count Boxes, Deep Learning Inference (Mech-DLK 2.1.0/2.0.0), Defect Detection, Detect Fiducial Markers, Detect Edges (Deep Learning), Image Classification, Generate Rectangle Candidates, Instance Segmentation, Load 2D Templates, Object Detection, Pallet Information Recognition, Validate and Calc Parameter Compensation, Evaluate 2D Poses, Generate Rectangular Edge Templates of Specified Sizes, Remove Overlapped Polygons, Remove Polygons outside Mask, Calc Calibration Board Poses

Remove Overlapped Objects, Remove Overlapped Objects V2, 3D Coarse Matching, 3D Fine Matching (Multiple Models), 3D Coarse Matching (Multiple Models), Adjust Poses, Sort 3D Poses

The deep learning Steps in earlier versions, such as Instance Segmentation and Object Detection, are no longer supported in Mech-Vision 1.8.0. Please use the upgraded “Deep Learning Model Package Inference” Step instead.

Improvements in Other Steps

Mech-Vision 1.8.0 improved some Steps. See the following for details.

No.	Step	Description
1	3D Workpiece Recognition	A “Matched Point Cloud” port was added to output object point clouds in which overlapping objects are removed.
2	3D Laser Profiler	Re-adapted 3D laser profilers for better scanning of objects to capture images.
3	3D Matching Related Steps	Improved the “Sampling Interval” of the “3D Fine Matching” and “3D Fine Matching (Multiple Models)” Steps and improved the matching speed.
4	Predict Pick Points V2	Improved the picking configuration file. Projects downloaded in the solution library of Mech-Vision 1.7.4 or earlier versions cannot be run in Mech-Vision 1.8.0 or further versions. Please re-download the corresponding project in the new version.
5	Predict Pick Points (Any Objects)	Improved the deep learning model file. Reduced the model optimization time during Step execution. However, when you execute the Step for the first time after the software is installed or the IPC is changed, the time it takes to optimize the model can be longer.
6	Create Object Model	Rectangle and cylinder curved surface object types are supported. You can also select the point cloud model type.
7	From Depth Map to Point Cloud	Improved the execution speed of the Step.
8	Point Filter	Improved the execution speed of the Step.
9	Calc Normals and Estimate Edges of Point Cloud	Improved the edge estimation effect and improved the Step execution speed.

No.

Step

Description

3D Workpiece Recognition

A “Matched Point Cloud” port was added to output object point clouds in which overlapping objects are removed.

3D Laser Profiler

Re-adapted 3D laser profilers for better scanning of objects to capture images.

3D Matching Related Steps

Improved the “Sampling Interval” of the “3D Fine Matching” and “3D Fine Matching (Multiple Models)” Steps and improved the matching speed.

Predict Pick Points V2

Improved the picking configuration file. Projects downloaded in the solution library of Mech-Vision 1.7.4 or earlier versions cannot be run in Mech-Vision 1.8.0 or further versions. Please re-download the corresponding project in the new version.

Predict Pick Points (Any Objects)

Improved the deep learning model file. Reduced the model optimization time during Step execution. However, when you execute the Step for the first time after the software is installed or the IPC is changed, the time it takes to optimize the model can be longer.

Create Object Model

Rectangle and cylinder curved surface object types are supported. You can also select the point cloud model type.

From Depth Map to Point Cloud

Improved the execution speed of the Step.

Point Filter

Improved the execution speed of the Step.

Calc Normals and Estimate Edges of Point Cloud

Improved the edge estimation effect and improved the Step execution speed.

Improved the 3D Visualization Window and Tips

Mech-Vision 1.8.0 improved the 3D visualization window and operation tips. See the following for details.

Function improvements: improved the operations such as zooming views, rotating views, panning views, and rotating/moving objects.
Tip improvements: prompted 3D animation to display the preceding operations when users enter the tools such as “Matching Model and Pick Point Editor” and “3D Workpiece Recognition Visualized Configurator”, helping users to learn about the operations in the 3D visualization window.

The operation methods for the Debug Output panel remain the same.

Resolved Issues

Mech-Vision 1.8.0 has fixed the following issues:

When the robot model was changed in the path planning tool, the soft limits of the previous robot model were wrongly applied to the new robot model.
When a version from Mech-Vision 1.7.0 to 1.7.4 was used, hand-eye calibration for 2D cameras was not supported.
When two Mech-Vision projects shared the same 2D camera, the images captured using auto exposure were abnormal in a project if the other project had just captured images using fixed exposure.
When the camera failed to be connected in the “Capture Images from Camera” Step, the error code was missing.