Mech-Vision Release Notes
Mech-Vision 2.2.0 Release Notes
This section introduces the new features, improvements, and resolved issues of Mech-Vision 2.2.0.
New Features
Support for Trajectory Scene Operations
Mech-Vision 2.2.0 supports trajectory scene operations with multiple new trajectory-related features as follows.
-
Added trajectory target object configuration workflow to the target object library, supporting quick completion of trajectory target object configuration by importing STEP files, importing processed point clouds, and collecting point clouds with cameras.
-
Added 3D Trajectory Recognition Step. This Step integrates point cloud preprocessing, 3D matching, and other vision processing functions, enabling quick target object recognition and trajectory generation.
-
Improved Output Step functionality to support outputting trajectory-type vision results.
-
Added multiple trajectory processing Steps as follows.
Step Description This Step generates trajectory points from input point clouds for subsequent trajectory processing.
This Step optimizes the original trajectory based on actual point clouds, making the trajectory better fit the target object surface. It supports multiple operations such as smoothing, sorting, and simplification, improving trajectory accuracy.
This Step smooths trajectory points to reduce the impact of noise in the trajectory and generates a smoother trajectory.
This Step simplifies trajectory shape by reducing the number of trajectory points while maintaining the overall trajectory form.
This Step sorts trajectory points based on their positions to optimize the trajectory point sequence.
This Step removes overlapped points with excessive proximity in the trajectory to reduce the number of invalid points and optimize trajectory point distribution.
This Step inserts more intermediate points between trajectory points to make the trajectory smoother and more continuous.
This Step obtains trajectory information related to trajectory-type target objects from the target object library based on the target object center point and target object name output by the previous Step, for use in subsequent Steps.
Support for Bin Recognition in Picking Scenarios
Mech-Vision 2.2.0 supports bin recognition in picking scenarios, mainly including:
-
Added bin target object configuration workflow to the target object library, supporting quick completion of bin target object configuration by importing STL files, collecting point clouds with cameras, and configuring without point cloud models.
-
Added 3D Bin Recognition Step. This Step integrates point cloud preprocessing, 3D matching, deep learning and other vision processing functions, enabling quick bin recognition.
Support for Pose Transformation in Assembly Scenarios
-
Added Advanced Pose Transformation Step. This Step can be used to calculate pose relationships between different coordinate systems, including coordinate system transformation, dual-robot relative pose calibration, and pose correction (assembly).
-
Added Frame Transformation Calculator Step. This Step automatically finds the shortest transformation path based on multiple known pose relationships and calculates transformation relationships between two target coordinate systems.
Extended 2D Vision Features
Mech-Vision 2.2.0 added multiple 2D vision-related features, mainly including:
-
Added 2D Smart Camera Step to acquire image data through a 2D smart camera as input for subsequent vision processing tasks.
-
Added 2D Camera Calibration feature, including distortion calibration and hand-eye calibration (external parameter calibration), to establish the mapping relationship between camera imaging and actual space, improving image measurement and positioning accuracy, and serving as the foundation for the vision system to achieve high-precision recognition and positioning.
-
Added 2D Camera Management feature to centrally manage 2D cameras in a solution, serving as the device configuration and debugging entry point before image acquisition.
-
Added 2D Template Editor feature to create and manage 2D matching templates.
-
Added 2D Target Object Recognition Step. This Step can quickly complete target object recognition and detection in positioning picking, placement deviation correction, error prevention inspection, and information reading scenarios.
-
Added multiple 2D-related Steps covering image preprocessing, image post-processing, and measurement scenarios. The Step library supports grouping Steps by 2D and 3D dimensions.
Category Step Description Preprocessing
This Step splits a three-channel image into three single-channel images, or merges three single-channel images into one three-channel image.
This Step performs preprocessing operations on input images such as enhancement, denoising, morphological transformation, grayscale inversion, and edge extraction.
This Step composes multiple images by placing them at specified positions into a single image.
This Step crops, pads, or resizes input images.
This Step flips and rotates images as required.
This Step rotates an image by a specified angle around the set rotation center.
This Step extracts target regions in a specified color space based on color range (upper and lower threshold limits) of three channels to generate a binary image (target region pixels have a value of 255, other pixels have a value of 0).
Post-processing
This Step performs logical operations on two groups of masks with the same dimensions to merge two groups of masks, extract the same part, or remove the same part.
This Step counts the number of pixels in a specified region of a grayscale image that meet the grayscale threshold range.
This Step outputs information about the input grayscale image including grayscale histogram, number of pixels, minimum and maximum grayscale values, grayscale median, grayscale mode (most frequent grayscale value), grayscale mean, grayscale standard deviation, and contrast. A grayscale histogram is a statistical analysis of the grayscale level distribution in a grayscale image, reflecting the frequency of a certain grayscale value appearing in the image.
This Step performs pixel-wise arithmetic operations (addition, subtraction, multiplication, or division) or combined operations (maximum, minimum, inversion) on input grayscale images, and can adjust results through multiplier and addend. The two images must have the same dimensions.
Measurement
This Step detects line edges from an image and fits a line.
This Step detects circular edges from an image and fits a circle.
This Step detects rectangular edges from an image and fits a rectangle.
This Step detects an edge point that meets the requirements on the vertical center line of a specified region in an image.
This Step detects oblong hole edges from an image and fits an oblong hole.
This Step detects a pair of edge points from an image and measures the distance between them as the edge-to-edge width.
This Step measures the geometric distance between two specified feature types. Supported feature combinations include: point-to-point, point-to-line, point-to-circle, line-to-line, line-to-circle, and circle-to-circle.
This Step measures the angle between line segments.
Other
This Step searches for and locates features matching the template in a 2D image, calculates object poses, and provides data for subsequent Steps that need to perform 2D pose transformation simultaneously. Supports multi-object positioning and recognition.
This Step aligns input images with the template through translation and rotation to achieve consistency between the two.
This Step detects blobs in images and filters them based on their geometric characteristics such as area and roundness.
This Step detects blobs in images, filters them based on their geometric characteristics, and adjusts image pose so that the blob centroid coincides with the original image center point.
This Step overlays features, images, text, poses and other results on 2D images to realize custom visualization.
This Step recognizes 1D/2D barcodes in a specified image region and outputs the barcode content and its position in the image.
This Step detects and counts target objects in an image through 2D template matching.
This Step performs deformation status detection for target objects in an image through 2D template matching.
This Step performs pose deviation detection for target objects in an image through 2D template matching.
This Step locates the feature points of a mask.
This Step creates 2D points based on geometric features such as points and lines.
This Step creates 2D lines based on geometric features such as points and lines.
This Step converts the 2D pose or 2D shape of a target object recognized by a 2D camera into a 3D target object pose in the robot coordinate system by combining external parameter calibration data and teaching poses.
Extended Deep Learning Features
Enhanced Deep Learning Model Package Inference Features
Mech-Vision 2.2.0 enhanced the Deep Learning Model Package Inference Step as follows.
-
Supports inference of Fast Positioning, Text Detection, Text Recognition, and Unsupervised Segmentation model packages, as well as inference of Multi-Model Package.
-
Enhanced inference configuration functionality, supporting post-processing parameter settings in the Inference Configuration Tool to improve inference result accuracy.
|
Starting from Mech-Vision 2.2.0, this Step only supports loading model packages exported from Mech-DLK 2.5.4 or later. |
Added Lite AI Steps
Mech-Vision 2.2.0 added the following Lite AI Steps.
| To use the "Lite AI" related Steps, please contact Mech-Mind Sales to obtain software licenses supporting this feature. After updating the software license, you can use this Step. |
| Step | Description |
|---|---|
This Step performs multi-class intelligent classification on target objects in an image. |
|
This Step performs binary classification (e.g., front/back, present/absent) on target objects in an image. |
|
This Step performs intelligent recognition on letters, numbers, and other characters in an image. |
Added Other Deep Learning Steps
| To use the "Pick Anything V2" Step, please contact Mech-Mind Sales to obtain software licenses supporting this feature. After updating the software license, you can use this Step. |
| Step | Description |
|---|---|
This Step performs surface segmentation on input depth images and color images based on the Pick Anything V2 model package, identifying each independent graspable surface and overlapped surfaces, and outputs a list of masks sorted by grasping priority. |
|
This Step performs segmentation of target objects and bins on input depth images and color images based on the target object and bin segmentation model package, outputting target object masks and bin masks, and provides visualization results. |
Support for Global Variables
Mech-Vision 2.2.0 supports using Global Variable Steps and Global Variable Viewer, enabling data sharing between different projects, achieving unified data management and transmission.
Added Other Steps and Procedures
Mech-Vision 2.2.0 added several other Steps and Procedures as follows.
| Category | Name | Description |
|---|---|---|
Step |
This Step calculates the overall optimal pose of a target object by performing best-fit matching between multiple real-time feature points on the target object and template feature points, improving adaptability in deformation scenarios. |
|
This Step is used to connect and configure a light source controller to control the light source operating mode and output brightness. |
||
This Step saves data to a local path. |
||
This Step can perform logical judgment on input data according to set rules and output boolean results (True or False). |
||
Procedure |
This Procedure performs preprocessing operations such as point filtering, point cloud merging, and image filtering on raw point clouds to delete interfering points, thus speeding up the processing of subsequent Steps. |
|
This Procedure extracts point clouds of target objects on the highest layer from multi-layer target object point clouds. By filtering point cloud height in the specified direction, it removes point cloud interference from lower-layer target objects. |
||
This Procedure filters the original image through point cloud information and extracts 2D images containing only the highest layer target objects, effectively removing background and lower-layer target object interference. |
||
This Procedure obtains relatively flat surface point clouds by removing point clouds far from the target object flat surface. |
||
This Procedure sorts target object poses by multiple dimensions based on target object height and angle, optimizing grasping order to improve grasping stability and success rate. |
||
This Procedure validates whether the pose angle and position of target objects meet requirements. |
||
This Procedure counts the number of elements (such as poses) in input data for use in subsequent logical judgment. |
||
This Procedure establishes a target object reference coordinate system through three feature points (primary reference point, Y-direction auxiliary reference point, and XY-plane auxiliary reference point), thereby determining the position and orientation of the target object in space. |
||
This Procedure transforms point clouds from camera or robot coordinate systems to a user-specified coordinate system. |
Support for Solution Switching
Mech-Vision supports Solution Switching Management functionality, enabling automatic switching of solutions based on received solution IDs. Each solution switching rule defines the mapping relationship between a solution ID and the solution path.
Support for Procedure Locking
Mech-Vision 2.2.0 supports Locking and Protecting Procedures. After a Procedure is locked, non-administrator users cannot view or modify the Step logic inside the Procedure. They can only call and adjust the Procedure through the exposed parameters.
Added Mech-Vision Secondary Development Features
Mech-Vision 2.2.0 added Mech-Vision SDK documentation and interface capabilities, supporting integration of solution, project, and Step vision capabilities into client applications, with C++, C#, and Python interfaces available.
Improvements
Enhanced Case Library
Mech-Vision 2.2.0 added the following cases:
| Case Category | Case Name |
|---|---|
Hands-on Examples - 3D Locating |
3D Trajectory Generation, Clothing Bin Picking, 3D Bin Recognition (Standard Bins), 3D Bin Recognition (Other Bins) |
Hands-on Examples - 2D Locating |
2D Target Object Recognition (Positioning and Picking), 2D Target Object Recognition (Placement Correction) |
Hands-on Examples - Deep Learning |
2D Target Object Recognition (Information Reading), 2D Target Object Recognition (Error-Proofing Check) |
Typical Cases - Randomly-Stacked Part Picking |
Automotive Sheet Metal Parts Loading |
Typical Cases - Locating and Assembly |
Double Hook Loading, Composite Locating and Assembly, Automotive Windshield Assembly |
Enhanced Target Object Library Picking Configuration Workflow
Mech-Vision 2.2.0 optimized the target object library picking configuration workflow as follows.
| Target Object Configuration Workflow | Optimization Details |
|---|---|
|
|
|
|
|
|
|
|
Import Processed Point Cloud - Obtain Pick Points from Steps |
|
Import Processed Point Cloud - Edit Manually to Set Pick Points |
|
Import Processed Point Cloud - Jog the Robot to Set Pick Points |
|
Enhanced Advanced Component Steps
Mech-Vision 2.2.0 optimized advanced component Steps as follows.
| Step | Optimization Details |
|---|---|
In the point cloud preprocessing workflow, support retrieving the highest-layer point cloud. |
|
|
Enhanced Other Steps
| Step | Optimization Details |
|---|---|
|
|
Adjusted step parameter order to improve ease of use. |
|
Support asynchronous saving. After enabling the "Asynchronous Save" parameter, saving will execute asynchronously in the background. Project completion status will no longer wait for this step to finish. |
|
Generate Target Object Picking Strategy |
|
Enhanced Step Library
Mech-Vision 2.2.0 optimized the step library, including optimizing some step names and removing some steps, as follows.
Optimized Step Names
Mech-Vision 2.2.0 optimized the following step names:
| Original Name | New Name |
|---|---|
Generate Target Object Picking Strategy |
Acquire Target Object Information |
Filter |
Filter Data Based on Boolean Value |
Binary Classification of Numerical Values Based on Threshold |
Determine Whether Numerical Value Exceeds Threshold |
Read Object Dimensions |
Quick Create Target Object Dimensions |
Convert Cloud (XYZ-Normal) to Cloud (XYZ-RGB) |
Convert Point Cloud with Normal to Colored Point Cloud |
Calculate Angle Between Two Vector3D |
Calculate Angle Between Two 3D Vectors |
Calculate Cross Product of Vector3D |
Calculate Cross Product of 3D Vectors |
Calculate Dot Product of Vector3D |
Calculate Dot Product of 3D Vectors |
Calculate Length of Vector3D |
Calculate Length of 3D Vector |
Calculate Unit Vector of Vector3D |
Calculate Unit Vector of 3D Vector |
Compose Vector3D from Numerical Values |
Compose 3D Vector from Numerical Values |
Decompose Vector3D into Numerical Values |
Decompose 3D Vector into Numerical Values |
Removed Steps
Mech-Vision 2.2.0 removed the following steps:
| Removed Step | Alternative Step |
|---|---|
Predict Grasp Point |
|
Detect and Measure Line |
|
Fit Line |
|
Detect and Measure Circle |
|
Fit Circle |
|
Measure Circle |
|
Detect and Measure Oblong Hole |
|
Measure Circle to Circle Distance |
|
Measure Circle to Line Distance |
|
Measure Point to Point Distance |
|
Measure Point to Line Distance |
|
Measure Point to Circle Distance |
|
Measure Line to Line Distance |
|
Measure Angle Between Lines |
|
Calculate Intersection of Line and Circle |
|
Calculate Intersection of Two Lines |
|
2D Camera |
N/A |
Measure Point to Point Height Difference |
N/A |
Measure Point to Reference Line Height Difference |
N/A |
Measure Longest Line |
N/A |
Caliper Tool |
N/A |
Detect Vertex |
N/A |
Convert Circle to 2D Pose |
N/A |
Screenshot |
N/A |
Visualize Information on Image |
N/A |
Enhanced Production Interface Configurator
Mech-Vision 2.2.0 optimized the production interface configurator as follows.
-
When configuring execution screens, support selecting "2D Image Visualization" screens to display 2D images and measurement results in the production interface.
-
Support setting data "save mode" and "subfolder creation method" in general settings.
Enhanced Adapter Program Generator
Mech-Vision 2.2.0 optimized the Adapter program generator functionality. When using the Adapter program generator to obtain robot names from Mech-Viz, it is no longer necessary to set the Mech-Viz project to auto-load.
Enhanced 3D Camera Hand-Eye Calibration
Mech-Vision 2.2.0 optimized 3D camera hand-eye calibration functionality, supporting use of calibration plate/camera offset relative to flange in the set motion path workflow to reduce displacement in camera view during rotation and ensure the calibration plate remains within the camera field of view.
Enhanced Auxiliary Tools
Mech-Vision 2.2.0 optimized some auxiliary tools as follows.
| Tool | Optimization Details |
|---|---|
|
|
Optimized tool interface and access point for improved user experience. |
|
|
|
Optimized tool interface and access point for improved user experience. |
Resolved Issues
Mech-Vision 2.2.0 resolved the following issues:
-
Renaming a project or solution could fail to save normally if the name ended with two spaces.
-
When saving a solution, if the temporary solution directory hierarchy is too deep, Windows flickering may occur.
-
When opening a solution where multiple projects use different parameter sets for the same camera, there is a small probability that parameter sets will be incorrectly modified.
-
When triggering project execution using standard interface, there is a small probability of delay of several seconds before execution begins.
-
After switching and applying a robot model in robot communication configuration, if the solution is not saved, the switched robot model will not be saved.
-
There is a small probability of software crash during production interface execution.
-
In the target object library "Import Processed Point Cloud - Robot Teaching" workflow, if the camera lacks internal and external parameters, the target object library may crash.
-
In the target object library "Import STL File" workflow, after switching model views and then switching point cloud generation modes, generating target object point cloud from STL file may fail.
-
During continuous execution, the "3D Matching" Step may occasionally crash.
-
After enabling the "Consider Holes in Surface Matching" feature in the "3D Matching" Step, OpenCV errors may occur with specific data.
-
In the "Output" Step, abnormalities may occur when outputting target object pick point information that does not require point cloud models.
-
After customizing port names in the "Output" Step, switching to "Predefined (Vision Result)" and checking "Other Input", the "Output" Step did not add corresponding input ports.
-
After setting the port type of the "Output" Step to "Predefined (Vision Result)" and checking "Other Input", Mech-Viz may incorrectly generate multiple target object models when outputting array pick points.
-
When multiple pick points correspond to the same target object center point, the "Output" Step output duplicate target object information.
-
Before the first execution of the "Read Image" Step, after setting "Read Mode" to "Repeat Single Image", images could not be read normally.
-
Saving images with the "Save Images" Step took a long time.
-
Opening the deep learning model package management tool caused the software to be unresponsive.
-
After deleting a Procedure containing custom alert settings, enabling custom alerts may cause the software to crash.
-
After enabling or disabling the custom alert feature, the settings did not take effect.