Mech-Vision 1.6.0 Release Notes

New Features

Data Storage Restructured

The Data Storage feature is restructured for better performance in troubleshooting and regression testing on-site. The detailed changes are as follows:

• Data Storage features no longer require the Procedure Save Images and Step Parameters to be used.

• Abnormal data are now stored in the new error_data folder in the data folder.

• Data from multiple cameras can be saved altogether.

• The naming conventions of the file and folder names in the data folder are changed.

Added Example Projects

Example Projects of typical applications in various industries are introduced in Mech-Vision V1.6.0 to provide a general picture of what Mech-Vision can do and how Steps work. These projects are highly user-friendly for beginners as they can be run with one click.

New 2D Matching Steps

The following 2D Matching Steps are added:

• Make Template

• Template Matching

• Record Criterion Pose and Calc Transformation

• Transform Image

These Steps can be used to obtain the positions of target objects in an image and align the target objects/ROIs in different images to the same position/orientation.

New Step - Deep Learning Model Package Inference

Mech-Vision V1.6.0 provides built-in deep learning inference models. With the Deep Learning Model Package Inference Step, deep learning inference can be enabled without installing Mech-Mind Software Environment.

New Step - 3D Coarse Matching V2

A newer version of the 3D coarse matching Step, 3D Coarse Matching V2, is added. This Step is used to coarsely match the point cloud model with the original point cloud and output the coarsely calculated candidate poses of the target objects in the scene.

New Step - Calc Results by Python

Mech-Vision V1.6.0 now has the Python 3.6.8 environment integrated, which allows you to run a Python script directly in Mech-Vision with the Calc Results by Python Step.

Note

If extra Python libraries are required when you use this Step, these libraries must be installed to the python folder under the installation directory of Mech-Vision.

Tip

A Python library can be installed as follows:

1. Open the Command Prompt or PowerShell program.

2. Switch to the “python” directory of Mech-Vision, such as C:\Mech-Mind\Mech-Vision-1.6.x\python.

3. Execute the following command: ./python -m pip install library_name.

Debug Output Improved

The function of the Debug Output panel is improved to better facilitate you in building and analyzing your Mech-Vision project.

Improvements include:

• When Debug Output mode is disabled, output results (shown by running a Step with Visualize Output or by clicking on the connection between two Steps) are also displayed in the Debug Output panel.

• Display output in a separate window: click on the icon in the upper right of the Debug Output panel to open a separate maximized window.

  When Debug Output mode is enabled, the separate window also allows for the following usage:

  • With a separate window opened, the result of the same Step is updated every time you run the Step/project and displayed in the same window.

  • Viewpoint adjustment of a 3D result in a separate window is memorized, and when you run the Step/project again, the result of the same Step is displayed with the same viewpoint, so that you can compare results from multiple executions easily.

  • You can open the results from multiple Steps in separate windows simultaneously for easy comparison.

Matching Model and Pick Point Editor Improved

• Add point cloud from camera:

  • Adaptation for applications with target objects that are heavy and/or not easily movable: now you can directly generate a point cloud of the entire scene and then edit the point cloud, separate depth map acquisition of the background and target object not mandatory.

  • When acquiring depth maps of the background and target object separately, background depth map is acquired first to better fit actual usage.

• New features for point cloud editing:

  • Remove selected points button added

  • Invert selection: select the points you want to keep first, and then invert the selection to quickly delete the unwanted points.

  • Generate edge point cloud: you can now generate a point cloud model of the object edges in the editor.

Improvements

User Interface Improved

The UI of Mech-Vision V1.6.0 is improved in the following aspects for better user experience:

• Layout and color scheme redesigned

• Default Layout option added to the View menu: automatically adjust the interface layout with a single click

• Appearance and user interaction of the Step Parameters tab improved

• Save, Undo, and Redo buttons added to the project toolbar at the top of the graphical programming workspace

• Appearance of the search box in Step Library improved

• Interface of Camera Viewer improved

• Layout and text in Camera Calibration improved

Edit Procedure Parameter Feature Improved

In Mech-Vision V1.6.0, the Edit Procedure Parameter feature is improved in the following aspects:

• Added Parameter Info Customization feature that enables users to customize the display name, value tooltip, and key tooltip of Procedures

• Added Custon Mapped Parameter for customizing the mapping relationship in a Procedure with JavaScript code

• Added Custom Recipe Parameter for setting the selected parameters as a recipe parameter of a Prcedure

Model Selection Parameter Added to the 3D Coarse Matching and 3D Fine Matching Steps

In Mech-Vision V1.6.0, the 3D Coarse Matching and 3D Fine Matching Steps now have a Model Settings parameter category for easy model selection from the model gallery.

The Check Pose Repeatability by Statistics Step Improved

The Check Pose Repeatability by Statistics Step is improved in the following aspects for better usability:

• Name updated: from Poses Repeatability Statistics to Check Pose Repeatability by Statistics

• Threshold for Acceptable Position Coordinate Deviations and Threshold for Acceptable Euler Angle Deviations parameters added for filtering out errors

• Output orientation data now in Euler angles

• Number of decimal places of the output data customizable

• Layout of output XLSX table improved

• Inputting multiple pose data simultaneously and saving results of different poses to different table sheets now supported

• Restart option now automatically unchecked after the Step is run once with the option checked

• Step Quick Info and Parameter tooltips updated

Compatibility for V4 and UHP Cameras

Mech-Vision V1.6.0 now supports image capturing and hand-eye calibration with V4 and UHP cameras.

Performance of the Instance Segmentation Step Improved

In Mech-Vision V1.6.0, instance segmentation conducted with the new Deep Learning Model Package Inference Step has better performance compared to the old Instance Segmentation Step.

Updated Step Names

The names of the following Steps are updated for easier understanding of the functions. When you open a project in Mech-Vision V1.6.0, the name of the Steps in your project are automatically updated.

Before V1.6.0	V1.6.0
Map to Multi Pick Points	Map to Multiple Pick Points
Calc Included Angle Between Specified Axis of Poses	Calc Included Angles between Specified Axes of Poses
Validate Box Masks	Validate Box Object Masks
Segment Depth Image	Segment Depth Map
Get Highest Areas in Depth Image	Get Highest Layer Regions in Depth Map
Pose Adjustment Collection	Adjust Poses
Rotate Poses to Goal Direction	Rotate Poses’ Axes to Specified Directions
Classification by Point Clouds’ Sizes	Classify Point Clouds by Dimensions
Sort and Output Index List	Sort List and Output Index List
Grasp Pose Estimation	Estimate Pick Points
Set Pose Quaternion	Set Pose Quaternions
Inverse Poses	Invert Poses
Validate Poses by Included Angle to Reference Direction	Validate Poses by Included Angles to Reference Direction
Binarize Image	Image Thresholding
CloudXYZ To CloudNormal	From CloudXYZ to CloudNormal
Cloud Num Limit	Trim Point Cloud List
From Poses to Euler Angles	Convert Quaternions to Euler Angles in Poses
Merge Cloud Vector	Merge Point Cloud Lists
Find 2D Contour at Specified Hierarchical Level	Find 2D Contour at Specified Inner-Outer Level
Calc Length Along Axis	Calc Point Cloud Spans along Axes
Poses Repeatability Statistics	Check Pose Repeatability by Statistics
Calc Minimum Bounding Rectangles of Masks	Calc Minimum Circumscribed Rectangles of Masks
Calc Distance to Reference Pose	Calc Distances between Poses
Smooth Trajectory	Smooth Path
Make Poses Point to Reference Place	Point Poses to Reference Positions
From Disparity Image to Depth Image	Convert Disparity Image to Depth Map
Compose Pose From Quaternion and Translation	Compose Poses from Quaternions and Translation Vectors
Merge LineSegment Vector	Merge LineSegment Lists
Merge Depth Images	Merge Depth Maps
Calc Angle Between Vector3D	Calc Angles between Vector3Ds
Mask Cluster	Mask Clustering
Cloud Smooth And Normal Estimation	Smooth Point Cloud and Estimate Normals
Compose Vector3D From Numbers	Compose Vector3Ds from Numbers
Compare Two Depth Image	Compare Two Depth Maps
Generate Point Cloud of Ring	Generate Ring Point Cloud
Adjust Poses to Obtain Acurate Trajectory	Adjust Targets to Get Correct Path
Load Poses in Trajectory and Apply Affine Transform	Load Targets in Path and Apply Affine Transform
Calc Poses from Heat Map of Graspability	Calc Poses from Heat Map of Pickability
Reverse	Reverse List
Cloud Filter By Model And Pose	Filter Point Cloud by Model and Poses
Merge Point Clouds with Similar Height	Merge Point Clouds with Similar Heights
Calc Mask’s Span on Given Line	Calc Mask Spans on Given Lines
Save Local Areas Around Poses as 3D ROI	Save Regions around Poses as 3D ROIs
Adjust Trajectory Circular Motion	Change Circular Motion Direction of Path
Detect Bin (Inscribed Rect Sides)	Detect Bin (Max Inscribed Rect)
Detect Bin (Largest Inscribed Rect)	Detect Bin (Max Inscribed Rect) V2
Filter Poses Outside Bin	Remove Poses outside Bin
Pixel-wise Graspability Evaluation	Pixel-Wise Pickability Evaluation
Mask Filter	Filter Masks
2D Poses To 3D Poses Base Orthographic Projection	Convert Poses 2D to 3D According to Orthographic Projection
Get Valid Ring Clouds	Filter Ring Point Cloud List
Project A Point onto A Plane	Project Points onto Plane
Compose Quaternion From Two Axis (Right-Hand)	Compose Quaternions from Two Axes (Right Hand Rule)
Calc Diagonal Length	Calc Diagonal Lengths
Generate Rect Traj	Generate Rect Path
Generate Traj By Contour	Generate Path from Contour
Is Z Value of Input Greater than Threshold	Compare Z Values of Poses with Threshold
Rectify Ring Pose	Rectify Ring Object Poses
Calc Center Point of Non-zero Areas	Calc Center Points of Non-Zero Regions
Calc Rect 2D Pose	Calc 2D Poses of Rectangles
Calc Pixel Size at Specified Height	Calc Pixel Sizes at Specified Heights
Coherent Line Drawing	Draw Coherent Lines
Image Transform	Adjust Image
Determine Pixel Size	Determine Pixel Sizes
Extract Empty Areas in Depth Image within 3D ROI	Extract Empty Regions in Depth Map within 3D ROI
Rotate Images By Provided Poses	Rotate Images by Specified Poses
Perspective Transform	Perspective Transformation
Clouds In 3d Box	Extract Point Cloud in 3D Box
Move Cloud Along Set Dir	Move Point Cloud along Specified Direction
Cloud Distortion Correction	Correct Point Cloud Distortion
Detect Obscured Objects	Detect Occluded Objects
Transform Plane Cloud To Align Direction	Align Plane Point Clouds
Cloud Scale	Scale Point Cloud
Calc Point Cloud Curvature	Calc Point Cloud Curvatures
Calc Edge Points Normal	Calc Edge Point Cloud Normals
Pose Transformed by Quaternion in Object Coordinate	Rotate Poses by Quaternion Vectors in Object Frames
Calc Length of Vector3D	Calc Lengths of Vector3Ds
Pose Transformed by Pose2 in Object Coordinate	Transform Poses by Matrix in Object Frames
Calc Cross Product of Vector3D	Calc Vector3D Cross Products
Calc Normalized Vector3D	Calc Normalized Vector3Ds
Compose Quaternion From Axis and Angle	Compose Quaternions from Axes and Angles
Rotate Poses to Directions With Symmetry Constraint	Rotate Poses’ Axes to Specified Directions under Symmetry Constraints
Easy Point to Reference Place	Easy Point Poses to Reference Position
Easy Coordinate Transform	Easy Frame Transformation
Inverse Quaternions	Get Inverses of Quaternions
Calc Dot Product of Vector3D	Calc Vector3D Dot Products
Calc Distance from 3D Points to Plane	Calc Distances from 3D Points to Plane
Measure Circle	Measure Circles
Calc Distance from 3D Points to Intersection of Two Planes	Calc Distances from 3D Points to Intersection of Two Planes
Replace Element In Vector	Replace Elements in List
Generate Test Cloud	Generate Test Point Cloud
Save Trajectory Points	Save Path Targets
Evaluate Variation of Depth Image	Evaluate Depth Map Fluctuation
Calculate Calib-board Pose	Calc Calibration Board Poses
Insert End Points And Send Motion Params	Insert End Target and Send Motion Params
Load 2d Trajectory	Load 2D Path
Generate Spiral Traj	Generate Spiral Path
Generate Trajectory Given Depth Image	Generate Path Given Depth Map
Generate Zigzag Traj	Generate Zigzag Path
Auto Trajectory	Extract 2D Path
Merge Label List	Merge Label Lists
Adjust Poses by Obstacles	Adjust Targets by Obstacles
Trajectory Points Matching	Path Target Matching
Adjust Poses by Obstacles V2	Adjust Targets by Obstacles V2
Validate 2D Poses Within Mask	Validate 2D Poses by Mask
Calc Mask Distance	Calc Mask Distances
Varying Normal Area	Extract Regions of Large Normal Deviations
Map Mask Non-zero Area	Extract Image Regions by Mask
Detect Graspable Rectangles	Detect Pickable Rectangles
Depth Cluster along Scan Lines	Depth Clustering along Scan Lines
Divide Cloud into Smaller Parts Evenly	Divide Point Cloud into Smaller Parts Evenly
Deep Learning Inference	Deep Learning Inference (DLK 2.1.0/2.0.0)