Hand-Eye Calibration
In this tutorial, you will perform the automatic hand-eye calibration in the Eye-To-Hand setup.
|
The hand-eye calibration establishes the transformation relationship between the camera and robot reference frames (that is camera extrinsic parameters). With this relationship, the object pose determined by the vision system can be transformed into that in the robot reference frame, which guides the robot in performing its tasks. |
Preparation before Calibration
In this section, you will mount the calibration board, adjust the camera parameters, and complete the pre-calibration configuration.
Mount the Calibration Board
|
In the Eye to Hand scenario, the calibration board needs to be mounted to the robot flange adapter. |
Follow these steps:
-
Take the calibration board and flange adapter out of the system delivery package.
-
Use screws, gaskets, and nuts to fasten the flange adapter to the flange adapter.
-
Use screws, gaskets, and nuts to secure the calibration board to the flange adapter.
-
After mounting the calibration board, move the robot to the top of the lowest workpiece in the work area and into the camera FOV.
Adjust Camera Parameters
-
In Mech-Eye Viewer, connect to the camera, and then set Parameter Group to “calib”.
-
Adjust the 2D parameters to make sure that the 2D image of the calibration board is clear and neither overexposed nor underexposed.
-
Adjust 3D parameters to make sure that the obtained point clouds of the circles on the calibration board are complete and have clear contours. It is recommended to set the Surface Smoothing and Outlier Removal parameters of Point Cloud Processing to Normal to reduce point cloud fluctuation.
| Normal | Overexposed | Underexposed | |
|---|---|---|---|
2D image |
|
|
|
Point cloud |
|
|
|
Create a Mech-Vision Project and Save It
-
Open Mech-Vision. If the welcome interface as shown below is displayed, it indicates that Mech-Vision is started successfully.
-
In the Welcome interface of Mech-Vision, click Create from solution library to open the Solution Library.
The Solution Library is a resource library that provides typical solutions or projects (with sample data) from various application scenarios.
-
Select the General Workpiece Recognition project from the Solution Library.
If you cannot find this project in the Solution Library, click More at the bottom of the Solution Library interface.
-
After this project is selected, the information introducing this project will be displayed on the lower part of the Solution Library interface. Set the solution name and path, and then click Create.
After the project is created, the created solution and project will be displayed in the project list in the upper-left corner of the Mech-Vision main interface.
-
A solution is a set of configurations and data related to robots and robot communication, vision processing, path planning, etc. that are required for the machine vision application.
-
A project is a workflow of vision processing in the solution. Normally, a solution only contains one project, but it may contain more than one project in complex application scenarios. In this tutorial, the solution requires only one project.
In the Graphical Programming Workspace of the main interface, the workflow of the “General Workpiece Recognition” project will be displayed.
-
-
On the menu bar, select .
Pre-calibration Configuration
-
In Mech-Vision, click the Camera Calibration (Standard) on the toolbar. The Configuration before Calibration window will be prompted.
-
In the Select how to calibrate window, select the New calibration radio button, and then click the Next button.
-
In the Select calibration task window, select Hand-eye calibration for listed robot from the drop-down list box, and click the Select robot model button.
-
Click the Brand dropdown box, and select “ABB”. Then, at the right panel, select model “ABB_IRB_1300_11_0_9”, click the Select button, and then click the Next button.
-
In the Select camera setup window, select the Eye to hand radio button, and then click the Next button.
-
In the Calibration method and robot control window, select Automatic and Master-control, and then click the Next button.
-
In the Communication Settings window, set the Robot IP address parameter to the real robot’s IP address.
-
On the robot teach pendant, confirm that the MM main program has been started.
-
Return to Mech-Vision, and click the Connect the robot button in the Connect the robot area. The button will turn into Waiting for the robot to connect...
-
Wait until the “Connected” status message is displayed in the Connect the robot area, and then click the Calibrate button. The Calibration (Eye to Hand) window will be prompted.
Calibration Procedure
Connect to the Camera
-
In the Connect to Camera step, find the camera to connect in the Detected Cameras list, and click the
button.
-
After the camera is connected, click the Capture once or Capture live button.
-
In the right Image viewer panel, ensure that the captured 2D image and depth map meet the calibration requirements and click the Next button on the bottom bar.
|
If the captured image does not meet the calibration requirements, you need to open the Mech-Eye Viewer software to adjust the 2D and 3D exposure parameters and re-capture images. |
Mount the Calibration Board
-
In the Mount calibration board step, select the Standard radio button and select the calibration board model according to its model nameplate in the 1. Select calibration board type area.
-
Make sure that the calibration board has been attached to the robot flange securely, and then click the Confirm button in the 2. Mount calibration board area.
-
Confirm that the calibration board is in the center of the camera’s field of view (the red rectangle), and then click the Confirm button in the 3. Position calibration board into red rectangle area.
-
After all the operations related to the calibration board are completed, click the Next button on the bottom bar.
Check Intrinsic Parameters
-
In the Check intrinsic parameters step, click the Start checking button.
-
Confirm that the camera intrinsic parameters check passes, and then click the Next button on the bottom bar.
|
If the camera intrinsic parameter check fails, adjust detection parameters by drawing aid circles or manually adjusting the calibration circle detection parameters. |
Set Robot Path
-
In the Set robot path step, specify the Height span parameter. Please set this parameter according to the working distance range of the calibration board in the camera height direction.
-
Set the Path type to ToHand, specify the pyramid parameters Height span, Num of layers, Bottom-layer dimensions X/Y, Top-layer dimensions X/Y, and Motion grid cols and rows per layer, specify the satellite parameter Rotation angle, and then click the Confirm button.
The set robot path should cover the work area. -
In the right Scene Viewer panel, confirm that the waypoints of the automatically generated motion path will not collide with obstacles in the environment, and then click the Next button.
Collect Images and Poses
-
In the Collect images and poses step, select the Save images checkbox.
-
Click the Control robot to auto move along path and capture images button.
-
Read the safety window carefully and click the OK button.
-
Wait until the robot finishes moving along the preset path and the camera finishes capturing images on all waypoints. Captured images can be seen in the right Marker Image and Pose List panel during this process.
-
After the automatic image capturing finishes, click the OK button, and then click the Next button on the bottom bar.
Calculate Camera Parameters
-
In the Calculate camera parameters step, click the Calculate camera extrinsic parameters button.
-
In the prompted window indicating calibration success, click the OK button.
-
View the calibration error of the point cloud in the right Point cloud viewer panel.
The error point cloud shows the deviation between the calculated value and the actual value of the circles on the calibration board. -
Confirm that the calibration accuracy meets the project requirements.
Determine the error value with 100% to obtain the rough calibration accuracy. For example, the calibration accuracy in the following figure is within 2 mm.
To improve the calibration accuracy, please refer to the section Calibration Result Check and Analysis.
Validate and Save the Calibration Result
-
After calibration, move the robot arms into the FOV of the camera.
-
In the Calculate camera parameters step, click the Calculate camera extrinsic parameters button. This operation triggers the camera to capture images.
-
Click Scene viewer to check the coincidence degree between the point cloud of the robot and the robot model.
If the point cloud of the robot approximately coincides with the robot model, the calibration is successful.
-
Click the Save button on the bottom bar. In the prompted Save Calibration Files dialog box, click the OK button. The camera calibration result will be automatically saved in the “calibration” directory of the project.