Complete Automatic Calibration in the Eye in Hand Scenario

This how-to guide introduces how to complete the automatic calibration in the Eye in Hand (EIH) scenario.

Preparation before Calibration

Before hand-eye calibration, you need to finish the following preparations:

• Construct the Mech-Mind Vision System.

• Prepare the materials required for calibration.

• Adjust the effect of calibration board’s point cloud.

• Complete the robot and communication configuration.

Construct the Mech-Mind Vision System

Construct the Mech-Mind Vision System by referring to the section Construct Mech-Mind Vision System.

You need to use Mech-Eye Viewer, Mech-Center, Mech-Vision and Mech-Viz during hand-eye calibration. Please ensure that they have been installed and are running the latest versions.

Prepare the Materials Required for Calibration

The automatic calibration in the EIH scenario needs to use the calibration board.

Please prepare the calibration board according to the following requirements:

• Ensure that the circles of the calibration board are clearly visible and without obvious scratches, and the board does not suffer from deformations.

• In the EIH scenario, please place the calibration board in the center of the object plane, where target objects are to be placed.

In addition, before calibration, move the robot to the starting point for calibration. For the automatic calibration in the EIH scenario, the starting point is the camera’s scanning position (the robot will move up during calibration).

Adjust the Effect of Calibration Board’s Point Cloud

1. Open Mech-Eye Viewer to adjust camera parameters.

2. Adjust 2D parameters to make sure that the 2D image of the calibration board is clear and neither overexposed nor underexposed.

3. Adjust 3D parameters to make sure that the point clouds of the circles on the calibration board are complete and have clear contours. It is recommended to set Point Cloud Smoothing and Outlier Removal to Normal in the Point Cloud Processing section to reduce the point cloud fluctuation range.

   Note

   If the on-site ambient lights are not ideal and affect the quality of 2D images and point clouds, you can use shading or supplemental light to improve the lighting conditions.

4. Make sure the obtained point clouds of the circles on the calibration board are complete and have clear contours, as shown in the following figure.

Hint

Because the 2D image type in the calibration process is the internal IR image, the captured image and generated scene point cloud are black and white.

Complete Robot and Communication Configuration

If the robot uses Standard Interface to communicate with the vision side, please complete the robot’s Standard Interface communication configuration. For details, refer to the section Standard Interface Communication.

If the robot uses Master-control to communicate with the vision side, please complete the robot’s Master-control communication configuration. For details, refer to the section Master-control Communication.

Pre-calibration Configuration

If the robot uses Standard Interface to communicate with the vision side, complete Pre-calibration Configuration (Standard Interface).

If the robot uses Master-control to communicate with the vision side, complete Pre-calibration Configuration (Master-control).

Pre-calibration Configuration (Standard Interface)

If Standard Interface is used to communicate with the robot, follow these steps:

1. Open Mech-Vision, and click the Camera Calibration (Standard) button on the toolbar. The Configuration before Calibration window will be prompted.

2. In the Select how to calibrate window, select the New calibration radio button, and then click the Next button.

   

3. In the Select calibration task window, select Hand-eye calibration for listed robot from the drop-down list box, click the Select robot model button to select the robot model used by the project, and then click the Next button.

   

4. In the Select camera setup window, select the Eye in hand radio button, and then click the Next button.

   

5. In the Calibration method and robot control window, select Automatic and Standard Interface, and then click the Next button.

   

6. In the Interface Settings window, set parameters Protocol and Host IP address, and then click the Start interface service button. The button will turn into Waiting for the robot to connect….

   

   Note

   In the above figure, TCP server and Hex（Little-endian） are only the Protocol setting example for the ABB robot. The Protocol settings for other robots are different. The software automatically adds the default setting of Protocol for robots here. Therefore, you only need to keep the default setting.

7. On the robot teach pendant, select the automatic calibration program, teach the starting point and run the program. For details on the calibration programs of different robots, refer to the section Standard Interface Communication. After the program is started, the log message “Entering the calibration process, please start the calibration in Mech-Vision” will be printed in the console log panel of Mech-Vision.

8. Return to Mech-Vision, confirm that “Connected” status message is displayed in the Connect the robot area, and click the Calibrate button. The Calibration (Eye in Hand) window will be prompted.

Pre-calibration Configuration (Master-control)

If Master-control is used to communicate with the robot, follow these steps:

1. Open Mech-Vision, and click the Camera Calibration (Standard) button on the toolbar. The Configuration before Calibration window will be prompted.

2. In the Select how to calibrate window, select the New calibration radio button, and then click the Next button.

   

3. In the Select calibration task window, select Hand-eye calibration for listed robot from the drop-down list box, click the Select robot model button to select the robot model used by the project, and then click the Next button.

   

4. In the Select camera setup window, select the Eye in hand radio button, and then click the Next button.

   

5. In the Calibration method and robot control window, select Automatic and Master-control, and then click the Next button.

   

6. In the prompted window, set the Robot IP address parameter.

   

7. On the robot teach pendant, select and run the Master-control program. For the instructions for different robots, refer to the section Master-control Communication.

8. Return to Mech-Vision, and click the Connect the robot button in the Connect the robot area. The button will turn into Waiting for the robot to connect….

   

9. Wait until the “Connected” status message is displayed in the Connect the robot area, and then click the Calibrate button. The Calibration (Eye in Hand) window will be prompted.

Till now, you have completed the pre-calibration configuration and can start the calibration procedure.

Calibration Procedure

Connect to the Camera

1. In the Connect to Camera step, select the camera to connect in the Detected cameras list, and then click the button or double-click the camera entry to connect to it.

   

2. After the camera is connected, click the Capture once or Capture live button.

   

3. In the right Image viewer panel, ensure that the captured 2D image and depth map meet the calibration requirements and click the Next button on the bottom bar.

   Note

   If the captured image does not meet the calibration requirements, you need to open the Mech-Eye Viewer software to adjust the 2D and 3D scanning parameters and re-capture images.

Mount the Calibration Board

1. In the Mount calibration board step, select the Standard radio button and select the calibration board model according to its model nameplate in the 1. Select calibration board type area.

   

2. Make sure that the calibration board has been placed to the plane of the workobject, and then click the Confirm button in the 2. Mount calibration board area.

3. Confirm that the calibration board is in the center of the camera’s field of view (the red rectangle), and then click the Confirm button in the 3. Position calibration board into red rectangle area.

4. After all operations related to the calibration board are completed, click the Next button on the bottom bar.

Check Intrinsic Parameters

1. In the Check intrinsic parameters step, click the Start checking button.

   

2. Confirm the results of the camera intrinsic parameter check.

   • If the camera intrinsic parameter check passes, click the OK button and then click the Next button on the bottom bar.

     

   • If the camera intrinsic parameter check fails, adjust detection parameters by drawing aid circles or manually adjusting the calibration circle detection parameters, and then click the Check again button.

Draw an Aid Circle

1. To draw an aid circle, click the Draw an aid circle button.

2. In the right Image viewer panel, right-click the calibration board image and clear the Fit to window checkbox, and press the Ctrl key and drag the roller to adjust the image to a suitable size.

3. Move the mouse pointer to the cross center point of the calibration circle, press the left mouse button and make the aid circle completely include the calibration circle and then release it.

   

4. Click the Check again button, and confirm that the camera intrinsic parameter check passes.

Manually Adjust Detection Parameters

To manually adjust the detection parameters, click Calibration circle detection parameters (advanced) and change the values of detection parameters accordingly.

If the calibration circles are still not detected, please adjust the camera settings according to the on-site operating conditions.

Set Robot Path

1. In the Set Robot Path step, specify the Height span parameter.

   

   Tip

   The Height span parameter should be set according to the recommended working distance range of the camera and the size of the robot’s working space.

2. If the Z axes of the camera and flange frames are not parallel, click the Auto-align the path button.

3. Set the Path type parameter to InHand, specify the pyramid parameters Height span, Num of layers, Bottom-layer dimensions X/Y, Top-layer dimensions X/Y and Motion grid cols and rows per layer, specify the satellite parameters Number of steps, Center height to board and Radius, and then click the Confirm button.

4. Confirm that waypoints are reasonable and will not collide with obstacles in the right scene viewer panel.

   

5. Click the Next button on the bottom bar.

Collect Images and Poses

1. In the Collect images and poses step, select the Save images checkbox, and then click the Auto move robot along path and capture images button.

   

2. Read the safety window carefully and click the OK button.

   

3. Wait until the robot finishes moving along the preset path and the camera finishes capturing images on all waypoints. In the right Image viewer panel, all the captured images are displayed.

   Attention

   • Please stay away from the robot working area to keep safe.

   • Clicking the Stop Robot button can exit the calibration process. But the robot will not stop until it finishes the current waypoint. In case of emergency, please tap the emergency stop button on the robot teach pendant to stop the robot immediately.

4. After all images are captured, and click the OK button in the prompted window.

   

5. Confirm that the number of the currently recognized calibration circles meets the requirements, and then click the Next button on the bottom bar.

Tip

If the requirements are not met, please manually move the robot (through the teach pendant or Mech-Viz) , select the Manually add more images checkbox, and click the Add an image and record flange pose button to add the calibration board image and enter the robot flange pose.

Calculate Camera Parameters

1. In the Calculate camera parameters step, select the Calculate rotation and translation separately checkbox, and then click the Calculate camera extrinsic parameters button.

   

2. In the prompted window indicating calibration success, click the OK button.

   

Confirm that the Calibration Accuracy Meets Requirements and Save Calibration Data

1. In the Calculate camera parameters step, view the calibration error of point cloud in the right Point cloud viewer panel after the calibration calculation is complete.

2. Find the error value with the 100% percentage to get the rough calibration accuracy. For example, the calibration accuracy in the following figure is less than 2.5 mm.

   

3. Confirm that the calibration accuracy meets the project requirements, and then click the Save button on the bottom bar.

   

4. In the prompted Save Calibration Files dialog box, click the OK button to save the calibration result to the project folder.

Till now, the calibration process is completed. If you want to enhance the calibration accuracy, please refer to the section Calibration Result Check and Analysis.