Hand-Eye Calibration

You are viewing an old version of the documentation. You can switch to the documentation of the latest version by clicking the top-right corner of the page.

In this tutorial, you will perform the automatic hand-eye calibration in the Eye-To-Hand setup.

The hand-eye calibration establishes the transformation relationship between the camera and robot reference frames (that is camera extrinsic parameters). With this relationship, the object pose determined by the vision system can be transformed into that in the robot reference frame, which guides the robot in performing its tasks.
Video Tutorial: Hand-Eye Calibration (Master-Control)

Preparation before Calibration

In this section, you will mount the calibration board, adjust the camera parameters, and complete the pre-calibration configuration.

Mount the Calibration Board

In the Eye to Hand scenario, the calibration board needs to be mounted to the robot flange adapter.

Follow these steps:

  1. Take the calibration board and flange adapter out of the system delivery package.

  2. Use screws, gaskets, and nuts to fasten the flange adapter to the flange adapter.

  3. Use screws, gaskets, and nuts to secure the calibration board to the flange adapter.

  4. After mounting the calibration board, move the robot to the top of the lowest workpiece in the work area and into the camera FOV.

Adjust Camera Parameters

  1. In Mech-Eye Viewer, connect to the camera, and then set Parameter Group to “calib”.

    viewer parameter group calib

  2. Adjust the 2D parameters to make sure that the 2D image of the calibration board is clear and neither overexposed nor underexposed.

  3. Adjust 3D parameters to make sure that the obtained point clouds of the circles on the calibration board are complete and have clear contours. It is recommended to set the Surface Smoothing and Outlier Removal parameters of Point Cloud Processing to Normal to reduce point cloud fluctuation.

Normal Overexposed Underexposed

2D image

normal-2d

overexposure-2d

underexposure-2d

Point cloud

normal-3d

overexposure-3d

underexposure-3d

Create a Mech-Vision Project and Save It

  1. Open Mech-Vision. When the welcome interface as shown below is displayed, it indicates that Mech-Vision is started successfully.

    project build welcome interface

  2. In the Welcome interface of Mech-Vision, click Create from solution library to open the Solution Library.

    project build welcome interface library

    Solution Library is a resource library that provides typical solutions or projects (with example data) coming from different application scenarios.

  3. Select the Single-Case Cartons project from the Solution Library.

    project build select project

    If you cannot find this project in the Solution Library, click More at the bottom of the Solution Library interface.

  4. After this project is selected, the information introducing this project will be displayed on the lower part of the Solution Library interface. Set the solution name and path, and then click Create.

    project build check project info

    After the project is created, the created solution and project will be displayed in the project list in the upper-left corner of the Mech-Vision main interface.

    • A solution is a set of configurations and data concerning robot and robot communication, vision processing, path planning, etc. that are required for the application of machine vision.

    • A project is a workflow of vision processing in the solution. Usually, a solution only contains a project, but it may contain more than one project in complex application scenarios. In this tutorial, the solution only needs one project.
    project build check project list

    In the Graphical Programming Workspace of the main interface, the workflow of the “Single-Case Cartons” project will be displayed.

    project build check project step

  5. On the menu bar, select File  Save Solution.

    project build save solution

Pre-calibration Configuration

  1. In Mech-Vision, click the Camera Calibration (Standard) on the toolbar. The Configuration before Calibration window will be prompted.

  2. In the Select how to calibrate window, select the New calibration radio button, and then click the Next button.

    getting start calib how

  3. In the Select calibration task window, select Hand-eye calibration for listed robot from the drop-down list box, and click the Select robot model button.

    getting start calib task

  4. Click the Brand dropdown box, and select “ABB”. Then, at the right panel, select model “ABB_IRB_1300_11_0_9”, click the Select button, and then click the Next button.

    getting start select robot model

  5. In the Select camera setup window, select the Eye to hand radio button, and then click the Next button.

    getting start calib eth

  6. In the Calibration method and robot control window, select Automatic and Master-control, and then click the Next button.

    getting start calib auto master

  7. In the Communication Settings window, set the Robot IP address parameter to the real robot’s IP address.

    getting start calib communication

  8. On the robot teach pendant, confirm that the MM main program has been started.

  9. Return to Mech-Vision, and click the Connect the robot button in the Connect the robot area. The button will turn into Waiting for the robot to connect...

  10. Wait until the “Connected” status message is displayed in the Connect the robot area, and then click the Calibrate button. The Calibration (Eye to Hand) window will be prompted.

Calibration Procedure

Connect to the Camera

  1. In the Connect to Camera step, find the camera to connect in the Detected Cameras list, and click the Connect button.

    getting start connect camera

  2. After the camera is connected, click the Capture once or Capture live button.

  3. In the right Image viewer panel, ensure that the captured 2D image and depth map meet the calibration requirements and click the Next button on the bottom bar.

If the captured image does not meet the calibration requirements, you need to open the Mech-Eye Viewer software to adjust the 2D and 3D exposure parameters and re-capture images.

Mount the Calibration Board

  1. In the Mount calibration board step, select the Standard radio button and select the calibration board model according to its model nameplate in the 1. Select calibration board type area.

  2. Make sure that the calibration board has been attached to the robot flange securely, and then click the Confirm button in the 2. Mount calibration board area.

  3. Confirm that the calibration board is in the center of the camera’s field of view (the red rectangle), and then click the Confirm button in the 3. Position calibration board into red rectangle area.

    getting start install calib board

  4. After all the operations related to the calibration board are completed, click the Next button on the bottom bar.

Check Intrinsic Parameters

  1. In the Check intrinsic parameters step, click the Start checking button.

    getting start check intri params

  2. Confirm that the camera intrinsic parameters check passes, and then click the Next button on the bottom bar.

    getting start check intri pass

If the camera intrinsic parameter check fails, adjust detection parameters by drawing aid circles or manually adjusting the calibration circle detection parameters.

Set Robot Path

  1. In the Set robot path step, specify the Height span parameter. Please set this parameter according to the working distance range of the calibration board in the camera height direction.

    getting start set motion path

  2. Set the Path type to ToHand, specify the pyramid parameters Height span, Num of layers, Bottom-layer dimensions X/Y, Top-layer dimensions X/Y, and Motion grid cols and rows per layer, specify the satellite parameter Rotation angle, and then click the Confirm button.

    The set robot path should cover the work area.

  3. In the right Scene Viewer panel, confirm that the waypoints of the automatically generated motion path will not collide with obstacles in the environment, and then click the Next button.

Collect Images and Poses

  1. In the Collect images and poses step, select the Save images checkbox.

    getting start add image poses

  2. Click the Control robot to auto move along path and capture images button.

  3. Read the safety window carefully and click the OK button.

    getting start security warnning

  4. Wait until the robot finishes moving along the preset path and the camera finishes capturing images on all waypoints. Captured images can be seen in the right Marker Image and Pose List panel during this process.

  5. After the automatic image capturing finishes, click the OK button, and then click the Next button on the bottom bar.

    getting start auto capture finished

Calculate Camera Parameters

  1. In the Calculate camera parameters step, click the Calculate camera extrinsic parameters button.

    getting start calculate extri params

  2. In the prompted window indicating calibration success, click the OK button.

  3. View the calibration error of the point cloud in the right Point cloud viewer panel.

    The error point cloud shows the deviation between the calculated value and the actual value of the circles on the calibration board.

  4. Confirm that the calibration accuracy meets the project requirements.

    Determine the error value with 100% to obtain the rough calibration accuracy. For example, the calibration accuracy in the following figure is within 2 mm.

    getting start check calib accuracy

    To improve the calibration accuracy, please refer to the section Calibration Result Check and Analysis.

Validate and Save the Calibration Result

  1. After calibration, move the robot arms into the FOV of the camera.

  2. In the Calculate camera parameters step, click the Calculate camera extrinsic parameters button. This operation triggers the camera to capture images.

  3. Click Scene viewer to check the coincidence degree between the point cloud of the robot and the robot model.

    If the point cloud of the robot approximately coincides with the robot model, the calibration is successful.

    robot point cloud coincide

  4. Click the Save button on the bottom bar. In the prompted Save Calibration Files dialog box, click the OK button. The camera calibration result will be automatically saved in the “calibration” directory of the project.

We Value Your Privacy

We use cookies to provide you with the best possible experience on our website. By continuing to use the site, you acknowledge that you agree to the use of cookies. If you decline, a single cookie will be used to ensure you're not tracked or remembered when you visit this website.

  • Preference
  • I accept
  • I don't accept