Hand-Eye Calibration

In this tutorial, you will perform the automatic hand-eye calibration in the eye-to-hand setup.

The hand-eye calibration establishes the transformation relationship between the camera and robot reference frames (that is camera extrinsic parameters). With this relationship, the object pose determined by the vision system can be transformed into that in the robot reference frame, which guides the robot in performing its tasks.
Video Tutorial: Hand-Eye Calibration (Standard Interface Communication)

Preparation before Calibration

In this section, you will modify the robot calibration program, mount the calibration board, adjust the camera parameters, and complete the pre-calibration configuration.

Modify the Robot Calibration Program (MM_AUTO_CALIB)

Select the Calibration Program

  1. In the TEACH mode, select JOB  SELECT JOB. Move the cursor to MM_AUTO_CALIB in the JOB LIST and press the SELECT key on the teach pendant.

    choose caribrate file 1
    choose caribrate file 2

    The interface is as follows after the program file is selected.

    choose caribrate file 3
You have already modified the IP address and port of the IPC in the section “Test Standard Interface Communication”. Therefore, you just only need to teach the calibration start point in this tutorial.

Teach the Calibration Start Point

In the calibration process, the robot poses sent by the robot to Mech-Vision must be flange poses. Therefore, before this step, you need to switch the tool No. to that (usually 00) of the tool whose ToolOffset values are all 0. The Tool Center Point (TCP) of Tool00 is at the center of the flange.

  1. Move the robot to the start point for calibration. Move the cursor to MOVJ P099 VJ=5, and press the DIRECT OPEN key on the teach pendant.

    calibrate start point 1

  2. Change the value of P099 to the current pose of the robot: Press the SERVO ON READY key on the teach pendant. Press the MODIFY key and then the ENTER key while holding the enable switch on the back. Make sure the value for TOOL is 00.

    calibrate start point 1 1
    calibrate start point 1 2
    calibrate start point 1 3
    calibrate start point 2

  3. Press the DIRECT OPEN key on the teach pendant again to return to JOB CONTENT.

So far, you have successfully modified the calibration program necessary for the automatic calibration process of YASKAWA robots.

Mount the Calibration Board

In the eye-to-hand setup, the calibration board needs to be mounted to the robot flange adapter.

Follow these steps:

  1. Take the calibration board and flange adapter out of the system delivery package.

  2. Use screws, gaskets, and nuts to fasten the flange adapter to the flange adapter.

  3. Use screws, gaskets, and nuts to secure the calibration board to the flange adapter.

  4. After mounting the calibration board, move the robot to the top of the lowest workpiece in the work area and into the camera’s field of view (FOV).

Adjust Camera Parameters

  1. In Mech-Eye Viewer, connect to the camera, and then set Parameter Group to “calib”.

    viewer parameter group calib

  2. Adjust the 2D parameters to make sure that the 2D image of the calibration board is clear and neither overexposed nor underexposed.

  3. Adjust 3D parameters to make sure that the obtained point clouds of the circles on the calibration board are complete and have clear contours. It is recommended to set the Surface Smoothing and Outlier Removal parameters of Point Cloud Processing to Normal to reduce point cloud fluctuation.

Normal Overexposed Underexposed

2D image

normal-2d

overexposure-2d

underexposure-2d

Point cloud

normal-3d

overexposure-3d

underexposure-3d

Pre-calibration Configuration

  1. In Mech-Vision, click the Camera Calibration (Standard) button on the toolbar. The Configuration before Calibration window will be prompted.

  2. In the Select how to calibrate window, select the New calibration radio button, and then click the Next button.

    getting start calib how

  3. In the Select calibration task window, select Hand-eye calibration for listed robot from the drop-down list box, and click the Select robot model button.

    getting start calib task

  4. Click the Brand dropdown box, and select “YASKAWA”. Then, at the right panel, select model “YASKAWA_GP8”, click the Select button, and then click the Next button.

    getting start select robot model

  5. In the Select camera setup window, select the Eye to hand radio button, and then click the Next button.

    getting start calib eth

  6. In the Calibration method and robot control window, select Automatic and Standard Interface, and then click the Next button.

    getting start calib auto interface

  7. In the Interface settings window, set the port of the Host IP address parameter to 50000.

    getting start calib communication

    • In this tutorial, the vision system acts as the TCP server. Therefore, the Host IP address parameters should be set to the IP address and port of the IPC. Since the IP address of the IPC is a local IP address, you do not need to set it and the software uses “0.0.0.0” to indicate it.

    • By default, the IPC uses port 50000 to provide the interface service. You can change the port number as required. If you have changed the port number, remember to use the correct port number when you try to establish communication with the vision system on the robot side.

  8. Click the Connect the robot button in the Connect the robot area. The button will turn into Waiting for the robot to connect...

  9. Run the calibration program on the robot teach pendant.

    Operation instructions

    1. Move the cursor back to the first line (0000), turn the mode switch to PLAY mode to enter the PLAY mode, press the SERVO ON READY key on the teach pendant.

      run program 1
      run program 2

    2. To move the robot at a low speed, select UTILITY  SETUP SPECIAL RUN on the teach pendant screen, and change SPEED LIMIT to VALID.

      low speed run 1
      low speed run 2

    3. Press the green START button on the teach pendant. when the start LED is lit, the robot is running the program.

      run program 3

      If the calibration program runs normally, “Connected” status message will be displayed in the Connect the robot area of Mech-Vision.

  10. After the robot is connected, click the Calibrate button. The Calibration (Eye to hand) window will be prompted.

Calibration Procedure

Connect to a Camera

  1. In the Connect to Camera step, find the camera to connect in the Detected Cameras list, and click the Connect button.

    getting start connect camera

  2. After the camera is connected, click the Capture once or Capture live button.

  3. In the right Image viewer panel, ensure that the captured 2D image and depth map meet the calibration requirements and click the Next button on the bottom bar.

If the captured image does not meet the calibration requirements, you need to open the Mech-Eye Viewer software to adjust the 2D and 3D exposure parameters and re-capture images.

Mount the Calibration Board

  1. In the Mount calibration board step, select the Standard radio button and then select the calibration board model according to its model nameplate in the 1. Select calibration board type area.

  2. Make sure that the calibration board has been attached to the robot flange securely, and then click the Confirm button in the 2. Mount calibration board area.

  3. Confirm that the calibration board is in the center of the camera’s field of view (the red rectangle), and then click the Confirm button in the 3. Position calibration board into red rectangle area.

    getting start install calib board

  4. After all the operations related to the calibration board are completed, click the Next button on the bottom bar.

Check Intrinsic Parameters

  1. In the Check intrinsic parameters step, click the Start checking button.

    getting start check intri params

  2. Confirm that the camera intrinsic parameters check passes, and then click the Next button on the bottom bar.

    getting start check intri pass

If the camera intrinsic parameter check fails, adjust detection parameters by drawing aid circles or manually adjusting the calibration circle detection parameters.

Set Robot Path

  1. In the Set robot path step, specify the Height span parameter. Please set this parameter according to the working distance range of the calibration board in the camera height direction.

    getting start set motion path

  2. Set the Path type to ToHand, specify the pyramid parameters Height span, Num of layers, Bottom-layer dimensions X/Y, Top-layer dimensions X/Y, and Motion grid cols and rows per layer, specify the satellite parameter Rotation angle, and then click the Confirm button.

    The set robot path should cover the work area.

  3. In the right Scene Viewer panel, confirm that the waypoints of the automatically generated motion path will not collide with obstacles in the environment, and then click the Next button.

Collect Images and Poses

  1. In the Collect images and poses step, select the Save images checkbox.

    getting start add image poses

  2. Click the Control robot to auto move along path and capture images button.

  3. Read the safety window carefully and click the OK button.

    getting start security warnning

  4. Wait until the robot finishes moving along the preset path and the camera finishes capturing images on all waypoints. Captured images can be seen in the right Marker Image and Pose List panel during this process.

  5. After the automatic image capturing finishes, click the OK button, and then click the Next button on the bottom bar.

    getting start auto capture finished

Calculate Camera Parameters

  1. In the Calculate camera parameters window, click the Calculate camera extrinsic parameters button.

    getting start calculate extri params

  2. In the prompted window indicating calibration success, click the OK button.

  3. View the calibration error of the point cloud in the right Point cloud viewer panel.

    The error point cloud shows the disparity between the calculated value and the actual value of the circles on the calibration board for each pose.

  4. Confirm that the calibration accuracy meets the project requirements.

    Determine the error value with 100% to obtain the rough calibration accuracy. For example, the calibration accuracy in the following figure is within 2.5 mm.

    getting start check calib accuracy

    To improve the calibration accuracy, please refer to the section Calibration Result Check and Analysis.

Validate and Save the Calibration Result

  1. After calibration, move the robot arms into the FOV of the camera.

  2. In the Calculate camera parameters step, click the Re-calculate camera extrinsic parameters button. This operation triggers the camera to capture images.

  3. Click Scene viewer to check the coincidence degree between the point cloud of the robot and the robot model.

    If the point cloud of the robot approximately coincides with the robot model, the calibration is successful.

    robot point cloud coincide

  4. Click the Save button on the bottom bar. In the prompted Save Calibration Files dialog box, click the OK button. The camera calibration result will be automatically saved in the “calibration” directory of the project.

We Value Your Privacy

We use cookies to provide you with the best possible experience on our website. By continuing to use the site, you acknowledge that you agree to the use of cookies. If you decline, a single cookie will be used to ensure you're not tracked or remembered when you visit this website.

  • Preference
  • I accept
  • I don't accept