Use HALCON to Control the Camera

This topic provides instructions on controlling Mech-Eye Industrial 3D Camera with HALCON and introduction on the HALCON samples.

Prerequisites

  • The camera and IPC are properly connected.

    Hint

    For Nano and Pro XS, it is recommended to connect the camera directly to the IPC, without using a network switch.

  • HALCON 20.11 or above has been installed on IPC.

  • Mech-Eye SDK 2.0.0 or above has been installed on IPC.

    Hint

    The version of the camera firmware must be consistent with that of Mech-Eye SDK. If not, please upgrade the camera firmware.

  • The IP addresses of the camera and IPC are in the same subnet.

    Hint

    For instructions on setting the IP addresses, please refer to Connect to Camera and Set IP Address.

In HALCON, you can control the camera with Image Acquisition Assistant or the sample program.

Control the Camera with Image Acquisition Assistant

Using the image acquisition assistant provided by HDevelop, you can quickly connect to the camera, perform image capturing and adjust camera parameters.

To open the image acquisition assitant: open HDevelop, and in the menu bar, select Assistants ‣ Open New Image Acquisition.

Connect to the Camera

  1. In the Image Acquisition window, under the Source tab, select Image Acquisition Interface, and select GigEVision2 from the drop-down menu.

    Hint

    If the GigEVision2 option is unavailable, the GigEVision2 image acquisition interface is not installed. Please refer to HALCON’s Installation Guide and install the interface through MVTec Software Manager (SOM).

    ../../_images/halcon_assistant_1.png

  2. Under the Connection tab, from the drop-down menu of Device, select the device you would like to connect. Then, click Connect in the lower left.

    ../../_images/halcon_assistant_2.png

    Hint

    • If the camera is successfully connected, the Connect button changes to Disconnect. If this change does not occur, the camera connection failed.

    • If the camera is already connected by another client, it cannot be connected by HDevelop. Please disconnect the camera from the other client first.

    • To disconnect from the camera in HDevelop, click Disconnect.

    • The camera name displayed here can be customized in Mech-Eye Viewer. It is recommended to avoid using special characters such as letters with accent marks, as these characters cannot be displayed in HDevelop.

Perform Image Capturing

To perform image capturing once: click Snap under the Connection tab in the Image Acquisition window.

Note

If the image capturing process takes too long, you can enable jumbo frames on your IPC.

You can also perform image capturing multiple times or continuously. To do this, you need to adjust the AcquisitionMode parameter first.

  • To perform image capturing multiple times:

    1. Switch to the Parameters tab, set the value of the AcquisitionMode parameter to MultiFrame.

    2. Click Refresh in the upper right, and then set the number of times for image capturing in the AcquisitionFrameCount parameter.

    3. Switch to the Connection tab and click Live to perform image capturing.

    4. After the set times of image capturing are completed, the Live button becomes Stop. Click Stop to stop image capturing.

  • To perform image capturing continuously:

    1. Switch to the Parameters tab, set the value of the AcquisitionMode parameter to Continuous.

    2. Switch to the Connection tab and click Live to perform image capturing.

    3. The Live button becomes Stop. Click Stop to stop image capturing.

Note

  • You can still use Snap to perform single-time image capturing when AcquisitionMode is set to MultiFrame or Continuous.

  • When AcquisitionMode is set to SingleFrame, you can only perform single-time image capturing.

Select Data Type

After the camera is connected, you can select whether to obtain 2D image or depth map by adjusting the DeviceScanType parameter.

  1. Click the Parameters tab and find the DeviceScanType parameter. Set its value according to your needs:

    Value

    Data type

    Areascan

    2D image

    Areascan3D

    Depth map (an image containing depth information)

  2. Perform image capturing to obtain the data type that you selected.

    Hint

    If Update Image in the upper right of the Parameters tab is checked, the image in the Canvas winodw is automatically updated as you adjust the parameters.

Set Capture Region

After the camera is connected, if you need to trim the obtained image, you can set a capture region by adjusting the Height, Width, OffsetX and OffsetY parameters.

To set a capture region, follow these steps:

  1. Select the data type for setting a capture region.

  2. Perform image capturing with Snap to see the current image.

  3. Switch to the Parameters tab and adjust the Height, Width, OffsetX and OffsetY parameters. The following figure shows the four parameters and the capture region defined (orange box) relative to the original image.

    ../../_images/set_capture_region_1.png

    • Width: the width of the capture region

    • Height: the height of the capture region

    • OffsetX: the x-coordinate of the upper-left corner of the capture region (the upper-left corner of the original image being (0, 0))

    • OffsetY: the y-coordinate of the upper-left corner of the capture region

    Note

    The above four parameters must satisfy the following requirements:

    • (Width + OffsetX) not greater than the width of the original image

    • (Height + OffsetY) not greater than the height of the original image

    The width and height of the original image are displayed in the WidthMax and HeightMax parameters under Read-only parameters (Visibility level must be set to Expert or higher).

  4. Perform image capturing again to see the trimming result.

    Hint

    If Update Image in the upper right is checked, the image in the Canvas winodw is automatically updated as you adjust the parameters.

  5. Switch to the Code Generation tab, click Inser Code to generate the corresponding code.

  6. If you need to set a capture region for the other data type:

    1. Disconnect from the camera in the current image acquisition assistant.

    2. Open a new image acquisition assistant, and connect to the camera.

    3. Select the other data type and repeat the above steps.

Note

  • The values of the four capture region parameters are not saved to parameter groups and are reset if the camera is powered off. Please generate the corresponding code and save the code for future reference.

  • The values of these parameters are also reset if the values of the DeviceScanType and Scan3DBinningEnable parameters are changed.

Comparison of Capture Region and Scan3DROI

Mech-Eye Industrial 3D Camera provides another set of parameters for setting an ROI: Scan3DROILeft, Scan3DROITop, Scan3DROIHeight, and Scan3DROIWidth (collectively referred to as “Scan3DROI”).

The differences between the capture region parameters and Scan3DROI parameters are summarized below. Please select the set of parameters that suit your needs.

Capture Region

Scan3DROI

Not saved to parameter groups, reset if camera powered off

Can be saved to parameter groups

Applicable to 2D image and depth map

Not applicable to 2D image

Image is trimmed

Image not trimmed

Can only be set in HDevelop

Can be set with visualized tool in Mech-Eye Viewer

Adjust Parameters

If the quality of the obtained data is unsatisfactory, you can adjust the camera parameters under the Parameters tab.

Hint

  • The camera supports the parameter group function; by selecting a different parameter group, you can quickly change the parameter configuration for the camera. Camera parameters must be set and saved under a parameter group.

  • You can add or delete a parameter group in Mech-Eye Viewer. After configuring the parameter group in Mech-Eye Viewer, click the Refresh button in the upper-right corner of the Parameters tab in Image Acquisition Assistant to obtain the latest camera configuration.

To adjust the camera parameters, follow these steps:

  1. After the camera is connected, click the Parameters tab, set the UserSetSelector parameter to the parameter group that you want to modify.

    ../../_images/select_parameter_group.png

  2. Find the UserSetLoad parameter, and click Apply to the right to read in the configuration.

    Note

    If parameter values are not updated after you click Apply, please click it again.

  3. Find the parameter that you want to adjust and change its value.

  4. Find the UserSetSave parameter, and click Apply to the right to save the configuration.

  5. Switch to the Code Generation tab, click Inser Code to generate the corresponding code.

References:

  • For descriptions and explanations of camera parameters supported for GenICam, download and read Parameters in GenICam Client.

  • Camera parameters supported for GenICam correspond with those provided by Mech-Eye Viewer. For their correspondence, refer to Camera Parameters Supported for GenICam.

  • Setting camera parameters such as Auto-Exposure ROI, Depth Range and ROI usually requires the assistance of visualized tools for several rounds of fine tuning. Since HALCON does not provide visualized tools, you can set these parameters using the visualized tools provided by Mech-Eye Viewer. For detailed information, refer to Use Mech-Eye Viewer to Adjust Camera Parameters for GenICam Client.

    Hint

    You must disconnect from the camera in HDevelop before you can connect to it in Mech-Eye Viewer and adjust parameters. If you fail to connect the camera in Mech-Eye Viewer, close HDevelop and try again.

Control the Camera with Sample Program

A HALCON sample program is provided on GitHub. With this sample program, you can connect to the camera, capture images, adjust parameters and save the data. You can run the program as is or modify it to better suit your needs.

Note

The HALCON sample program is written in C++ and can be executed in HDevelop of HALCON directly.

Run the HALCON Sample Program

  1. Obtain the HALCON sample program.

  2. Open the sample program in any of the following ways:

    • Open HDevelop, and drag the sample program into HDevelop.

    • Right-click the sample program and select Open with ‣ HDevelop.

  3. Obtain all the available cameras in any of the following ways:

    • Select the line containing the info_framegrabber operator in Program Windows and click step_over in the toolbar.

    • Select the line containing the info_framegrabber operator in Program Windows and press the F6 key to execute the first operator.

    Hint

    All the obtained cameras can be viewed in the DeviceInfos variable in the Control Variables area of the Variable Window.

  4. Select a camera to connect in any of the following ways:

    • Select a camera by its unique_name:

      (1) In the Control Variables area, double-click DeviceInfos to display a list of all the available cameras.

      ../../_images/halcon_program_1.png

      (2) In the list, double-click the camera that you want to connect, and copy the camera name after unique_name:.

      ../../_images/halcon_program_2.png

      (3) In the Program Window, locate the line containing the tuple_regexp_select operator, and replace MechEye in single quotation marks with the copied unique_name.

    • Select a camera by its index number:

      (1) In the Control Variables area, double-click DeviceInfos to display a list of all the available cameras.

      ../../_images/halcon_program_1.png

      (2) Check the index number of the camera that you want to connect.

      ../../_images/halcon_program_3.png

      (3) In the Program Window, locate the line containing the open_framegrabber operator, and change the number in MechEyeCameraInfos[X] accordingly.

  5. Run the sample program by clicking run or pressing the F5 key.

  6. If the program is successfully executed, you can find the saved 2D image (image2d.bmp by default) and point cloud (PointCloud.ply by default) in the folder where the sample program is located.

Note

  • After executing the entire sample program, please click reset in the toolbar to reset program execution. Otherwise, the camera cannot be connected by Mech-Eye Viewer.

  • When the point cloud is displayed in the Canvas window, please click the orange Continue button in this window to continue the program execution. Otherwise, the program is stuck in the visualize_object_model_3d operator and will not proceed.

  • If the image capturing process takes too long, you can enable jumbo frames on your IPC.

  • If the network connection is poor, you can reduce data loss by increasing the maximum number of allowed packet gaps with the following operator. Replace ParameterValues with the number of allowed packet gaps.

    set_framegrabber_param (AcqHandle, '[Stream]GevStreamMaxPacketGaps', ParameterValues)

Explanation of the Sample Program

  • Connect to the camera:

    info_framegrabber ('GigEVision2', 'device', Info, DeviceInfos)
tuple_regexp_select(DeviceInfos, 'MechEye', MechEyeCameraInfos)
open_framegrabber ('GigEVision2', 1, 1, 0, 0, 0, 0, 'default', -1, 'default', -1, 'false', 'default', MechEyeCameraInfos[0],   0, -1, AcqHandle)

  • Perform image capturing:

    grab_image (Image2d, AcqHandle)
grab_data(Image3d, Region, Contours, AcqHandle, ObjectModel3D)

  • Obtain available parameters:

    get_framegrabber_param (AcqHandle, 'available_param_names', ParameterValues)

  • Modify parameters:

    Parameters included in the sample program are 2D parameters, 3D parameters, ROI, depth range and point cloud settings.

    • 2D parameters:

      * Switch to the 2D scanning mode.
set_framegrabber_param (AcqHandle, 'DeviceScanType', 'Areascan')
set_framegrabber_param (AcqHandle, 'AcquisitionMode', 'SingleFrame')

get_framegrabber_param (AcqHandle, 'Width', Width)
get_framegrabber_param (AcqHandle, 'Height', Height)
get_framegrabber_param (AcqHandle, 'PixelFormat', PixeLFormat)

* Set the 2D scanning parameters.
set_framegrabber_param (AcqHandle, 'Scan2DExposureMode', 'Timed')
set_framegrabber_param (AcqHandle, 'Scan2DExposureTime', 100)

    • 3D parameters:

      * Switch to the 3D scanning mode:
set_framegrabber_param (AcqHandle, 'DeviceScanType', 'Areascan3D')

* Open the 3D object model generator.
set_framegrabber_param (AcqHandle, 'create_objectmodel3d', 'enable')
set_framegrabber_param (AcqHandle, 'add_objectmodel3d_overlay_attrib', 'enable')

get_framegrabber_param (AcqHandle, 'Width', Width)
get_framegrabber_param (AcqHandle, 'Height', Height)
get_framegrabber_param (AcqHandle, 'PixelFormat', PixeLFormat)

* Set the 3D scanning exposure time parameter.
set_framegrabber_param (AcqHandle, 'Scan3DExposureCount', 1)
set_framegrabber_param (AcqHandle, 'Scan3DExposureTime', 8)

    • Depth range:

      * Set the 3D scanning depth range parameter(unit: mm).
set_framegrabber_param (AcqHandle, 'DepthLowerLimit', 1)
set_framegrabber_param (AcqHandle, 'DepthUpperLimit', 3000)

    • Point cloud settings:

      * Set the 3D scanning point cloud filter parameter.
set_framegrabber_param (AcqHandle, 'CloudOutlierFilterMode', 'Normal')
set_framegrabber_param (AcqHandle, 'CloudSmoothMode', 'Normal')

  • Save the obtained data:

    • Save 2D image:

      * Acquire the 2D image from the camera.
grab_image (Image2d, AcqHandle)

* Save the 2D image result in "Image2d.bmp"
write_image( Image2d , 'bmp' , 0 , 'Image2d' )

    • Save point cloud:

      * Acquire the 3D data from the camera.
grab_data(Image3d, Region, Contours, AcqHandle, ObjectModel3D)

get_object_model_3d_params(ObjectModel3D, 'num_points', NumOfPoints)

if(NumOfPoints != 0)
    * Save the point cloud result in "PointCloud.ply"
    write_object_model_3d (ObjectModel3D, 'ply', 'PointCloud.ply', [], [])
endif

  • Disconnect from the camera:

    clear_object_model_3d (ObjectModel3D)
close_framegrabber (AcqHandle)

Adjust Parameters with HALCON Program

If the quality of the obtained data is unsatisfactory, you can adjust the camera parameters.

Hint

  • The camera supports the parameter group function; by selecting a different parameter group, you can quickly change the parameter configuration for the camera. Camera parameters must be set and saved under a parameter group.

  • You can add or delete a parameter group in Mech-Eye Viewer.

To adjust the camera parameters, follow these steps:

  1. Use the following operator to obtain a list of available parameter groups.

    get_framegrabber_param (AcqHandle, 'UserSetSelector_values', ParameterValues)

  2. View the list of parameter groups in the ParameterValues variable in the Control Variables area.

  3. Use the following operator to select the parameter group to modify. UserSetSelector and UserSetLoad are the camera parameters used to select and load a parameter group, respectively. Replace ParameterGroupName with the actual parameter group name. .. code-block:

    set_framegrabber_param (AcqHandle, 'UserSetSelector','ParameterGroupName')
set_framegrabber_param (AcqHandle, 'UserSetLoad','ParameterGroupName')

  4. Use the following operator to obtain the value of the specified parameter. Replace ParameterName with the actual camera parameter name. ParameterValues is the variable used to save the parameter value, and you can change it according to actual needs.

    get_framegrabber_param (AcqHandle, 'ParameterName', ParameterValues)

  5. View the parameter value in the ParameterValues variable in the Control Variables area.

  6. Use the following operator to adjust the setting of the specified parameter. Replace ParameterName with the actual camera parameter name and NewParameterValue with the new parameter value.

    set_framegrabber_param (AcqHandle, 'ParameterName', 'NewParameterValue')

  7. Use the following operator to save the parameter value modification into the parameter group. UserSetSave is the camera parameter used to save the parameter group. Replace ParameterGroupName with the actual parameter group name.

    set_framegrabber_param (AcqHandle, 'UserSetSave', 'ParameterGroupName')

References:

  • For descriptions and explanations of camera parameters supported for GenICam, download and read Parameters in GenICam Client.

  • Camera parameters supported for GenICam correspond with those provided by Mech-Eye Viewer. For their correspondence, refer to Camera Parameters Supported for GenICam.

  • Setting camera parameters such as Auto-Exposure ROI, Depth Range and ROI usually requires the assistance of visualized tools for several rounds of fine tuning. Since HALCON does not provide visualized tools, you can set these parameters by using the visualized tools provided by Mech-Eye Viewer. For detailed information, refer to Use Mech-Eye Viewer to Adjust Camera Parameters for GenICam Client.

    Hint

    You must disconnect from the camera in HDevelop before you can connect to it in Mech-Eye Viewer and adjust parameters. If you fail to connect the camera in Mech-Eye Viewer, close HDevelop and try again.