Capture Images from Camera

You are viewing an old version of the documentation. You can switch to the documentation of the latest version by clicking the top-right corner of the page.

Function

This Step is used to obtain color images, depth maps, and point clouds from a real or virtual camera.

When this Step is used to connect a DEEP (V4) or LSR (V4) series camera, and the 2D Image Type is set to Internal IR Image, the output “color image” is actually a 2D image (Depth Source). If you want to adjust the 2D image (Depth Source), adjust the 2D Image (Depth Source) Exposure Time of the 2D Image (Depth Source) Exposure Mode in Mech-Eye Viewer.

Usage Scenario

Used as the initial input entry of a Mech-Vision project, this Step collects data from a Mech-Eye Industrial 3D Camera or third-party camera or acts as a virtual camera for inputting simulated data source.

Input and Output

capture images from camera input and output

Parameter Tuning

Camera Settings

Camera Type

Description: This parameter is used to select the type of camera in use.

Value list:

  • External2D: external 2D camera

  • LMI: LMI camera

  • Mech-Eye (default value): Mech-Eye Industrial 3D Camera

  • Mech-EyeTOF: customized Mech-Eye Industrial 3D Camera

  • If the firmware of Mech-Eye Industrial 3D Camera is updated to 2.0.0, it is recommended to use Mech-Vision of version 1.6.2 and above. If you use Mech-Vision 1.6.1 or older versions, the software may crash when you connect the Mech-Eye Industrial 3D Camera in the External2D EBUSCamera category in the Choose the camera and parameter group to use window.

  • The Virtual Mode is supported only when Mech-Eye camera type is selected.

Tuning recommendation: Different camera types have different parameters. Please adjust according to the actual situation.


Mech-Eye

When the Virtual Mode is disabled, a real camera can be used. When the Virtual Mode is enabled, local data will be used to run the Step. The Virtual Mode is disabled by default.

Use a Real Camera

Camera ID

Description: This parameter is used to connect the camera and select the camera calibration parameters.

Instructions:

  1. Click Select camera to open the Choose the camera and parameter group to use window.

  2. Select the camera you want to connect and click capture images from camera icon 2.

    If the camera is connected successfully, capture images from camera icon 2 will become capture images from camera icon 3.

  3. After the camera is connected successfully, you can click Select from and select the camera calibration parameter group in the drop-down list and then click OK in the lower right corner of the window.

    Once the calibration parameter group is selected successfully, the Camera ID, Camera Calibration Parameters, IP Address, and Port parameters will be automatically filled.

Camera Calibration Parameters

Description: This parameter is used to specify the calibration parameter group of the camera.

IP Address

Description: This parameter is used to display the IP address of the camera. Please do not modify.

Port

Description: This parameter is used to display the port number of the camera. Please do not modify.

IP Address and Port are only used for display. Modifying these two parameters will disconnect the camera.

Timeout

Description: This parameter is used to set the maximum response timeout period (in seconds) from the camera receives the data to it sends the data to the client. The two types of timeout are as follows:

  • The camera fails to be connected successfully within the set time

  • The camera fails to capture any images within the set time

Default value: 20 s

Num of Reconnection Attempts

Description: This parameter is used to specify the maximum number of attempts to reconnect the camera if the camera fails to be connected within the timeout period.

Default value: 3

Camera Config Group

Description: This parameter is used to select the configuration parameter group of the camera. The camera will capture the data according to the parameters in this parameter group.

You can add/delete a parameter group in Mech-Eye Viewer.

Camera Model

Description: This parameter is only used to display the model of the currently connected camera.

2D Image Type

Description: This parameter is used to set the 2D image type output by the camera.

Value list:

  • Internal IR Image (default value): 2D images (depth source) whose reference frame is the same as that of the depth map and does not need to be rectified.

  • External Color Image: 2D images (texture) that should be rectified before use. You should select Rectify to Depth Map when 2D images of this type are output.

This parameter is only available when a DEEP (V4) or LSR (V4) series camera is used.

Rectify to Depth Map

Description: This parameter is used to rectify the reference frame of the external color image to the reference frame of the depth map.

Default value: Selected.

This parameter is only available when External Color Image is set as the 2D Image Type.

Background Removal Settings

Remove Background by Depth

Description: This parameter is used to remove the information of the background from the depth map.

Default Value: Disabled.

Depth Background Image File

Description: This parameter is used to capture or read the depth map of the background.

Default value: depth_background.png

Instructions:

  1. Click the Open an Editor button to open the Set Background window.

    If there is an available image of the background, please see step 2.

    • If there is not an available image of the background, please see step 3.

  2. Click ... and select the local image map of the background. Then, click the OK button to confirm.

  3. Select a camera in the Detected cameras panel on the right and click capture images from camera icon 2 to connect the camera.

    Once the camera is connected successfully, the depth map obtained from the camera will be displayed in the Image viewer panel on the left.

  4. If you want to update the depth map, click the Capture once or Capture live button.

  5. If you want to modify the file name of the depth map, enter an Image Name in the text box.

  6. Click OK to finish selecting the depth map of the background.

Variation of Background Depth

Description: This parameter is used to compare the depth (mm) of a point in the newly-obtained depth map and the depth of the point in the background depth map. When the difference in depths between the point and its counterpart in the background is less than this value, the point will be considered a point in the background and removed.

Default value: 10.0 mm

Max Capture Attempts

Description: This parameter is used to specify the total number of attempts to capture the image if the camera fails to capture any image within the timeout period.

Default value: 3

Recommended value: 3

Robot Service Name in Mech-Center

Description: This parameter is used to specify the robot service name, which should be the same as the name of the robot connected in Mech-Center.

Default value: Null.

Use a Virtual Camera

When the Virtual Mode is enabled, you will need to adjust the following parameters.

Camera Calibration Parameters

Instruction: Once the Data Path is selected, Camera Calibration Parameters will be automatically filled. If there are multiple sets of camera calibration parameters under the Data Path, please select the one you need in the drop-down list.

Data Path

Description: This parameter is used to select the folder where images, intrinsic parameters, and extrinsic parameters are stored.

Instruction: Click the capture images from camera icon 1 icon on the right and select the folder where images, intrinsic parameters, and extrinsic parameters are stored.

In the virtual mode, the naming conventions for the images as follows should be followed. The serial number of the color image should be the same as its depth map counterpart.

  • Color images: rgb_image_xxxx.jpg

  • Depth maps: depth_image_xxxx.png

If the camera mounting method is Eye-in-Hand, note that the serial number of images should correspond to that of flange poses.

Virtual Camera Assistant:

  • If any one of the following descriptions is true when you select the data path, a Virtual Camera Assistant will be triggered to help you select the image data.

    • You have not entered a path in the Data Path box.

    • There are multiple sets of data in the selected folder.

    • There is no color image, depth map, or intrinsic and extrinsic parameters in the selected folder.

  • Please follow the steps below to configure in Virtual Camera Assistant:

    • In the Virtual Camera Assistant window, click capture images from camera icon 4 to select the data path.

      Once the path is entered successfully, the Camera ID and Parameter group will be automatically filled.

    • Click Verify, and a Camera parameter group updated message will appear.

    • Click capture images from camera icon 4 to select the folders where color images, depth maps, and the flange pose are stored one by one. Then, click Confirm after selecting the data.

The flange pose is needed only when the camera is mounted in the eye-in-hand mode.

  • Lastly, click OK in the pop-up Setup Complete window.

In the virtual mode, please re-select the data path if there is any modification of the image in the selected data path, or else the new image data cannot be read.

Play Back Mode

Description: This parameter is used to specify the order to read the images.

Value list:

  • Sequential (default value): Read the images in the order of the images in the folder.

  • Repeat One: Read the current image repeatedly.

  • Repeat All: Read all images in the order of the images in the folder, and then read them from the beginning after all images are read.

  • Random: Read images randomly.

Tuning recommendation: Please select the mode according to image reading order you need.

Current Frame Name

Description: This parameter is used to display the serial number and timestamp of the currently read image.

Image Name Type

Description: This parameter is used to select the type of image name output by the Color Image Path port.

Value list: Complete Path, File Name, and Base Name.

Default value: Complete Path.

Background Removal Settings

Remove Background by Depth

Description: This parameter is used to remove the information of the background from the depth map.

Default Value: Disabled.

Depth Background Image File

Description: This parameter is used to capture or read the depth map of the background.

Default value: depth_background.png

Instructions:

  1. Click the Open an Editor button to open the Set Background window.

    • If there is an available image of the background, please see step 2.

    • If there is not an available image of the background, please see step 3.

  2. Click ... and select the local image map of the background. Then, click the OK button to confirm.

  3. Select a camera in the Detected cameras panel on the right and click capture images from camera icon 2 to connect the camera.

    Once the camera is connected successfully, the depth map obtained from the camera will be displayed in the Image viewer panel on the left.

  4. If you want to update the depth map, click the Capture once or Capture live button.

  5. If you want to modify the file name of the depth map, enter an Image Name in the text box.

  6. Click OK to finish selecting the depth map of the background.

Variation of Background Depth

Description: This parameter is used to compare the depth (mm) of a point in the newly-obtained depth map and the depth of the point in the background depth map. When the difference in depths between the point and its counterpart in the background is less than this value, the point will be considered a point in the background and removed.

Default value: 10.0 mm

External2D

Camera ID

Description: This parameter is used to connect the camera and select the camera calibration parameters.

Instructions:

  1. Click Select camera to open the Choose the camera and parameter group to use window.

  2. Select the camera you want to connect and click capture images from camera icon 2.

    If the camera is connected successfully, capture images from camera icon 2 will become capture images from camera icon 3.

  3. After the camera is connected successfully, you can click Select from and select the camera calibration parameter group in the drop-down list and then click OK in the lower right corner of the window.

    Once the calibration parameter group is selected successfully, the Camera ID, Camera Calibration Parameters, IP Address, and Port parameters will be automatically filled.

Camera Calibration Parameters

Description: This parameter is used to specify the calibration parameter group of the camera.

IP Address

Description: This parameter is used to display the IP address of the camera. Please do not modify.

Port

Description: This parameter is used to read the port number of the camera.

IP Address and Port are only used for reading. Modifying these two parameters will disconnect the camera.

Timeout

Description: This parameter is used to set the maximum response timeout period (in seconds) from the camera receives the data to it sends the data to the client. The two types of timeout are as follows:

  • The camera fails to be connected successfully within the set time

  • The camera fails to capture any images within the set time

Default value: 20 s

Num of Reconnection Attempts

Description: This parameter is used to specify the maximum number of attempts to reconnect the camera if the camera fails to be connected within the timeout period.

Default value: 3

Auto Exposure

Description: This parameter is used to determine whether to enable automatic exposure.

Default value: Selected.

Mean Gray Value

Description: This parameter is used to set the expected gray value of the image. The larger the value, the brighter the image. The smaller the value, the darker the image.

Default value: 100

Exposure Time

Description: This parameter is used to set the exposure time which affects the brightness of the image. The larger the value, the brighter the image. The smaller the value, the darker the image.

Default value: 10

Max Capture Attempts

Description: This parameter is used to specify the total number of attempts to capture the image if the camera fails to capture any image within the timeout period.

Default value: 3

Recommended value: 3

Robot Service Name in Mech-Center

Description: This parameter is used to specify the robot service name, which should be the same as the name of the robot connected in Mech-Center.

Default value: Null.

LMI

Camera ID

Description: This parameter is used to connect the camera and select the camera calibration parameters.

Instructions:

  1. Click Select camera to open the Choose the camera and parameter group to use window.

  2. Select the camera you want to connect and click capture images from camera icon 2.

    If the camera is connected successfully, capture images from camera icon 2 will become capture images from camera icon 3.

  3. After the camera is connected successfully, you can click Select from and select the camera calibration parameter group in the drop-down list and then click OK in the lower right corner of the window.

    Once the calibration parameter group is selected successfully, the Camera ID, Camera Calibration Parameters, IP Address, and Port parameters will be automatically filled.

Camera Calibration Parameters

Description: This parameter is used to specify the calibration parameter group of the camera.

IP Address

Description: This parameter is used to display the IP address of the camera. Please do not modify.

Max Capture Attempts

Description: This parameter is used to specify the total number of attempts to capture the image if the camera fails to capture any image within the timeout period.

Default value: 3

Recommended value: 3

Robot Service Name in Mech-Center

Description: This parameter is used to specify the robot service name, which should be the same as the name of the robot connected in Mech-Center.

Default value: Null.

Mech-EyeTOF

Please contact Mech-Mind Technical Support for assistance in using the parameters of this mode.

We Value Your Privacy

We use cookies to provide you with the best possible experience on our website. By continuing to use the site, you acknowledge that you agree to the use of cookies. If you decline, a single cookie will be used to ensure you're not tracked or remembered when you visit this website.