Capture Tutorial¶

This section describes how to use Mech-Eye API to capture 2D images, depth maps, and 3D point clouds, and save the data.

Requirements¶

For C++, C# and Python, please install Mech-Eye SDK first.

Also for Python, please complete the relevant configurations.

Detect Cameras¶

Once the requirements are satisfied, you can detect the connectable cameras using the following commands.

Detect Cameras (C++)

std::vector<mmind::api::MechEyeDeviceInfo> deviceInfoList = mmind::api::MechEyeDevice::enumerateMechEyeDeviceList();

Detect Cameras (C#)

List<MechEyeDeviceInfo> deviceInfoList = MechEyeDevice.enumerateMechEyeDeviceList();

Detect Cameras (Python)

self.device_list = self.device.get_device_list()

Connect to a Camera¶

After finding the connectable cameras, you can connect to a camera using the following commands.

Connect to a Camera (C++)

mmind::api::MechEyeDevice device;
device.connect(deviceInfoList[0]);

Connect to a Camera (C#)

MechEyeDevice device = new MechEyeDevice();
device.connect(deviceInfoList[0]);

Connect to a Camera (Python)

self.device.connect(self.device_list[int(0)])

Set the Camera Parameters¶

After connecting to the camera, please set the camera parameters using the following commands.

2D Scanning Parameters¶

Before capturing 2D images, you need to set the 2D scanning parameters, including Exposure Mode, Exposure Time, etc.

Set 2D Scanning Parameters (C++)
device.setScan2DExposureMode(mmind::api::Scanning2DSettings::Scan2DExposureMode::Timed);
device.setScan2DExposureTime(100);
Set 2D Scanning Parameters (C#)
device.setScan2DExposureMode(Scan2DExposureMode.Timed);
device.setScan2DExposureTime(100);
Set 2D Scanning Parameters (Python)
self.device.set2D_exposure_mode("Timed")
self.device.set2D_exposure_time(100.0)

3D Scanning Parameters¶

Before capturing 3D data, you need to set the 3D scanning parameters, including Scan Multiplier, Exposure Time**(s), **Scan 3D ROI, Cloud Smooth Mode, etc.

Set 3D Scanning Parameters (C++)

device.setScan3DExposure(std::vector<double>{5, 10});
device.setDepthRange(mmind::api::DepthRange(100, 1000));
device.setScan3DROI(mmind::api::ROI(0, 0, 500, 500));
device.setCloudSmoothMode(
    mmind::api::PointCloudProcessingSettings::CloudSmoothMode::Normal);
device.setCloudOutlierFilterMode(
    mmind::api::PointCloudProcessingSettings::CloudOutlierFilterMode::Normal);

Set 3D Scanning Parameters (C#)

device.setScan3DExposure(new List<double> {5, 10});
device.setDepthRange(new DepthRange(100, 1000));
device.setScan3DROI(new ROI(0, 0, 500, 500));
device.setCloudSmoothMode(CloudSmoothMode.Normal);
device.setCloudOutlierFilterMode(CloudOutlierFilterMode.Normal);

Set 3D Scanning Parameters (Python)

self.device.set3D_exposure([5.0, 10.0])
self.device.set_depth_range(100, 1000)
self.device.set_3D_roi(0, 0, 500, 500)
self.device.set_cloud_smooth_mode("Normal")
self.device.set_cloud_outlier_filter_mode("Normal")

Capture Images¶

After setting the parameters, you can start capturing images by sending the commands to control the camera.

Capture 2D Images¶

Capture 2D Images (C++)

mmind::api::ColorMap color;
device.captureColorMap(color);

Capture 2D Images (C#)

ColorMap colorMap = new ColorMap();
device.captureColorMap(ref color);

Capture 2D Images (Python)

color_map = device.capture_color()

Capture 3D Images¶

Capture 3D Images (C++)

mmind::api::PointXYZMap pointXYZMap;
device.capturePointXYZMap(pointXYZMap);

Capture 3D Images (C#)

PointXYZMap pointXYZMap = new PointXYZMap();
device.capturePointXYZMap(ref pointXYZMap);

Capture 3D Images (Python)

point_xyz = device.capture_point_xyz()

Save Data¶

Save 2D Images¶

You can convert captured 2D image data into OpenCV image data and save them as .png files.

Save 2D Images (C++)

const std::string colorFile = "ColorMap.png";
cv::Mat color8UC3 = cv::Mat(colorMap.height(), colorMap.width(), CV_8UC3, colorMap.data());
cv::imwrite(colorFile, color8UC3);

Save 2D Images (C#)

string colorFile = "colorMap.png";
Mat color8UC3 = new Mat(unchecked((int)colorMap.height()), unchecked((int)colorMap.width()), DepthType.Cv8U, 3, colorMap.data(), unchecked((int)colorMap.width()) * 3);
CvInvoke.Imwrite(colorFile, color8UC3);

Save 2D Images (Python)

color_file = "ColorMap.png"
cv2.imwrite(color_file, color_map.data())

Save Point Clouds¶

You can generate point clouds from the captured images and save the point clouds to .ply files.

Save Point Clouds (C++)

std::string pointCloudPath = "PointCloudXYZ.ply";
savePLY(pointXYZMap, pointCloudPath);

Save Point Clouds (C#)

Mat depth32FC3 = new Mat(unchecked((int)pointXYZMap.height()), unchecked((int)pointXYZMap.width()), DepthType.Cv32F, 3, pointXYZMap.data(), unchecked((int)pointXYZMap.width()) * 12);
string pointCloudPath = "pointCloudXYZ.ply";
CvInvoke.WriteCloud(pointCloudPath, depth32FC3);

Save Point Clouds (Python)

point_cloud_xyz = o3d.geometry.PointCloud()
points_xyz = np.zeros(
    (point_xyz.width() * point_xyz.height(), 3), dtype=np.float64)

pos = 0
for dd in np.nditer(point_xyz_data):
    points_xyz[int(pos / 3)][int(pos % 3)] = 0.001 * dd
    pos = pos + 1

point_cloud_xyz.points = o3d.utility.Vector3dVector(points_xyz)
o3d.io.write_point_cloud("PointCloudXYZ.ply", point_cloud_xyz)