Generate Point Cloud Model

There are two ways for you to generate a point cloud model in Matching Model and Pick Point Editor: from images acquired by the camera, or from an imported CAD file.

---

Generate from Images Acquired by the Camera

This feature first acquires a depth map of the background and one with the target object in it, and then generates a point cloud of the target object through comparing the two depth maps.

Hint

This feature is available only when the opened project contains the Capture Images from Camera Step, AND the Camera Settings parameters of this Step are fully configured.

The procedure differs depending on whether you are using a real camera or a virtual camera. Click on the options below for detailed instructions.

Real camera

Depending on whether you have added pick points by teaching, the procedure for using a real camera to acquire images for point cloud model generation differs. Click on the option that fits your situation for detailed instructions.

Have NOT added pick point by teaching

1. Remove all target objects in the camera’s field of view and retain the background only. If the robot picks from a bin, then you need to remove all the target objects from the bin but keep the bin in place.

   

2. Click on in the Toolbar to open the Capture Point Cloud Model Window.

3. Click on Capture background without target object to acquire a background depth map, and then click on Next.

   

4. Place a target object in the camera’s field of view (in the bin).

   

5. Click on Capture target object to acquire a depth map of the target object, and then click on Next.

   

6. Click on Generate Model to generate a point cloud model of the target object.

   

   Hint

   You can check the point cloud by adjusting your view using the mouse in this page.

7. Adjust Point count threshold:

   • If the target object in the point cloud is not intact, you can increase the value here and then click on Generate Model to check the result.

   • If the point cloud contains too many points of the background, you can decrease the value here and then click on Generate Model to check the result.

8. Click on Finish to exit the window. Now you can see the newly generated point cloud model in the Visualizing Space and in the Model Files List on the left.

   

   Hint

   Double-click on the model name in the Model Files List to rename a point cloud model.

Already added pick point by teaching

1. Make sure that the target object has not been moved. Click on in the Toolbar, and then click on Next in the window that appears.

2. Click on Capture target object to acquire a depth map of the target object, and then click on Next.

   

3. Remove all target objects in the camera’s field of view and retain the background only. If the robot picks from a bin, then you need to remove all the target objects from the bin but keep the bin in place.

   

4. Click on Capture background without target object to acquire a background depth map. Then, click on Next, and click on Next AGAIN to reach the Generate Model page.

   

5. Click on Generate Model to generate a point cloud model of the target object.

   

   Hint

   You can check the point cloud by adjusting your view using the mouse in this page.

6. Adjust Point count threshold:

   • If the point cloud of the target object is incomplete, you can increase the value here, and then click on Generate Model to check the result.

   • If the point cloud contains too many points of the background, you can decrease the value here, and then click on Generate Model to check the result.

7. Click on Finish to exit the window. Now you can see the newly generated point cloud model in the Visualizing Space and in the Model Files List on the left.

   

   Hint

   Double-click on the model name in the Model Files List to rename a point cloud model.

Hint

If you reached this page through clicking on the link in Add Pick Point by Teaching, you can use the Back button on your browser to return and continue reading the instructions of Add Pick Point by Teaching.

Virtual camera

When using a virtual camera, since you will not be able to acquire images in real time, you need to get images collected previously ready for use.

Tip

The background depth map doesn’t have to contain no target objects at all; it just needs to contain one fewer target object than the depth map of the target object. For example, in the figure below, the left image can be used as the background depth map, and the right one as the depth map of the target object.

1. In the Graphical Programming Workspace, select the Capture Images from Camera Step, and adjust the following in the Parameter panel to the right:

   • For Data Path, select the parent folder of the folder containing the depth maps. For detailed instructions, please see Capture Images from Camera.

   • Set Play Mode to Repeat All.

2. Click on Toolkit ‣ Matching Model and Pick Point Editor in the Menu Bar to open the Editor, and then click on in the Toolbar to open the Capture Point Cloud Model Window.

   Note

   As saved images are usually acquired during actual picking, the n th image usually contains one more target object than the (n-1) th image. Therefore, we recommend that you read in the depth map of the target object first, and then use the next image as the background depth map.

3. Click on Next, and then click on Capture target object to read in a depth map of the target object.

   Click on Capture target object again to read in the next image in the folder. Repeat until the displayed depth map meets your needs, and then click on Previous.

   

4. Click on Capture background without target object to read in a background depth map. Then, click on Next, and click on Next AGAIN to reach the Generate Model page.

   

5. Click on Generate Model to generate a point cloud model of the target object.

   

   Hint

   You can check the point cloud by adjusting your view using the mouse in this page.

6. Adjust Point count threshold:

   • If the point cloud of the target object is incomplete, you can increase the value here, and then click on Generate Model to check the result.

   • If the point cloud contains too many points of the background, you can decrease the value here, and then click on Generate Model to check the result.

7. Click on Finish to exit the window. Now you can see the newly generated point cloud model in the Visualizing Space and in the Model Files List on the left.

   

   Hint

   Double-click on the model name in the Model Files List to rename a point cloud model.

   Due to factors like lighting variations in the images, the generated point cloud model is likely to contain unwanted points. You can trim the model during editing.

---

Generate from imported CAD files

If you already have the CAD model file of the target object, you can import this file into Matching Model and Pick Point Editor, and generate a point cloud model from the CAD file.

Hint

Currently, only STL format is supported.

Click here for detailed instructions

1. Click on in the Toolbar, locate the CAD file you need, select this file and click on Open.

2. Select the measurement unit of the CAD model in the pop-up window.

   

3. The imported CAD file will show up in the CAD Files List on the left.

   

   Hint

   Only one CAD file can be imported at a time. If you need to import multiple CAD files, please repeat steps 1 to 2.

4. Select the CAD file for point cloud model generation in the CAD Files List, and then click on or in the Toolbar to generate a point cloud model.

   • : generate a point cloud of the entire surface of the CAD model.

     After clicking, you will need to adjust the sampling interval for the point cloud in the pop-up window, and then click on OK to generate the point cloud model.

   

   • : generate a point cloud of only the part visible in the current view.

     In the figure below, the left image is the view used for point cloud generation, and the right image is the generated point cloud. You can see that the point cloud doesn’t contain points from the part that is not visible in the left image.

     

   Hint

   Occasionally, and may become disabled if you already have point cloud model(s) generated.

   In such case, please click on anywhere in the Editor interface to deselect the file in the CAD Files List, and then reselect it. The above buttons should become enabled.

---

Hint

Other than generating point cloud models through the methods described above, you can also import existing point cloud files in PLY format directly by clicking on in the Toolbar.

The following are some actions you can take once you have generated a point cloud model:

• Save all the files in the Lists: click on File ‣ Save or use the shortcut Ctrl+S.

  Hint

  Files are save to the following directory: Project Folder/resource/model_editor.

• Delete a point cloud model: right-click on the point cloud you wish to delete in the Model Files List, and select Delete in the pop-up menu.

• Toggle the visibility of a file: click on to the right of the file name will make the file invisible in the Visualizing Space. Clicking again makes the file visible.

• Show the bounding box: select a file in the Files List, and then check the box in front of Show bounding box below the CAD Files List.

• Change the display color of a point cloud: select a point cloud model in the Model Files List, double-click on the white rectangle next to Cloud color below the CAD Files List, and then change the color in the pop-up window.

  Hint

  The display color of an imported color point cloud cannot be changed.

---

Now that you have a point cloud model, you can start editing it to meet your actual needs. Please click on Next in the lower right for a detailed instruction for editing a point cloud model.