Introduction to Instance Segmentation

What Instance Segmentation Does

Instance segmentation solves the problems of what is there, what it is, and where it is, i.e., whether there is a target object in a picture, what kind of object it is, and where it is in the picture.

Examples:

• If the target objects are simply cartons, an instance segmentation model will judge whether there are cartons in an image (it will not recognize any other objects). If so, it will draw the contour of each carton and output the carton label indicating the object class (the label has been created during the data labeling process); otherwise, it will not output any results.

  

  Figure 1. An instance segmentation model recognizing and labeling every carton

• If the target objects are of multiple classes, such as soap, toothbrush, shampoo, etc., an instance segmentation model will judge whether there are objects of these classes in an image. If so, it will draw the contour of each item and apply the corresponding label; otherwise, it will not output any results.

  

  Figure 2. An instance segmentation model recognizing and labeling various objects

Typical Industrial Application Scenarios of Instance Segmentation

• Palletizing/depalletizing: Objects such as cartons, turnover boxes, sacks, etc. need to be removed from a pallet and placed on another pallet or equipment, such as bag break station, conveyor belt, etc.

  

  Figure 3. Segmenting sacks in an image in a palletizing/depalletizing project

• Machine tending: Handle and grasp complex workpieces, structural parts, irregular parts, etc., in the automotive, steel, machinery, and other industries.

  

  Figure 4. Segmenting workpieces in an image in a machine tending project

• Order picking: Frequently seen picking scenarios in various e-commerce warehouses include batch picking, discrete picking, sorting, etc. Supports various objects, including inflatable packaging, transparent packaging, bottles, aluminum cans, and irregularly-shaped goods like pots and pans.

  

  Figure 5. Segmenting goods in an image in an order picking project

• Logistic parcel picking: Supports picking various commonly seen packages such as shipping bags, postal envelopes, shipping boxes, padded envelopes, etc., and a variety of irregularly-shaped packages.

  

  Figure 6. Segmenting shipping boxes and bags in an image in a logistic parcel picking project

Application Process of Instance Segmentation

Given enough image data from actual usage scenarios with object contours and classes correctly labeled, an instance segmentation model will learn how to segment the objects by itself. The application process of instance segmentation is as follows:

• Collect the Training Data: Take enough pictures of target objects with the camera.

• Label the Training Data: Label the contour and class of each object on each picture.

• Train the Model: Feed the labeled data to the instance segmentation model for training.

• Use the Model: Apply the trained model in an actual project.