Deep Learning Model Package Management Tool

This section offers a guide on using the deep learning model package management tool and some notes worth your attention.

Introduction

Deep learning model package management tool is designed to manage all deep learning model packages. It can be used to optimize single model packages or cascaded model packages exported by Mech-DLK 2.2.0 or above and manage and monitor the operation mode, hardware type, model efficiency, and model package status. Besides, this tool can be used to monitor the GPU memory usage of the IPC.

If deep learning Steps are used in the project, you can import the model package to the deep learning model package management tool first and then use the models in the relative Steps. Importing the model package to the tool facilitates optimizing the model package in advance.

Note

From Mech-DLK 2.4.1, model packages can be divided into single model packages and cascaded model packages.

• Single model package: There is only one deep learning model in the model package, such as an “Instance Segmentation” model.

• Cascaded model package: Multiple models are cascaded in the model package, and the output result of the previous model is input to the next model. For example, there are two models, “Object Detection” and “Instance Segmentation” in the model package and the inference sequence is Object Detection ‣ Instance Segmentation. The output of “Object Detection” is input to the “Instance Segmentation” model.

Interface Introduction

You can open the tool in either of the following two ways:

• Select Deep Learning ‣ Deep Learning Model Package Management Tool in the menu bar.

• In an opened project, select the Deep Learning Model Package Inference Step, and click the Model Package Management Tool button in the Step Parameters panel.

The options in the deep learning model package management tool are shown in the table below.

Option	Description
Available model package	The names of imported model packages
Project name	The Mech-Vision project in which the corresponding model package is used
Model package type	The type of the model package, such as “Object Detection” (single model package) and “Object Detection + Defect Segmentation” (cascaded model package)
Operation mode	The operation mode of the model package, including “Sharing mode” and “Performance mode”
Hardware type	The hardware type used for model package inference. If you are using a GPU model, you can modify the hardware type, i.e., GPU (default) and GPU (optimization)
Model efficiency	The inference efficiency of the model package
Model package status	The status of the model package, such as “Loading and optimizing”, “Loading completed”, and “Optimization failed”

Note

Operation mode

• Sharing mode: Once this option is selected, when multiple Steps use the same model package, the inferences will be done one by one in sequence and less memory will be used.

• Performance mode: Once this option is selected, when multiple Steps use the same model package, the inferences will be done simultaneously, and the inference speed will be relatively fast. However, more memory will be used in this mode.

Hardware type

• CPU: Use CPU for model package inference. Compared to GPU, the inference time is longer and the recognition accuracy is lower.

• GPU (default): Once this option is selected, the model package will not be optimized according to the hardware type and the deep learning inference will not be speeded up.

• GPU (optimization): Once this option is selected, the model package will be optimized according to the hardware type and the one-time optimization process takes about 5 to 15 minutes. When an optimized model package is used, the inference time will be reduced.

Usage

Follow the steps below to learn about common procedures for using the deep learning model package management tool.

Import the Deep Learning Model Package

1. Open the deep learning model package management tool, and click Import in the upper left corner.

2. Select the model package you want to import in the pop-up Select File window and click Open. Then the deep learning model package will appear in the list in the window, which suggests that the model package is imported successfully.

Attention

To import a model package successfully, the minimum version requirement for the graphics driver is 472.50, and the minimum requirement for the CPU is a 6th-generation Intel Core processor. It is not recommended to use a graphics driver above version 500, which may cause fluctuations in the execution time of deep learning Steps.

Select the Deep Learning Model Package in the Step

After importing the model package into the tool, if you want to use the model package in the Deep Learning Model Package Inference Step, you can select the model package in the drop-down list of the Model Package Name parameter in the Step Parameters panel.

Remove the Imported Deep Learning Model Package

If you want to remove an imported model package, select the model package first, and click the Remove button in the upper right corner.

Attention

When the deep learning model package is Loading and optimizing or the project using the deep learning model package is running, the model package cannot be removed.

Switch the Operation Mode

If you want to switch the Operation mode for deep learning model package inference, you can click in the Operation mode column in the deep learning model package management tool, and select Sharing mode or Performance mode.

Attention

• When the model package is Loading and optimizing or the project using the model package is running, the Operation mode cannot be changed.

• When the operation mode of the model package is Sharing mode, the GPU ID in the Deep Learning Model Package Inference Step cannot be changed.

Switch the Hardware Type

You can change the hardware type for deep learning model package inference to GPU (default), GPU (optimization), or CPU.

Click in the Hardware type column in the deep learning model package management tool, and select GPU (default), GPU (optimization), or CPU.

Attention

• Only model packages exported by Mech-DLK 2.4.1 or above support switching between CPU and GPU.

• You cannot use the GPU (default) for either of the following types of model packages.

  • Instance segmentation model package and instance segmentation super model package exported by Mech-DLK 2.2.0

  • Model package that only contains the DLKMT file exported by Mech-DLK

• When the model package is Loading and optimizing or the project using the model package is running, the Hardware type cannot be changed.

Configure the Model Efficiency

When a model package exported by Mech-DLK 2.4.1 or above is used, the model efficiency can be configured. The procedures for configuring the model efficiency are as follows.

1. Import the deep learning model package.

2. Click the Configure button in the Model efficiency column, a Model Efficiency configuration window will pop up, and you can configure the “Batch size” and “Precision”.

   The “Batch size” and “Precision” parameters can affect the model efficiency configuration.

   • Batch size: the number of images that will be passed through the neural network at once during inference, ranging from 1 to 128. Increasing the value will increase the model’s inference speed, but more video memory will be used. If the value is not set properly, the inference speed will be slowed down. The instance segmentation models do not support configuring “Batch size”, and the default “Batch size” must be set to 1.

   • Precision (Only available when the “Hardware type” is “GPU optimization”): FP32: high-precision model with lower inference speed. FP16: low-precision model with higher inference speed.

Hint

It is recommended to set the “Batch size” the same as the actual number of images that are passed through the neural network.

If the set “Batch size” is much larger than the actual number of images that are passed through the neural network, part of the resources will be wasted, resulting in slower inference.

For example, if the number of images is 26, and the “Batch size” is set to 20, two separate inferences will be performed. In the first inference, 20 images will be fed to the neural network, while in the second inference, 6 images will be fed to the neural network. For the second inference, the set “Batch size” is much larger than the actual number of images that are passed through the neural network, part of the resources will be wasted, resulting in slower inference. Therefore, please set the “Batch size” to a proper value to ensure the efficient use of the resources.

Notes

Fail to Import a Deep Learning Model Package

• When there is already an imported model package, and you want to import another one with the same name, please modify the name of the new model package or remove the model package with the same name, and then start to import.

  

• When you have imported a model package, and start to import another one with a different name but the same content, an error message “Failed to import the deep learning model package.” will pop up.

  

• When the software and hardware cannot meet the minimum requirement, the model package cannot be imported successfully. The minimum required version for the graphics driver is 472.50, and the minimum required CPU version is 6th generation Intel Core. If the version of the graphics driver is above 500, fluctuations may occur in the inference time.

Fail to Optimize a Deep Learning Model Package

When there is not enough memory, the model package may not be able to be optimized successfully. A pop-up window as shown below will appear. You can fix the problem according to the solutions in the window.

Attention

For cascaded model packages, if the optimization of any of the models fails, the optimization of the entire model package will fail. Then you will need to re-optimize the model package.

Notes for Compatibilities

• Deep learning model packages exported by Mech-DLK 2.4.1 can be used in Mech-Vision 1.7.1. However, there may be some compatibility issues. It is recommended to use deep learning model packages exported by Mech-DLK 2.4.1 or above with Mech-Vision 1.7.2 or above.

  • Cascaded model packages cannot be used in Mech-Vision.

  • The model efficiency cannot be configured.

  • The performance of image classification may be diminished.

  • Model packages cannot be used on CPU devices.

• If model packages optimized with Mech-Vision 1.7.1 are used in Mech-Vision 1.7.2, the execution time may be longer when the optimized model package is used in the “Deep Learning Model Package Inference” Step for the first time.