Deep Learning Model Package Management Tool

You are currently viewing the documentation for the latest version (2.2.0). To access a different version, click the "Switch version" button located in the upper-right corner of the page.

■ If you are not sure which version of the product you are currently using, please feel free to contact Mech-Mind Technical Support.

This section offers a guide on using the deep learning model package management tool and some notes worth your attention.

Introduction

The deep learning model package management tool is designed to manage all deep learning model packages in Mech-Vision. It can be used to optimize model packages exported from Mech-DLK 2.2.0 or later versions and manage the operation mode, hardware type, model efficiency, and model package status. Besides, this tool can be used to monitor the GPU memory usage of the IPC.

If deep learning Steps are used in the project (such as Deep Learning Model Package Inference or Pick Anything V2), you can import the model package to the deep learning model package management tool first and then use the models in the relative Steps. Importing the model package to the tool facilitates optimizing the model package in advance.

  • To use the Pick Anything V2 model package for deep learning inference, please ensure that you have the software license for this model package. If you do not have the deep learning authorization, please contact Mech-Mind sales to obtain the authorization.

  • Ensure that the minimum required version for the GPU driver is 526.98, and the minimum required CPU version is 6th-generation Intel Core. If the hardware cannot meet the requirement, the deep learning model package cannot be imported successfully.

Since Mech-DLK 3.0.0, model packages are available in two types: single model packages and multiple model packages. Versions from Mech-DLK 2.4.1 and Mech-DLK 3.0.0 support exporting only single-model packages and cascaded model packages (that is, serial models).

  • Single model package: There is only one deep learning model in the model package, such as an “Instance Segmentation” model.

  • Multiple model packages: A model package that contains models for multiple deep learning algorithm modules, which can be combined in serial, parallel, or serial-parallel configurations. The data flow between modules can be flexibly passed according to configuration, supporting both serial (the output of the previous model serves as the input of the next model) and parallel or serial-parallel data flow branches.

    As shown in the figure, the multi-model package contains one image classification module and several defect segmentation modules. The image classification module is connected in series with multiple defect segmentation modules, while the defect segmentation modules are connected in parallel with one another.

    multi model package example

Start the Feature

You can open the tool in the following ways:

  • After creating or opening a project, select Deep Learning  Deep Learning Model Package Management Tool in the menu bar.

  • In the graphical programming workspace of the software, click the Config wizard button on the deep learning Steps.

  • In the graphical programming workspace of the software, select the deep learning Step you need, and then click Open the editor button under the Model Manager Tool in the Parameters section.

entrance

Interface Description

The fields in this tool are described as follows:

Field Description

Available model package

The name of the imported model package.

Project name

The Mech-Vision project that uses the corresponding model package.

Model package type

The types of model packages, including single model packages (object detection, text recognition, etc.) and multiple model packages.

Operation mode

The operation mode of the model package during inference, including Sharing mode and Performance mode.

  • Sharing mode: Once this option is selected, when multiple Steps use the same model package, the inferences will be done one by one in sequence and less memory will be used.

  • Performance mode: Once this option is selected, when multiple Steps use the same model package, the inferences will be done simultaneously, and the inference speed will be relatively fast. However, more memory will be used in this mode.

Hardware type

The hardware type used for model package inference, including GPU (default), GPU (optimization), and CPU.

  • CPU: Use CPU for deep learning model inference, which will increase the inference time and reduce the recognition accuracy compared with GPU.

  • GPU (default): Once this option is selected, the model package will not be optimized according to the hardware type and the deep learning inference will not be speeded up.

  • GPU (optimization): Once this option is selected, the model package will be optimized according to the hardware type and the one-time optimization process takes about 5 to 15 minutes. When an optimized model package is used, the inference time will be reduced.

The tool determines the Hardware type option by detecting the IPC hardware type. The display rules for Hardware type option are as follows.

  • CPU: This option is shown when a computer with an Intel CPU is detected.

  • GPU (default), GPU (optimization): These options are shown when a computer with an NVIDIA discrete graphics card is detected, and the graphics card driver version is 526.98 or higher.

Model efficiency

The inference efficiency of the model package.

Model package status

The statuses of the model package, including Optimizing…​, Fail to optimize, Unloaded, and Ready for use.

  • Optimizing…​: The model package is under optimization.

  • Fail to optimize: The model package optimization failed.

  • Unloaded: The model package has not been used by any deep learning Steps.

  • Ready for use: The model package is already used by the deep learning Steps.

Operation

You can release or delete a model package.

  • Release: After clicking the Release button, the model package status changes from Loading completed to Unloaded, but the model package remains in the parameters of the deep learning Step. Re-running the Step restores it to loading completed.

  • Delete: Once the Delete button is clicked, the model package will be removed from the current solution. After deletion, Steps that depend on the model package may fail to run.

The model package cannot be released or deleted while it is being optimized. The software cannot be closed at this time either. Restart the operation after the optimization is completed.

Common Operations

Follow the steps below to learn about common operations for using the deep learning model package management tool.

Import the Deep Learning Model Package

  1. Open the deep learning model package management tool and click the Import button in the upper right corner.

  2. In the pop-up window, select the model package you want to import, and click the Open button. The model package will appear in the list.

Switch the Operation Mode

If you want to switch the Operation mode for deep learning model package inference, you can click icon 1 in the Operation mode column in the deep learning model package management tool, and select Sharing mode or Performance mode.

select operating mode

  • When the deep learning model package is Optimizing or In use(i.e., being used by a running project), the operation mode cannot be changed.

  • When the operation mode of the deep learning model package is Sharing mode, the GPU ID in the Parameters section of the deep learning Steps cannot be changed.

Switch the Hardware Type

You can change the hardware type for deep learning model package inference to GPU (default), GPU (optimization), or CPU.

Click the icon 1 button in the Hardware type column in the deep learning model package management tool, and select GPU (default), GPU (optimization), or CPU.

select hardware type

  • When the deep learning model package is Optimizing or In use (i.e., the project using the model package is running), the model package cannot be switched.

  • When the model package contains fast positioning, GPU (optimization) is not supported.

  • When the general model package for text detection or text recognition is used, GPU (default) is not supported.

Configure the Model Efficiency

The process of configuring model efficiency is as follows:

  1. Determine the deep learning model package to be configured.

  2. Click the corresponding Configure button under Model efficiency and set the Batch size and Precision in the pop-up window. The model execution efficiency is affected by batch size and precision.

    • Batch size: the number of images that will be passed through the neural network at once during inference, ranging from 1 to 128. Increasing the value will increase the model’s inference speed, but more video memory will be used. If the value is not set properly, the inference speed will be slowed down.

  • It is recommended to set the “Batch size” the same as the actual number of images that are passed through the neural network.

  • The instance segmentation models do not support configuring “Batch size”, and the default “Batch size” must be set to 1.

  • When a multi-model package is used for inference, the data dimension of the input/output ports is determined by the Batch size of the first model: the port dimension will only increase when the Batch size of the first model is greater than 1.

  • Precision (only available when the Hardware type is set to GPU (optimization)):

    • FP32: high-precision model with slow inference.

    • FP16: low-precision model with fast inference.

Troubleshooting

Fail to Import a Deep Learning Model Package

Symptom

After a deep learning model package to import was selected, the system shows the error message of “Failed to import the deep learning model package.”

Possible causes

  1. A model package with the same name has been imported.

  2. A model package with the same content has been imported.

  3. Hardware and software cannot meet the minimum requirements.

Solutions

  1. Modify the model package name or remove the imported model package with the same name.

  2. Check the content of the model package. If it is the same as the imported model package, you do not need to import it again.

  3. Ensure that the minimum required version of the graphics driver is 526.98, and the minimum required CPU version is 6th generation Intel Core.

Fail to Optimize a Deep Learning Model Package

Symptom

When optimizing a deep learning model package, an error message saying “Model package optimization failed” popped up.

Possible causes

Insufficient GPU memory.

Solutions

  • Remove the unused model packages in the tool and then re-import the model package for optimization.

  • Switch the “Operation mode” of other model packages to “Sharing mode” and then import the model package for optimization again.

Notes for Compatibilities

  • Mech-Vision 2.2.0 and above can use deep learning model packages exported by Mech-DLK 2.5.4 and above, supporting all algorithm modules in Mech-DLK, and can import and use multi-model packages for inference. However, the following compatibility issues may be encountered in actual use:

    • Fast positioning models do not support using the GPU (optimization) hardware type.

    • When importing model packages from Mech-DLK 2.5.4, the category names in the post-processing interface may display abnormally. For example, image classification models may only display a single category in the visualization configuration of the Inference Configuration Tool, and cannot display all categories properly.

  • Deep learning model packages exported by Mech-DLK 2.4.1 can be used in Mech-Vision 1.7.1. However, there may be some compatibility issues. It is recommended to use deep learning model packages exported by Mech-DLK 2.4.1 or above with Mech-Vision 1.7.2 or above.

    • Cascaded model packages cannot be used in Mech-Vision.

    • The model efficiency cannot be configured.

    • The performance of image classification may be diminished.

    • Model packages cannot be used on CPU devices.

  • If model packages optimized with Mech-Vision 1.7.1 are used in Mech-Vision 1.7.2, the execution time may be longer when the optimized model package is used in the “Deep Learning Model Package Inference” Step for the first time.

  • Please pay attention to the following issue when the Hardware type for model package inference is set to GPU (optimization).

    If the model package is not optimized in Mech-Vision 1.7.X, and it is optimized in Mech-Vision 1.8.0 or later versions, the model package cannot function properly in Mech-Vision 1.7.X. You should click Open folder in the Deep Learning Model Package Management Tool, delete the cache folder corresponding to the model package, and optimize the model package again.

    You can check the cache folder corresponding to the model package in the model_config.json file.

Is this page helpful?

You can give a feedback in any of the following ways:

We Value Your Privacy

We use cookies to provide you with the best possible experience on our website. By continuing to use the site, you acknowledge that you agree to the use of cookies. If you decline, a single cookie will be used to ensure you're not tracked or remembered when you visit this website.

  • Preference
  • I accept
  • I don't accept