Model Training and Optimization#

You can configure and retrain your trained models in the Models screen after entering your project. On the Models’ landing page, you will see all your active models.

Models’ Home Screen#

In the home screen of Models, you can reconfigure parameters for active model(s) by pressing Reconfigure and perform transfer learning (training a new model from the base model) by pressing Train a new model. Each mentioned button invokes a dialog box with the settings.

For anomaly projects, you will be presented with two accuracy scores:

Classification accuracy - this score pertains to accuracy on images i.e. it detects whether there is any anomaly in images
Localization accuracy - this score pertains to accuracy on objects i.e. it detects what regions in images contain anomalous objects

Reconfigure active model#

On the left hand side of the dialog box, you will find 2 options: Global and project-specific, whose name depends on a project type (e.g. Classification task, Detection task etc.). In both of these options, you will find a number of parameters you can tinker with. For instance:

Batch size - The number of training samples (e.g. annotated images) seen in each iteration of training. Increasing this value will reduce the computation time. However, a larger batch size has higher memory requirements.
Learning rate - The size of step at each iteration while reaching a minimum of a loss function. This parameter controls how fast the model adapts to the problem. Increasing this value may speed up training but may also lead to a suboptimal solution.
Maximum number of training epochs - The number of times an algorithm goes through the entire dataset. Increasing this value causes the results to be more robust but the training time becomes longer.

Image tiling#

Image tiling is available in detection and instance segmentation projects, where you can enable this feature in the configurable parameters accessible by clicking on Reconfigure active model.

Image tiling is a technique used to divide a large digital image into multiple smaller images or so called tiles. Tiling is most effective when dealing with large-scale, high-resolution images, such as satellite imagery and aerial photographs, which often contain minuscule objects of interest.

Train a new model#

In this dialog box, you will see model templates for creating a new model with the focus on accuracy, speed, balance, or custom configuration.

In the Recommended tab:

Accuracy option - produces a more accurate, but slower and larger model
Balance option - produces a model in balance with accuracy and speed
Speed option - produces a faster and smaller, but less accurate model

In the All tab, you can see all available model architectures for training a new model.

When you check Use the latest model configuration box, it will allow you to utilize the most recent model’s configurable parameters for the training. If there hasn’t been any model trained for this architecture previously, the system will default to using its pre-set configurable parameters.

The Set configurations manually box, on the other hand, is an option for those seeking to input custom training parameters, but this is currently only supported for the existing pre-defined architectures. When setting configurations manually, you can also customize the new model by clicking on the Train from scratch box, which ignores the training history and retrains the model from the beginning.

Note

If you would like to train a custom model with exported data from the Intel® Geti™ platform, see this example leveraging PyTorch Lightning.

View model’s version history#

You can also see the versions of your active model by clicking on the show more icon to see the accuracy and the creation date of each model version.

Note

Sometimes, you may notice that the system accepts a model with a lower accuracy than the previous model version. You may think that the selected model with the lower accuracy will perform worse. That is not the case. When the next version of a model is made the platform computes the performance of the new version based on the current dataset and also computes the performance of the old model on that new dataset.

For instance:

Model 1 is trained on simple images and it shows performance of 100%.
After training, active learning suggested more complicated images to annotate.
Model 2 is trained on simple and more complicated images.
The system collects the performances of model 1 and model 2 on the more complicated dataset.
Let’s assume, model 2 yields 90% while model 1 yields 60%. The system will accept model 2 as the latest model.

The model screen will show the history of the models with their respective dataset. In this example, the screen will show that model 1 has 100% performance and model 2 has 90% performance. Rest assured that the latest model performs better based on the latest dataset.

Change active model#

If you have trained a couple of models with different architectures, you can change the active model between these architectures. When you change the active model, the model parameters configuration will be passed from a previously active model to a newly set active model.

If you have not trained any model yet, you can change the configuration that will produce the first model in the Models screen - click on Train model to do so. The Balance (SSD) model is set by default as active model before any training starts.

Models’ Details Screen#

You can view more options for each of your model by simply clicking on the model of your choice. You will see the details for the model version you selected but you can switch between versions any time by clicking on the show more icon in the Model breadcrumbs at the top of the screen.

Upon clicking on the model, you will land in the Optimized models tab from which you can select other tabs:

Model variants - where you can view all your models
Model metrics - where you can view the statistics for your model
Parameters - where you can view hyperparameters your model was trained with
Labels - where you can view labels your model contains

Model variants#

You will see your active model with the following information:

baseline model - the name of the model with which the Intel® Geti™ platform started training
precision - a computer number format that occupies space in computer memory
accuracy - the score of the model (measured in %) that indicates a ratio of correctly predicted observation to the total observations
size - the amount of storage capacity the model consumes
download button - allows you to export the trained model

Below your baseline model, you will see your optimized models. If you do not see any optimized model, it means you have not optimized any model yet. You can optimize a model by clicking on Start optimization.

Start optimization option#

When you click on Start optimization, you will be able to convert your model to OpenVINO. Additionally, you can set the optimization to include:

POT (Post-Training Optimization Tool)
NNCF (Neural Network Compression Framework)

Training dataset#

In the Training dataset tab, you can see which images or video frames are used as part of a given subset. You can see the file information by hovering over the thumbnails and view the images or frames by clicking on the thumbnails.

The training dataset is split into three parts (subsets):

Training subset - the system will train on these items
Validation subset - used during training on which performance and optimization is validated
Testing subset - this subset is used to validate performance after training

By default, the split between these subsets depends on the dataset size. If there are less than 10 images, the split is (1/3, 1/3, 1/3). If there are between 10 and 40 images, the split is (1/2, 1/4, 1/4). If there are over 40 images, the split is (8/10, 1/10, 1/10). Most of the measures in the UI state on which subset they are computed (validation or test subset). In general, the score on a validation subset is higher since that is used to optimize the model.

Important

After training a new model, the subset distribution remains unchanged even if manual modifications are made to the subset distribution in the configuration screen. To ensure that the modified distribution takes effect, it is necessary to adjust the distribution sets prior to training any model.

Model metrics#

You can export all the metrics graphs in a PDF format or individually selected metrics graphs in a PDF or CSV format. To download all metrics graphs, click on Download all graphs in the top right-hand corner of the Model metrics panel. To download an individual graph, click on in the in the top right-hand corner of the graph box.

Configurable parameters#

You will find the same options as in Reconfigure in the home screen of Models with the exception of these parameters to be applied to one specific model you selected.