Tests Management

Testing is a critical component in the development of AI models. It evaluates the model’s performance on unseen data, simulating real-world scenarios. This ensures the model is robust, accurate, and ready for deployment.

Why is Testing Crucial?

Testing serves several vital purposes:

• Evaluating Accuracy: Determines how precise the model’s predictions are.

• Identifying Weaknesses: Reveals areas where the model underperforms, providing insights for improvement.

• Refinement: Offers opportunities to enhance the model through algorithm adjustments or data augmentation.

Run Tests

In the Tests screen, you can perform two types of tests:

1. Live predictions

2. Tests on testing sets

Live Predictions

Live predictions allow you to quickly evaluate an AI model’s prediction on an individual image. This requires:

• A trained model

• An image for testing

To receive a prediction for a single image:

1. Click Upload to submit your image in the Live prediction tab.

2. Upon uploading, the system instantly generates a prediction, assuming your model is already trained.

3. (Optional) To enhance prediction visibility, adjust the Canvas Settings as needed.

When you toggle the Explain button, a saliency map is displayed. A saliency map visually highlights the areas within the image most influential to the model’s prediction, providing insights into how the model is interpreting different elements of the image. This tool is invaluable for understanding the model’s focus and decision-making process. Saliency maps can be viewed for each label, offering a detailed analysis of the model’s considerations for every potential outcome.

You can also perform a live prediction on an image taken with your device camera.

Tests on testing sets

Note

It’s essential that the testing set is annotated prior to the test, as this data serves as the benchmark for assessing the model’s predictive capabilities.

To initiate tests on testing sets, ensure you have:

• A designated testing set, pre-annotated to reflect the data you aim to evaluate.

• A fully trained model ready for testing.

To start the test:

1. Click Run Test in the Live prediction tab to open the configuration dialog box. Here, you can select the specific model and its version you wish to test.

2. After configuration, start the test. The system will process the testing set, which may take some time depending on the size of the data and complexity of the model.

3. Upon completion, access the test’s outcome by clicking on the result. This action reveals detailed insights into the model’s performance.

Upon completing the test, the following test attributes are displayed:

• Model: The specific model used for the test.

• Architecture: The underlying structure of the selected model.

• Testing Set Name: Identifier for the data set used.

• Creation Date: When the test or model was created.

• Number of Labels: The total categories or labels within the test.

• Number of Images/Frames: Quantity of visual data tested.

• Score of the Model: Overall performance metric of the model.

The results are organized into two segments, differentiated by the model’s performance score. These two categories are created based on a default threshold of 50% model score. Adjust this threshold using a slider to refine what is considered above or below average performance.

• Above Threshold: Media items with scores exceeding the set threshold are shown to the right.

• Below Threshold: Media items scoring below the threshold are displayed to the left.

You can also enhance your analysis through sorting options:

• Score Sorting: Order media items by their model score, either ascending or descending.

• Label Filtering: Focus on specific labels to evaluate model performance per category.