tests.json
Learn how to write the tests.json config for your project
The tests.json file allows you to specify your Openlayer tests.
This allows you to fully control your project’s tests with a single configuration file that can be version-controlled, without needing to use the Openlayer UI to create and update tests.
This guide shows how you can write the tests.json for your project.
Discoverability
First, it is important to signal to Openlayer that you have a test configuration file.
You can do so in the testsPath field of your openlayer.json.
Make sure that you point to your tests.json file, as in:
Structure
The tests.json file should contain an array of Test objects, each representing
a test in your project.
The Test objects have a set of common attributes and a set of attributes that
depend on the mode (i.e., if it is a development test or monitoring test).
The best way to write a test configuration is to copy the examples from the documentation and edit them to fit your use case. Each test has a documentation page with information about it and configuration examples. They are all listed here.
Test object common attributes
The common attributes must always be present, regardless of the mode.
name
Type: string, required
The test name.
description
Type: string.
The test description.
type
Type: string, required.
The test type, which represents the test category. Must be one of integrity, consistency, or performance.
subtype
Type: string, required.
The test subtype, which identifies the test on the platform. Must be one of the valid subtypes.
thresholds
Type: array of Threshold objects, required.
The thresholds that
subpopulationFilters
Type: object.
Filters that define the subpopulation.
syncId
Type: string, required.
An id (UUID) that identifies the test.
mode
Type: string, required.
Defines to which mode the test applies to. Must be one of development or monitoring.
Test object mode-specific attributes
The additional attributes you must specify for a test depend on its mode, i.e., if it is a
development or monitoring mode test.
usesValidationDataset
Type: bool, required
Indicates if the test uses the validation dataset.
usesTrainingDataset
Type: bool, required
Indicates if the test uses the training dataset.
usesMlModel
Type: bool, required
Indicates if the test uses model.
Was this page helpful?