The tests.json file allows you to specify your Openlayer tests. This allows you to fully control your project’s tests with a single configuration file that can be version-controlled, without needing to use the Openlayer UI to create and update tests. This guide shows how you can write the tests.json for your project.

​
Discoverability

First, it is important to signal to Openlayer that you have a test configuration file. You can do so in the testsPath field of your openlayer.json. Make sure that you point to your tests.json file, as in:
openlayer.json
{
  "testsPath": "tests.json",
  ...
}

​
Structure

The tests.json file should contain an array of Test objects, each representing a test in your project. The Test objects have a set of common attributes and a set of attributes that depend on the mode (i.e., if it is a development test or monitoring test).
The best way to write a test configuration is to copy the examples from the documentation and edit them to fit your use case. Each test has a documentation page with information about it and configuration examples. They are all listed here.

​
Test object common attributes

The common attributes must always be present, regardless of the mode.

​
name

Type: string, required The test name.

​
description

Type: string. The test description.

​
type

Type: string, required. The test type, which represents the test category. Must be one of integrity, consistency, or performance.

​
subtype

Type: string, required. The test subtype, which identifies the test on the platform. Must be one of the valid subtypes.

​
thresholds

Type: array of Threshold objects, required. The thresholds that

​
subpopulationFilters

Type: object. Filters that define the subpopulation.

​
syncId

Type: string, required. An id (UUID) that identifies the test.

​
mode

Type: string, required. Defines to which mode the test applies to. Must be one of development or monitoring.

​
Test object mode-specific attributes

The additional attributes you must specify for a test depend on its mode, i.e., if it is a development or monitoring mode test.

​
usesValidationDataset

Type: bool, requiredIndicates if the test uses the validation dataset.

​
usesTrainingDataset

Type: bool, requiredIndicates if the test uses the training dataset.

​
usesMlModel

Type: bool, requiredIndicates if the test uses model.