tests.json file allows you to specify your Openlayer tests.
This allows you to fully control your project’s tests with a single configuration file that
can be version-controlled, without needing to use the Openlayer UI to create and update tests.
This guide shows how you can write the tests.json for your project.
Discoverability
First, it is important to signal to Openlayer that you have a test configuration file. You can do so in the testsPath field of your openlayer.json. Make sure that you point to yourtests.json file, as in:
openlayer.json
Structure
Thetests.json file should contain an array of Test objects, each representing
a test in your project.
The Test objects have a set of common attributes and a set of attributes that
depend on the mode (i.e., if it is a development test or monitoring test).
The best way to write a test configuration is to copy the examples from
the documentation and edit them to fit your use case. Each
test has a documentation page with information about it and configuration
examples. They are all listed here.
Test object common attributes
The common attributes must always be present, regardless of the mode.
name
Type: string, required
The test name.
description
Type: string.
The test description.
type
Type: string, required.
The test type, which represents the test category. Must be one of integrity, consistency, or performance.
subtype
Type: string, required.
The test subtype, which identifies the test on the platform. Must be one of the valid subtypes.
thresholds
Type: array of Threshold objects, required.
The thresholds that
Threshold object attributes
Threshold object attributes
insightName
Type: string, required.Name of the insight from which the test is based. Must be one of the valid insight names.measurement
Type: string, required.Key from the insightName on top of which the threshold will be applied. Must be one of the valid measurement names.operator
Type: string, required.Operator used to compare the measurement and value. Must be one of is, >, >=, <, <=.value
Type: number | string, required.Threshold value.insightParameters
Type: object.Parameters needed to compute the insight. Might be null depending on the insight.subpopulationFilters
Type: object.
Filters that define the subpopulation.
syncId
Type: string, required.
An id (UUID) that identifies the test.
mode
Type: string, required.
Defines to which mode the test applies to. Must be one of development or monitoring.
Test object mode-specific attributes
The additional attributes you must specify for a test depend on its mode, i.e., if it is a
development or monitoring mode test.