> ## Documentation Index
> Fetch the complete documentation index at: https://docs.openlayer.com/llms.txt
> Use this file to discover all available pages before exploring further.

# openlayer.json

> Learn how to write the `openlayer.json` config for your project

The `openlayer.json` contains the information Openlayer needs to
validate your artifacts, run your AI system on your datasets, and evaluate your tests.

This guide shows how you can write the `openlayer.json` for your project.

<Tip>
  If you prefer, you can **pick a template from our [Template
  gallery](https://github.com/openlayer-ai/examples)** that feels closest to
  your use case and make edits to its `openlayer.json`.
</Tip>

The `openlayer.json` file has five parts:

* [taskType](#tasktype)
* [model](#model)
* [datasets](#datasets)
* [testsPath](#testspath)
* [metrics](#metrics)

## `taskType`

**Type**: `string`

The `taskType` must be one of `llm-base`,
`tabular-classification`, `tabular-regression`, and `text-classification`. It corresponds
to your Openlayer project's task type.

Example:

```json openlayer.json theme={null}
{
  "taskType": "llm-base",
  ...
}
```

This is needed so that Openlayer can validate the information provided in the `model`
and `datasets` sections.

***

## `model`

**Type**: `object`

The `model` part of the `openlayer.json` specifies the commands Openlayer will use to generate
predictions with your AI system, and metadata about it.

### Object attributes

#### `modelType`

**Type**: `string`, required

The type of model. Must be one of `shell` or `full`.

You must specify it
as `full` if you are providing a script in `batchCommand` to run your model and get its predictions.
You must specify it as `shell` if you
already computed the model predictions and are uploading model metadata
only.

#### `runtime`

**Type**: `string`

The environment runtime to execute the commands specified in `installCommand` and `batchCommand`.

This is only required if you want Openlayer to run your model to get its outputs. Refer
to the [Configuring output generation page](/development/configuring-output-generation) for more information.
Currently, the supported runtimes are:

| Runtime | Available options                                                        |
| ------- | ------------------------------------------------------------------------ |
| Python  | `python_3_13`, `python_3_12`, `python_3_11`, `python_3_10`, `python_3_8` |
| NodeJS  | `node_20`                                                                |

#### `installCommand`

**Type**: `string`

The command that gets executed before the run script. Serves the
purpose of installing the dependencies needed by your `batchCommand` script.

For more information about the `installCommand`, refer to the [Configuring output generation](/development/configuring-output-generation) guide.

Examples:

<CodeGroup>
  ```json openlayer.json using a Python runtime theme={null}
  {
    ...
    "model": {
        "installCommand": "pip install -r requirements.txt",
        ...
    }
  }
  ```

  ```json openlayer.json using a NodeJS runtime with TypeScript theme={null}
  {
    ...
    "model": {
        "installCommand": "npm i && npx tsc",
        ...
    }
  }
  ```
</CodeGroup>

#### `batchCommand`

**Type**: `string`

The command that executes your script to get your model predictions.

In general, if you are using one of [Openlayer's SDKs](/api-reference/sdk) to write your script,
it is followed by the
placeholder arguments `--dataset-path {{ path }} --output-dir {{ outputDirectory }}/{{ name }}`.

For more information about the `batchCommand` and the placeholder arguments, refer to the [Configuring output generation](/development/configuring-output-generation) guide.

Examples:

<CodeGroup>
  ```json openlayer.json with a Python script theme={null}
  {
    ...
    "model": {
        "batchCommand": "python run.py --dataset-path {{ path }} --output-dir {{ outputDirectory }}/{{ name }}",
        ...
    }
  }
  ```

  ```json openlayer.json with a TypeScript script theme={null}
  {
    ...
    "model": {
        "batchCommand": "node run.js --dataset-path {{ path }} --output-dir {{ outputDirectory }}/{{ name }}",
        ...
    }
  }
  ```
</CodeGroup>

#### `outputDirectory`

**Type**: `string`, default `output`

Directory where the file with model outputs will be saved.

#### `metadata`

**Type**: `object`

Object with model metadata.

***

## `datasets`

**Type**: `array` of `Dataset objects`

The `datasets` part of the `openlayer.json` has an array of `Dataset` objects. Openlayer
will iterate over this array to get your model's outputs for each dataset.

The `Dataset` object has a set of **common attributes** and a set of attributes that
**depend on the `taskType`**.

### `Dataset` object common attributes

The common attributes must always be present, regardless of the `taskType`.

#### `name`

**Type**: `string`, required

Dataset name.

#### `label`

**Type**: `string`, required

Dataset label. Must be one of `validation`, `training`, or `fine-tuning`.

The non-validation label depends on the `taskType`: `llm-base` projects use `fine-tuning`
(the platform rejects `training` for LLM datasets), while tabular and text classification
projects use `training` (and reject `fine-tuning`).

#### `path`

**Type**: `string`, required

Path to the dataset file. The accepted file formats are `.csv`, `.tsv`, and `.json`
(a JSON array of row objects). Note that `.jsonl` (newline-delimited JSON) is **not**
supported — convert it to a JSON array first.

#### `metadata`

**Type**: `object`

Object with dataset metadata.

### `Dataset` object task-specific attributes

The additional attributes you must specify for a dataset depend on the `taskType` of your
Openlayer project.

<Tabs>
  <Tab title="llm-base">
    #### `inputVariableNames`

    **Type**: `array[string]`, required

    Array of input variable names. Each input variable should be in a dataset column.

    #### `outputColumnName`

    **Type**: `string`, required for `shell` models

    Name of the dataset column that holds the model's (precomputed) output for each row.

    This is required when your `model` has `"modelType": "shell"` — i.e., you have already
    computed the outputs and are uploading them rather than letting Openlayer run your model.
    Without it, any test that depends on the model output (for example, the LLM-as-a-judge
    and Ragas metrics) silently skips or errors, since Openlayer cannot find the generated
    text to evaluate.

    For `full` models, Openlayer runs your `batchCommand` and populates the output column for
    you, so you do not need to set `outputColumnName`.

    #### `groundTruthColumnName`

    **Type**: `string | null`

    Name of the dataset column with the ground truths. This attribute is specific to
    `llm-base` datasets — for classification task types, use `labelColumnName` instead.
  </Tab>

  <Tab title="tabular-classification">
    #### `categoricalFeatureNames`

    **Type**: `array[string] | []`

    Array containing the names of all categorical features in the dataset.

    For example, `[“Gender”, “Geography”]`.

    #### `classNames`

    **Type**: `array[string]`, required

    Array of class names indexed by label integer in the dataset.

    For example, `[“Retained”, “Exited”]` when class `0` is `"Retained"` and
    class `1` is `"Exited"`.

    #### `featureNames`

    **Type**: `array[string] | []`, required

    Array of all input feature names.

    #### `labelColumnName`

    **Type**: `string`, required

    Name of the dataset column with the ground-truth class label (an integer index into
    `classNames`). For classification task types, use `labelColumnName` for the ground
    truths — **not** `groundTruthColumnName`.

    #### `predictionsColumnName`

    **Type**: `string`

    Name of the dataset column with the model's predicted class label (an integer index
    into `classNames`).

    #### `predictionScoresColumnName`

    **Type**: `string`

    Name of the dataset column with the model's per-class predicted probabilities. Each row's
    value is a list of floats — one score per class, ordered to match `classNames`. For
    example, `[0.1, 0.9]` for a two-class problem.

    <Note>
      For tabular and text classification, the `featureNames`,
      `categoricalFeatureNames`, and `classNames` you set on each dataset must
      **also** be set on the top-level [`model`](#model) object. If they are present
      on the datasets but missing from the `model`, the server fails the commit.
    </Note>
  </Tab>

  <Tab title="tabular-regression">
    #### `categoricalFeatureNames`

    **Type**: `array[string] | []`

    Array containing the names of all categorical features in the dataset.

    For example, `[“Gender”, “Geography”]`.

    #### `featureNames`

    **Type**: `array[string] | []`, required

    Array of all input feature names.

    #### `targetColumnName`

    **Type**: `string`, required

    Name of the dataset column with the ground-truth (numeric) target value. For
    regression task types, use `targetColumnName` for the ground truths — **not**
    `groundTruthColumnName` (which the server rejects as an unknown field for tabular
    regression).

    #### `predictionsColumnName`

    **Type**: `string`

    Name of the dataset column with the model's predicted (numeric) value. The values must
    be **floats** (e.g. `275000.0`, not `275000`) — a column of integers is rejected at commit
    with "values … that are not floats".

    <Note>
      For tabular regression, the `featureNames` and `categoricalFeatureNames` you
      set on each dataset must **also** be set on the top-level [`model`](#model)
      object. If they are present on the datasets but missing from the `model`, the
      server fails the commit.
    </Note>
  </Tab>

  <Tab title="text-classification">
    #### `classNames`

    **Type**: `array[string]`, required

    Array of class names indexed by label integer in the dataset.

    For example, `[“Retained”, “Exited”]` when class `0` is `"Retained"` and
    class `1` is `"Exited"`.

    #### `textColumnName`

    **Type**: `string`, required

    Name of the column with the text.

    #### `labelColumnName`

    **Type**: `string`, required

    Name of the dataset column with the ground-truth class label (an integer index into
    `classNames`). For classification task types, use `labelColumnName` for the ground
    truths — **not** `groundTruthColumnName`.

    #### `predictionsColumnName`

    **Type**: `string`

    Name of the dataset column with the model's predicted class label (an integer index
    into `classNames`).

    #### `predictionScoresColumnName`

    **Type**: `string`

    Name of the dataset column with the model's per-class predicted probabilities. Each row's
    value is a list of floats — one score per class, ordered to match `classNames`. For
    example, `[0.1, 0.9]` for a two-class problem.

    <Note>
      For text classification, the `classNames` you set on each dataset must
      **also** be set on the top-level [`model`](#model) object. If it is present on
      the datasets but missing from the `model`, the server fails the commit.
    </Note>
  </Tab>
</Tabs>

***

## testsPath

**Type**: `string`

Path to a JSON file with test configurations. This field is not needed if you are
only creating tests via the UI.

Read more about test configurations on the
[tests.json guide](/development/tests-json).

Example:

```json openlayer.json theme={null}
{
  "testsPath": "tests.json",
  ...
}
```

***

## metrics

**Type**: `object`

The `metrics` part of the `openlayer.json` allows you to control the metric settings for your
project. You can control which metrics are "starred" and which are "selected" for your project,
which defines the metrics that appear on the top panel of the project and metrics that should be computed, respectively.

### Object attributes

#### `settings`

**Type**: `array` of `Setting` objects

### `Setting` object attributes

#### `key`

**Type**: `string`

Metric name. For example, `"conciseness"` or `"accuracy"`.

#### `starred`

**Type**: `bool`

Bool indicating if the metric is "starred." Starred metrics are the ones shown on the top
panel of your project.

#### `selected`

**Type**: `bool`

Bool indicating if the metric is "selected." Selected metrics are computed, which allow you
to create tests based on them. Unselected metrics are skipped.

Example:

```json openlayer.json theme={null}
{
  "metrics": {
    "settings": [
      {
        "key": "conciseness",
        "starred": true,
        "selected": true
      },
      {
        "key": "maxCost",
        "starred": false,
        "selected": true
      }
    ]
  }
  ...
}
```

***

## Examples

Below are a few examples of `openlayer.json`. For additional examples, check out our
[Template gallery](https://github.com/openlayer-ai/examples).

<Accordion title="View example openlayer.json">
  <CodeGroup>
    ```json Python theme={null}
    {
      "taskType": "llm-base",
      "model": {
        "modelType": "full",
        "runtime": "python_3_10",
        "installCommand": "pip install -r requirements.txt",
        "batchCommand": "python run.py --dataset-path {{ path }} --output-dir {{ outputDirectory }}/{{ name }}",
        "outputDirectory": "output"
      },
      "datasets": [
        {
          "name": "validation_set_october_november",
          "label": "validation",
          "path": "dataset.json",
          "inputVariableNames": ["userQuery"],
          "groundTruthColumnName": "groundTruth"
        }
      ]
    }
    ```

    ```json TypeScript theme={null}
    {
      "taskType": "llm-base",
      "model": {
        "modelType": "full",
        "runtime": "node_20",
        "installCommand": "npm i && npx tsc",
        "batchCommand": "node run.js --dataset-path {{ path }} --output-dir {{ outputDirectory }}/{{ name }}",
        "outputDirectory": "output"
      },
      "datasets": [
        {
          "name": "validation_set_october_november",
          "label": "validation",
          "path": "dataset.json",
          "inputVariableNames": ["userQuery"],
          "groundTruthColumnName": "groundTruth"
        }
      ]
    }
    ```

    ```json llm-base (shell model) theme={null}
    {
      "taskType": "llm-base",
      "model": {
        "modelType": "shell"
      },
      "datasets": [
        {
          "name": "validation_set_october_november",
          "label": "validation",
          "path": "dataset.json",
          "inputVariableNames": ["userQuery"],
          "outputColumnName": "modelOutput",
          "groundTruthColumnName": "groundTruth"
        }
      ]
    }
    ```

    ```json tabular-classification theme={null}
    {
      "taskType": "tabular-classification",
      "model": {
        "modelType": "shell",
        "featureNames": ["CreditScore", "Age", "Gender", "Geography"],
        "categoricalFeatureNames": ["Gender", "Geography"],
        "classNames": ["Retained", "Exited"]
      },
      "datasets": [
        {
          "name": "validation_set",
          "label": "validation",
          "path": "dataset.csv",
          "featureNames": ["CreditScore", "Age", "Gender", "Geography"],
          "categoricalFeatureNames": ["Gender", "Geography"],
          "classNames": ["Retained", "Exited"],
          "labelColumnName": "Exited",
          "predictionsColumnName": "prediction",
          "predictionScoresColumnName": "predictionScores"
        }
      ]
    }
    ```

    ```json tabular-regression theme={null}
    {
      "taskType": "tabular-regression",
      "model": {
        "modelType": "shell",
        "featureNames": ["sqft", "bedrooms", "bathrooms", "age_years", "lot_size"],
        "categoricalFeatureNames": []
      },
      "datasets": [
        {
          "name": "validation_set",
          "label": "validation",
          "path": "dataset.csv",
          "featureNames": ["sqft", "bedrooms", "bathrooms", "age_years", "lot_size"],
          "categoricalFeatureNames": [],
          "targetColumnName": "price",
          "predictionsColumnName": "prediction"
        }
      ]
    }
    ```
  </CodeGroup>
</Accordion>