> ## Documentation Index
> Fetch the complete documentation index at: https://docs.openlayer.com/llms.txt
> Use this file to discover all available pages before exploring further.

# openlayer.json

> Learn how to write the `openlayer.json` config for your project

The `openlayer.json` contains the information Openlayer needs to
validate your artifacts, run your AI system on your datasets, and evaluate your tests.

This guide shows how you can write the `openlayer.json` for your project.

<Tip>
  If you prefer, you can **pick a template from our [Template
  gallery](https://github.com/openlayer-ai/examples)** that feels closest to
  your use case and make edits to its `openlayer.json`.
</Tip>

The `openlayer.json` file has five parts:

* [taskType](#tasktype)
* [model](#model)
* [datasets](#datasets)
* [testsPath](#testspath)
* [metrics](#metrics)

## `taskType`

**Type**: `string`

The `taskType` must be one of `llm-base`,
`tabular-classification`, `tabular-regression`, and `text-classification`. It corresponds
to your Openlayer project's task type.

Example:

```json openlayer.json theme={null}
{
  "taskType": "llm-base",
  ...
}
```

This is needed so that Openlayer can validate the information provided in the `model`
and `datasets` sections.

***

## `model`

**Type**: `object`

The `model` part of the `openlayer.json` specifies the commands Openlayer will use to generate
predictions with your AI system, and metadata about it.

### Object attributes

#### `modelType`

**Type**: `string`, required

The type of model. Must be one of `shell` or `full`.

You must specify it
as `full` if you are providing a script in `batchCommand` to run your model and get its predictions.
You must specify it as `shell` if you
already computed the model predictions and are uploading model metadata
only.

#### `runtime`

**Type**: `string`

The environment runtime to execute the commands specified in `installCommand` and `batchCommand`.

This is only required if you want Openlayer to run your model to get its outputs. Refer
to the [Configuring output generation page](/development/configuring-output-generation) for more information.
Currently, the supported runtimes are:

| Runtime | Available options                                                        |
| ------- | ------------------------------------------------------------------------ |
| Python  | `python_3_13`, `python_3_12`, `python_3_11`, `python_3_10`, `python_3_8` |
| NodeJS  | `node_20`                                                                |

#### `installCommand`

**Type**: `string`

The command that gets executed before the run script. Serves the
purpose of installing the dependencies needed by your `batchCommand` script.

For more information about the `installCommand`, refer to the [Configuring output generation](/development/configuring-output-generation) guide.

Examples:

<CodeGroup>
  ```json openlayer.json using a Python runtime theme={null}
  {
    ...
    "model": {
        "installCommand": "pip install -r requirements.txt",
        ...
    }
  }
  ```

  ```json openlayer.json using a NodeJS runtime with TypeScript theme={null}
  {
    ...
    "model": {
        "installCommand": "npm i && npx tsc",
        ...
    }
  }
  ```
</CodeGroup>

#### `batchCommand`

**Type**: `string`

The command that executes your script to get your model predictions.

In general, if you are using one of [Openlayer's SDKs](/api-reference/sdk) to write your script,
it is followed by the
placeholder arguments `--dataset-path {{ path }} --dataset-name {{ name }}`.

For more information about the `batchCommand` and the placeholder arguments, refer to the [Configuring output generation](/development/configuring-output-generation) guide.

Examples:

<CodeGroup>
  ```json openlayer.json with a Python script theme={null}
  {
    ...
    "model": {
        "batchCommand": "python run.py --dataset-path {{ path }} --dataset-name {{ name }}",
        ...
    }
  }
  ```

  ```json openlayer.json with a TypeScript script theme={null}
  {
    ...
    "model": {
        "batchCommand": "node run.js --dataset-path {{ path }} --dataset-name {{ name }}",
        ...
    }
  }
  ```
</CodeGroup>

#### `outputDirectory`

**Type**: `string`, default `output`

Directory where the file with model outputs will be saved.

#### `metadata`

**Type**: `object`

Object with model metadata.

***

## `datasets`

**Type**: `array` of `Dataset objects`

The `datasets` part of the `openlayer.json` has an array of `Dataset` objects. Openlayer
will iterate over this array to get your model's outputs for each dataset.

The `Dataset` object has a set of **common attributes** and a set of attributes that
**depend on the `taskType`**.

### `Dataset` object common attributes

The common attributes must always be present, regardless of the `taskType`.

#### `name`

**Type**: `string`, required

Dataset name.

#### `label`

**Type**: `string`, required

Dataset label. Must be one of `validation` or `training`.

#### `path`

**Type**: `string`, required

Path to the dataset file.

#### `groundTruthColumnName`

**Type**: `string | null`

Name of the dataset column with the ground truths.

#### `metadata`

**Type**: `object`

Object with dataset metadata.

### `Dataset` object task-specific attributes

The additional attributes you must specify for a dataset depend on the `taskType` of your
Openlayer project.

<Tabs>
  <Tab title="llm-base">
    #### `inputVariableNames`

    **Type**: `array[string]`, required

    Array of input variable names. Each input variable should be in a dataset column.
  </Tab>

  <Tab title="tabular-classification">
    #### `categoricalFeatureNames`

    **Type**: `array[string] | []`

    Array containing the names of all categorical features in the dataset.

    For example, `[“Gender”, “Geography”]`.

    #### `classNames`

    **Type**: `array[string]`, required

    Array of class names indexed by label integer in the dataset.

    For example, `[“Retained”, “Exited”]` when class `0` is `"Retained"` and
    class `1` is `"Exited"`.

    #### `featureNames`

    **Type**: `array[string] | []`, required

    Array of all input feature names.
  </Tab>

  <Tab title="tabular-regression">
    #### `categoricalFeatureNames`

    **Type**: `array[string] | []`

    Array containing the names of all categorical features in the dataset.

    For example, `[“Gender”, “Geography”]`.

    #### `featureNames`

    **Type**: `array[string] | []`, required

    Array of all input feature names.
  </Tab>

  <Tab title="text-classification">
    #### `classNames`

    **Type**: `array[string]`, required

    Array of class names indexed by label integer in the dataset.

    For example, `[“Retained”, “Exited”]` when class `0` is `"Retained"` and
    class `1` is `"Exited"`.

    #### `textColumnName`

    **Type**: `string`, required

    Name of the column with the text.
  </Tab>
</Tabs>

***

## testsPath

**Type**: `string`

Path to a JSON file with test configurations. This field is not needed if you are
only creating tests via the UI.

Read more about test configurations on the
[tests.json guide](/development/tests-json).

Example:

```json openlayer.json theme={null}
{
  "testsPath": "tests.json",
  ...
}
```

***

## metrics

**Type**: `object`

The `metrics` part of the `openlayer.json` allows you to control the metric settings for your
project. You can control which metrics are "starred" and which are "selected" for your project,
which defines the metrics that appear on the top panel of the project and metrics that should be computed, respectively.

### Object attributes

#### `settings`

**Type**: `array` of `Setting` objects

### `Setting` object attributes

#### `key`

**Type**: `string`

Metric name. For example, `"conciseness"` or `"accuracy"`.

#### `starred`

**Type**: `bool`

Bool indicating if the metric is "starred." Starred metrics are the ones shown on the top
panel of your project.

#### `selected`

**Type**: `bool`

Bool indicating if the metric is "selected." Selected metrics are computed, which allow you
to create tests based on them. Unselected metrics are skipped.

Example:

```json openlayer.json theme={null}
{
  "metrics": {
    "settings": [
      {
        "key": "conciseness",
        "starred": true,
        "selected": true
      },
      {
        "key": "maxCost",
        "starred": false,
        "selected": true
      }
    ]
  }
  ...
}
```

***

## Examples

Below are a few examples of `openlayer.json`. For additional examples, check out our
[Template gallery](https://github.com/openlayer-ai/examples).

<Accordion title="View example openlayer.json">
  <CodeGroup>
    ```json Python theme={null}
    {
      "taskType": "llm-base",
      "model": {
        "modelType": "full",
        "runtime": "python_3_10",
        "installCommand": "pip install -r requirements.txt",
        "batchCommand": "python run.py --dataset-path {{ path }} --dataset-name {{ name }}",
        "outputDirectory": "output"
      },
      "datasets": [
        {
          "name": "validation_set_october_november",
          "label": "validation",
          "path": "dataset.json",
          "inputVariableNames": ["userQuery"],
          "groundTruthColumnName": "groundTruth"
        }
      ]
    }
    ```

    ```json TypeScript theme={null}
    {
      "taskType": "llm-base",
      "model": {
        "modelType": "full",
        "runtime": "node_20",
        "installCommand": "npm i && npx tsc",
        "batchCommand": "node run.js --dataset-path {{ path }} --dataset-name {{ name }}",
        "outputDirectory": "output"
      },
      "datasets": [
        {
          "name": "validation_set_october_november",
          "label": "validation",
          "path": "dataset.json",
          "inputVariableNames": ["userQuery"],
          "groundTruthColumnName": "groundTruth"
        }
      ]
    }
    ```
  </CodeGroup>
</Accordion>