Skip to main content
Tests evaluate a whole column. When your output is structured — for example, a transcription object with a summary inside it — a test on the output column measures the whole serialized object, not the key you care about. A custom column gives one key a name. It is an RFC 9535 JSONPath expression, scoped to an inference pipeline, that points into the output or an input column. Once it exists, you use its name wherever a column name goes: in tests, in filters, and in the records table. For example, a character length test on a custom column measures transcription.summary alone.

​
Prerequisites

  • An inference pipeline that is already receiving rows whose output, or one of whose input columns, holds a JSON object or array — or JSON stored as a string. Path suggestions and the preview both read the pipeline’s recent rows.
  • The Update projects permission to create and delete custom columns. See Roles and permissions. Without it, the list is read-only.

​
Create a custom column

Custom columns live under the project’s Settings → Custom columns, one list per inference pipeline. The Add custom columns entry in the records table’s options menu opens the same page for the pipeline you are viewing. The New custom column form asks for:
  • Name — what tests and filters refer to. Use letters, numbers, spaces, underscores, and hyphens, starting with a letter or underscore. Names are unique per pipeline, and a name can’t start with openlayer_ or match a column that already exists on the pipeline’s rows.
  • Source column — the column the path is evaluated against: the output (openlayer_output, the default) or one of the pipeline’s input columns.
  • Path — a JSONPath relative to the source column, starting with $. Paths seen in recent data are offered as suggestions. Paths inside stringified JSON are not suggested, so type those by hand and check them with the preview.

​
Example

Suppose each row’s output looks like this:
With Source column set to openlayer_output: Save the last one as summary_en_gb, and a test on summary_en_gb measures the English summary on every row, whichever position it appears in.

​
Path syntax

Paths follow RFC 9535 and are at most 512 characters long. The parts you are most likely to need: The path is relative to the source column, so it never repeats the column’s name.

​
Check the match rate before saving

Select Check path in the form’s Preview section to resolve the path against the pipeline’s most recent rows. The preview reports Matched X of Y recent rows and shows sample values, so you can catch a path that matches only some row shapes, or the wrong key.
A path that matches none of the recent rows is rejected when you create the column, even if you skip the preview. A column that matches nothing would produce a test that reports nothing, which looks like missing data rather than a wrong path. Fix the path against a recent row and try again. The only exception is a pipeline with no recent rows at all: there is nothing to check against, so the column can still be saved.
On rows where the path matches nothing, the custom column is empty (null).

​
When a path matches several values

A path with a wildcard, a filter, or .. can match several values on one row, but a column holds one value per row. The column’s reduction picks it: Non-string matches are compared and joined as their JSON text. The create form saves columns with first; a column with any other reduction shows it as a tag in the list’s Options column.

​
Stringified JSON

Some payloads hold JSON as a string, such as {"transcript": "{\"summary\": \"...\"}"}. With Parse stringified JSON on (the default), the path still resolves through those strings:
  • The path runs against the raw value first. If it matches, that value is used, so a field whose JSON text you want to measure as text stays text.
  • Only on a miss does Openlayer parse string values that hold a JSON object or array and retry, up to five levels deep. Strings such as "42" or "true" stay strings.
When a path resolves only this way, the preview says Resolves by parsing stringified JSON, and the column shows an unwraps JSON tag in the list.

​
Use a custom column

  • Tests — custom columns appear in the column picker of monitoring tests as soon as they are created. In a monitoring tests.json, use the custom column’s name as the column_name value.
  • Filters — you can filter tests and records on a custom column whose path is a chain of plain keys (such as $.transcription.summary) and that has Parse stringified JSON turned off. Filtering on any other custom column returns an error instead of empty results, and its filter offers no value suggestions.
  • Records — custom columns appear in the records table and in the record detail. A custom column can start out hidden in the table, for example when you saved a column layout before it existed. Enable it in the table’s column options.

​
Delete a custom column

Custom columns can’t be edited. To change one, delete it and create it again.
Tests that use a deleted custom column stop evaluating and are skipped as missing a column until a custom column with the same name exists again. Remove or repoint those tests before you delete the column.

​
Limits

  • Exports do not include custom columns.
  • Values keep the type they have in your data. A number stored as a string stays a string.