module.exports = {
    extends: ["plugin:prettier/recommended"],
    plugins: ["mdx", "prettier"],
    rules: {
        "prettier/prettier": "error",
    },
    overrides: [
        {
            files: ["*.mdx"], // Target MDX files
            processor: "mdx/remark",
        },
        {
            files: ["**/*.md?(x)"],
            extends: ["plugin:mdx/recommended"],
        },
    ],
    ignorePatterns: ["api-reference/rest/*"],
};


A collection of answers to frequently asked questions.

Overview

Openlayer

Introduction

Quickstart

Learn how to create and load projects in Openlayer

Create projects

Learn how to create API keys to interact with Openlayer

Create API keys

Explore the tests available on the platform

Tests overview

Understanding tests

Learn how to create custom metrics for your project

Custom metrics

Learn how you can use Development mode with Git, the Openlayer CLI, or the Openlayer REST API

Development mode overview

Learn how to write the `openlayer.json` config for your project

openlayer.json

Learn how to write the `tests.json` config for your project

tests.json

Learn how to configure output generation for your model

Configuring output generation

Learn how to use Openlayer with GitHub Actions

GitHub Actions with Openlayer

Learn how to push and poll your artifacts using the Openlayer CLI

Push and poll using the Openlayer CLI

Learn how to push and poll your artifacts using the Openlayer REST API

Push and poll using the Openlayer REST API

Learn how to monitor your AI system with Openlayer

Monitoring mode overview

Learn how to publish data to the Openlayer platform

Publishing data

Learn how to trace your system with Openlayer

Tracing

Annotating data

Learn about evaluation and delay windows in Openlayer

Evaluation and delay windows

Learn how to upload a reference dataset to your inference pipeline on Openlayer

Uploading a reference dataset

Learn how to update data previously published to the Openlayer platform

Updating data

Learn how Openlayer estimates the costs associated with your LLM calls

LLM cost estimation

Glossary

Learn how to set up the Openlayer MCP server.

Openlayer MCP

Learn how to interact with Openlayer programmatically

Use the Openlayer SDKs to interact with Openlayer

Use the Openlayer CLI to interact with Openlayer

Learn about the global options available for the Openlayer CLI

CLI global options

Use the Openlayer REST API to interact with Openlayer

Integrate Openlayer with your favorite tools and platforms

Learn how to evaluate OpenAI LLMs with Openlayer

OpenAI & Azure OpenAI

Learn how Openlayer integrates with Anthropic

Anthropic

Learn how to evaluate LangChain applications with Openlayer

LangChain

Learn how to export Semantic Kernel traces to Openlayer

Semantic Kernel

Learn how to export Spring AI traces to Openlayer

Spring AI

Learn how to evaluate Mistral AI LLMs with Openlayer

Mistral AI

Learn how to evaluate Groq LLMs with Openlayer

Groq

Learn how to upload models deployed in Amazon SageMaker to Openlayer

Amazon SageMaker

Learn how to export data from Openlayer and import it into Label Studio

Label Studio

Learn how to use Great Expectations with Openlayer

Great Expectations

Learn how to use datasets stored in Amazon S3 with Openlayer

Amazon S3

Learn how to use datasets available in a Databricks IPython kernel with Openlayer

Databricks

Learn how to export OpenTelemetry data to Openlayer

OpenTelemetry

Learn how to export OpenLLMetry traces to Openlayer

OpenLLMetry

Learn how to export OpenLIT traces to Openlayer

OpenLIT

Learn how to trace Pydantic AI agents with Openlayer

Pydantic AI

Learn how to connect Openlayer to Slack to receive notifications

Slack

Learn how to connect your GitHub repository to your Openlayer project

GitHub

Learn how to evaluate LLM applications with Ragas

Ragas

Learn about the latest updates to Openlayer

Product updates

Reference

Integrations

Changelog

Get started

Community

Log in

Learn how to use the character length test

Character length

Learn how to use the class imbalance ratio test

Class imbalance ratio

Column average

Learn how to use the conflicting labels test

Conflicting labels

Learn how to use the column contains string test

Column contains string

Learn how to use the correlated features test

Correlated features

Learn how to use the data type validation test

Data type validation

Duplicate rows

Empty features

Learn how to use the empty feature count test

Empty feature count

Learn how to use the features missing values test

Features missing values

Feature values

Great expectations

Learn how to use the ill-formed rows test

Ill-formed rows

Is code

Is JSON

Null rows

Number of rows

Learn how to use the personal identifiable information (PII) test

Personal identifiable information (PII)

Learn how to use the predictive power score (PPS) test

Predictive power score (PPS)

Learn how to use the quasi-constant features test

Quasi-constant features

Learn how to use the quasi-constant feature count test

Quasi-constant feature count

Learn how to use the special characters ratio test

Special characters ratio

Learn how to use the string validation test

String validation

Valid URLs

Learn how to use the column drift test to detect drift in your data

Column drift

Learn how to use the column values match test

Column values match

Learn how to use the feature drift test to detect drift in your data

Feature drift

Learn how to use the label/target drift test to detect drift in your data

Label/target drift

Learn how to use the new categories test to detect new categories in your data

New categories

Learn how to use the new labels test to detect new labels in your data

New labels

Size ratio

Learn how to use the training-validation leakage test

Training-validation leakage

Learn how to use aggregate metrics to evaluate your model

Aggregate metrics

LLM evaluation

Max cost

Max latency

Max tokens

Mean cost

Mean latency

Mean tokens

Total cost

Total tokens

Learn more about our data security practices and compliance measures.

Security

Set up enterprise-grade SAML Single Sign-On (SSO) for secure authentication in Openlayer with step-by-step instructions for Okta, Azure AD, and Google Workspace

SAML SSO Authentication

See examples of templates with common AI patterns ready for Openlayer

Templates

See how to use Openlayer's monitoring mode

Monitoring

Use the Openlayer Python SDK to interact with the Openlayer platform

Python

Use the Openlayer TypeScript SDK to interact with the Openlayer platform

TypeScript

Use the Openlayer Go SDK to interact with the Openlayer platform

Use the Openlayer Java SDK to interact with the Openlayer platform

Java

Run your model on all the datasets referenced in the openlayer.json file

batch

Download monitoring data from an inference pipeline

export

Get help information about the Openlayer CLI

help

Retrieve information about a commit on Openlayer

inspect

install

Link your local directory to an Openlayer project

link

Log into your Openlayer account through Openlayer CLI

login

Push, pull, and run your custom metrics through Openlayer CLI

metrics

Push the current local directory to the Openlayer project

push

Run a series of validations on your local directory

validate

Display the user information of the user currently logged into the Openlayer CLI

Get started

Set up tests

Test your system offline

Monitor your live system

Other resources

Overview

How can I use GitHub Actions with Openlayer?

How can I use the Openlayer CLI for the development mode workflow?

How can I use the Openlayer REST API for the development mode workflow?