You must set up a way to publish the requests your AI system is receiving to the Openlayer platform to use Openlayer’s monitoring mode.

This guide expands on the use of Openlayer’s SDKs to publish data to the Openlayer platform. Alternatively, you can use Openlayer’s REST API, as discussed in the Monitoring overview.

Data publishing methods

The data publishing methods are categorized as streamlined approaches and the manual approach.

The streamlined approaches exist for common AI patterns and frameworks. To use them, you need to wrap or decorate your code a certain way, and Openlayer automatically captures relevant data and metadata, such as the number of tokens, cost, latency, etc. This data is then published to the Openlayer platform.

The manual approach is system-agnostic. It is equivalent to hitting the relevant endpoint from Openlayer’s REST API but via Openlayer’s SDKs.

Streamlined approaches

There is a streamlined approach for each of the frameworks below:

Manual approach

To manually stream data to Openlayer, you must:

  1. Retrieve an existing inference pipeline:
import openlayer
from openlayer.tasks import TaskType

client = openlayer.OpenlayerClient("YOUR_API_KEY_HERE")

project = client.create_project(

inference_pipeline = project.create_inference_pipeline(
  1. Stream data:
# If you have a data stream, where `data` is a single row as a dictionary

You can also send additional information, such as the latency and the number of tokens used in the inference. Refer to the Data config guide for your task type of interest for more information about the config in the code snippet above.