Publish records

Python

from openlayer import Openlayer

# Let's say we want to stream the following row, which represents a model prediction:
data = {
  "user_query": "what's the meaning of life?",
  "output": "42",
  "tokens": 7,
  "cost": 0.02,
  "timestamp": 1620000000
}

# Prepare the config for the data, which depends on your project's task type. In this
# case, we have an LLM project:
from openlayer.types.inference_pipelines import data_stream_params

config = data_stream_params.ConfigLlmData(
    input_variable_names=["user_query"],
    output_column_name="output",
    num_of_token_column_name="tokens",
    cost_column_name="cost",
    timestamp_column_name="timestamp",
    prompt=[{"role": "user", "content": "{{ user_query }}"}],
)

client = Openlayer()
data_stream_response = client.inference_pipelines.data.stream(
    inference_pipeline_id="182bd5e5-6e1a-4fe4-a799-aa6d9a6ab26e",
    rows=[data],
    config=config,
)

{
  "success": true
}

POST

inference-pipelines

{inferencePipelineId}

data-stream

Python

from openlayer import Openlayer

# Let's say we want to stream the following row, which represents a model prediction:
data = {
  "user_query": "what's the meaning of life?",
  "output": "42",
  "tokens": 7,
  "cost": 0.02,
  "timestamp": 1620000000
}

# Prepare the config for the data, which depends on your project's task type. In this
# case, we have an LLM project:
from openlayer.types.inference_pipelines import data_stream_params

config = data_stream_params.ConfigLlmData(
    input_variable_names=["user_query"],
    output_column_name="output",
    num_of_token_column_name="tokens",
    cost_column_name="cost",
    timestamp_column_name="timestamp",
    prompt=[{"role": "user", "content": "{{ user_query }}"}],
)

client = Openlayer()
data_stream_response = client.inference_pipelines.data.stream(
    inference_pipeline_id="182bd5e5-6e1a-4fe4-a799-aa6d9a6ab26e",
    rows=[data],
    config=config,
)

{
  "success": true
}

Use this endpoint to stream individual inference data points to Openlayer. If you want to upload many inferences in one go, please use the batch upload method instead.

Authorizations

Authorization

string

header

required

Bearer authentication header of the form Bearer <token>, where <token> is your workspace API key. See Find your API key for more information.

Path Parameters

inferencePipelineId

string<uuid>

required

The inference pipeline id (a UUID).

Body

application/json

rows

object[]

required

A list of inference data points with inputs and outputs

Example:

[
  {
    "user_query": "what is the meaning of life?",
    "output": "42",
    "tokens": 7,
    "cost": 0.02,
    "timestamp": 1620000000
  }
]

config

LLM · object

required

Configuration for the data stream. Depends on your Openlayer project task type.

LLM
Tabular classification
Tabular regression
Text classification

Show child attributes

config.outputColumnName

string

required

Name of the column with the model outputs.

Example:

"output"

config.numOfTokenColumnName

string | null

Name of the column with the total number of tokens.

Example:

"num_tokens"

config.contextColumnName

string

Name of the column with the context retrieved. Applies to RAG use cases. Providing the context enables RAG-specific metrics.

Example:

"context"

config.costColumnName

string

Name of the column with the cost associated with each row.

Example:

"cost"

config.groundTruthColumnName

string

Name of the column with the ground truths.

Example:

"ground_truth"

config.inferenceIdColumnName

string

Name of the column with the inference ids. This is useful if you want to update rows at a later point in time. If not provided, a unique id is generated by Openlayer.

Example:

"id"

config.inputVariableNames

string[]

Array of input variable names. Each input variable should be a dataset column.

Example:

["user_query"]

config.latencyColumnName

string

Name of the column with the latencies.

Example:

"latency"

config.metadata

object

Object with metadata.

config.prompt

object[]

Prompt for the LLM.

Show child attributes

config.prompt.role

string

Role of the prompt.

Example:

"user"

config.prompt.content

string

Content of the prompt.

Example:

"{{ user_query }}"

Example:

[
  {
    "role": "user",
    "content": "{{ user_query }}"
  }
]

config.questionColumnName

string

Name of the column with the questions. Applies to RAG use cases. Providing the question enables RAG-specific metrics.

Example:

"question"

config.timestampColumnName

string

Name of the column with the timestamps. Timestamps must be in UNIX sec format. If not provided, the upload timestamp is used.

Example:

"timestamp"

config.userIdColumnName

string | null

Name of the column with the user id.

Example:

"user_id"

config.sessionIdColumnName

string | null

Name of the column with the session id.

Example:

"session_id"

Example:

{
  "prompt": [
    {
      "role": "user",
      "content": "{{ user_query }}"
    }
  ],
  "inputVariableNames": ["user_query"],
  "outputColumnName": "output",
  "timestampColumnName": "timestamp",
  "costColumnName": "cost",
  "numOfTokenColumnName": "tokens"
}

Response

Status OK.

success

enum<boolean>

required

Available options:

true,

false

Delete data source

Update record

⌘I

Reference

SDKs

CLI

REST API

Authorizations

Path Parameters

Body

Response