To monitor your AI system in production, Openlayer needs to see the requests it is handling. Instrumenting your code with Openlayer’s SDKs is how you make that happen, and it takes just a couple of lines of code. Once done, you will be able to view traces and set up tests that run continuously on top of them.

​
How to integrate

Prerequisites:
The canonical recipe to integrate is:
1

Configure credentials

Set the following environment variables to tell the Openlayer SDKs where to upload captured the traces:
OPENLAYER_API_KEY=YOUR_OPENLAYER_API_KEY
OPENLAYER_INFERENCE_PIPELINE_ID=YOUR_OPENLAYER_INFERENCE_PIPELINE_ID
2

Instrument the code you want to trace

Annotate all the functions you want to trace with Openlayer’s SDK.
import openai
from openlayer.lib import trace, trace_openai

# Wrap the OpenAI client Openlayer's `trace_openai`
openai_client = trace_openai(openai.OpenAI())

# Decorate all the functions you want to trace
@trace()
def main(user_query: str) -> str:
    context = retrieve_context(user_query)
    answer = generate_answer(user_query, context)
    return answer

@trace()
def retrieve_context(user_query: str) -> str:
    return "Some context"

@trace()
def generate_answer(user_query: str, context: str) -> str:
    result = openai_client.chat.completions.create(
        messages=[{"role": "user", "content": user_query + " " + context}],
        model="gpt-4o"
    )
    return result.choices[0].message.content
Not using OpenAI? The steps are similar for other LLM providers and frameworks.
3

Use the instrumented code

All data that goes through the instrumented code is automatically sent to the Openlayer platform, where your tests and alerts are defined.In the example above, if we call main:
main("what is the meaning of life?")
the resulting trace is:TraceNote how the main function has two nested steps: retrieve_context, and generate_answer. The generate_answer has a chat completion call within it. The cost, number of tokens, latency, and other metadata are all captured automatically.

​
Framework integrations

In the example above, we wrapped an OpenAI client. If you are using a different provider or framework, the process is the same but the wrapper might be different. Pick your stack below for the exact snippet:

OpenAI

Azure OpenAI

Anthropic

LangChain

LangGraph

Amazon Bedrock

LiteLLM

Oracle OCI GenAI

Mistral AI

Groq

OpenAI Agents SDK

Don’t see your framework? Check out the Integrations page for more details or reach out.

​
What the instrumentation is doing

When you integrate, you are telling Openlayer two things:
  1. “Here are my AI calls.” These are handled by Openlayer SDKs integrations for frameworks like OpenAI, Anthropic, or LangChain. Leveraging them, you get automatic capture of inputs, outputs, tokens, costs, latency, model parameters, and more.
  2. “Here’s the rest of my workflow.” (Optional) Retrieval steps, ranking, filtering, post-processing — anything that’s not an LLM call. These are marked with the @trace decorator, so they show up as steps in the same trace.
Together, wrappers and decorators give you a full picture: not just model calls, but how your whole system behaves in production.