Tracing
Traces help you understand your system, particularly when it contains multiple steps, such as in RAG, LLM chains, and agents.
In the monitoring mode of an Openlayer project, you can view the traces for the live requests your AI system receives. This allows you to log the inputs, outputs, latency, and other metadata such as cost and number of tokens associated with every step of your system.
This guide shows how you can set up tracing with Openlayer’s SDKs to achieve a result similar to the one below.
If you prefer, feel free to refer to a notebook example. Our templates gallery also has complete sample projects that show how tracing works for development and monitoring.
How to set up tracing
You must use one of Openlayer’s SDKs to trace your system. After installing the SDK in your language of choice, follow the steps:
Set environment variables
Openlayer needs to know where to upload the traces to. This information is in the following environment variables:
Annotate the code you want to trace
Annotate all the functions you want to trace with Openlayer’s SDK.
The traced generate_answer function in the example above uses an OpenAI LLM.
However, tracing also works for other LLM providers. If you set up any of the
streamlined approaches described in the Publishing data
guide, it will get added to the trace as well.
Use the annotated code
All data that goes through the decorated code is automatically streamed to the Openlayer platform, where your tests and alerts are defined.
In the example above, if we call main:
the resulting trace would be:
The main function has two nested steps: retrieve_context, and
generate_answer. The generate_answer has a chat completion call within it. The cost,
number of tokens, latency, and other metadata are all computed automatically behind the
scenes.
Was this page helpful?