> ## Documentation Index
> Fetch the complete documentation index at: https://docs.openlayer.com/llms.txt
> Use this file to discover all available pages before exploring further.
# LiteLLM
> Learn how to trace and monitor LiteLLM completions with Openlayer
Openlayer integrates seamlessly with [LiteLLM](https://docs.litellm.ai/), which
provides a unified interface to call 100+ LLM APIs using the same input/output format.
LiteLLM supports providers including OpenAI, Azure OpenAI, Anthropic,
Cohere, Replicate, PaLM, and many more, making it easy to switch between
different LLM providers while maintaining consistent evaluation with Openlayer.
## Evaluating LiteLLM applications
You can set up Openlayer tests to evaluate your Lit
eLLM applications in [monitoring](/monitoring/overview) and [development](/development/overview).
### Monitoring
To use the [monitoring mode](/monitoring/overview), you must instrument your code to publish
the requests your AI system receives to the Openlayer platform.
To set it up, you must follow the steps in the code snippet below:
```python theme={null}
import litellm
# 1. Set the environment variables
import os
os.environ["OPENAI_API_KEY"] = "YOUR_OPENAI_API_KEY_HERE" # Or other provider keys
os.environ["OPENLAYER_API_KEY"] = "YOUR_OPENLAYER_API_KEY_HERE"
os.environ["OPENLAYER_INFERENCE_PIPELINE_ID"] = "YOUR_OPENLAYER_INFERENCE_PIPELINE_ID_HERE"
# 2. Enable Openlayertracing for all LiteLLM completions
from openlayer.lib import trace_litellm
trace_litellm()
# 3. Now use LiteLLM normally - tracing happens automatically
response = litellm.completion(
model="gpt-3.5-turbo",
messages=[{"role": "user", "content": "Hello, how are you?"}]
)
```
Once the code is instrumented, all your LiteLLM completions are automatically published to Openlayer,
along with metadata, such as latency, number of tokens, cost estimate, and more.
If you navigate to the "Data" page of your Openlayer data source, you can see
the traces for each request.
If the LiteLLM completions are just one of the steps of your AI system, you
can use the code snippets above together with [tracing](/monitoring/tracing).
In this case, your LiteLLM completions get added as a step of a larger trace.
Refer to the [Tracing guide](/monitoring/tracing) for details.
After your AI system requests are continuously published and logged by Openlayer, you can
[create tests](/tests/overview) that run at a regular cadence on top of them.
Refer to the [Monitoring overview](/monitoring/overview), for details on Openlayer's
monitoring mode, to the [Publishing data guide](/monitoring/publishing-data), for more
information on setting it up, or to the [Tracing guide](/monitoring/tracing), to
understand how to trace more complex systems.
### Development
In [development mode](/development/overview), Openlayer becomes a step in your CI/CD
pipeline, and your tests get automatically evaluated after being triggered by some events.
Openlayer tests often rely on your AI system's outputs on a validation dataset.
As discussed in the [Configuring output generation guide](/development/configuring-output-generation), you have two options:
1. either provide a way for Openlayer to run your AI system on your datasets, or
2. before pushing, generate the model outputs yourself and push them alongside your artifacts.
For LiteLLM applications, if you are **not** computing your system's outputs yourself,
you must provide the required **API credentials** for the LLM providers you're using.
For example, if your application uses OpenAI models through LiteLLM, you
provide an `OPENAI_API_KEY`, if it uses Anthropic models, you must provide an `ANTHROPIC_API_KEY`, and so on.
To provide the required API credentials, navigate to "**Workspace settings**" -> "**Environment variables**,"
and add the credentials as variables.
## Next steps
* Explore the [LiteLLM tracing example notebook](https://colab.research.google.com/github/openlayer-ai/openlayer-python/blob/main/examples/tracing/litellm/litellm_tracing.ipynb)
* Learn about [Openlayer's testing capabilities](/tests/overview)
* Set up [monitoring](/monitoring/overview) for your production LiteLLM applications
* Check out other [integrations](/integrations/overview) that work well with LiteLLM