> ## Documentation Index
> Fetch the complete documentation index at: https://docs.openlayer.com/llms.txt
> Use this file to discover all available pages before exploring further.
# OpenAI & Azure OpenAI
> Learn how to evaluate OpenAI LLMs with Openlayer
Openlayer integrates with [OpenAI](https://openai.com/) in two different ways:
* If you are building an AI system with OpenAI LLMs and want to evaluate it,
you can use the [SDKs](/api-reference/sdk/overview) to make Openlayer part of your
workflow.
* Some tests on Openlayer are based on a score produced by an LLM judge. You can
set any of OpenAI's LLMs as the LLM judge for these tests.
This integration guide explores each of these paths.
Using **OpenAI Agents SDK**? Check out the [OpenAI Agents SDK
integration](/integrations/openai-agents-sdk) page.
## Evaluating OpenAI LLMs
You can set up Openlayer tests to evaluate your OpenAI LLMs
in [monitoring](/monitoring/overview) and [development](/development/overview).
### Monitoring
To use the [monitoring mode](/monitoring/overview), you must instrument your code to publish
the requests your AI system receives to the Openlayer platform.
To set it up, you must follow the steps in the code snippet below:
```python Python theme={null}
# 1. Set the environment variables
import os
import openai
os.environ["OPENAI_API_KEY"] = "YOUR_OPENAI_API_KEY_HERE"
os.environ["OPENLAYER_API_KEY"] = "YOUR_OPENLAYER_API_KEY_HERE"
os.environ["OPENLAYER_INFERENCE_PIPELINE_ID"] = "YOUR_OPENLAYER_INFERENCE_PIPELINE_ID_HERE"
# 2. Import the `trace_openai` function and wrap the OpenAI client with it
from openlayer.lib import trace_openai
openai_client = trace_openai(openai.OpenAI())
# 3. From now on, every chat completion/completion call with
# the `openai_client` is traced and published to Openlayer. E.g.,
completion = openai_client.chat.completions.create(
model="gpt-3.5-turbo",
messages=[
{"role": "user", "content": "How are you doing today?"},
]
)
```
```javascript TypeScript theme={null}
import { OpenAIMonitor } from "openlayer";
const monitor = new OpenAIMonitor({
openAiApiKey: 'YOUR_OPENAI_API_KEY',
openlayerApiKey: 'YOUR_OPENLAYER_API_KEY',
// EITHER specify an existing inference pipeline ID
openlayerInferencePipelineId: 'YOUR_OPENLAYER_INFERENCE_PIPELINE_ID',
// OR the project and inference pipeline names to create or load one
openlayerInferencePipelineName: 'production',
openlayerProjectName: 'YOUR_OPENLAYER_PROJECT_NAME',
});
await monitor.initialize();
// From now on, every time that you call `monitor.createCompletion` or
//`monitor.createChatCompletion`, the data is automatically published to Openlayer.
```
}
iconType="duotone"
href="https://github.com/openlayer-ai/openlayer-ts/blob/main/examples/openai-monitor.mjs"
/>
For Azure OpenAI, check out [this code
example](https://colab.research.google.com/github/openlayer-ai/openlayer-python/blob/main/examples/tracing/azure-openai/azure_openai_tracing.ipynb)
instead.
Once the code is instrumented, all your OpenAI calls are automatically published to Openlayer,
along with metadata, such as latency, number of tokens, cost estimate, and more.
If you navigate to the "Data" page of your Openlayer data source, you can see
the traces for each request.
If the OpenAI LLM call is just one of the steps of your AI system, you can use
the code snippets above together with [tracing](/monitoring/tracing). In this
case, your OpenAI LLM calls get added as a step of a larger trace.
After your AI system requests are continuously published and logged by Openlayer, you can
[create tests](/tests/overview) that run at a regular cadence on top of them.
Refer to the [Monitoring overview](/monitoring/overview), for details on Openlayer's
monitoring mode, to the [Publishing data guide](/monitoring/publishing-data), for more
information on setting it up, or to the [Tracing guide](/monitoring/tracing), to
understand how to trace more complex systems.
### Development
In [development mode](/development/overview), Openlayer becomes a step in
your CI/CD pipeline, and your tests get automatically evaluated after being triggered
by some events.
Openlayer tests often rely on your AI system's outputs on a validation
dataset. As discussed in the
[Configuring output generation guide](/development/configuring-output-generation),
you have two options:
1. either provide a way for Openlayer to run your AI system on your datasets, or
2. before pushing, generate the model outputs yourself and push them alongside your
artifacts.
For AI systems built with OpenAI LLMs, if you are **not** computing
your system's outputs yourself, you must provide your **API credentials**.
To do so, navigate to "**Workspace settings**" -> "**Environment variables**," and add the `OPENAI_API_KEY`
variable.
For Azure OpenAI, add the `AZURE_OPENAI_API_KEY`, and `AZURE_OPENAI_ENDPOINT`
secrets instead.
If you don't add the required OpenAI API key, you'll encounter a "Missing API key"
error when Openlayer tries to run your AI system to get its outputs.
You can use one of the **OpenAI templates** to check out how a sample project
fully set up with Openlayer looks like. We have templates in
[Python](https://github.com/openlayer-ai/templates/tree/main/python/llms/openai-chatbot),
and
[TypeScript](https://github.com/openlayer-ai/templates/tree/main/typescript/llms/openai-chatbot).
## Using OpenAI LLMs as the LLM judge
Some tests on Openlayer rely on scores produced by an LLM judge. For example,
tests that use [Ragas metrics](/integrations/ragas)
and the custom [LLM evaluator test](/tests/performance/l-l-m-rubric-threshold).
You can use any of OpenAI’s LLMs as the underlying LLM judge for these tests.
You can change the default LLM evaluator for a project in the project settings page. To
do so, navigate to "Settings" > Select your project in the left sidebar > click on "Metrics" to
go to the metric settings page. Under "LLM evaluator," choose the OpenAI LLM you
want to use.
Furthermore, make sure to add your `OPENAI_API_KEY` as an environment variable.
