
Evaluating Mistral AI LLMs
You can set up Openlayer tests to evaluate your Mistral AI LLMs in monitoring and development.Monitoring
To use the monitoring mode, you must instrument your code to publish the requests your AI system receives to the Openlayer platform. To set it up, you must follow the steps in the code snippet below:Python
See full Python example

If the Mistral AI LLM call is just one of the steps of your AI system, you can
use the code snippets above together with tracing. In
this case, your Mistral LLM calls get added as a step of a larger trace. Refer
to the Tracing guide for details.
Development
In development mode, Openlayer becomes a step in your CI/CD pipeline, and your tests get automatically evaluated after being triggered by some events. Openlayer tests often rely on your AI system’s outputs on a validation dataset. As discussed in the Configuring output generation guide, you have two options:- either provide a way for Openlayer to run your AI system on your datasets, or
- before pushing, generate the model outputs yourself and push them alongside your artifacts.
MISTRAL_API_KEY
.
If you don’t add the required Mistral API key, you’ll encounter a “Missing API key”
error when Openlayer tries to run your AI system to get its outputs.
The Mistral AI client does not read the
MISTRAL_API_KEY
directly from the
environment. Therefore, make sure to manually read it in the script you provide as the batchCommand
in
the openlayer.json with: