You can use Openlayer in two modes:

  • development: to evaluate an AI system as you develop it, iterating through their versions.
  • monitoring: to evaluate a deployed AI system serving live requests.

This guide shows how you can set up Openlayer’s development mode as a step in your CI/CD pipeline using a Git repository and how you can leverage the same setup to monitor your live AI system with monitoring mode.

Prerequisite: you need an Openlayer account to follow this guide.

1. Find a template

To accelerate the setup, we use templates. Templates are sample projects that use common AI patterns and tools, and that already contain the configurations required by Openlayer.

2. Create tests

You have a GitHub repo linked to an Openlayer project. You can view the artifacts on Openlayer and start creating tests.

3. Make changes

From now on, every version of your system will get evaluated against your tests. All you need to do is keep pushing to your connected GitHub repo.

4. Ship

After some development iterations, you’ll want to deploy your AI system. You can continuously run your tests on your live data.