Here, we define some terms used in Openlayer.
When you create an account, you are given a workspace. A workspace is your home in Openlayer, where all your projects live and where your team collaborates.
A project lives inside a workspace. It represents a problem you are tackling with AI/ML and houses all the models, data, and tests related to it. You can create multiple projects in your workspace, and your team members have access to all of them.
A project has two modes: development and monitoring.
The development mode of a project hosts your efforts during model development. It is where you push models and datasets for testing and keep track of their versions. Refer to the Versioning guide for a deep dive into the process followed in development.
The monitoring mode of a project allows you to monitor a model deployed in production. It receives production data and evaluates it with the tests you defined. In the monitoring mode, you can also set up alerts to get notified when tests start failing.
Tests materialize expectations around models and data and exist to measure different aspects of quality and performance.
In development, they ensure you are systematically making progress and avoiding regressions in your quest toward high-quality models. In monitoring, they help measure the model’s health in production and trigger notifications when they fail. Refer to the Development and Monitoring test guides to learn more about them.
A commit bundle is a relevant concept in a project’s development mode. It is comprised of the bundle with all the artifacts produced during an iteration of model development — generally, a model and datasets (training/fine-tuning, and validation). Refer to the commit bundle section of the Versioning guide for details.
The inference pipeline is part of a project’s monitoring mode. It represents a model that is deployed in production making inferences.
A common setup for many teams is to have two inference pipelines: one named
staging, and the other
When you publish/stream production data to Openlayer, you need to specify which inference pipeline it belongs to.