Projects and goals
Navigate to the Openlayer platform
Before starting this part of the tutorial, please navigate and log into the Openlayer platform.
After running the notebook from the previous part and logging into the Openlayer platform, the first thing you see is the “Hockey x Baseball - 20 newsgroups” project and our model’s aggregate metrics.
It seems like our model is not performing as well as it could be. But how should we begin improving it? As mentioned earlier, aggregate metrics are limited to the extent that they can help us figure out how to improve our models.
To avoid relying on trial and error as the main driver for model development, we need to start by identifying the issues affecting our model. Enter the world of goals.
What are goals?
In the context of the Openlayer platform, goals are objectives or checkpoints set up to ensure our data and models meet specific standards.
There are different types of goals, each designed to cover a distinct aspect of quality and performance.
For now, let’s focus on how the process of creating goals can help us identify issues negatively affecting our model.
Click create goals
Click the “Create goals” button at the center of the page. Feel free to spend some time browsing through the different goal-type tabs.
Note how there are issues that likely have ripple effects on the model’s performance. To mention a few, notice there are duplicate rows in the training set, rows with ill-formed text, data leakage, and more.
For now, the supported goal types are:
- Data integrity goals: assesses the data quality of the training and validation sets.
- Data consistency goals: related to the consistency between the training and validation data.
- Performance goals: guardrails on the model’s performance on the whole validation set or specific subpopulations of it.
- Fairness goals: assess the model using different fairness metrics.
- Robustness goals: assert robustness to different data perturbation recipes.
As a final comment, notice how there is a hierarchy between the goal types. Our recommended approach involves beginning with the easily achievable data integrity goals, before moving on to each successive goal type and ultimately concluding with robustness.
Let’s move to the next part of the tutorial, where we will start creating goals that help us build a high-quality model!
Updated about 2 months ago