Projects and goals

✅

Navigate to the Openlayer platform

Before starting this part of the tutorial, please navigate and log into the Openlayer platform.

After running the notebook from the previous part and logging into the Openlayer platform, the first thing you see is the “Churn Prediction” project and our model’s aggregate metrics.

It seems like our model is not performing as well as it could be. But how should we begin improving it? As mentioned earlier, aggregate metrics are limited to the extent that they can help us figure out how to improve our models.

To avoid relying on trial and error as the main driver for model development, we need to start by identifying the issues affecting our model. Enter the world of goals.

Goals

💡

What are goals?

In the context of the Openlayer platform, goals are objectives or checkpoints set up to ensure our data and models meet specific standards.

There are different types of goals, each designed to cover a distinct aspect of quality and performance.

For now, let’s focus on how the process of creating goals can help us identify issues negatively affecting our model.

✅

Click create goals

Click the “Create goals” button at the center of the page. Feel free to spend some time browsing through the different goal-type tabs.

🚀

Actionable insights

Note how there are issues that likely have ripple effects on the model’s performance. To mention a few, notice there are duplicate rows in the training set, quasi-constant features, feature drift, and more.

Goal types

For now, the supported goal types are:

• Data integrity goals: assesses the data quality of the training and validation sets.
• Data consistency goals: related to the consistency between the training and validation data.
• Performance goals: guardrails on the model’s performance on the whole validation set or specific subpopulations of it.
• Fairness goals: assess the model using different fairness metrics.
• Robustness goals: assert robustness to different data perturbation recipes.

As a final comment, notice how there is a hierarchy between the goal types. Our recommended approach involves beginning with the easily achievable data integrity goals, before moving on to each successive goal type and ultimately concluding with robustness.

Let’s move to the next part of the tutorial, where we will start creating goals that help us build a high-quality model!