Skip to main content
Every Openlayer project has a Dashboard, a single page that gathers the project’s metrics, test results, cost, and activity in one place. Use it to review performance metrics, pass rate, column distributions, failing tests, recent activity, and cost for the commit and data source you select.
Prerequisite: you need a project to follow this guide.

​
Open the dashboard

  1. Log into your Openlayer account.
  2. Open the project you want to inspect.
  3. Click Dashboard.
Projects you keep as favorites list Dashboard, described as Project dashboard, in the sidebar. You can also search for Dashboard in the command palette.

​
Choose commit and data source

Two comboboxes at the top of the page set what the sections below report on:
  • Select a project version picks the commit — the platform also calls a commit a project version. Use Find commit… to search among them.
  • Select a data source picks the data source. Use Find data source… to search among them.
Usage and cost is the exception. It compares every data source or commit in the project side by side instead of following these selections.

​
Sections

The dashboard is made up of these sections.
  • Performance metrics — “Key usage metrics across development and production”
  • Pass rate over time — the percentage of tests passing over time, with a Development / Monitoring toggle
  • Column distributions — “Record counts broken down by category across development and production”
  • Failing tests — the tests that fail most often, with columns Test name, Failures, Latest result, Threshold, and Created by, and a Development / Monitoring toggle
  • Tests — “Performance of pinned tests over time across development and production”
  • Recent activity — “Project updates and notifications”
  • Usage and cost — token consumption and estimated cost, with a Monitoring / Development toggle
  • LLM-as-a-judge costs — evaluator model spend from LLM-as-a-judge tests, kept separate from inference usage and cost
Performance metrics, Column distributions, and Tests stay empty until you pin what they report on.

​
Pin metrics and distributions

Until you pin anything, the three sections show their empty states:
  • Performance metrics shows No metrics pinned and prompts you to Pin metrics.
  • Column distributions shows No distributions pinned and prompts you to Pin distributions.
  • Tests shows No tests pinned, Pin tests to view their results over time, and a Pin tests button.
Click Pin metrics to choose what appears in Performance metrics. The picker has Development and Monitoring tabs, plus Search metrics to pin… and Search pinned metrics…. Each card carries a DEVELOPMENT or MONITORING badge and shows Results for the selected commit or Results for the selected data source, matching the tab. Pin the metrics, distributions, and tests you care about, and each section renders them in place of its empty state.

​
Track usage and cost

Usage and cost reports how many tokens the project consumes and what they cost. The toggle in the section header picks Monitoring or Development, and the two modes are never combined, so production usage never mixes with offline experiments. Monitoring groups production traces by data source, with an Unassigned bucket for traces no data source claims. Development groups offline traces by commit, newest first, and compares every commit in the project. Data sources and commits you have removed still appear, labeled (deleted) or (archived), so past spend stays accounted for. Each mode shows the same four views:
  • A daily time series of usage and cost.
  • Bars, titled By data source in monitoring and By commit in development.
  • A donut titled Tokens by type, splitting tokens into Input, Output, Cached, and Other, with total tokens in the center and the estimated cost on the legend.
  • A comparison list, with columns Data source or Commit, Tokens, Share, Cost, and Cost share.
Select a data source or a commit to open its traces on the data page. Unassigned is the one bucket you cannot open this way. Before any traces arrive, the section reads No monitoring usage yet or No development usage yet, depending on the mode, and points at where the data comes from: production traces collected by data sources, and offline traces associated with commits. Individual charts have their own empty states when a mode has traces but not the detail a chart needs — No token data on the donut, for example, or No time series yet on the series.

​
Track LLM-as-a-judge costs

LLM-as-a-judge costs tracks the evaluator model spend from your LLM-as-a-judge tests. It stays separate from Usage and cost, so judging your outputs never looks like a change in the cost of producing them. The section header carries the same Monitoring / Development toggle, and again the modes are never combined: monitoring reports on evaluator runs against production traces, development on evaluator runs against a commit. Two charts split the spend — By test bars and By model donut. Spend that Openlayer cannot attribute to a test lands in Unassigned, and when a single evaluator run covers several tests, its cost is split equally across them. Until an LLM-as-a-judge test runs, the section reads No judge costs tracked yet and offers Learn about LLM-as-a-judge.

​
Customize display

Click Display (keyboard shortcut d) to open the Customize display popover. Use it to choose and reorder sections: “Choose and reorder sections to customize your dashboard”. The checkboxes show or hide each of the eight sections, and the drag handles reorder them.