Project dashboards

Every Openlayer project starts with a default dashboard and can have additional dashboards. Each dashboard gathers the project’s metrics, test results, cost, and activity in a view with its own name, description, visibility, sections, and pinned content. Use separate dashboards to focus on different audiences or workflows, such as operational health, test performance, or cost review.

​

Open dashboards

Open a project and select Dashboards. This entry opens the dashboard you viewed most recently, or the dashboards index if you have not opened one. Selecting Dashboards while you are already in this area returns to the index. The index lists each dashboard as a card with its visibility, name, description, pinned content, and last update. The default dashboard also has a Default tag. Search appears when the project has at least seven dashboards. Projects you keep as favorites list Dashboards, described as Project dashboards, in the sidebar. The command palette lists the project’s dashboards individually. On a dashboard page, use the Dashboards › dashboard name breadcrumb to switch dashboards. Press ← or → to move between dashboards in index order. On the index, press C to create a dashboard if you have permission.

​

Manage multiple dashboards

Create a custom dashboard from Default layout to copy the project’s default dashboard, or choose Blank to start from scratch. Use the dashboard options menu to Edit details, Duplicate, Copy link, or Copy dashboard ID. Visible to sets the minimum workspace role that can open the dashboard: Viewer, Member Restricted, Member, or Admin. The corresponding card tags are Everyone, Restricted, Members, and Admins. This picker appears only if you can update dashboards. You cannot set visibility above your own role, and the visibility requirement applies when editing a dashboard as well as when opening it. Deleting a dashboard requires confirmation and removes the dashboard and its widget layout for everyone in the project; it does not affect tests or metrics. Default dashboards cannot be deleted, and Delete dashboard only appears if you have permission.

​

Choose commit and data source

Two comboboxes at the top of the page set what the sections below report on:

• Select a commit picks the commit — the platform also calls a commit a project version. Use Find commit… to search among them.
• Select a data source picks the data source. Use Find data source… to search among them.

Usage and cost is the exception. It compares every data source or commit in the project side by side instead of following these selections.

​

Sections

Each dashboard can contain these sections. Where available, use the Development / Monitoring toggle to change the mode.

• Performance metrics — “Key usage metrics across development and production”
• Pass rate over time — the percentage of tests passing over time, with a Development / Monitoring toggle
• Column distributions — “Record counts broken down by category across development and production”
• Failing tests — the tests that fail most often, with columns Test name, Failures, Latest result, Threshold, and Created by, and a Development / Monitoring toggle
• Tests — “Performance of pinned tests over time across development and production”
• Recent activity — “Project updates and notifications”
• Usage and cost — token consumption and estimated cost, with a Monitoring / Development toggle
• LLM-as-a-judge costs — evaluator model spend from LLM-as-a-judge tests, kept separate from inference usage and cost

Performance metrics, Column distributions, and Tests stay empty until you pin what they report on.

​

Pin metrics and distributions

Pin controls appear only if you can edit dashboards. Until you pin anything, the three sections show their empty states:

• Performance metrics shows No metrics pinned and prompts you to Pin metrics.
• Column distributions shows No distributions pinned and prompts you to Pin distributions.
• Tests shows No tests pinned, Pin tests to view their results over time, and a Pin tests button.

Click Pin metrics to choose what appears in Performance metrics. The picker has Development and Monitoring tabs, plus Search metrics to pin… and Search pinned metrics…. Each card carries a DEVELOPMENT or MONITORING badge and shows Results for the selected commit or Results for the selected data source, matching the tab. Pin the metrics, distributions, and tests you care about, and each section renders them in place of its empty state.

​

Track usage and cost

Usage and cost reports how many tokens the project consumes and what they cost. The toggle in the section header picks Monitoring or Development, and the two modes are never combined, so production usage never mixes with offline experiments. Monitoring groups production traces by data source, with an Unassigned bucket for traces no data source claims. Development groups offline traces by commit, newest first, and compares every commit in the project. Data sources and commits you have removed still appear, labeled (deleted) or (archived), so past spend stays accounted for. When the selected mode has usage data, the section shows three separate chart cards:

• Token usage over time shows Daily totals for the selected mode. Modes are never combined. Use the Tokens / Cost toggle to change the value plotted.
• By data source in monitoring describes Tokens and estimated cost by data source. By commit in development describes Tokens and estimated cost by commit. Up to five groups appear by name, with longer tails folded into Other. Click a navigable bar to open that data source or commit on the data page. Unassigned is not clickable.
• Tokens by type describes Input, output, cached, and other tokens plus estimated cost. Its legend categories are Input, Output, Cached, and Other. The center shows total tokens, and the list includes Estimated cost.

Before any traces arrive, the section shows No monitoring usage yet with Production traces collected by data sources will appear here., or No development usage yet with Offline traces associated with commits will appear here. Individual cards have their own empty states when the selected mode has usage but not the details that a chart needs:

• Token usage over time shows No time series yet and Daily token and cost totals appear here once traces are collected.
• By data source shows No token data and No data source has recorded token usage yet. By commit shows No token data and No commit has recorded token usage yet.
• Tokens by type shows No token categories and Input, output, and cached tokens appear here when usage details are available.

If the section cannot load, it shows Couldn’t load usage breakdown and Usage and cost could not be loaded. Try again in a moment. When only some sources load, it warns Some sources could not be loaded. Totals may be incomplete.

​

Track LLM-as-a-judge costs

LLM-as-a-judge costs tracks the evaluator model spend from your LLM-as-a-judge tests. It stays separate from Usage and cost, so judging your outputs never looks like a change in the cost of producing them. The section header carries the same Monitoring / Development toggle, and again the modes are never combined: monitoring reports on evaluator runs against production traces, development on evaluator runs against a commit. Two charts split the spend — By test bars and By model donut. Spend that Openlayer cannot attribute to a test lands in Unassigned, and when a single evaluator run covers several tests, its cost is split equally across them. Until an LLM-as-a-judge test runs, the section reads No judge costs tracked yet and offers Learn about LLM-as-a-judge.

​

Customize display

Click Display (keyboard shortcut v) to open the Customize display popover. Admins can choose and reorder sections on any dashboard. On custom dashboards, other users with dashboard editing access can add or remove preset sections but cannot reorder them. Viewers do not see Display or pin controls.