Definition
The session latency test monitors cumulative model latency per session. For each session, Openlayer sums per-trace latency across all turns, then exposes window-level aggregates you can threshold against. No LLM evaluator is involved — it’s a deterministic aggregation.Taxonomy
- Task types: LLM.
- Availability: and .
- Evaluation level: session.
- Computation: deterministic aggregation.
Why it matters
- Per-trace latency is the standard monitoring signal, but total latency within a session is what the user actually waits through.
- A session that accumulates 15 turns × 3 seconds is a materially different user experience from a single 45-second turn — this metric captures the former.
Available measurements
| Measurement | What it means |
|---|---|
totalLatency | Sum of per-trace latency across all traces in the window (global, not per-session) |
meanLatencyPerSession | Mean of per-session latency sums across sessions in the window |
medianLatencyPerSession | Median of per-session latency sums across sessions in the window |
Per-session latency is the sum of all trace latencies in that session, not
a per-turn average.
meanLatencyPerSession is then the mean across sessions
of those per-session sums.Required columns
- Session ID: Groups turns belonging to the same conversation.
- Latency: Per-trace latency (units follow the
openlayer_latencycolumn; usually populated automatically by the Openlayer client or via OpenTelemetry).
Test configuration examples
Related
- Session duration — wall-clock session length, different signal.
- Mean latency, Max latency, p95 latency — trace-level latency metrics.