The max latency test ensures that the maximum latency for the data is within a given range.
To enable this test, the data published/streamed to Openlayer must have a
column with the latency for each row. In the data
config, the name of this column is
specified in the
- Category: Performance.
- Task types: LLM, tabular classification, tabular regression, text classification.
- Availability: .
Why it matters
- The latency of a model directly impacts the user experience. Setting up tests to monitor the latency ensures that the latency is kept within a given range.