99th latency percentile

Definition

The 99th latency percentile test ensures that the 99th percentile latency for the data is within a given range.

Taxonomy

Task types: LLM, tabular classification, tabular regression, text classification.
Availability: .

Why it matters

The 99th percentile latency captures the performance experienced by nearly all users, excluding only the worst 1% of cases.
This metric helps identify performance issues that could affect even a small percentage of users, which is important for maintaining high service quality.
Monitoring the 99th percentile is crucial for applications where consistent performance is critical, even for edge cases.

Test configuration examples

If you are writing a tests.json, here are a few valid configurations for the 99th latency percentile test:

[
  {
    "name": "99th percentile latency below 12000 msec",
    "description": "Make sure that the 99th percentile latency is below 12000 msec",
    "type": "performance",
    "subtype": "metricThreshold",
    "thresholds": [
      {
        "insightName": "metrics",
        "measurement": "p99Latency",
        "operator": "<",
        "value": 12000
      }
    ],
    "subpopulationFilters": null,
    "mode": "development",
    "usesValidationDataset": true,
    "usesTrainingDataset": false,
    "usesMlModel": true,
    "syncId": "b4dee7dc-4f15-48ca-a282-63e2c04e0689"
  }
]

[
  {
    "name": "99th percentile latency below 12000 msec",
    "description": "Make sure that the 99th percentile latency is below 12000 msec",
    "type": "performance",
    "subtype": "metricThreshold",
    "thresholds": [
      {
        "insightName": "metrics",
        "measurement": "p99Latency",
        "operator": "<",
        "value": 12000
      }
    ],
    "subpopulationFilters": null,
    "mode": "monitoring",
    "usesProductionData": true,
    "evaluationWindow": 3600,
    "delayWindow": 0,
    "syncId": "b4dee7dc-4f15-48ca-a282-63e2c04e0689"
  }
]

95th latency percentile

Predictive power score (PPS)

⌘I

​Definition

​Taxonomy

​Why it matters

​Test configuration examples

​Related

Definition

Taxonomy

Why it matters

Test configuration examples

Related