​
Definition

The number of rows test allows you to set a threshold on the number of rows in your dataset.

​
Taxonomy

  • Category: Integrity.
  • Task types: LLM, tabular classification, tabular regression, text classification.
  • Availability: and .

​
Why it matters

  • During development, datasets are often updated and modified. It is important to keep track of the number of rows in your dataset to ensure that it has the expected size as you iterate on it.
  • In monitoring, it is important to keep track of the number of rows received within a given evaluation window, as sudden shifts in traffic can be a sign that actions need to be taken.

​
Test configuration examples

If you are writing a tests.json, here are a few valid configurations for the character length test: 
[
  {
    "name": "Data has at least 100 rows",
    "description": "Asserts that the data has at least 100 rows",
    "type": "integrity",
    "subtype": "rowCount",
    "thresholds": [
      {
        "insightName": "subpopulationRowCount",
        "insightParameters": null,
        "measurement": "subpopulationRowCount",
        "operator": ">=",
        "value": 100
      }
    ],
    "subpopulationFilters": null,
    "mode": "development",
    "usesValidationDataset": true, // Apply test to the validation set
    "usesTrainingDataset": false,
    "usesMlModel": false,
    "syncId": "b4dee7dc-4f15-48ca-a282-63e2c04e0689" // Some unique id
  }
]