Sentence length

Definition

The sentence length test validates that sentences in your text data fall within specified length boundaries. You can set thresholds on both the maximum and minimum sentence lengths to ensure text quality and consistency. This test analyzes individual sentences within your text columns and measures their character length, allowing you to detect overly long or short sentences that might indicate data quality issues or generation problems.

Taxonomy

Task types: LLM, text classification.
Availability: and .

Why it matters

Text quality assurance: Ensures generated or processed text maintains appropriate sentence lengths for readability
Model output validation: Prevents LLMs from generating extremely long run-on sentences or incomplete fragments
Data consistency: Maintains uniform text formatting and structure across your dataset
User experience: Ensures text outputs are readable and well-formatted for end users
Detection of generation issues: Identifies when models produce malformed or truncated text

Test configuration examples

If you are writing a tests.json, here are a few valid configurations for the sentence length test:

[
  {
    "name": "Sentences not too long",
    "description": "Ensures no sentences exceed 200 characters to maintain readability",
    "type": "integrity",
    "subtype": "sentenceLength",
    "thresholds": [
      {
        "insightName": "sentenceLength",
        "measurement": "maxSentenceLength",
        "operator": "<=",
        "value": 200
      }
    ],
    "subpopulationFilters": null,
    "mode": "development",
    "usesValidationDataset": true,
    "usesTrainingDataset": false,
    "usesMlModel": false,
    "syncId": "b4dee7dc-4f15-48ca-a282-63e2c04e0689"
  },
  {
    "name": "Sentences have minimum content",
    "description": "Ensures sentences are at least 10 characters to avoid fragments",
    "type": "integrity",
    "subtype": "sentenceLength",
    "thresholds": [
      {
        "insightName": "sentenceLength",
        "measurement": "minSentenceLength",
        "operator": ">=",
        "value": 10
      }
    ],
    "subpopulationFilters": null,
    "mode": "development",
    "usesValidationDataset": true,
    "usesTrainingDataset": false,
    "usesMlModel": false,
    "syncId": "96622fba-ea00-4e42-8f42-5e8f5f60805f"
  },
  {
    "name": "Balanced sentence lengths",
    "description": "Ensures sentences are between 20-150 characters for optimal readability",
    "type": "integrity",
    "subtype": "sentenceLength",
    "thresholds": [
      {
        "insightName": "sentenceLength",
        "measurement": "minSentenceLength",
        "operator": ">=",
        "value": 20
      },
      {
        "insightName": "sentenceLength",
        "measurement": "maxSentenceLength",
        "operator": "<=",
        "value": 150
      }
    ],
    "subpopulationFilters": null,
    "mode": "development",
    "usesValidationDataset": true,
    "usesTrainingDataset": false,
    "usesMlModel": false,
    "syncId": "a1b2c3d4-e5f6-47g8-h9i0-j1k2l3m4n5o6"
  }
]

[
  {
    "name": "Generated text length monitoring",
    "description": "Monitors that generated sentences stay within acceptable length bounds",
    "type": "integrity",
    "subtype": "sentenceLength",
    "thresholds": [
      {
        "insightName": "sentenceLength",
        "measurement": "maxSentenceLength",
        "operator": "<=",
        "value": 300
      }
    ],
    "subpopulationFilters": null,
    "mode": "monitoring",
    "usesProductionData": true,
    "evaluationWindow": 3600,
    "delayWindow": 0,
    "syncId": "b4dee7dc-4f15-48ca-a282-63e2c04e0689"
  },
  {
    "name": "Sentence completeness check",
    "description": "Alerts when sentences are suspiciously short, indicating potential truncation",
    "type": "integrity",
    "subtype": "sentenceLength",
    "thresholds": [
      {
        "insightName": "sentenceLength",
        "measurement": "minSentenceLength",
        "operator": ">=",
        "value": 5
      }
    ],
    "subpopulationFilters": null,
    "mode": "monitoring",
    "usesProductionData": true,
    "evaluationWindow": 3600,
    "delayWindow": 0,
    "syncId": "96622fba-ea00-4e42-8f42-5e8f5f60805f"
  }
]

Semantic similarity

Session coherence

⌘I

Get started

Workspace setup

Governance

Observability

Offline testing

Tests

Gateway

Data quality monitoring

Administration

Notifications

Other resources

Definition

Taxonomy

Why it matters

Test configuration examples

​Definition

​Taxonomy

​Why it matters

​Test configuration examples

Definition

Taxonomy

Why it matters

Test configuration examples