Skip to main content

Definition

The SQL query test allows you to write custom SQL queries to validate your data and set thresholds on the query results. This test executes a user-defined SQL query that returns a numerical result, which can then be compared against specified thresholds. The SQL query must reference your dataset as df (the table name) and should return a single numerical value.
Inside df, columns use their canonical platform names, not the raw column names from your dataset config. In particular, the model output column is openlayer_output (not the outputColumnName you declared), and the system columns are openlayer_latency, openlayer_num_of_tokens, and openlayer_cost. A query like SELECT COUNT(*) FROM df WHERE output IS NULL errors with "output" not found in FROM clause — use openlayer_output. Your own feature columns keep the names you gave them.

Taxonomy

  • Task types: LLM, tabular classification, tabular regression, text classification.
  • Availability: and .

Why it matters

  • The SQL query test provides flexibility for creating custom data validation rules tailored to your specific use case.
  • You can implement complex business logic and data quality checks that aren’t covered by standard tests.
  • This enables domain-specific validation rules, such as checking data consistency across multiple columns, validating ranges, or ensuring specific business constraints are met.
  • Custom SQL queries allow you to leverage the full power of SQL for data analysis and validation within your testing pipeline.

Restrictions

  • Creating this test, or editing its SQL, requires the View data permission, which Admin and Member have. Member Restricted cannot view data, so they cannot create the test or change its SQL. Changing only a threshold value on an existing test still works for them, and tests they already created keep running. See Roles and permissions.
  • The query must be a single SELECT statement that reads only the dataset table df. Common table expressions you define in the query are fine.
  • Statements that write, multiple statements, table functions, and metadata or system functions are rejected. The test fails with an error instead of running.
  • Syntax that cannot be parsed is also rejected when your data is backed by Snowflake or BigQuery. Other data runs the query in memory and accepts only one SELECT statement.
  • A query that runs longer than 300 seconds is cancelled.

Test configuration examples

If you are writing a tests.json, here are a few valid configurations for the SQL query test: