The mean cost test ensures that the average request cost (in USD) for the data is within a given range.


  • Category: Performance.
  • Task types: LLM.
  • Availability: and .

Why it matters

  • If you use an external API to get the model predictions, you might be charged for each request. This is the case for most third-party LLMs, such as OpenAI GPTs. Setting up tests for the cost of the requests is important to avoid unexpected costs.