Total cost

​

Definition

The total cost test ensures that the total request cost (in USD) for the data is within a given range.

​

Taxonomy

• Category: Performance.
• Task types: LLM.
• Availability: development and monitoring.

​

Why it matters

• If you use an external API to get the model predictions, you might be charged for each request. This is the case for most third-party LLMs, such as OpenAI GPTs. Setting up tests for the cost of the requests is important to avoid unexpected costs.

​

Related

• Max cost.
• Mean cost.