| Answer relevancy | Evaluates how well the model’s answer aligns with the intent of the question. The evaluator LLM infers possible questions from the answer and compares them to the actual question using semantic similarity. | input, outputs | answerRelevancy |
| Answer correctness | Measures factual alignment between the generated answer and the ground truth reference. The evaluator breaks both into factual statements and compares them (true positives, false positives, false negatives). | outputs, ground truths | answerCorrectness |
| Context relevancy | Assesses whether the retrieved context is relevant to the ground truth answer. Each context chunk is judged independently by an LLM for its relevance. | input, ground truth, context | contextRelevancy |
| Context recall | Evaluates how completely the retrieved context supports all the claims present in the ground truth. High recall means most ground truth claims are supported by the retrieved context. | ground truth, context | contextRecall |
| Faithfulness | Measures how factually consistent the model’s answer is with the retrieved context. The evaluator identifies factual claims in the output and verifies if each is supported by the context. | outputs, context | faithfulness |
| Correctness | Judges the general factual soundness of the answer. The evaluator LLM rates the output’s correctness based on an aspect-based critique. | input, outputs | correctness |
| Harmfulness | Evaluates whether the answer contains harmful, unsafe, or toxic content. The evaluator LLM critiques the response through a safety-oriented lens. | input, outputs | harmfulness |
| Coherence | Measures how logically and linguistically coherent the answer is — i.e., whether it flows naturally and maintains internal consistency. | input, outputs | coherence |
| Conciseness | Evaluates whether the answer is clear and to the point, without unnecessary verbosity or repetition. | input, outputs | conciseness |
| Maliciousness | Detects whether the answer exhibits malicious intent, manipulation, or socially undesirable behavior. | input, outputs | maliciousness |
