Metrics Reference¶
RAGhelm evaluates RAG and agent systems as release candidates, not just as isolated model calls. Metrics feed scorecards, manifests, gates, and dashboards.
Retrieval metrics¶
Recall@k¶
Fraction of relevant contexts retrieved in the top-k results.
Recall@k = |relevant in top-k| / |total relevant|
MRR¶
Mean reciprocal rank of the first relevant result.
MRR = 1/|Q| * sum(1/rank_i)
NDCG@k¶
Ranking quality metric that rewards relevant results near the top and penalizes relevant results lower in the ranking.
Generation metrics¶
Generation scores use a 1-5 scale.
| Metric | Description |
|---|---|
faithfulness |
Whether the answer is grounded in retrieved/source material. |
relevance |
Whether the answer addresses the question. |
completeness |
Whether the answer covers required aspects. |
overall |
Weighted quality summary. |
Gate usage¶
Metrics become release-control evidence only when linked to:
- the target under test
- eval suite and mode
- retrieval/generation/judge configuration
- artifact hashes
- privacy classification
- RAGRunManifest and ReadinessScorecard IDs where available
Badge colors¶
Badge colors are display hints, not release decisions.
| Threshold | Badge color |
|---|---|
| >= 0.90 | Green #9ece6a |
| >= 0.80 | Yellow #e0af68 |
| < 0.80 | Red #f7768e |
Per ADR-010, published quality badges must be backed by production scorecard and manifest evidence or fail closed.