Retrieval-Augmented Question Answering on Unique Canonical User-Facing Queries n=170 (Overall)
97FaithfulnessClaude Opus 4.7
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Claude Opus 4.72026.06 | 97 | — | 45 | 33 | |
| Naive RAG2026.06 | 87 | 72 | 72 | 77 | |
| I-GUIDE SSmode=Smart Search2026.06 | 81 | 98 | 91 | 97 | |
| Naive RAG2026.06 | 78 | 38 | 41 | 41 | |
| I-GUIDE SSmode=Smart Search2026.06 | 77 | 73 | 72 | 71 | |
| Claude Opus 4.72026.06 | 76 | — | 56 | 50 | |
| Qwen2.52026.06 | 69 | — | 39 | 51 | |
| GPT-5.52026.06 | 64 | — | 55 | 72 | |
| Qwen2.52026.06 | 62 | — | 41 | 57 | |
| GPT-4o2026.06 | 58 | — | 44 | 61 | |
| GPT-4o2026.06 | 20 | — | 27 | 82 | |
| GPT-5.52026.06 | 20 | — | 26 | 85 |