Memory Question Answering on ATM-Bench
86QS ScoreClaude Opus 4.5
Evaluation results
| Method | Links | |
|---|---|---|
| Claude Opus 4.5Retriever=Oracle, Reasoning Effort=medium2026.03 | 86 | |
| GPT-5Retriever=Oracle, Reasoning Effort=medium2026.03 | 85.3 | |
| Claude Sonnet 4.5Retriever=Oracle, Reasoning Effort=medium2026.03 | 83.3 | |
| Gemini 2.5 ProRetriever=Oracle, Reasoning Effort=medium2026.03 | 78.6 | |
| Qwen3-VL-8BRetriever=Oracle, Reasoning Effort=medium, Variant=Instruct2026.03 | 77.8 | |
| Gemini 2.5 FlashRetriever=Oracle, Reasoning Effort=medium2026.03 | 76.9 | |
| Qwen3-VL-2BRetriever=Oracle, Reasoning Effort=medium, Variant=Instruct2026.03 | 34.8 |