Memory Question Answering on ATM-Bench Hard
74.7QS ScoreGPT-5
Evaluation results
| Method | Links | |
|---|---|---|
| GPT-5Retriever=Oracle, Reasoning Effort=medium2026.03 | 74.7 | |
| Gemini 2.5 ProRetriever=Oracle, Reasoning Effort=medium2026.03 | 64.3 | |
| Claude Opus 4.5Retriever=Oracle, Reasoning Effort=medium2026.03 | 62.7 | |
| Claude Sonnet 4.5Retriever=Oracle, Reasoning Effort=medium2026.03 | 61.9 | |
| Gemini 2.5 FlashRetriever=Oracle, Reasoning Effort=medium2026.03 | 55.6 | |
| Qwen3-VL-8BRetriever=Oracle, Reasoning Effort=medium, Variant=Instruct2026.03 | 47.3 | |
| Qwen3-VL-2BRetriever=Oracle, Reasoning Effort=medium, Variant=Instruct2026.03 | 34.1 |