Multi-Doc QA on LongBench v2 (test)
86.7Finance AccuracyBase
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| BaseBase Model=DeepSeek-V4-Pro2026.06 | 86.7 | 47.8 | 44.9 | 76.2 | 64 | 63.9 | |
| DELMBase Model=DeepSeek-V4-Pro2026.06 | 86.7 | 52.2 | 59.4 | 73.8 | 65.3 | 67.5 | |
| Claude CodeBase Model=DeepSeek-V4-Pro2026.06 | 84.4 | 36.2 | 56.5 | 64.3 | 48.7 | 58 | |
| DELMBase Model=Gemini 3 Flash2026.06 | 82.2 | 37.8 | 52.2 | 71.4 | 64 | 61.5 | |
| BaseBase Model=Gemini 3 Flash2026.06 | 80 | 30.4 | 43.5 | 71.4 | 60 | 57.1 | |
| ReadAgentBase Model=Gemini 3 Flash2026.06 | 75.6 | 21.7 | 46.4 | 69 | 62 | 54.9 | |
| DELMBase Model=Claude Sonnet 4.62026.06 | 73.3 | 46.4 | 55.1 | 59.5 | 64.7 | 59.8 | |
| Claude CodeBase Model=Gemini 3 Flash2026.06 | 73.3 | 26.1 | 39.1 | 57.1 | 52 | 49.5 | |
| DELMBase Model=GPT-5.42026.06 | 71.1 | 43.5 | 65.2 | 64.3 | 56 | 60.1 | |
| ReadAgentBase Model=DeepSeek-V4-Pro2026.06 | 71.1 | 43.5 | 30.4 | 38.1 | 50.7 | 46.8 | |
| BaseBase Model=Claude Sonnet 4.62026.06 | 68.9 | 34.8 | 46.4 | 52.4 | 64 | 53.3 | |
| Claude CodeBase Model=Claude Sonnet 4.62026.06 | 66.7 | 42 | 47.8 | 50 | 52 | 51.7 | |
| Claude CodeBase Model=GPT-5.42026.06 | 65 | 36.2 | 50.7 | 59.5 | 60.7 | 54.4 | |
| BaseBase Model=GPT-5.42026.06 | 62.2 | 39.1 | 50.5 | 62 | 56 | 53.9 | |
| ReadAgentBase Model=Claude Sonnet 4.62026.06 | 62.2 | 44.9 | 43.5 | 57.1 | 64.7 | 54.5 | |
| ReadAgentBase Model=GPT-5.42026.06 | 57.8 | 34.8 | 52.2 | 61.9 | 58 | 53 |