Long-context Question Answering on FRAMES
73.54Avg@4 ScoreGPT5-Nano
Evaluation Results
| Method | Links | |
|---|---|---|
| GPT5-NanoBackbone=GPT5-Nano, Category=Frontier Models2026.01 | 73.54 | |
| GPT-OSS-120BBackbone=GPT-OSS-120B, Category=Frontier Models2026.01 | 72.69 | |
| LONGPASBackbone=Qwen3-30B-A3B-Thinking, Category=Reasoning Models, Training Strategy=LONGPAS2026.01 | 71.84 | |
| Qwen3-30B-A3B-ThinkingBackbone=Qwen3-30B-A3B-Thinking, Category=Reasoning Models, Training Strategy=Vanilla2026.01 | 69.66 | |
| RLVRBackbone=Qwen3-30B-A3B-Thinking, Category=Reasoning Models, Training Strategy=RLVR2026.01 | 69.66 | |
| LONGPASBackbone=Qwen3-30B-A3B-Instruct, Category=Instruct Models, Training Strategy=LONGPAS2026.01 | 68.93 | |
| RLVRBackbone=Qwen3-30B-A3B-Instruct, Category=Instruct Models, Training Strategy=RLVR2026.01 | 66.26 | |
| Gemini-2.5-Flash-ThinkingBackbone=Gemini-2.5-Flash-Thinking, Category=Frontier Models2026.01 | 65.78 | |
| LONGPASBackbone=Qwen3-4B-Instruct, Category=Instruct Models, Training Strategy=LONGPAS2026.01 | 64.9 | |
| LONGPASBackbone=Qwen3-4B-Thinking, Category=Reasoning Models, Training Strategy=LONGPAS2026.01 | 64.75 | |
| GPT-OSS-20BBackbone=GPT-OSS-20B, Category=Frontier Models2026.01 | 64.44 | |
| Qwen3-30B-A3B-InstructBackbone=Qwen3-30B-A3B-Instruct, Category=Instruct Models, Training Strategy=Vanilla2026.01 | 62.96 | |
| RLVRBackbone=Qwen3-4B-Thinking, Category=Reasoning Models, Training Strategy=RLVR2026.01 | 62.74 | |
| RLVRBackbone=Qwen3-4B-Instruct, Category=Instruct Models, Training Strategy=RLVR2026.01 | 60.92 | |
| Qwen3-4B-ThinkingBackbone=Qwen3-4B-Thinking, Category=Reasoning Models, Training Strategy=Vanilla2026.01 | 60.84 | |
| LONGPASBackbone=LLaMA3.1-8B-Instruct, Category=Instruct Models, Training Strategy=LONGPAS2026.01 | 60.62 | |
| RLVRBackbone=LLaMA3.1-8B-Instruct, Category=Instruct Models, Training Strategy=RLVR2026.01 | 55.98 | |
| LONGPASBackbone=Qwen2.5-7B-Instruct, Category=Instruct Models, Training Strategy=LONGPAS2026.01 | 55.76 | |
| RLVRBackbone=Qwen2.5-7B-Instruct, Category=Instruct Models, Training Strategy=RLVR2026.01 | 50.97 | |
| Qwen3-4B-InstructBackbone=Qwen3-4B-Instruct, Category=Instruct Models, Training Strategy=Vanilla2026.01 | 46.81 | |
| Qwen2.5-7B-InstructBackbone=Qwen2.5-7B-Instruct, Category=Instruct Models, Training Strategy=Vanilla2026.01 | 45.08 | |
| LLaMA3.1-8B-InstructBackbone=LLaMA3.1-8B-Instruct, Category=Instruct Models, Training Strategy=Vanilla2026.01 | 41.84 |