Science on GPQA Diamond (Acc, AMSR)
67AccuracyVP2O
Evaluation results
| Method | Links | ||
|---|---|---|---|
| VP2OContext Length=16K2026.06 | 67 | — | |
| GRPO-baselineContext Length=16K2026.06 | 65.2 | — | |
| GRPO-baselineContext Length=8K2026.06 | 64.2 | — | |
| VP2OContext Length=8K2026.06 | 63.9 | — | |
| DenseModel=DeepSeek R1 Distill Qwen 32B2025.07 | 63.1 | 0 | |
| DenseModel=QwQ 32B2025.07 | 61.1 | 0 | |
| ReasonCacheModel=DeepSeek R1 Distill Qwen 32B2025.07 | 60.7 | 15.54 | |
| ReasonCacheModel=QwQ 32B2025.07 | 58.1 | 18.12 | |
| QuestModel=DeepSeek R1 Distill Qwen 32B2025.07 | 56.1 | 13.8 | |
| QuestModel=QwQ 32B2025.07 | 53.5 | 18.1 | |
| StreamingLLMModel=QwQ 32B2025.07 | 53 | 17.5 | |
| DenseModel=Phi 4 reasoning plus2025.07 | 52 | 0 | |
| ReasonCacheModel=Phi 4 reasoning plus2025.07 | 50.8 | 17.6 | |
| StreamingLLMModel=DeepSeek R1 Distill Qwen 32B2025.07 | 40.9 | 16.5 | |
| StreamingLLMModel=Phi 4 reasoning plus2025.07 | 33.3 | 18.9 | |
| QuestModel=Phi 4 reasoning plus2025.07 | 31.2 | 17.8 | |
| SnapKVModel=DeepSeek R1 Distill Qwen 32B2025.07 | 29.3 | 8.2 | |
| SnapKVModel=Phi 4 reasoning plus2025.07 | 25.3 | 7.29 | |
| SnapKVModel=QwQ 32B2025.07 | 14.6 | 4.27 |