Scientific Question Answering on GPQA Diamond (test)
49Pass@1Transformers
Evaluation results
| Method | Links | ||
|---|---|---|---|
| TransformersBackbone=R1-Qwen2026.01 | 49 | 349.4 | |
| H2OBackbone=R1-Qwen2026.01 | 48.1 | 593.6 | |
| DYNTSBackbone=R1-Qwen2026.01 | 48.1 | 625.7 | |
| R-KVBackbone=R1-Qwen2026.01 | 47.2 | 615.1 | |
| SnapKVBackbone=R1-Qwen2026.01 | 46.5 | 622 | |
| TransformersBackbone=R1-Llama2026.01 | 46.4 | 207.9 | |
| DYNTSBackbone=R1-Llama2026.01 | 46.3 | 397.6 | |
| WindowBackbone=R1-Qwen2026.01 | 45.9 | 634.1 | |
| StreamingLLMBackbone=R1-Qwen2026.01 | 45.9 | 631.1 | |
| SepLLMBackbone=R1-Qwen2026.01 | 45.6 | 639.5 | |
| R-KVBackbone=R1-Llama2026.01 | 44.5 | 394.9 | |
| SnapKVBackbone=R1-Llama2026.01 | 41.9 | 394.1 | |
| H2OBackbone=R1-Llama2026.01 | 41.6 | 381.5 | |
| SepLLMBackbone=R1-Llama2026.01 | 39.7 | 406.6 | |
| StreamingLLMBackbone=R1-Llama2026.01 | 37.8 | 407.4 | |
| WindowBackbone=R1-Llama2026.01 | 37.6 | 408.8 |