Science Question Answering on GPQA D (val)
65.9Accuracy (%)Base
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| BaseBackbone=Qwen3-32B, Generations=642026.04 | 65.9 | 1.89 | 9.7 | |
| ControlBackbone=Qwen3-32B, Generations=642026.04 | 64.6 | 2.45 | 15.7 | |
| BaseBackbone=Phi-4-r (14B), Generations=642026.04 | 64.1 | 0.8 | 3.7 | |
| ControlBackbone=Phi-4-r (14B), Generations=642026.04 | 64.1 | 2.11 | 14.2 | |
| Mem. + RLBackbone=Qwen3-8B, Generations=642026.04 | 62.9 | 1.24 | 9.2 | |
| MEMENTOBackbone=Qwen3-32B, Generations=642026.04 | 62.1 | 1.07 | 7.6 | |
| MEMENTOBackbone=Phi-4-r (14B), Generations=642026.04 | 61.6 | 0.8 | 6.2 | |
| BaseBackbone=Qwen3-8B, Generations=642026.04 | 61.4 | 1.23 | 6.6 | |
| ControlBackbone=Qwen3-8B, Generations=642026.04 | 57.8 | 1.6 | 11.8 | |
| MEMENTOBackbone=Qwen3-8B, Generations=642026.04 | 55.8 | 0.56 | 4 | |
| BaseBackbone=Olmo 3 (7B), Generations=642026.04 | 50.8 | 3.21 | 30.8 | |
| MEMENTOBackbone=Olmo 3 (7B), Generations=642026.04 | 49.5 | 2.72 | 25.2 | |
| ControlBackbone=Olmo 3 (7B), Generations=642026.04 | 45.7 | 2.94 | 23.8 |