Question Answering on ARC (test) with Generation Metrics
93Pass@1GRPO
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| GRPOBackbone=Qwen3-4B-Instruct2026.02 | 93 | 161.94 | 8.04 | |
| DRGRPOBackbone=Qwen3-4B-Instruct2026.02 | 92.6 | 268.47 | 12.34 | |
| GRPOBackbone=Qwen2.5-7B-Instruct2026.02 | 90.4 | 309.3 | 14.72 | |
| DRGRPOBackbone=Qwen2.5-7B-Instruct2026.02 | 90.4 | 226.11 | 11.51 | |
| WS-GRPOBackbone=Qwen3-4B-Instruct2026.02 | 88.6 | 20.23 | 2 | |
| WS-GRPOBackbone=Qwen2.5-7B-Instruct2026.02 | 87.9 | 16 | 2 | |
| DRGRPOBackbone=Llama-3.1-8B-Instruct2026.02 | 84.1 | 8 | 1 | |
| WS-GRPOBackbone=Llama-3.1-8B-Instruct2026.02 | 82.4 | 20 | 2 | |
| GRPOBackbone=Llama-3.1-8B-Instruct2026.02 | 81.6 | 13.05 | 1 | |
| GRPOBackbone=Llama-3.2-3B-Instruct2026.02 | 79.4 | 137.2 | 5.07 | |
| DRGRPOBackbone=Llama-3.2-3B-Instruct2026.02 | 78.2 | 97.91 | 4.22 | |
| WS-GRPOBackbone=Llama-3.2-3B-Instruct2026.02 | 74.8 | 14 | 2.04 |