Pass@1 Evaluation on BBEH General Reasoning
9.94Pass@1OpenSIR
Evaluation Results
| Method | Links | |
|---|---|---|
| OpenSIRBackbone=Llama-3.1-8B-Instruct2025.11 | 9.94 | |
| GRPOgsm8kBackbone=Llama-3.1-8B-Instruct2025.11 | 7.93 | |
| GRPOmathBackbone=Llama-3.1-8B-Instruct2025.11 | 7.79 | |
| OpenSIRBackbone=Llama-3.2-3B-Instruct2025.11 | 7.13 | |
| Absolute ZeroBackbone=Llama-3.1-8B-Instruct2025.11 | 6.76 | |
| R-ZeroBackbone=Llama-3.1-8B-Instruct2025.11 | 6.63 | |
| BaseBackbone=Llama-3.1-8B-Instruct2025.11 | 6.48 | |
| GRPOgsm8kBackbone=Llama-3.2-3B-Instruct2025.11 | 6.12 | |
| GRPOmathBackbone=Llama-3.2-3B-Instruct2025.11 | 5.98 | |
| R-ZeroBackbone=Llama-3.2-3B-Instruct2025.11 | 4.73 | |
| Absolute ZeroBackbone=Llama-3.2-3B-Instruct2025.11 | 4.54 | |
| BaseBackbone=Llama-3.2-3B-Instruct2025.11 | 4.49 |