Multilingual Reasoning on MT-AIME 24
44.4Accuracy (%)ROSA2
Evaluation Results
| Method | Links | |
|---|---|---|
| ROSA2Model=Qwen3-8B2026.03 | 44.4 | |
| ROSAModel=Qwen3-8B, Update Location=LM Head, Reward Model=Model-based2025.09 | 43.93 | |
| ROSAModel=Qwen3-8B, Update Location=Hidden States, Reward Model=Rule-based2025.09 | 42.27 | |
| ROSAModel=Qwen3-8B, Update Location=LM Head, Reward Model=Rule-based2025.09 | 40.16 | |
| TextGradModel=Qwen3-8B2026.03 | 40 | |
| ROSAModel=Qwen3-8B2026.03 | 38.6 | |
| ROSA2Model=Qwen2.5-7B-Base2026.03 | 37.8 | |
| TextGradModel=Qwen2.5-7B-Base2026.03 | 37.6 | |
| ROSAModel=Qwen2.5-7B-Base2026.03 | 37 | |
| BaselineModel=Qwen3-8B2025.09 | 30.37 | |
| ROSAModel=Qwen2.5-7B-Instruct, Update Location=LM Head, Reward Model=Model-based2025.09 | 30.21 | |
| BaselineModel=Qwen3-8B2026.03 | 29.4 | |
| ROSAModel=Qwen2.5-7B-Instruct, Update Location=LM Head, Reward Model=Rule-based2025.09 | 27.14 | |
| ROSAModel=Qwen2.5-7B-Instruct, Update Location=Hidden States, Reward Model=Rule-based2025.09 | 26.75 | |
| ROSA2Model=DeepSeek-R1-Distill-Llama-8B2026.03 | 21.4 | |
| BaselineModel=Qwen2.5-7B-Instruct2025.09 | 19.24 | |
| ROSAModel=DeepSeek-R1-Distill-Llama-8B2026.03 | 17.2 | |
| BaselineModel=Qwen2.5-7B-Base2026.03 | 17 | |
| TextGradModel=DeepSeek-R1-Distill-Llama-8B2026.03 | 16.2 | |
| ROSA2Model=Qwen3-0.6B-Instruct2026.03 | 9.6 | |
| ROSAModel=Qwen3-0.6B, Update Location=LM Head, Reward Model=Model-based2025.09 | 9.43 | |
| ROSAModel=DeepSeek-R1-Distill-Llama-8B, Update Location=LM Head, Reward Model=Model-based2025.09 | 8.19 | |
| ROSAModel=Qwen3-0.6B, Update Location=Hidden States, Reward Model=Rule-based2025.09 | 7.9 | |
| ROSAModel=Qwen3-0.6B, Update Location=LM Head, Reward Model=Rule-based2025.09 | 7.58 | |
| ROSAModel=DeepSeek-R1-Distill-Llama-8B, Update Location=Hidden States, Reward Model=Rule-based2025.09 | 7.27 | |
| ROSAModel=Qwen3-0.6B-Instruct2026.03 | 7.2 | |
| TextGradModel=Qwen3-0.6B-Instruct2026.03 | 7 | |
| ROSAModel=Qwen2.5-0.5B-Instruct, Update Location=LM Head, Reward Model=Model-based2025.09 | 6.13 | |
| ROSAModel=DeepSeek-R1-Distill-Llama-8B, Update Location=LM Head, Reward Model=Rule-based2025.09 | 6.13 | |
| ROSAModel=Qwen2.5-0.5B-Instruct, Update Location=Hidden States, Reward Model=Rule-based2025.09 | 4.9 | |
| BaselineModel=Qwen3-0.6B2025.09 | 4.8 | |
| BaselineModel=DeepSeek-R1-Distill-Llama-8B2026.03 | 4.8 | |
| BaselineModel=DeepSeek-R1-Distill-Llama-8B2025.09 | 4.73 | |
| ROSA2Model=Qwen2.5-0.5B-Instruct2026.03 | 4.4 | |
| ROSAModel=Qwen2.5-0.5B-Instruct2026.03 | 3.8 | |
| ROSAModel=Qwen2.5-0.5B-Instruct, Update Location=LM Head, Reward Model=Rule-based2025.09 | 3.67 | |
| BaselineModel=Qwen2.5-0.5B-Instruct2025.09 | 3.48 | |
| BaselineModel=Qwen3-0.6B-Instruct2026.03 | 3.2 | |
| BaselineModel=Qwen2.5-0.5B-Instruct2026.03 | 2.6 | |
| TextGradModel=Qwen2.5-0.5B-Instruct2026.03 | 2.2 |