Code Reasoning on CRUX
87.37AccuracyRMoA
Evaluation Results
| Method | Links | |
|---|---|---|
| RMoAModel=GPT-4o2025.05 | 87.37 | |
| SMoAModel=GPT-4o2025.05 | 86.93 | |
| MoAModel=GPT-4o2025.05 | 86.66 | |
| GPT-4oModel=GPT-4o2025.05 | 75.8 | |
| RMoAModel=Qwen2.5-7B-Instruct2025.05 | 61 | |
| SMoAModel=Qwen2.5-7B-Instruct2025.05 | 59.93 | |
| Qwen2.5-7B-InstructModel=Qwen2.5-7B-Instruct2025.05 | 57.31 | |
| MoAModel=Qwen2.5-7B-Instruct2025.05 | 56.81 | |
| MoAModel=Gemma2-9B-Instruct2025.05 | 51.5 | |
| SMoAModel=Gemma2-9B-Instruct2025.05 | 51.25 | |
| RMoAModel=Gemma2-9B-Instruct2025.05 | 50.5 | |
| Gemma2-9B-InstructModel=Gemma2-9B-Instruct2025.05 | 47.5 | |
| MoAModel=Llama3.1-8B-Instruct2025.05 | 46.12 | |
| SMoAModel=Llama3.1-8B-Instruct2025.05 | 44.81 | |
| RMoAModel=Llama3.1-8B-Instruct2025.05 | 42.65 | |
| Llama3.1-8B-InstructModel=Llama3.1-8B-Instruct2025.05 | 40.62 | |
| INTUITORBackbone=Qwen2.5-3B-Base, Training Dataset=CodeContests, RL Strategy=INTUITOR2026.05 | 39.38 | |
| VIGORBackbone=Qwen2.5-3B-Base, Training Dataset=CodeContests, RL Strategy=VIGOR2026.05 | 35.62 | |
| INTUITORModel=Llama3.2-3B-Ins2025.05 | 29.3 | |
| GRPO-PVModel=Llama3.2-3B-Ins2025.05 | 28.1 | |
| GRPOModel=Llama3.2-3B-Ins2025.05 | 26.6 | |
| BaselineModel=Llama3.2-3B-Ins2025.05 | 26.5 | |
| Before RL (Base)Backbone=Qwen2.5-3B-Base, Training Dataset=CodeContests, RL Strategy=None2026.05 | 24.38 | |
| BaselineModel=OLMo2-7B-SFT2025.05 | 23.8 | |
| GRPOModel=OLMo2-7B-SFT2025.05 | 21.8 | |
| INTUITORModel=OLMo2-7B-SFT2025.05 | 21.5 |