Logical Reasoning on KK
47.5Test Accuracygemma-3-12b-it
Evaluation Results
| Method | Links | |
|---|---|---|
| gemma-3-12b-itBackbone=Gemma 3, RL Type=/, Supervision=/, Environment=/2026.05 | 47.5 | |
| wd1Generation Length=5122026.05 | 39.71 | |
| StableDRLGeneration Length=5122026.05 | 39.29 | |
| GDPOGeneration Length=5122026.05 | 38.86 | |
| SPGGeneration Length=5122026.05 | 38.29 | |
| wd1Generation Length=2562026.05 | 38 | |
| d1Generation Length=5122026.05 | 37.57 | |
| StableDRLGeneration Length=2562026.05 | 37.57 | |
| AdaBlock-dLLMGeneration Length=5122026.05 | 37.43 | |
| Diffu-GRPOGeneration Length=5122026.05 | 37.43 | |
| LLaDA-8B-InstructGeneration Length=5122026.05 | 37.29 | |
| ESPOGeneration Length=5122026.05 | 37.29 | |
| d1Generation Length=2562026.05 | 37 | |
| Diffu-GRPOGeneration Length=2562026.05 | 36.86 | |
| GDPOGeneration Length=2562026.05 | 36.86 | |
| SFTGeneration Length=5122026.05 | 36.57 | |
| SPGGeneration Length=2562026.05 | 36.57 | |
| ESPOGeneration Length=2562026.05 | 36.14 | |
| AdaBlock-dLLMGeneration Length=2562026.05 | 33.71 | |
| SimpleSEBackbone=Gemma 3, RL Type=Offline, Supervision=No, Environment=No2026.05 | 33.2 | |
| SFTGeneration Length=2562026.05 | 32 | |
| gemma-3-4b-itBackbone=Gemma 3, RL Type=/, Supervision=/, Environment=/2026.05 | 31 | |
| LLaDA-8B-InstructGeneration Length=2562026.05 | 30 | |
| Qwen2.5-14B-InstructBackbone=Qwen 2.5, RL Type=/, Supervision=/, Environment=/2026.05 | 26.4 | |
| Qwen2.5-7B-InstructBackbone=Qwen 2.5, RL Type=/, Supervision=/, Environment=/2026.05 | 18.1 | |
| SimpleSEBackbone=Qwen 2.5, RL Type=Offline, Supervision=No, Environment=No2026.05 | 17.6 | |
| Base + AZR-CoderBackbone=Qwen 2.5, RL Type=Online, Supervision=No, Environment=Yes2026.05 | 8.5 | |
| Base + AZRBackbone=Qwen 2.5, RL Type=Online, Supervision=No, Environment=Yes2026.05 | 5.1 |