Multitask Reasoning on MMLU-Pro (Accuracy)
61.1Accuracy (MMLU-Pro)SPIRAL
Evaluation results
| Method | Links | |
|---|---|---|
| SPIRALBackbone=Qwen3-8B, Training Data=Multi-game2025.06 | 61.1 | |
| SPIRALBackbone=DeepSeek-Distill-Qwen-7B, Training Data=Multi-game2025.06 | 58.9 | |
| SFTBackbone=Qwen3-8B, Training Data=Multi-game2025.06 | 58.8 | |
| SPIRALBackbone=Qwen3-4B, Training Data=Multi-game2025.06 | 58.5 | |
| SPIRALBackbone=Qwen3-4B, Training Data=Kuhn Poker2025.06 | 57.7 | |
| DeepSeek-Distill-Qwen-7BBackbone=DeepSeek-Distill-Qwen-7B2025.06 | 57.1 | |
| Qwen3-8B-BaseBackbone=Qwen3-8B, Evaluation Protocol=Few-shot2025.06 | 55.7 | |
| SFTBackbone=DeepSeek-Distill-Qwen-7B, Training Data=Multi-game2025.06 | 55.6 | |
| SFTBackbone=Qwen3-4B, Training Data=Multi-game2025.06 | 51.3 | |
| SPIRALBackbone=Llama-3.1-8B-Instruct, Training Data=Multi-game2025.06 | 50.4 | |
| SPIRALBackbone=Octothinker-8B, Training Data=Multi-game2025.06 | 49.3 | |
| Llama-3.1-8B-InstructBackbone=Llama-3.1-8B-Instruct2025.06 | 49.1 | |
| SFTBackbone=Llama-3.1-8B-Instruct, Training Data=Multi-game2025.06 | 48.9 | |
| SFTBackbone=Qwen3-4B, Training Data=Kuhn Poker2025.06 | 48.8 | |
| Qwen3-4B-BaseBackbone=Qwen3-4B, Evaluation Protocol=Few-shot2025.06 | 47.2 | |
| SFTBackbone=Octothinker-8B, Training Data=Multi-game2025.06 | 39.1 | |
| Octothinker-8B-BaseBackbone=Octothinker-8B2025.06 | 30.8 |