Multistep Reasoning on MuSR
73.33AccuracySpurious Universal
Evaluation Results
| Method | Links | |
|---|---|---|
| Spurious UniversalModel=Qwen/Qwen3.5-27B2026.05 | 73.33 | |
| PromptWizardModel=Qwen/Qwen3.5-27B2026.05 | 71.33 | |
| Step-BackModel=Qwen/Qwen3.5-27B2026.05 | 70 | |
| Plan-and-SolveModel=Qwen/Qwen3.5-27B2026.05 | 70 | |
| SpuriousModel=Qwen/Qwen3.5-27B2026.05 | 69.33 | |
| PromptWizardModel=allenai/Olmo-3-7B-Instruct2026.05 | 67.33 | |
| SpuriousModel=allenai/Olmo-3-7B-Instruct2026.05 | 66 | |
| Re-ReadingModel=meta-llama/Llama-3.2-1B-Instruct2026.05 | 64 | |
| Re-ReadingModel=Qwen/Qwen3.5-27B2026.05 | 64 | |
| Plan-and-SolveModel=meta-llama/Llama-3.2-1B-Instruct2026.05 | 62 | |
| Spurious UniversalModel=allenai/Olmo-3-7B-Instruct2026.05 | 62 | |
| Least-to-MostModel=Qwen/Qwen3.5-27B2026.05 | 62 | |
| SpuriousModel=meta-llama/Llama-3.2-1B-Instruct2026.05 | 60.67 | |
| Zero-Shot CoTModel=Qwen/Qwen3.5-0.8B2026.05 | 60 | |
| SpuriousModel=Qwen/Qwen3.5-0.8B2026.05 | 59.2 | |
| Plan-and-SolveModel=Qwen/Qwen3.5-0.8B2026.05 | 58 | |
| AnalogicalModel=Qwen/Qwen3.5-27B2026.05 | 58 | |
| Zero-Shot CoTModel=Qwen/Qwen3.5-27B2026.05 | 58 | |
| Step-BackModel=meta-llama/Llama-3.2-1B-Instruct2026.05 | 56 | |
| AnalogicalModel=meta-llama/Llama-3.2-1B-Instruct2026.05 | 56 | |
| Self-AskModel=meta-llama/Llama-3.2-1B-Instruct2026.05 | 56 | |
| Self-AskModel=allenai/Olmo-3-7B-Instruct2026.05 | 56 | |
| Step-BackModel=Qwen/Qwen3.5-0.8B2026.05 | 54 | |
| Spurious UniversalModel=Qwen/Qwen3.5-0.8B2026.05 | 54 | |
| PromptWizardModel=meta-llama/Llama-3.2-1B-Instruct2026.05 | 54 | |
| Spurious UniversalModel=meta-llama/Llama-3.2-1B-Instruct2026.05 | 54 | |
| Step-BackModel=allenai/Olmo-3-7B-Instruct2026.05 | 54 | |
| Re-ReadingModel=allenai/Olmo-3-7B-Instruct2026.05 | 54 | |
| Self-AskModel=Qwen/Qwen3.5-27B2026.05 | 54 | |
| RTNModel=Qwen3.5-27B, Group Size (g)=162026.05 | 53.7 | |
| PromptWizardModel=Qwen/Qwen3.5-0.8B2026.05 | 53.33 | |
| BF16Model=Qwen3.5-27B2026.05 | 52.5 | |
| AWQModel=Qwen3.5-27B, Group Size (g)=162026.05 | 52 | |
| Self-AskModel=Qwen/Qwen3.5-0.8B2026.05 | 52 | |
| Zero-Shot CoTModel=meta-llama/Llama-3.2-1B-Instruct2026.05 | 52 | |
| AAACModel=Qwen3.5-27B, Group Size (g)=162026.05 | 51.6 | |
| IF4Model=Qwen3.5-27B, Group Size (g)=162026.05 | 51.3 | |
| AnalogicalModel=Qwen/Qwen3.5-0.8B2026.05 | 50 | |
| AnalogicalModel=allenai/Olmo-3-7B-Instruct2026.05 | 50 | |
| Re-ReadingModel=Qwen/Qwen3.5-0.8B2026.05 | 48 | |
| Least-to-MostModel=Qwen/Qwen3.5-0.8B2026.05 | 46 | |
| Plan-and-SolveModel=allenai/Olmo-3-7B-Instruct2026.05 | 46 | |
| Zero-Shot CoTModel=allenai/Olmo-3-7B-Instruct2026.05 | 46 | |
| AWQModel=Qwen3.5-9B, Group Size (g)=162026.05 | 44.2 | |
| AAACModel=Qwen3.5-9B, Group Size (g)=162026.05 | 44.2 | |
| BF16Model=Qwen3.5-9B2026.05 | 43.4 | |
| RTNModel=Qwen3.5-9B, Group Size (g)=162026.05 | 42.1 | |
| Least-to-MostModel=meta-llama/Llama-3.2-1B-Instruct2026.05 | 42 | |
| Least-to-MostModel=allenai/Olmo-3-7B-Instruct2026.05 | 42 | |
| SePTBackbone=Qwen2.5-Math-7B2025.10 | 41.5 | |
| BaseBackbone=Qwen2.5-Math-7B2025.10 | 41.4 | |
| GRPOBackbone=Qwen2.5-Math-7B2025.10 | 41.4 | |
| IF4Model=Qwen3.5-9B, Group Size (g)=162026.05 | 40.5 |