Reasoning on GSM→FOL
85.8AccuracyDIN-ICL
Evaluation results
| Method | Links | |
|---|---|---|
| DIN-ICLModel Series=Qwen-3, Parameters=8B2026.04 | 85.8 | |
| DIN-ICLModel Series=Qwen-3, Parameters=8B2026.04 | 85.8 | |
| DIN-ICLModel Series=Qwen-3, Parameters=14B2026.04 | 85 | |
| DIN-ICLModel Series=Qwen-3, Parameters=14B2026.04 | 85 | |
| ZERO-SHOTModel Series=Qwen-3, Parameters=14B2026.04 | 84.5 | |
| DIN-ICLModel Series=Qwen-3, Parameters=32B2026.04 | 84 | |
| DIN-ICLModel Series=Qwen-3, Parameters=32B2026.04 | 84 | |
| X-ICLModel Series=Qwen-3, Parameters=32B2026.04 | 83.6 | |
| SET-BSRModel Series=Qwen-3, Parameters=32B2026.04 | 83.6 | |
| X-ICLModel Series=Qwen-3, Parameters=14B2026.04 | 83.1 | |
| SET-BSRModel Series=Qwen-3, Parameters=14B2026.04 | 83.1 | |
| ZERO-SHOTModel Series=Qwen-3, Parameters=32B2026.04 | 82.2 | |
| ZERO-SHOTModel Series=Qwen-3, Parameters=8B2026.04 | 81.7 | |
| X-ICLModel Series=Qwen-3, Parameters=8B2026.04 | 81.2 | |
| SET-BSRModel Series=Qwen-3, Parameters=8B2026.04 | 81.2 | |
| DIN-ICLModel Series=Qwen-2.5, Parameters=32B2026.04 | 71.4 | |
| DIN-ICLModel Series=Qwen-2.5, Parameters=32B2026.04 | 71.4 | |
| DIN-ICLModel Series=Qwen-2.5, Parameters=14B2026.04 | 70.4 | |
| DIN-ICLModel Series=Qwen-2.5, Parameters=14B2026.04 | 70.4 | |
| ZERO-SHOTModel Series=Qwen-2.5, Parameters=32B2026.04 | 70.3 | |
| DIN-ICLModel Series=Gemma-3, Parameters=27B2026.04 | 68.9 | |
| DIN-ICLModel Series=Gemma-3, Parameters=27B2026.04 | 68.9 | |
| X-ICLModel Series=Gemma-3, Parameters=27B2026.04 | 68.4 | |
| SET-BSRModel Series=Gemma-3, Parameters=27B2026.04 | 68.4 | |
| ZERO-SHOTModel Series=Gemma-3, Parameters=27B2026.04 | 67.9 | |
| ZERO-SHOTModel Series=Qwen-2.5, Parameters=14B2026.04 | 67.4 | |
| X-ICLModel Series=Qwen-2.5, Parameters=14B2026.04 | 67.4 | |
| SET-BSRModel Series=Qwen-2.5, Parameters=14B2026.04 | 67.4 | |
| X-ICLModel Series=Qwen-2.5, Parameters=32B2026.04 | 66 | |
| SET-BSRModel Series=Qwen-2.5, Parameters=32B2026.04 | 66 | |
| DIN-ICLModel Series=Gemma-3, Parameters=12B2026.04 | 65.5 | |
| DIN-ICLModel Series=Gemma-3, Parameters=12B2026.04 | 65.5 | |
| DIN-ICLModel Series=Qwen-2.5, Parameters=7B2026.04 | 63.5 | |
| DIN-ICLModel Series=Qwen-2.5, Parameters=7B2026.04 | 63.5 | |
| DIN-ICLModel Series=LLaMA-3.1, Parameters=8B2026.04 | 63.3 | |
| DIN-ICLModel Series=LLaMA-3.1, Parameters=8B2026.04 | 63.3 | |
| X-ICLModel Series=Gemma-3, Parameters=12B2026.04 | 62.5 | |
| SET-BSRModel Series=Gemma-3, Parameters=12B2026.04 | 62.5 | |
| ZERO-SHOTModel Series=Qwen-2.5, Parameters=7B2026.04 | 61.8 | |
| ZERO-SHOTModel Series=Gemma-3, Parameters=12B2026.04 | 61 | |
| X-ICLModel Series=Qwen-2.5, Parameters=7B2026.04 | 59.7 | |
| SET-BSRModel Series=Qwen-2.5, Parameters=7B2026.04 | 59.7 | |
| ZERO-SHOTModel Series=LLaMA-3.1, Parameters=8B2026.04 | 56.3 | |
| X-ICLModel Series=LLaMA-3.1, Parameters=8B2026.04 | 55.5 | |
| SET-BSRModel Series=LLaMA-3.1, Parameters=8B2026.04 | 55.5 |