Simulatability on Tabular Datasets Aggregate (test)
74.63Accuracy (w/o explanation)Gemini 3
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| Gemini 3Variant=Flash2026.02 | 74.63 | 82.44 | 7.81 | — | 30.79 | — | |
| Claude 4.5Variant=Opus2026.02 | 74 | 81.38 | 7.38 | — | 28.38 | — | |
| Qwen 3Model Size=14B2026.02 | 72.89 | 82.08 | 9.19 | — | 33.9 | — | |
| Claude 4.5Variant=Sonnet2026.02 | 72.53 | 81.81 | 9.28 | — | 33.78 | — | |
| GPT-5.22026.02 | 72.5 | 82.05 | 9.55 | — | 34.72 | — | |
| GPT-5Variant=mini2026.02 | 72.38 | 81.84 | 9.46 | — | 34.25 | — | |
| Qwen 3Model Size=8B2026.02 | 72.15 | 80.3 | 8.14 | — | 29.24 | — | |
| Claude 4.5Variant=Haiku2026.02 | 71.98 | 79.49 | 7.51 | — | 26.8 | — | |
| Gemma 3Model Size=12B2026.02 | 71.64 | 81.21 | 9.57 | — | 33.74 | — | |
| Gemini 3Variant=Pro2026.02 | 71.62 | 79.68 | 8.06 | — | 28.38 | — | |
| Qwen 3Model Size=32B2026.02 | 71.18 | 81.44 | 10.27 | — | 35.61 | — | |
| Gemma 3Model Size=27B2026.02 | 71.01 | 81.14 | 10.12 | — | 34.92 | — | |
| Gemma 3Model Size=4B2026.02 | 70.37 | 81.19 | 10.82 | — | 36.51 | — | |
| GPT-5Variant=nano2026.02 | 69.48 | 79.49 | 10.02 | — | 32.81 | — | |
| Qwen 3Model Size=4B2026.02 | 69.25 | 76.69 | 7.44 | — | 24.19 | — | |
| Qwen 3Model Size=1.7B2026.02 | 66.17 | 71.5 | 5.34 | — | 15.77 | — | |
| Qwen 3Model Size=0.6B2026.02 | 65.73 | 69.51 | 3.78 | — | 11.03 | — | |
| Gemma 3Model Size=1B2026.02 | 63.8 | 70.14 | 6.34 | — | 17.51 | — |