Question Answering on ARC Challenge (Accuracy, Drop, TC2b)
95.22AccuracyQwen3-32B (FP16)
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Qwen3-32B (FP16)Model=Qwen3-32B, Precision=FP162026.06 | 95.22 | — | — | |
| Qwen3-32B (+P)Model=Qwen3-32B, Precision=2-bit + FP16 plan2026.06 | 93.77 | -1.79 | 99.83 | |
| Qwen3-32B (+P + L)Model=Qwen3-32B, Precision=2-bit + FP16 plan + loop rescue2026.06 | 93.69 | -1.79 | 99.83 | |
| Qwen3-32B (+L)Model=Qwen3-32B, Precision=2-bit + loop rescue2026.06 | 93.6 | -1.79 | 99.83 | |
| Qwen3-32B (2bit)Model=Qwen3-32B, Precision=2-bit2026.06 | 93.43 | -1.79 | 99.83 | |
| Qwen3-8B (FP16)Model=Qwen3-8B, Precision=FP162026.06 | 93.09 | — | — | |
| Qwen3-8B (+P)Model=Qwen3-8B, Precision=2-bit + FP16 plan2026.06 | 81.31 | -17.49 | 76.02 | |
| Qwen3-8B (+P + L)Model=Qwen3-8B, Precision=2-bit + FP16 plan + loop rescue2026.06 | 80.03 | -17.49 | 76.02 | |
| Qwen3-8B (+L)Model=Qwen3-8B, Precision=2-bit + loop rescue2026.06 | 76.44 | -17.49 | 76.02 | |
| Qwen3-8B (2bit)Model=Qwen3-8B, Precision=2-bit2026.06 | 75.6 | -17.49 | 76.02 |