Question Answering on ARC Challenge (Accuracy and Accuracy Drop)
73.21AccuracyMXFP4-W2FP8+Dyn
Evaluation results
| Method | Links | ||
|---|---|---|---|
| MXFP4-W2FP8+DynModel=Qwen3-30B2026.04 | 73.21 | 0.43 | |
| W8A8-DirectModel=Qwen3-30B2026.04 | 72.87 | 0.09 | |
| W16A16-DirectModel=Qwen3-30B2026.04 | 72.78 | 0 | |
| MXFP4-OSCModel=Qwen3-30B2026.04 | 72.61 | -0.17 | |
| MXFP4-DynamicModel=Qwen3-30B2026.04 | 71.76 | -1.02 | |
| MXFP4-W2FP8Model=Qwen3-30B2026.04 | 71.5 | -1.28 | |
| MXFP4-DirectModel=Qwen3-30B2026.04 | 70.31 | -2.47 | |
| W16A16-DirectModel=Qwen3-8B2026.04 | 67.58 | 0 | |
| W8A8-DirectModel=Qwen3-8B2026.04 | 67.58 | 0 | |
| MXFP4-OSCModel=Qwen3-8B2026.04 | 65.1 | -2.47 | |
| MXFP4-W2FP8Model=Qwen3-8B2026.04 | 64.42 | -3.16 | |
| MXFP4-W2FP8+DynModel=Qwen3-8B2026.04 | 63.99 | -3.58 | |
| MXFP4-DynamicModel=Qwen3-8B2026.04 | 63.99 | -3.58 | |
| MXFP4-DirectModel=Qwen3-8B2026.04 | 61.43 | -6.14 |