Zero-shot Evaluation on Zero-Shot Aggregate
59.52AccuracyFloat
Evaluation results
| Method | Links | |
|---|---|---|
| FloatModel=SmolLM2 1.7B, Average Bit-Width=BF162026.06 | 59.52 | |
| dMXModel=SmolLM2 1.7B, Average Bit-Width=5.542026.06 | 58.25 | |
| MXFP6Model=SmolLM2 1.7B, Average Bit-Width=6.02026.06 | 58.18 | |
| dMXModel=SmolLM2 1.7B, Average Bit-Width=5.062026.06 | 57.52 | |
| dMXModel=SmolLM2 1.7B, Average Bit-Width=4.522026.06 | 56.4 | |
| dMXModel=Qwen3 1.7B, Average Bit-Width=5.692026.06 | 54.86 | |
| dMXModel=Qwen3 1.7B, Average Bit-Width=5.032026.06 | 54.14 | |
| FloatModel=Qwen3 1.7B, Average Bit-Width=BF162026.06 | 54.12 | |
| MXFP6Model=Qwen3 1.7B, Average Bit-Width=6.02026.06 | 54.05 | |
| MXFP4Model=SmolLM2 1.7B, Average Bit-Width=4.02026.06 | 53.42 | |
| dMXModel=Qwen3 1.7B, Average Bit-Width=4.522026.06 | 52.9 | |
| FloatModel=Llama 3.2 1B, Average Bit-Width=BF162026.06 | 51.53 | |
| MXFP6Model=Llama 3.2 1B, Average Bit-Width=6.02026.06 | 51.39 | |
| MXFP4Model=Qwen3 1.7B, Average Bit-Width=4.02026.06 | 51.15 | |
| dMXModel=Llama 3.2 1B, Average Bit-Width=5.982026.06 | 50.86 | |
| dMXModel=Llama 3.2 1B, Average Bit-Width=5.022026.06 | 50.26 | |
| dMXModel=Llama 3.2 1B, Average Bit-Width=4.522026.06 | 49.43 | |
| MXFP4Model=Llama 3.2 1B, Average Bit-Width=4.02026.06 | 47.82 |