Zero-shot Language Understanding and Reasoning Suite (PIQA, ARC, HellaSwag, WinoGrande, BoolQ, LAMBADA, C4)
79.67PIQA AccuracyL2QER
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| L2QERRank=64, Zero-shot protocol=true, Base Model=LLaMA 3.1-8B2026.03 | 79.67 | 49.33 | 80.67 | 66.67 | 74.33 | 80.33 | 76 | 9.63 | 72.43 | |
| GlowQRank=64, Zero-shot protocol=true, Base Model=LLaMA 3.1-8B2026.03 | 79.67 | 51 | 81.33 | 66 | 74.33 | 82 | 79 | 9.59 | 73.33 | |
| GlowQ-SRank=64, Zero-shot protocol=true, Base Model=LLaMA 3.1-8B2026.03 | 79 | 50.33 | 81.67 | 66.33 | 72 | 82 | 77 | 9.78 | 72.62 | |
| FP16Rank=-, Zero-shot protocol=true, Base Model=LLaMA 3.1-8B2026.03 | 78.67 | 51.67 | 80.67 | 67.67 | 74.67 | 80.67 | 79 | 9 | 73.29 | |
| FP16Rank=-, zero-shot=true, Model=Qwen 3-14B2026.03 | 78.33 | 59.33 | 80.33 | 66.67 | 75.67 | 92 | 66.33 | 13.08 | 74.1 | |
| ZeroQuant-V2Rank=64, zero-shot=true, Model=Qwen 3-14B2026.03 | 78.33 | 59.33 | 78 | 65.67 | 73 | 92 | 62 | 13.79 | 72.62 | |
| L2QERRank=64, zero-shot=true, Model=Qwen 3-14B2026.03 | 78.33 | 56.33 | 79.67 | 66.67 | 75.33 | 91.67 | 64.67 | 13.8 | 73.24 | |
| ZeroQuant-V2Rank=64, Zero-shot protocol=true, Base Model=LLaMA 3.1-8B2026.03 | 78 | 51.33 | 81.67 | 68.67 | 76 | 84.33 | 74.33 | 9.87 | 73.48 | |
| GlowQzero-shot=true, Model=Qwen 3-14B2026.03 | 77.67 | 56.67 | 80 | 68.87 | 75.67 | 91.33 | 66.67 | 13.26 | 73.84 | |
| GlowQ-Szero-shot=true, Model=Qwen 3-14B2026.03 | 77.67 | 57 | 79.33 | 67.67 | 74.33 | 91.33 | 65.33 | 13.48 | 73.24 | |
| GlowQZero-shot evaluation protocol=true, Backbone=LLaMA 3.2-3B2026.03 | 77.67 | 39.67 | 72 | 64 | 70.33 | 74.33 | 70.33 | 10.98 | 66.9 | |
| FP16Rank=-, Zero-shot=true, Backbone=Qwen 3-8B2026.03 | 77.33 | 53 | 83 | 63.67 | 68.67 | 87 | 67.67 | 14.52 | 71.48 | |
| GlowQ-SZero-shot evaluation protocol=true, Backbone=LLaMA 3.2-3B2026.03 | 77.33 | 39.67 | 71.67 | 64 | 69.67 | 71.67 | 70.33 | 11.07 | 66.33 | |
| QERARank=64, Zero-shot protocol=true, Base Model=LLaMA 3.1-8B2026.03 | 77 | 51.33 | 80.33 | 69 | 74.33 | 82.67 | 75.33 | 9.68 | 72.86 | |
| QERAzero-shot=true, Model=Qwen 3-14B2026.03 | 76.98 | 57.67 | 79.33 | 67 | 74 | 92 | 65 | 13.29 | 73.14 | |
| GlowQZero-shot=true, Backbone=Qwen 3-8B2026.03 | 76.67 | 52.33 | 80.33 | 64.67 | 71 | 86.33 | 63.67 | 14.6 | 70.71 | |
| QERAZero-shot evaluation protocol=true, Backbone=LLaMA 3.2-3B2026.03 | 76.67 | 38.67 | 72.33 | 61.67 | 68.67 | 72.33 | 64.67 | 11.04 | 65.48 | |
| QERAZero-shot=true, Backbone=Qwen 3-8B2026.03 | 76.33 | 51.33 | 79 | 62.33 | 69.67 | 85.67 | 64.67 | 14.78 | 69.86 | |
| GlowQ-SZero-shot=true, Backbone=Qwen 3-8B2026.03 | 76.33 | 50.67 | 80.67 | 63.33 | 70.67 | 85 | 65.33 | 14.77 | 70.29 | |
| ZeroQuant-V2Rank=64, Zero-shot=true, Backbone=Qwen 3-8B2026.03 | 75.67 | 52.33 | 80.33 | 63 | 71 | 85.33 | 63.67 | 15 | 70.19 | |
| L2QERRank=64, Zero-shot=true, Backbone=Qwen 3-8B2026.03 | 75.67 | 51.33 | 79.33 | 62.67 | 67.67 | 85.33 | 64.67 | 14.82 | 69.52 | |
| ZeroQuant-V2Rank=64, Zero-shot evaluation protocol=true, Backbone=LLaMA 3.2-3B2026.03 | 75.33 | 39.33 | 73.33 | 60.67 | 68.67 | 74.67 | 65.67 | 11.45 | 65.38 | |
| L2QERRank=64, Zero-shot evaluation protocol=true, Backbone=LLaMA 3.2-3B2026.03 | 75.33 | 40 | 71.67 | 64 | 68.33 | 73.33 | 68.33 | 11.04 | 66.19 | |
| FP16Rank=-, Zero-shot evaluation protocol=true, Backbone=LLaMA 3.2-3B2026.03 | 72.33 | 39.33 | 72.33 | 63.67 | 70.33 | 77 | 71 | 10.3 | 67.14 |