Zero-shot Prediction on HellaSwag
76.36Zero-shot HellaSwag AccuracyBF16
Evaluation Results
| Method | Links | |
|---|---|---|
| BF16Model=Ouro 2.6B, Quantization=W4A42026.05 | 76.36 | |
| LoopQModel=Ouro 2.6B, Quantization=W4A82026.05 | 75.55 | |
| BF16Model=Ouro 1.4B, Quantization=W4A42026.05 | 71.59 | |
| BF16Model=Ouro 1.4B, Quantization=W4A82026.05 | 71.59 | |
| LoopQModel=Ouro 1.4B, Quantization=W4A82026.05 | 69.54 | |
| FlatQuantModel=Ouro 1.4B, Quantization=W4A82026.05 | 69.48 | |
| LoopQModel=Ouro 2.6B, Quantization=W4A42026.05 | 68.6 | |
| QuaRotModel=Ouro 1.4B, Quantization=W4A82026.05 | 68.17 | |
| LoopQModel=Ouro 1.4B, Quantization=W4A42026.05 | 65.82 | |
| SmoothQuantModel=Ouro 1.4B, Quantization=W4A82026.05 | 60.1 | |
| DenseSparsity=0%, Backbone=LLaMA-2-7B, Zero-shot=true2025.10 | 57.14 | |
| SymmetricModel=Ouro 1.4B, Quantization=W4A82026.05 | 50.22 | |
| MagnitudeSparsity=4:8, Backbone=LLaMA-2-7B, Zero-shot=true2025.10 | 50.05 | |
| ELSASparsity=4:8, Backbone=LLaMA-2-7B, Zero-shot=true2025.10 | 49.9 | |
| L-ADMMSparsity=4:8, Backbone=LLaMA-2-7B, Zero-shot=true2025.10 | 49.45 | |
| ALPSSparsity=2:4, Backbone=LLaMA-2-7B, Zero-shot=true2025.10 | 49.4 | |
| SAFESparsity=4:8, Backbone=LLaMA-2-7B, Zero-shot=true2025.10 | 48.55 | |
| SpinQuantModel=Ouro 1.4B, Quantization=W4A82026.05 | 48.38 | |
| SparseLLMSparsity=4:8, Backbone=LLaMA-2-7B, Zero-shot=true2025.10 | 48.28 | |
| SparseGPTSparsity=4:8, Backbone=LLaMA-2-7B, Zero-shot=true2025.10 | 48.26 | |
| ELSASparsity=2:4, Backbone=LLaMA-2-7B, Zero-shot=true2025.10 | 47.87 | |
| WandaSparsity=4:8, Backbone=LLaMA-2-7B, Zero-shot=true2025.10 | 46.98 | |
| ALPSSparsity=4:8, Backbone=LLaMA-2-7B, Zero-shot=true2025.10 | 45.96 | |
| MagnitudeSparsity=2:4, Backbone=LLaMA-2-7B, Zero-shot=true2025.10 | 45.45 | |
| L-ADMMSparsity=2:4, Backbone=LLaMA-2-7B, Zero-shot=true2025.10 | 45.05 | |
| SAFESparsity=2:4, Backbone=LLaMA-2-7B, Zero-shot=true2025.10 | 44.66 | |
| LoopQModel=Parcae 370M, Quantization=W4A82026.05 | 44.11 | |
| SparseLLMSparsity=2:4, Backbone=LLaMA-2-7B, Zero-shot=true2025.10 | 43.62 | |
| SparseGPTSparsity=2:4, Backbone=LLaMA-2-7B, Zero-shot=true2025.10 | 43.47 | |
| LoopQModel=Parcae 370M, Quantization=W4A42026.05 | 41.74 | |
| WandaSparsity=2:4, Backbone=LLaMA-2-7B, Zero-shot=true2025.10 | 41.28 | |
| LoopQModel=LoopFormer 3×8, Quantization=W4A82026.05 | 31.99 | |
| TMMFormer2026.05 | 31.82 | |
| LoopQModel=LoopFormer 3×8, Quantization=W4A42026.05 | 31.79 | |
| AdamFormer2026.05 | 30.96 | |
| VanillaTransformer2026.05 | 30.2 | |
| AdamWFormer2026.05 | 30.08 | |
| SpinQuantModel=Ouro 1.4B, Quantization=W4A42026.05 | 29.14 | |
| QuaRotModel=Ouro 1.4B, Quantization=W4A42026.05 | 29.08 | |
| SymmetricModel=Ouro 2.6B, Quantization=W4A42026.05 | 28.11 | |
| SymmetricModel=Ouro 1.4B, Quantization=W4A42026.05 | 27.77 | |
| FlatQuantModel=Ouro 1.4B, Quantization=W4A42026.05 | 26.98 | |
| SmoothQuantModel=Ouro 1.4B, Quantization=W4A42026.05 | 25.75 |