General Language Understanding on LAMBADA, ARC-C, HellaSwag, BoolQ, MMLU, and GSM8K
74.35AccuracyBF16-Baseline
Evaluation results
| Method | Links | |
|---|---|---|
| BF16-BaselineModel=Llama-3.1-8B-Instruct, Eff. Bits=162026.06 | 74.35 | |
| GRINQH-8b-RTNModel=Llama-3.1-8B-Instruct, Eff. Bits=4.012026.06 | 74.34 | |
| GRINQH-8b-AnyPModel=Llama-3.1-8B-Instruct, Eff. Bits=3.892026.06 | 74.19 | |
| BF16-BaselineModel=Qwen3-8B, Eff. Bits=162026.06 | 74.02 | |
| GRINQH-6b-RTNModel=Llama-3.1-8B-Instruct, Eff. Bits=3.922026.06 | 73.91 | |
| GRINQH-8b-AnyPModel=Qwen3-8B, Eff. Bits=3.952026.06 | 73.91 | |
| GRINQH-6b-RTNModel=Qwen3-8B, Eff. Bits=3.942026.06 | 73.81 | |
| GRINQH-8b-AnyPModel=Qwen3-8B, Eff. Bits=3.062026.06 | 73.74 | |
| GRINQH-8b-RTNModel=Qwen3-8B, Eff. Bits=3.972026.06 | 73.7 | |
| GRINQH-8b-AnyPModel=Llama-3.1-8B-Instruct, Eff. Bits=2.962026.06 | 73.69 | |
| NVFP4-G16Model=Qwen3-8B, Eff. Bits=42026.06 | 73.36 | |
| AnyPrecLLMModel=Qwen3-8B, Eff. Bits=42026.06 | 73.36 | |
| GRINQH-8b-RTNModel=Qwen3-8B, Eff. Bits=3.072026.06 | 73.33 | |
| GRINQH-8b-AnyPModel=Qwen3-8B, Eff. Bits=2.222026.06 | 73.27 | |
| NVFP4-G16Model=Llama-3.1-8B-Instruct, Eff. Bits=42026.06 | 73.24 | |
| GRINQH-8b-RTNModel=Llama-3.1-8B-Instruct, Eff. Bits=2.982026.06 | 73.19 | |
| GRINQH-6b-RTNModel=Qwen3-8B, Eff. Bits=3.062026.06 | 73.14 | |
| QuaRot+GPTQModel=Qwen3-8B, Eff. Bits=42026.06 | 72.99 | |
| QuaRot+GPTQModel=Llama-3.1-8B-Instruct, Eff. Bits=42026.06 | 72.86 | |
| GPTQModel=Llama-3.1-8B-Instruct, Eff. Bits=42026.06 | 72.85 | |
| AnyPrecLLMModel=Llama-3.1-8B-Instruct, Eff. Bits=42026.06 | 72.84 | |
| SliM-LLMModel=Llama-3.1-8B-Instruct, Eff. Bits=42026.06 | 72.82 | |
| GRINQH-6b-RTNModel=Llama-3.1-8B-Instruct, Eff. Bits=3.022026.06 | 72.75 | |
| SliM-LLMModel=Qwen3-8B, Eff. Bits=42026.06 | 72.5 | |
| GRINQH-4b-GPTQModel=Llama-3.1-8B-Instruct, Eff. Bits=3.042026.06 | 72.34 | |
| AutoRoundModel=Llama-3.1-8B-Instruct, Eff. Bits=42026.06 | 72.25 | |
| GPTQModel=Qwen3-8B, Eff. Bits=42026.06 | 72.18 | |
| AWQModel=Llama-3.1-8B-Instruct, Eff. Bits=42026.06 | 71.98 | |
| AutoRoundModel=Qwen3-8B, Eff. Bits=42026.06 | 71.79 | |
| AWQModel=Qwen3-8B, Eff. Bits=42026.06 | 71.77 | |
| GRINQH-4b-GPTQModel=Qwen3-8B, Eff. Bits=3.032026.06 | 71.75 | |
| GRINQH-8b-RTNModel=Qwen3-8B, Eff. Bits=2.252026.06 | 71.6 | |
| RTNModel=Llama-3.1-8B-Instruct, Eff. Bits=42026.06 | 71.44 | |
| GRINQH-4b-RTNModel=Llama-3.1-8B-Instruct, Eff. Bits=3.022026.06 | 71.34 | |
| GRINQH-4b-RTNModel=Qwen3-8B, Eff. Bits=3.032026.06 | 70.94 | |
| RTNModel=Qwen3-8B, Eff. Bits=42026.06 | 69.89 | |
| AnyPrecLLMModel=Qwen3-8B, Eff. Bits=32026.06 | 69.28 | |
| GRINQH-8b-AnyPModel=Llama-3.1-8B-Instruct, Eff. Bits=2.012026.06 | 69.06 | |
| SliM-LLMModel=Qwen3-8B, Eff. Bits=32026.06 | 67.64 | |
| SliM-LLMModel=Llama-3.1-8B-Instruct, Eff. Bits=32026.06 | 66.35 | |
| AutoRoundModel=Qwen3-8B, Eff. Bits=32026.06 | 65.45 | |
| QuaRot+GPTQModel=Qwen3-8B, Eff. Bits=32026.06 | 65.03 | |
| GPTQModel=Qwen3-8B, Eff. Bits=32026.06 | 64.67 | |
| AnyPrecLLMModel=Llama-3.1-8B-Instruct, Eff. Bits=32026.06 | 64.59 | |
| QuaRot+GPTQModel=Llama-3.1-8B-Instruct, Eff. Bits=32026.06 | 63.4 | |
| GPTQModel=Llama-3.1-8B-Instruct, Eff. Bits=32026.06 | 63.27 | |
| AutoRoundModel=Llama-3.1-8B-Instruct, Eff. Bits=32026.06 | 63 | |
| GRINQH-8b-RTNModel=Llama-3.1-8B-Instruct, Eff. Bits=2.112026.06 | 61.37 | |
| AWQModel=Qwen3-8B, Eff. Bits=32026.06 | 59.19 | |
| AWQModel=Llama-3.1-8B-Instruct, Eff. Bits=32026.06 | 56.05 | |
| RTNModel=Llama-3.1-8B-Instruct, Eff. Bits=32026.06 | 50.64 | |
| RTNModel=Qwen3-8B, Eff. Bits=32026.06 | 44.79 | |
| AnyPrecLLMModel=Qwen3-8B, Eff. Bits=22026.06 | 33.8 | |
| SliM-LLMModel=Qwen3-8B, Eff. Bits=22026.06 | 31.44 | |
| AnyPrecLLMModel=Llama-3.1-8B-Instruct, Eff. Bits=22026.06 | 22.22 | |
| SliM-LLMModel=Llama-3.1-8B-Instruct, Eff. Bits=22026.06 | 19.59 |