Large Language Model Evaluation on 10 tasks average
70.56Avg AccuracyDeltaLoss-only
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| DeltaLoss-onlyAvg. Bits=6, Model=Llama3.1-70B-I2025.12 | 70.56 | 100.8 | |
| SignRoundV2Avg. Bits=6, Model=Llama3.1-70B-I2025.12 | 70.5 | 100.71 | |
| SignRoundV2Avg. Bits=5, Model=Llama3.1-70B-I2025.12 | 70.31 | 100.45 | |
| DeltaLoss-onlyAvg. Bits=5, Model=Llama3.1-70B-I2025.12 | 70.1 | 100.15 | |
| 16bitsAvg. Bits=16, Model=Llama3.1-70B-I2025.12 | 70 | — | |
| SignRoundV2Avg. Bits=4.5, Model=Llama3.1-70B-I2025.12 | 69.96 | 99.95 | |
| DeltaLoss-onlyAvg. Bits=4.5, Model=Llama3.1-70B-I2025.12 | 69.78 | 99.7 | |
| SignRoundV2Avg. Bits=4, Model=Llama3.1-70B-I2025.12 | 69.32 | 99.04 | |
| SignRoundV1Avg. Bits=4, Model=Llama3.1-70B-I2025.12 | 69.01 | 98.6 | |
| RTNAvg. Bits=4, Model=Llama3.1-70B-I2025.12 | 68.71 | 98.16 | |
| SignRoundV2Avg. Bits=6, Model=Qwen3-32B2025.12 | 67.21 | 100.32 | |
| SignRoundV2Avg. Bits=5, Model=Qwen3-32B2025.12 | 67.17 | 100.25 | |
| 16bitsAvg. Bits=16, Model=Qwen3-32B2025.12 | 67 | — | |
| SignRoundV1Avg. Bits=4, Model=Qwen3-32B2025.12 | 66.92 | 99.88 | |
| SignRoundV2Avg. Bits=4.5, Model=Qwen3-32B2025.12 | 66.89 | 99.83 | |
| SignRoundV2Avg. Bits=4, Model=Qwen3-32B2025.12 | 66.86 | 99.79 | |
| DeltaLoss-onlyAvg. Bits=6, Model=Qwen3-32B2025.12 | 66.61 | 99.42 | |
| DeltaLoss-onlyAvg. Bits=4.5, Model=Qwen3-32B2025.12 | 66.05 | 98.58 | |
| DeltaLoss-onlyAvg. Bits=5, Model=Qwen3-32B2025.12 | 66 | 98.51 | |
| 16bitsAvg. Bits=16, Model=Qwen2.5-7B-I2025.12 | 65.67 | — | |
| DeltaLoss-onlyAvg. Bits=6, Model=Qwen2.5-7B-I2025.12 | 65.45 | 99.67 | |
| SignRoundV2Avg. Bits=6, Model=Qwen2.5-7B-I2025.12 | 65.3 | 99.43 | |
| RTNAvg. Bits=4, Model=Qwen3-32B2025.12 | 65.07 | 97.12 | |
| SignRoundV2Avg. Bits=5, Model=Qwen2.5-7B-I2025.12 | 65.04 | 99.03 | |
| SignRoundV2Avg. Bits=4.5, Model=Qwen2.5-7B-I2025.12 | 64.51 | 98.24 | |
| 16bitsAvg. Bits=16, Model=Llama3.1-8B-I2025.12 | 64.16 | — | |
| SignRoundV2Avg. Bits=6, Model=Llama3.1-8B-I2025.12 | 64.12 | 99.93 | |
| SignRoundV1Avg. Bits=4, Model=Qwen2.5-7B-I2025.12 | 64.06 | 97.55 | |
| DeltaLoss-onlyAvg. Bits=5, Model=Qwen2.5-7B-I2025.12 | 64.04 | 97.52 | |
| DeltaLoss-onlyAvg. Bits=6, Model=Llama3.1-8B-I2025.12 | 63.64 | 99.18 | |
| SignRoundV2Avg. Bits=4, Model=Qwen2.5-7B-I2025.12 | 63.37 | 96.5 | |
| 16bitsAvg. Bits=16, Model=Qwen3-8B2025.12 | 63.24 | — | |
| SignRoundV2Avg. Bits=5, Model=Llama3.1-8B-I2025.12 | 63.19 | 98.49 | |
| DeltaLoss-onlyAvg. Bits=4.5, Model=Qwen2.5-7B-I2025.12 | 63.13 | 96.13 | |
| DeltaLoss-onlyAvg. Bits=5, Model=Llama3.1-8B-I2025.12 | 62.57 | 97.51 | |
| SignRoundV2Avg. Bits=5, Model=Qwen3-8B2025.12 | 62.54 | 98.89 | |
| SignRoundV2Avg. Bits=4.5, Model=Llama3.1-8B-I2025.12 | 62.33 | 97.14 | |
| SignRoundV2Avg. Bits=4.5, Model=Qwen3-8B2025.12 | 62.28 | 98.49 | |
| SignRoundV2Avg. Bits=6, Model=Qwen3-8B2025.12 | 62.19 | 98.34 | |
| DeltaLoss-onlyAvg. Bits=6, Model=Qwen3-8B2025.12 | 62 | 98.04 | |
| SignRoundV2Avg. Bits=4, Model=Qwen3-8B2025.12 | 61.89 | 97.87 | |
| DeltaLoss-onlyAvg. Bits=5, Model=Qwen3-8B2025.12 | 61.45 | 97.18 | |
| SignRoundV2Avg. Bits=4, Model=Llama3.1-8B-I2025.12 | 61.34 | 95.59 | |
| SignRoundV1Avg. Bits=4, Model=Llama3.1-8B-I2025.12 | 60.72 | 94.64 | |
| DeltaLoss-onlyAvg. Bits=4.5, Model=Llama3.1-8B-I2025.12 | 60.64 | 94.5 | |
| RTNAvg. Bits=4, Model=Qwen2.5-7B-I2025.12 | 60.62 | 92.32 | |
| SignRoundV1Avg. Bits=4, Model=Qwen3-8B2025.12 | 60.25 | 95.28 | |
| DeltaLoss-onlyAvg. Bits=4.5, Model=Qwen3-8B2025.12 | 59.81 | 94.58 | |
| RTNAvg. Bits=4, Model=Qwen3-8B2025.12 | 58.54 | 92.57 | |
| RTNAvg. Bits=4, Model=Llama3.1-8B-I2025.12 | 58.31 | 90.88 |