Language Modeling on C4 Llama 2 & 3 (test)
15.022-7 ScoreOmniQ
Evaluation Results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| OmniQBits=2, Group=128, context length=20482024.07 | 15.02 | 11.05 | 8.52 | 22.46 | 15.06 | |
| ApiQBits=2, Group=128, context length=20482024.07 | 12.04 | 9.13 | — | — | — | |
| CBQBits=2, Group=64, context length=20482024.07 | 11.3 | — | — | — | — | |
| ApiQBits=2, Group=64, context length=20482024.07 | 10.56 | 8.92 | — | — | — | |
| DB-LLMBits=2, Group=64, context length=20482024.07 | 9.62 | 8.38 | 6.77 | 19.2 | — | |
| AQLMBits=2, Group=2x8, context length=20482024.07 | 8.96 | 7.8 | 6.36 | — | — | |
| EfficientQATBits=2, Group=128, context length=20482024.07 | 8.79 | 7.75 | 6.48 | 13.22 | 9.53 | |
| EfficientQATBits=2, Group=64, context length=20482024.07 | 8.5 | 7.59 | 6.38 | 12.77 | 9.23 | |
| QuIP#Bits=2, context length=20482024.07 | 8.35 | 7.45 | 6.12 | — | — | |
| GPTQBits=3, Group=128, context length=20482024.07 | 7.89 | 7 | 5.85 | 11.66 | 8.64 | |
| AWQBits=3, Group=128, context length=20482024.07 | 7.84 | 6.94 | 5.81 | 11.49 | 7.91 | |
| OmniQBits=3, Group=128, context length=20482024.07 | 7.75 | 6.98 | 5.85 | 11.66 | 7.97 | |
| EfficientQATBits=3, Group=128, context length=20482024.07 | 7.34 | 6.73 | 5.71 | 10.06 | 7.46 | |
| FP16Bits=16, context length=20482024.07 | 6.97 | 6.47 | 5.52 | 8.88 | 6.73 |