Zero-shot Evaluation on Evaluation Suite (ARC, BoolQ, HellaSwag, PIQA)
69.82Average ScoreBF16
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| BF16Bits=16, Finetuning=none, Incoherence processing=true, Backbone=Llama 3.1 8B Instruct2025.05 | 69.82 | 51.37 | 78.03 | 82.05 | 57.74 | 79.92 | |
| YAQA-ABits=4, Finetuning=none, Incoherence processing=true, Quantizer=INT4, Backbone=Llama 3.1 8B Instruct2025.05 | 69.11 | 50.68 | 78.11 | 79.65 | 57.13 | 79.98 | |
| YAQA-BBits=4, Finetuning=none, Incoherence processing=true, Quantizer=INT4, Backbone=Llama 3.1 8B Instruct2025.05 | 68.92 | 49.49 | 77.31 | 81.01 | 56.98 | 79.82 | |
| LDLQBits=4, Finetuning=none, Incoherence processing=true, Quantizer=INT4, Backbone=Llama 3.1 8B Instruct2025.05 | 67.99 | 50 | 76.94 | 77.01 | 56.83 | 79.16 | |
| DISCQUANTBits=4, Finetuning=none, Incoherence processing=true, Quantizer=INT4, Backbone=Llama 3.1 8B Instruct2025.05 | 67.68 | 50.34 | 77.44 | 74.12 | 56.55 | 79.92 |