Zero-shot Evaluation on Winogrande, OBQA, HellaSwag, BoolQ, ARC (c/e), and RTE
76.07Winogrande AccuracyBTC-LLM
Evaluation Results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| BTC-LLMBackbone=LLaMA-1-30B, W-Bits=0.80, Zero-shot protocol=true2025.05 | 76.07 | 45 | 76.07 | 71.71 | 73.99 | 45.39 | 66.06 | 64.48 | |
| FP16Backbone=LLaMA-1-30B, W-Bits=16, Zero-shot protocol=true2025.05 | 75.77 | 36 | 63.37 | 82.69 | 80.3 | 52.9 | 67.15 | 67.4 | |
| FP16Backbone=LLaMA-1-13B, W-Bits=16, Zero-shot protocol=true2025.05 | 72.69 | 33.2 | 59.91 | 77.89 | 77.4 | 46.42 | 70.4 | 63.8 | |
| FP16Backbone=LLaMA-2-13B, W-Bits=16, Zero-shot protocol=true2025.05 | 72.22 | 35.2 | 60.03 | 80.55 | 79.42 | 48.38 | 65.34 | 65 | |
| STBLLMBackbone=LLaMA-1-30B, W-Bits=0.80, Zero-shot protocol=true2025.05 | 71.59 | 41 | 69.85 | 77.37 | 71.55 | 41.3 | 48.01 | 60.1 | |
| BTC-LLMBackbone=LLaMA-1-13B, W-Bits=0.80, Zero-shot protocol=true2025.05 | 70.8 | 41.6 | 72.48 | 74.86 | 67.8 | 42.24 | 55.96 | 60.82 | |
| BTC-LLMBackbone=LLaMA-2-13B, W-Bits=0.80, Zero-shot protocol=true2025.05 | 69.46 | 71.53 | 72.63 | 71.53 | 70.75 | 42.75 | 64.62 | 61.91 | |
| STBLLMBackbone=LLaMA-1-13B, W-Bits=0.80, Zero-shot protocol=true2025.05 | 65.98 | 36.2 | 63.67 | 65.38 | 68.86 | 34.04 | 56.68 | 55.83 | |
| STBLLMBackbone=LLaMA-2-13B, W-Bits=0.80, Zero-shot protocol=true2025.05 | 63.93 | 37 | 57.76 | 71.53 | 60.56 | 31.99 | 54.15 | 53.85 |