Multiple Choice Question Answering on Arc-C
79.81AccuracyBLoB(N=10)
Evaluation results
| Method | Links | ||
|---|---|---|---|
| BLoB(N=10)Backbone=Llama3.1-8B, Fine-tuned Dataset=OBQA, Number of forward passes=102024.10 | 79.81 | 5.8 | |
| BLoB(Mean)Backbone=Llama3.1-8B, Fine-tuned Dataset=OBQA2024.10 | 79.51 | 11.371 | |
| UQ4CTBackbone=Llama3.1-8B, Fine-tuned Dataset=OBQA, Number of forward passes=12024.10 | 79 | 3.61 | |
| LABackbone=Llama3.1-8B, Fine-tuned Dataset=OBQA2024.10 | 78.7 | 7.2 | |
| EnsembleBackbone=Llama3.1-8B, Fine-tuned Dataset=OBQA2024.10 | 78.2 | 15.4 | |
| MixLoRABackbone=Llama3.1-8B, Fine-tuned Dataset=OBQA2024.10 | 78.1 | 14.5 | |
| LoRABackbone=Llama3.1-8B, Fine-tuned Dataset=OBQA2024.10 | 77.8 | 14.8 | |
| MC DropBackbone=Llama3.1-8B, Fine-tuned Dataset=OBQA2024.10 | 77.12 | 13.42 | |
| BF16Model Size=8B, Quantization Method=None (BF16), Adaptive Block Scaling (4/6)=false2025.12 | 56.7 | — | |
| AWQ + 4/6Model Size=8B, Quantization Method=AWQ, Adaptive Block Scaling (4/6)=true2025.12 | 55.8 | — | |
| AWQModel Size=8B, Quantization Method=AWQ, Adaptive Block Scaling (4/6)=false2025.12 | 55.2 | — | |
| SmoothQuantModel Size=8B, Quantization Method=SmoothQuant, Adaptive Block Scaling (4/6)=false2025.12 | 55.2 | — | |
| GPTQ + 4/6Model Size=8B, Quantization Method=GPTQ, Adaptive Block Scaling (4/6)=true2025.12 | 54.9 | — | |
| SmoothQuant + 4/6Model Size=8B, Quantization Method=SmoothQuant, Adaptive Block Scaling (4/6)=true2025.12 | 54.8 | — | |
| GPTQModel Size=8B, Quantization Method=GPTQ, Adaptive Block Scaling (4/6)=false2025.12 | 54.3 | — | |
| RTNModel Size=8B, Quantization Method=RTN, Adaptive Block Scaling (4/6)=false2025.12 | 54.1 | — | |
| RTN + 4/6Model Size=8B, Quantization Method=RTN, Adaptive Block Scaling (4/6)=true2025.12 | 53.7 | — | |
| BF16Model Size=1.7B, Quantization Method=None (BF16), Adaptive Block Scaling (4/6)=false2025.12 | 43 | — | |
| SmoothQuant + 4/6Model Size=1.7B, Quantization Method=SmoothQuant, Adaptive Block Scaling (4/6)=true2025.12 | 40.4 | — | |
| AWQ + 4/6Model Size=1.7B, Quantization Method=AWQ, Adaptive Block Scaling (4/6)=true2025.12 | 39.2 | — | |
| RTN + 4/6Model Size=1.7B, Quantization Method=RTN, Adaptive Block Scaling (4/6)=true2025.12 | 38.7 | — | |
| GPTQ + 4/6Model Size=1.7B, Quantization Method=GPTQ, Adaptive Block Scaling (4/6)=true2025.12 | 37.3 | — | |
| SmoothQuantModel Size=1.7B, Quantization Method=SmoothQuant, Adaptive Block Scaling (4/6)=false2025.12 | 37.2 | — | |
| GPTQModel Size=1.7B, Quantization Method=GPTQ, Adaptive Block Scaling (4/6)=false2025.12 | 36.3 | — | |
| AWQModel Size=1.7B, Quantization Method=AWQ, Adaptive Block Scaling (4/6)=false2025.12 | 36.3 | — | |
| RTNModel Size=1.7B, Quantization Method=RTN, Adaptive Block Scaling (4/6)=false2025.12 | 35.3 | — | |
| MDLM2025.10 | 25.17 | — | |
| LDDM-M2025.10 | 23.21 | — |