Question Answering on TriviaQA 5shots
80Accuracyfp16
Evaluation Results
| Method | Links | |
|---|---|---|
| fp16Size=70B2026.02 | 80 | |
| MagRSize=70B2026.02 | 77.5 | |
| AstroSize=70B2026.02 | 77.5 | |
| OmniQuantSize=70B2026.02 | 76.8 | |
| BNRMBase Model=Llama3.1-8B-Instruct, Evaluation Protocol=5-shot2026.02 | 71.99 | |
| fp16Size=13B2026.02 | 70.4 | |
| FP16Backbone=LLaMA-3.1-8B, Quantization=None, Protocol=5-shot2026.02 | 70.4 | |
| AstroSize=13B2026.02 | 66.6 | |
| MagRSize=13B2026.02 | 65.2 | |
| OmniQuantSize=13B2026.02 | 64.5 | |
| fp16Size=7B2026.02 | 64.4 | |
| AstroSize=7B2026.02 | 55.9 | |
| MagRSize=7B2026.02 | 55.3 | |
| OmniQuantSize=7B2026.02 | 54.4 | |
| BNRMBase Model=OpenRLHF-Llama3-8B-SFT, Evaluation Protocol=5-shot2026.02 | 54.3 | |
| SKBase Model=OpenRLHF-Llama3-8B-SFT, Evaluation Protocol=5-shot2026.02 | 52.86 | |
| PoEBase Model=OpenRLHF-Llama3-8B-SFT, Evaluation Protocol=5-shot2026.02 | 52.34 | |
| ALBMBase Model=Llama3.1-8B-Instruct, Evaluation Protocol=5-shot2026.02 | 52.09 | |
| ALBMBase Model=OpenRLHF-Llama3-8B-SFT, Evaluation Protocol=5-shot2026.02 | 51.52 | |
| AstroBackbone=LLaMA-3.1-8B, Quantization=W3A16g128, Protocol=5-shot2026.02 | 51.2 | |
| SKBase Model=Llama3.1-8B-Instruct, Evaluation Protocol=5-shot2026.02 | 49.01 | |
| BaseBase Model=OpenRLHF-Llama3-8B-SFT, Evaluation Protocol=5-shot2026.02 | 48.53 | |
| PoEBase Model=Llama3.1-8B-Instruct, Evaluation Protocol=5-shot2026.02 | 48.41 | |
| LPBase Model=OpenRLHF-Llama3-8B-SFT, Evaluation Protocol=5-shot2026.02 | 48.32 | |
| InfoRMBase Model=OpenRLHF-Llama3-8B-SFT, Evaluation Protocol=5-shot2026.02 | 48.16 | |
| OmniQuantBackbone=LLaMA-3.1-8B, Quantization=W3A16g128, Protocol=5-shot2026.02 | 47.6 | |
| LPBase Model=Llama3.1-8B-Instruct, Evaluation Protocol=5-shot2026.02 | 47.2 | |
| MagRBackbone=LLaMA-3.1-8B, Quantization=W3A16g128, Protocol=5-shot2026.02 | 45.5 | |
| BaseBase Model=Llama3.1-8B-Instruct, Evaluation Protocol=5-shot2026.02 | 32.64 | |
| InfoRMBase Model=Llama3.1-8B-Instruct, Evaluation Protocol=5-shot2026.02 | 30.56 |