Factuality and Reasoning on MMLU
55.8MMLU AccuracySpIEL
Evaluation results
| Method | Links | |
|---|---|---|
| SpIELBackbone=Llama-2 13B, SFT=true2024.06 | 55.8 | |
| LoRABackbone=Llama-2 13B, SFT=true2024.06 | 55.8 | |
| FullFTBackbone=Llama-2 13B, SFT=true2024.06 | 55.6 | |
| SATBackbone=Llama-2 13B, SFT=true2024.06 | 55.4 | |
| OriginalBackbone=Llama-2 13B, SFT=false2024.06 | 55.3 | |
| (IA)³Backbone=Llama-2 13B, SFT=true2024.06 | 54.3 | |
| SpIELBackbone=Llama-2 7B, SFT=true2024.06 | 50.7 | |
| FullFTBackbone=Llama-2 7B, SFT=true2024.06 | 50.5 | |
| SATBackbone=Llama-2 7B, SFT=true2024.06 | 50.5 | |
| LoRABackbone=Llama-2 7B, SFT=true2024.06 | 49.3 | |
| (IA)³Backbone=Llama-2 7B, SFT=true2024.06 | 46.7 | |
| OriginalBackbone=Llama-2 7B, SFT=false2024.06 | 45.8 |