Multiple Choice Question Answering on MMLU High school US history (val)
40.91Top-1 AccuracyDense
Evaluation results
| Method | Links | |
|---|---|---|
| DenseModel=Llama 3.1-8B (16FP), Rem.=0/322024.04 | 40.91 | |
| NEPENTHEModel=Llama 3.1-8B (16FP), Rem.=3/32, fine-tuning protocol=no ft2024.04 | 38.49 | |
| NEPENTHEModel=Llama 3.1-8B (16FP), Rem.=5/32, fine-tuning protocol=no ft2024.04 | 36.36 | |
| NEPENTHEModel=Llama 3.1-8B (16FP), Rem.=7/32, fine-tuning protocol=no ft2024.04 | 33.82 | |
| NEPENTHEModel=Llama 3.1-8B (16FP), Rem.=9/32, fine-tuning protocol=no ft2024.04 | 31.82 | |
| NEPENTHEModel=Llama 3.1-8B (16FP), Rem.=11/32, fine-tuning protocol=no ft2024.04 | 18.18 |