Multiple Choice Question Answering on MMLU High school US history (test)
93.84Top-1 AccuracyPSAO_LLM
Evaluation Results
| Method | Links | |
|---|---|---|
| PSAO_LLMModel=GPT-4o2026.05 | 93.84 | |
| MIPROModel=GPT-4o2026.05 | 93.61 | |
| COPROModel=GPT-4o2026.05 | 92.95 | |
| NEPENTHEModel=Llama 3.1-8B (16FP), Rem.=3/32, fine-tuning protocol=no ft2024.04 | 36.76 | |
| NEPENTHEModel=Llama 3.1-8B (16FP), Rem.=7/32, fine-tuning protocol=no ft2024.04 | 35.08 | |
| DenseModel=Llama 3.1-8B (16FP), Rem.=0/322024.04 | 33.82 | |
| NEPENTHEModel=Llama 3.1-8B (16FP), Rem.=5/32, fine-tuning protocol=no ft2024.04 | 29.9 | |
| NEPENTHEModel=Llama 3.1-8B (16FP), Rem.=9/32, fine-tuning protocol=no ft2024.04 | 28.92 | |
| NEPENTHEModel=Llama 3.1-8B (16FP), Rem.=11/32, fine-tuning protocol=no ft2024.04 | 27.45 |