Operational Metrics on MMLU (Multiple-choice Question Answering)
0Cost (cents)LLaMA-3 EXIT
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| LLaMA-3 EXITBase Model=LLaMA-3.2-3b, alpha=0.12025.11 | 0 | 84 | 3.97 | 3.97 | |
| CPOBase Model=LLaMA-3.2-3b, alpha=0.12025.11 | 6.846 | 86 | 1.01 | 1 | |
| LLaMA-3-2-3bBase Model=LLaMA-3.2-3b, alpha=0.12025.11 | 8.54 | 58 | 1.03 | 1 | |
| CCPOlambda=0, Base Model=LLaMA-3.2-3b, alpha=0.12025.11 | 10.82 | 90.1 | 1.34 | 2.66 | |
| CCPOlambda=1e−4, Base Model=LLaMA-3.2-3b, alpha=0.12025.11 | 10.9 | 90 | 1.35 | 2.7 | |
| CPO onlineBase Model=LLaMA-3.2-3b, alpha=0.12025.11 | 11.76 | 87.8 | 1.51 | 2.53 | |
| CPO batchBase Model=LLaMA-3.2-3b, alpha=0.12025.11 | 12.12 | 91.7 | 1.51 | 2.36 | |
| RandomBase Model=LLaMA-3.2-3b, alpha=0.12025.11 | 12.74 | 60.7 | 1.55 | 1 | |
| UALABase Model=LLaMA-3.2-3b, alpha=0.12025.11 | 16.1 | 89.3 | 2 | 2 | |
| GPT-4oBase Model=LLaMA-3.2-3b, alpha=0.12025.11 | 26.87 | 91.3 | 1.19 | 1 | |
| GPT-4o EXITBase Model=LLaMA-3.2-3b, alpha=0.12025.11 | 104.6 | 96 | 1.39 | 1.39 |