Question Answering on MMLU (test) (Cost and Coverage Evaluation)
0Cost (cents)LLaMA-2 EXIT
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| LLaMA-2 EXITBase Model=LLaMA-2-7B, alpha=0.12025.11 | 0 | 0.79 | 3.14 | 3.14 | |
| CCPOBase Model=LLaMA-2-7B, alpha=0.1, lambda=02025.11 | 7.949 | 0.92 | 1.08 | 2.14 | |
| CCPOBase Model=LLaMA-2-7B, alpha=0.1, lambda=2e-42025.11 | 8.071 | 0.907 | 1.11 | 1.48 | |
| LLaMA-2-7BBase Model=LLaMA-2-7B, alpha=0.12025.11 | 8.682 | 0.66 | 1.11 | 1 | |
| CPOBase Model=LLaMA-2-7B, alpha=0.12025.11 | 9.174 | 0.7 | 1.195 | 1 | |
| RandomBase Model=LLaMA-2-7B, alpha=0.12025.11 | 12.99 | 0.545 | 1.53 | 1 | |
| UALABase Model=LLaMA-2-7B, alpha=0.12025.11 | 16.99 | 0.92 | 2 | 2 | |
| CPO onlineBase Model=LLaMA-2-7B, alpha=0.12025.11 | 19.07 | 0.901 | 2.48 | 3.46 | |
| CPO batchBase Model=LLaMA-2-7B, alpha=0.12025.11 | 22.25 | 0.902 | 2.86 | 4.2 | |
| GPT-4oBase Model=LLaMA-2-7B, alpha=0.12025.11 | 25.35 | 0.93 | 1.08 | 1 | |
| GPT-4o EXITBase Model=LLaMA-2-7B, alpha=0.12025.11 | 104.6 | 0.96 | 1.39 | 1.39 |