Cost and Coverage Evaluation on HotpotQA
0Cost (cents)LLaMA-3 EXIT
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| LLaMA-3 EXITBase model=LLaMA-3.2-3b, alpha=0.12025.11 | 0 | 71.8 | 4 | 4 | |
| RandomBase model=LLaMA-3.2-3b, alpha=0.12025.11 | 1.411 | 61 | 1.51 | 1 | |
| LLaMA-3.2-3bBase model=LLaMA-3.2-3b, alpha=0.12025.11 | 4.449 | 61.5 | 1 | 1 | |
| CPOBase model=LLaMA-3.2-3b, alpha=0.12025.11 | 4.655 | 82.9 | 1 | 1 | |
| CCPO (λ = 0)Base model=LLaMA-3.2-3b, alpha=0.1, lambda=02025.11 | 7.061 | 90.1 | 1.59 | 3.18 | |
| CCPO (λ = 1e−4)Base model=LLaMA-3.2-3b, alpha=0.1, lambda=1e-42025.11 | 7.397 | 90.2 | 1.66 | 2.78 | |
| CPO batchBase model=LLaMA-3.2-3b, alpha=0.12025.11 | 8.007 | 93.2 | 1.79 | 3.58 | |
| UALABase model=LLaMA-3.2-3b, alpha=0.12025.11 | 8.012 | 92.3 | 2 | 2 | |
| CPO onlineBase model=LLaMA-3.2-3b, alpha=0.12025.11 | 8.829 | 87.5 | 1.856 | 3.11 | |
| GPT-4oBase model=LLaMA-3.2-3b, alpha=0.12025.11 | 18.09 | 85 | 1.15 | 1 | |
| GPT-4o EXITBase model=LLaMA-3.2-3b, alpha=0.12025.11 | 827 | 90.8 | 2.39 | 2.39 |