Multi-fidelity Multi-armed Bandit on NLI shared evaluation pool
3,277.7Mean Cost-Weighted Pseudo-RegretTACC
Evaluation Results
| Method | Links | |
|---|---|---|
| TACCHigh-fidelity cost (λ(H))=200, Final budget (Λ)=1280002026.05 | 3,277.7 | |
| DNCHigh-fidelity cost (λ(H))=200, Final budget (Λ)=1280002026.05 | 3,894 | |
| TACCHigh-fidelity cost (λ(H))=500, Final budget (Λ)=1280002026.05 | 4,023.4 | |
| UCBHigh-fidelity cost (λ(H))=200, Final budget (Λ)=1280002026.05 | 4,264.1 | |
| MF-UCBHigh-fidelity cost (λ(H))=200, Final budget (Λ)=1280002026.05 | 4,317 | |
| TACCHigh-fidelity cost (λ(H))=1000, Final budget (Λ)=1280002026.05 | 4,688.6 | |
| UCBHigh-fidelity cost (λ(H))=500, Final budget (Λ)=1280002026.05 | 5,083.2 | |
| UCBHigh-fidelity cost (λ(H))=1000, Final budget (Λ)=1280002026.05 | 5,190.7 | |
| DNCHigh-fidelity cost (λ(H))=500, Final budget (Λ)=1280002026.05 | 5,201 | |
| MF-UCBHigh-fidelity cost (λ(H))=1000, Final budget (Λ)=1280002026.05 | 5,334.4 | |
| MF-UCBHigh-fidelity cost (λ(H))=500, Final budget (Λ)=1280002026.05 | 5,359.1 | |
| DNCHigh-fidelity cost (λ(H))=1000, Final budget (Λ)=1280002026.05 | 5,532.7 | |
| Low-ImprovingHigh-fidelity cost (λ(H))=1000, Final budget (Λ)=1280002026.05 | 6,668.6 | |
| Low-ImprovingHigh-fidelity cost (λ(H))=500, Final budget (Λ)=1280002026.05 | 6,808.4 | |
| Low-ImprovingHigh-fidelity cost (λ(H))=200, Final budget (Λ)=1280002026.05 | 6,827.5 | |
| Weak-FixedHigh-fidelity cost (λ(H))=1000, Final budget (Λ)=1280002026.05 | 7,058.4 | |
| Weak-FixedHigh-fidelity cost (λ(H))=200, Final budget (Λ)=1280002026.05 | 7,331.9 | |
| Weak-FixedHigh-fidelity cost (λ(H))=500, Final budget (Λ)=1280002026.05 | 7,501.6 |