Retrieval on FEVER
93.8PrecisionGEH
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| GEHModel Pair=Mistral ↔ OpenAI, Overlap ratio (α)=0.3, Seed budget (|S|)=30, Hardware=NVIDIA A100 (80 GB), Runtime (s)=33282026.05 | 93.8 | 68.9 | — | |
| MDRhops=2, mode=direct2020.09 | 45.7 | 69.1 | 55 | |
| Entity Linkinghops=22020.09 | 30.6 | 53.8 | 39 | |
| TF-IDF + Linkedhops=2, reranking=BM252020.09 | 18.6 | 35.8 | 24.5 | |
| TF-IDFhops=22020.09 | 14.9 | 28.2 | 19.5 | |
| ProcModel Pair=Mistral ↔ OpenAI, Overlap ratio (α)=0.3, Seed budget (|S|)=30, Hardware=NVIDIA A100 (80 GB), Runtime (s)=44942026.05 | 9.03 | 0.73 | — | |
| RCSLSModel Pair=Mistral ↔ OpenAI, Overlap ratio (α)=0.3, Seed budget (|S|)=30, Hardware=NVIDIA A100 (80 GB), Runtime (s)=33482026.05 | 6.4 | 0.11 | — | |
| CCAModel Pair=Mistral ↔ OpenAI, Overlap ratio (α)=0.3, Seed budget (|S|)=30, Hardware=NVIDIA A100 (80 GB), Runtime (s)=16132026.05 | 5.22 | 0.61 | — | |
| LinearModel Pair=Mistral ↔ OpenAI, Overlap ratio (α)=0.3, Seed budget (|S|)=30, Hardware=NVIDIA A100 (80 GB), Runtime (s)=44202026.05 | 4.13 | 0 | — | |
| MLPModel Pair=Mistral ↔ OpenAI, Overlap ratio (α)=0.3, Seed budget (|S|)=30, Hardware=NVIDIA A100 (80 GB), Runtime (s)=44142026.05 | 1.93 | 0.01 | — |