Question Answering on HotpotQA (F1, Avg Cost)
41.02F1 ScoreRelevance
Evaluation results
| Method | Links | ||
|---|---|---|---|
| RelevanceBgt/k=20, Backbone=Llama3.1-8B-instruct2026.06 | 41.02 | 15.5 | |
| MMRBgt/k=25, Backbone=Llama3.1-8B-instruct2026.06 | 38.97 | 10.88 | |
| GreedyBgt/k=30, Backbone=Llama3.1-8B-instruct2026.06 | 37.8 | 18 | |
| RelevanceBgt/k=30, Backbone=Llama3.1-8B-instruct2026.06 | 37.73 | 18.1 | |
| RelevanceBgt/k=∞, Backbone=Llama3.1-8B-instruct2026.06 | 37.14 | 19.2 | |
| KnapsackBgt/k=∞, Backbone=Llama3.1-8B-instruct2026.06 | 37 | 19.24 | |
| RelevanceBgt/k=15, Backbone=Llama3.1-8B-instruct2026.06 | 36.99 | 12.9 | |
| KnapsackBgt/k=25, Backbone=Llama3.1-8B-instruct2026.06 | 36.65 | 17.1 | |
| KnapsackBgt/k=15, Backbone=Llama3.1-8B-instruct2026.06 | 36.62 | 12.65 | |
| GreedyBgt/k=15, Backbone=Llama3.1-8B-instruct2026.06 | 36.26 | 12.56 | |
| GreedyBgt/k=20, Backbone=Llama3.1-8B-instruct2026.06 | 36.19 | 15.08 | |
| GreedyBgt/k=∞, Backbone=Llama3.1-8B-instruct2026.06 | 35.74 | 19.24 | |
| MMRBgt/k=∞, Backbone=Llama3.1-8B-instruct2026.06 | 35.69 | 10.88 | |
| MMRBgt/k=15, Backbone=Llama3.1-8B-instruct2026.06 | 35.19 | 9.82 | |
| GreedyBgt/k=25, Backbone=Llama3.1-8B-instruct2026.06 | 35.17 | 16.98 | |
| MMRBgt/k=30, Backbone=Llama3.1-8B-instruct2026.06 | 34.68 | 10.88 | |
| MMRBgt/k=10, Backbone=Llama3.1-8B-instruct2026.06 | 34.52 | 8.04 | |
| RelevanceBgt/k=10, Backbone=Llama3.1-8B-instruct2026.06 | 34.47 | 9.2 | |
| KnapsackBgt/k=30, Backbone=Llama3.1-8B-instruct2026.06 | 34.43 | 18.06 | |
| GreedyBgt/k=10, Backbone=Llama3.1-8B-instruct2026.06 | 34.21 | 8.93 | |
| Top-kBgt/k=20, Backbone=Llama3.1-8B-instruct2026.06 | 33.99 | — | |
| KnapsackBgt/k=0, Backbone=Llama3.1-8B-instruct2026.06 | 33.91 | 0 | |
| KnapsackBgt/k=20, Backbone=Llama3.1-8B-instruct2026.06 | 33.59 | 15.38 | |
| KnapsackBgt/k=5, Backbone=Llama3.1-8B-instruct2026.06 | 33.56 | 4.77 | |
| RelevanceBgt/k=25, Backbone=Llama3.1-8B-instruct2026.06 | 33.55 | 17.2 | |
| MMRBgt/k=20, Backbone=Llama3.1-8B-instruct2026.06 | 33.51 | 10.61 | |
| RelevanceBgt/k=0, Backbone=Llama3.1-8B-instruct2026.06 | 33.45 | 0 | |
| RedundancyBgt/k=∞, Backbone=Llama3.1-8B-instruct2026.06 | 33.32 | 0.19 | |
| KnapsackBgt/k=10, Backbone=Llama3.1-8B-instruct2026.06 | 33.05 | 9.17 | |
| Top-kBgt/k=25, Backbone=Llama3.1-8B-instruct2026.06 | 32.98 | — | |
| Top-kBgt/k=30, Backbone=Llama3.1-8B-instruct2026.06 | 32.69 | — | |
| RedundancyBgt/k=20, Backbone=Llama3.1-8B-instruct2026.06 | 32.29 | 0.19 | |
| GreedyBgt/k=5, Backbone=Llama3.1-8B-instruct2026.06 | 31.35 | 4.74 | |
| RelevanceBgt/k=5, Backbone=Llama3.1-8B-instruct2026.06 | 31.23 | 4.8 | |
| RedundancyBgt/k=15, Backbone=Llama3.1-8B-instruct2026.06 | 30.71 | 0.19 | |
| RedundancyBgt/k=0, Backbone=Llama3.1-8B-instruct2026.06 | 30.62 | 0 | |
| GreedyBgt/k=0, Backbone=Llama3.1-8B-instruct2026.06 | 30.48 | 0 | |
| RedundancyBgt/k=10, Backbone=Llama3.1-8B-instruct2026.06 | 30.46 | 0.19 | |
| RedundancyBgt/k=5, Backbone=Llama3.1-8B-instruct2026.06 | 30.36 | 0.15 | |
| Top-kBgt/k=15, Backbone=Llama3.1-8B-instruct2026.06 | 30.24 | — | |
| MMRBgt/k=0, Backbone=Llama3.1-8B-instruct2026.06 | 29.69 | 0 | |
| MMRBgt/k=5, Backbone=Llama3.1-8B-instruct2026.06 | 28.78 | 4.67 | |
| RedundancyBgt/k=30, Backbone=Llama3.1-8B-instruct2026.06 | 28.74 | 0.19 | |
| RedundancyBgt/k=25, Backbone=Llama3.1-8B-instruct2026.06 | 28.53 | 0.19 | |
| Top-kBgt/k=10, Backbone=Llama3.1-8B-instruct2026.06 | 26.53 | — | |
| Top-kBgt/k=5, Backbone=Llama3.1-8B-instruct2026.06 | 15.36 | — | |
| Vanilla LLMBackbone=Llama3.1-8B-instruct2026.06 | 14.61 | — |