Question Answering on NQ (avg_cost, F1)
44.11F1 ScoreMMR
Evaluation results
| Method | Links | ||
|---|---|---|---|
| MMRBackbone=Qwen3-8B, Bgt/k=102026.06 | 44.11 | 8.8 | |
| RelevanceBgt/k=15, Backbone=Llama3.1-8B-instruct2026.06 | 42.44 | 13.6 | |
| RelevanceBgt/k=20, Backbone=Llama3.1-8B-instruct2026.06 | 41.36 | 16.8 | |
| RedundancyBackbone=Qwen3-8B, Bgt/k=∞2026.06 | 40.85 | 0.3 | |
| Top-kBackbone=Qwen3-8B, Bgt/k=152026.06 | 40.1 | — | |
| Top-kBgt/k=25, Backbone=Llama3.1-8B-instruct2026.06 | 39.6 | — | |
| Top-kBackbone=Qwen3-8B, Bgt/k=252026.06 | 39.6 | — | |
| RelevanceBgt/k=30, Backbone=Llama3.1-8B-instruct2026.06 | 39.51 | 20.9 | |
| RelevanceBgt/k=0, Backbone=Llama3.1-8B-instruct2026.06 | 39.48 | 0 | |
| Top-kBackbone=Qwen3-8B, Bgt/k=202026.06 | 39.06 | — | |
| MMRBgt/k=∞, Backbone=Llama3.1-8B-instruct2026.06 | 38.33 | 12.56 | |
| Top-kBackbone=Qwen3-8B, Bgt/k=302026.06 | 38.25 | — | |
| KnapsackBgt/k=30, Backbone=Llama3.1-8B-instruct2026.06 | 37.65 | 20.72 | |
| Top-kBackbone=Qwen3-8B, Bgt/k=52026.06 | 37.14 | — | |
| RelevanceBgt/k=10, Backbone=Llama3.1-8B-instruct2026.06 | 37.12 | 9.7 | |
| KnapsackBgt/k=20, Backbone=Llama3.1-8B-instruct2026.06 | 36.88 | 16.49 | |
| GreedyBgt/k=20, Backbone=Llama3.1-8B-instruct2026.06 | 36.87 | 16.1 | |
| Top-kBackbone=Qwen3-8B, Bgt/k=102026.06 | 36.87 | — | |
| GreedyBgt/k=30, Backbone=Llama3.1-8B-instruct2026.06 | 36.66 | 20.54 | |
| Top-kBgt/k=15, Backbone=Llama3.1-8B-instruct2026.06 | 36.57 | — | |
| MMRBgt/k=30, Backbone=Llama3.1-8B-instruct2026.06 | 36.55 | 12.56 | |
| MMRBgt/k=20, Backbone=Llama3.1-8B-instruct2026.06 | 36.45 | 12.04 | |
| GreedyBgt/k=15, Backbone=Llama3.1-8B-instruct2026.06 | 36.34 | 12.94 | |
| RedundancyBgt/k=20, Backbone=Llama3.1-8B-instruct2026.06 | 36.23 | 0.3 | |
| RedundancyBgt/k=30, Backbone=Llama3.1-8B-instruct2026.06 | 36.23 | 0.3 | |
| KnapsackBgt/k=5, Backbone=Llama3.1-8B-instruct2026.06 | 35.91 | 4.97 | |
| RedundancyBgt/k=∞, Backbone=Llama3.1-8B-instruct2026.06 | 35.73 | 0.3 | |
| KnapsackBgt/k=15, Backbone=Llama3.1-8B-instruct2026.06 | 35.7 | 13.21 | |
| GreedyBgt/k=0, Backbone=Llama3.1-8B-instruct2026.06 | 35.42 | 0 | |
| KnapsackBgt/k=∞, Backbone=Llama3.1-8B-instruct2026.06 | 35.31 | 23.46 | |
| RedundancyBgt/k=10, Backbone=Llama3.1-8B-instruct2026.06 | 35.01 | 0.3 | |
| RedundancyBgt/k=25, Backbone=Llama3.1-8B-instruct2026.06 | 34.87 | 0.3 | |
| RelevanceBgt/k=∞, Backbone=Llama3.1-8B-instruct2026.06 | 34.71 | 23.5 | |
| MMRBgt/k=15, Backbone=Llama3.1-8B-instruct2026.06 | 34.7 | 11.12 | |
| RedundancyBgt/k=15, Backbone=Llama3.1-8B-instruct2026.06 | 34.7 | 0.3 | |
| Top-kBgt/k=20, Backbone=Llama3.1-8B-instruct2026.06 | 34.52 | — | |
| KnapsackBgt/k=10, Backbone=Llama3.1-8B-instruct2026.06 | 34.52 | 9.46 | |
| GreedyBgt/k=10, Backbone=Llama3.1-8B-instruct2026.06 | 34.43 | 9.4 | |
| GreedyBgt/k=5, Backbone=Llama3.1-8B-instruct2026.06 | 34.25 | 4.89 | |
| MMRBgt/k=10, Backbone=Llama3.1-8B-instruct2026.06 | 34.21 | 8.78 | |
| GreedyBgt/k=25, Backbone=Llama3.1-8B-instruct2026.06 | 34.17 | 18.72 | |
| RedundancyBgt/k=5, Backbone=Llama3.1-8B-instruct2026.06 | 34.12 | 0.26 | |
| RelevanceBgt/k=5, Backbone=Llama3.1-8B-instruct2026.06 | 33.9 | 5 | |
| GreedyBgt/k=∞, Backbone=Llama3.1-8B-instruct2026.06 | 33.6 | 23.46 | |
| Top-kBgt/k=10, Backbone=Llama3.1-8B-instruct2026.06 | 33.55 | — | |
| MMRBgt/k=0, Backbone=Llama3.1-8B-instruct2026.06 | 33.52 | 0 | |
| KnapsackBgt/k=25, Backbone=Llama3.1-8B-instruct2026.06 | 33.35 | 18.94 | |
| Top-kBgt/k=30, Backbone=Llama3.1-8B-instruct2026.06 | 33.33 | — | |
| MMRBgt/k=25, Backbone=Llama3.1-8B-instruct2026.06 | 33.31 | 12.46 | |
| MMRBgt/k=5, Backbone=Llama3.1-8B-instruct2026.06 | 33.25 | 4.86 | |
| KnapsackBgt/k=0, Backbone=Llama3.1-8B-instruct2026.06 | 32.21 | 0 | |
| RedundancyBgt/k=0, Backbone=Llama3.1-8B-instruct2026.06 | 32.14 | 0 | |
| RelevanceBgt/k=25, Backbone=Llama3.1-8B-instruct2026.06 | 31.93 | 19.2 | |
| Top-kBgt/k=5, Backbone=Llama3.1-8B-instruct2026.06 | 27.55 | — | |
| Vanilla LLMBackbone=Qwen3-8B, Bgt/k=-2026.06 | 19.81 | — | |
| Vanilla LLMBackbone=Llama3.1-8B-instruct2026.06 | 12.52 | — |