Question Answering on MMLU (F1 and Cost Evaluation)
84F1 ScoreMMR
Evaluation results
| Method | Links | ||
|---|---|---|---|
| MMRBackbone=Qwen3-8B, Bgt/k=252026.06 | 84 | 9.6 | |
| KnapsackBackbone=Qwen3-8B, Bgt/k=52026.06 | 83 | 4.9 | |
| KnapsackBgt/k=30, Backbone=Llama3.1-8B-instruct2026.06 | 79 | 27.68 | |
| GreedyBgt/k=25, Backbone=Llama3.1-8B-instruct2026.06 | 76 | 22.99 | |
| GreedyBgt/k=∞, Backbone=Llama3.1-8B-instruct2026.06 | 75 | 48.24 | |
| Vanilla LLMBackbone=Qwen3-8B, Bgt/k=-2026.06 | 75 | — | |
| Top-kBgt/k=15, Backbone=Llama3.1-8B-instruct2026.06 | 72 | — | |
| KnapsackBgt/k=20, Backbone=Llama3.1-8B-instruct2026.06 | 72 | 18.97 | |
| KnapsackBgt/k=∞, Backbone=Llama3.1-8B-instruct2026.06 | 72 | 48.24 | |
| Top-kBgt/k=30, Backbone=Llama3.1-8B-instruct2026.06 | 71 | — | |
| RelevanceBgt/k=15, Backbone=Llama3.1-8B-instruct2026.06 | 71 | 14.7 | |
| GreedyBgt/k=30, Backbone=Llama3.1-8B-instruct2026.06 | 71 | 27.08 | |
| RelevanceBgt/k=25, Backbone=Llama3.1-8B-instruct2026.06 | 70 | 24.1 | |
| GreedyBgt/k=20, Backbone=Llama3.1-8B-instruct2026.06 | 70 | 18.16 | |
| MMRBgt/k=5, Backbone=Llama3.1-8B-instruct2026.06 | 70 | 4.62 | |
| KnapsackBgt/k=15, Backbone=Llama3.1-8B-instruct2026.06 | 69 | 14.1 | |
| KnapsackBgt/k=25, Backbone=Llama3.1-8B-instruct2026.06 | 69 | 23.59 | |
| MMRBgt/k=10, Backbone=Llama3.1-8B-instruct2026.06 | 69 | 7.85 | |
| Top-kBgt/k=20, Backbone=Llama3.1-8B-instruct2026.06 | 68 | — | |
| RedundancyBgt/k=0, Backbone=Llama3.1-8B-instruct2026.06 | 68 | 0 | |
| Vanilla LLMBackbone=Llama3.1-8B-instruct2026.06 | 67 | — | |
| Top-kBgt/k=25, Backbone=Llama3.1-8B-instruct2026.06 | 67 | — | |
| GreedyBgt/k=0, Backbone=Llama3.1-8B-instruct2026.06 | 67 | 0 | |
| GreedyBgt/k=10, Backbone=Llama3.1-8B-instruct2026.06 | 67 | 9.13 | |
| GreedyBgt/k=15, Backbone=Llama3.1-8B-instruct2026.06 | 67 | 13.68 | |
| RelevanceBgt/k=10, Backbone=Llama3.1-8B-instruct2026.06 | 66 | 9.9 | |
| RelevanceBgt/k=∞, Backbone=Llama3.1-8B-instruct2026.06 | 66 | 48.2 | |
| MMRBgt/k=15, Backbone=Llama3.1-8B-instruct2026.06 | 66 | 9.12 | |
| MMRBgt/k=25, Backbone=Llama3.1-8B-instruct2026.06 | 66 | 9.63 | |
| MMRBgt/k=∞, Backbone=Llama3.1-8B-instruct2026.06 | 66 | 9.68 | |
| RedundancyBgt/k=15, Backbone=Llama3.1-8B-instruct2026.06 | 66 | 0.91 | |
| RelevanceBgt/k=30, Backbone=Llama3.1-8B-instruct2026.06 | 65 | 28.1 | |
| MMRBgt/k=30, Backbone=Llama3.1-8B-instruct2026.06 | 65 | 9.68 | |
| RedundancyBgt/k=25, Backbone=Llama3.1-8B-instruct2026.06 | 65 | 0.91 | |
| Top-kBgt/k=5, Backbone=Llama3.1-8B-instruct2026.06 | 64 | — | |
| KnapsackBgt/k=5, Backbone=Llama3.1-8B-instruct2026.06 | 64 | 4.88 | |
| RedundancyBgt/k=10, Backbone=Llama3.1-8B-instruct2026.06 | 64 | 0.91 | |
| RedundancyBgt/k=20, Backbone=Llama3.1-8B-instruct2026.06 | 64 | 0.91 | |
| RedundancyBgt/k=∞, Backbone=Llama3.1-8B-instruct2026.06 | 64 | 1.76 | |
| Top-kBgt/k=10, Backbone=Llama3.1-8B-instruct2026.06 | 63 | — | |
| MMRBgt/k=20, Backbone=Llama3.1-8B-instruct2026.06 | 63 | 9.52 | |
| RedundancyBgt/k=30, Backbone=Llama3.1-8B-instruct2026.06 | 63 | 0.91 | |
| RelevanceBgt/k=5, Backbone=Llama3.1-8B-instruct2026.06 | 62 | 5 | |
| RelevanceBgt/k=20, Backbone=Llama3.1-8B-instruct2026.06 | 62 | 19.6 | |
| RelevanceBgt/k=0, Backbone=Llama3.1-8B-instruct2026.06 | 61 | 0 | |
| KnapsackBgt/k=0, Backbone=Llama3.1-8B-instruct2026.06 | 60 | 0 | |
| RedundancyBgt/k=5, Backbone=Llama3.1-8B-instruct2026.06 | 59 | 0.64 | |
| GreedyBgt/k=5, Backbone=Llama3.1-8B-instruct2026.06 | 58 | 4.82 | |
| KnapsackBgt/k=10, Backbone=Llama3.1-8B-instruct2026.06 | 57 | 9.55 | |
| MMRBgt/k=0, Backbone=Llama3.1-8B-instruct2026.06 | 55 | 0 |