Multilingual Question Answering on TyDiQA (1-shot macro-averaged)
48.86F1 Score (1-shot macro)Filter-then-Weight
Evaluation Results
| Method | Links | |
|---|---|---|
| Filter-then-WeightBackbone=Llama-3.2-1B, Selection Protocol=Best-of-run2026.03 | 48.86 | |
| Filter-then-WeightBackbone=Llama-3.2-1B, Selection Protocol=Fixed-budget2026.03 | 48.67 | |
| GREATSBackbone=Llama-3.2-1B, Selection Protocol=Best-of-run2026.03 | 47.5 | |
| LESSBackbone=Llama-3.2-1B, Selection Protocol=Best-of-run2026.03 | 47.21 | |
| GREATSBackbone=Llama-3.2-1B, Selection Protocol=Fixed-budget2026.03 | 47.16 | |
| TracIncBackbone=Llama-3.2-1B, Selection Protocol=Best-of-run2026.03 | 47.01 | |
| TracIncBackbone=Llama-3.2-1B, Selection Protocol=Fixed-budget2026.03 | 47.01 | |
| LESSBackbone=Llama-3.2-1B, Selection Protocol=Fixed-budget2026.03 | 46.88 | |
| Filter-then-WeightBackbone=Qwen3-0.6B, Selection Protocol=Best-of-run2026.03 | 46.29 | |
| GRAD-MATCHBackbone=Llama-3.2-1B, Selection Protocol=Best-of-run2026.03 | 45.35 | |
| GREATSBackbone=Qwen3-0.6B, Selection Protocol=Best-of-run2026.03 | 45.25 | |
| TracIncBackbone=Qwen3-0.6B, Selection Protocol=Best-of-run2026.03 | 45.17 | |
| LESSBackbone=Qwen3-0.6B, Selection Protocol=Best-of-run2026.03 | 45.07 | |
| Filter-then-WeightBackbone=Qwen3-0.6B, Selection Protocol=Fixed-budget2026.03 | 44.82 | |
| GRAD-MATCHBackbone=Qwen3-0.6B, Selection Protocol=Best-of-run2026.03 | 44.23 | |
| GRAD-MATCHBackbone=Llama-3.2-1B, Selection Protocol=Fixed-budget2026.03 | 44.15 | |
| FullBackbone=Qwen3-0.6B, Selection Protocol=Best-of-run2026.03 | 43.65 | |
| LESSBackbone=Qwen3-0.6B, Selection Protocol=Fixed-budget2026.03 | 43.57 | |
| GRAD-MATCHBackbone=Qwen3-0.6B, Selection Protocol=Fixed-budget2026.03 | 43.14 | |
| TracIncBackbone=Qwen3-0.6B, Selection Protocol=Fixed-budget2026.03 | 43.07 | |
| GREATSBackbone=Qwen3-0.6B, Selection Protocol=Fixed-budget2026.03 | 42.7 | |
| FullBackbone=Llama-3.2-1B, Selection Protocol=Best-of-run2026.03 | 42.47 | |
| FullBackbone=Qwen3-0.6B, Selection Protocol=Fixed-budget2026.03 | 41.14 | |
| FullBackbone=Llama-3.2-1B, Selection Protocol=Fixed-budget2026.03 | 38.68 | |
| OriginBackbone=Qwen3-0.6B, Selection Protocol=Best-of-run2026.03 | 20.94 | |
| OriginBackbone=Qwen3-0.6B, Selection Protocol=Fixed-budget2026.03 | 20.94 | |
| OriginBackbone=Llama-3.2-1B, Selection Protocol=Best-of-run2026.03 | 12.69 | |
| OriginBackbone=Llama-3.2-1B, Selection Protocol=Fixed-budget2026.03 | 12.69 |