Question Answering on NQ-Open (EM, F1, Retrieval, Latency)
57.6EMMargin
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| MarginBackbone=Llama-3.1-8B-Instruct, tau=0.502025.11 | 57.6 | 54.7 | 0.8 | 0.012 | |
| VarianceBackbone=Llama-3.1-8B-Instruct, tau=0.602025.11 | 56.8 | 53.7 | 2.6 | 0.059 | |
| EntropyBackbone=Llama-3.1-8B-Instruct, tau=0.952025.11 | 55.4 | 53.1 | 13.2 | 0.175 | |
| NeverBackbone=Llama-3.1-8B-Instruct2025.11 | 53.8 | 51.7 | 0 | 10.299 | |
| AlwaysBackbone=Llama-3.1-8B-Instruct2025.11 | 48.6 | 46.1 | 100 | 1.248 | |
| EntropyBackbone=Qwen2.5-7B-Instruct, tau=0.852025.11 | 39.6 | 39.1 | 4.6 | 0.964 | |
| MarginBackbone=Qwen2.5-7B-Instruct, tau=0.202025.11 | 39.6 | 38.8 | 30.4 | 1.295 | |
| NeverBackbone=Qwen2.5-7B-Instruct2025.11 | 38.8 | 37.7 | 0 | 3.293 | |
| VarianceBackbone=Qwen2.5-7B-Instruct, tau=0.602025.11 | 38.6 | 38 | 1.2 | 0.291 | |
| AlwaysBackbone=Qwen2.5-7B-Instruct2025.11 | 37.4 | 36.7 | 100 | 2.922 | |
| Expanded DropBackbone=DeepSeek-V2-Lite-Chat, Capacity Factor=1.02025.03 | 26.8 | — | — | — | |
| BaselineBackbone=DeepSeek-V2-Lite-Chat, Capacity Factor=1.02025.03 | 26.5 | — | — | — | |
| Token DropBackbone=DeepSeek-V2-Lite-Chat, Capacity Factor=1.02025.03 | 25.2 | — | — | — | |
| BaselineBackbone=Qwen1.5-MoE-Chat, Capacity Factor=1.02025.03 | 22.6 | — | — | — | |
| Expanded DropBackbone=Qwen1.5-MoE-Chat, Capacity Factor=1.02025.03 | 22.4 | — | — | — | |
| Token DropBackbone=Qwen1.5-MoE-Chat, Capacity Factor=1.02025.03 | 21.4 | — | — | — | |
| BaselineBackbone=OLMoE-Instruct, Capacity Factor=1.02025.03 | 17.5 | — | — | — | |
| Expanded DropBackbone=OLMoE-Instruct, Capacity Factor=1.02025.03 | 17 | — | — | — | |
| Token DropBackbone=OLMoE-Instruct, Capacity Factor=1.02025.03 | 15.9 | — | — | — |