Question Answering on NewsQA (F1, EM)
80.5F1Oracle
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| Oracle2026.04 | 80.5 | 60.67 | |
| EvolveRouterPrompt Optimization=Enabled, Adaptive K=Enabled2026.04 | 64.74 | 41.67 | |
| EvolveRouter (w/o Adaptive K)Prompt Optimization=Enabled, Adaptive K=Disabled2026.04 | 64.48 | 42 | |
| EvolveRouter (w/o PO)Prompt Optimization=Disabled, Adaptive K=Enabled2026.04 | 63.81 | 41.33 | |
| AgentRouter2026.04 | 63.02 | 41.33 | |
| Best Agent2026.04 | 62.8 | 37.11 | |
| GraphRouter2026.04 | 62.66 | 42.33 | |
| DOPAModel=Qwen3-1.7B2026.04 | 61.49 | 37.8 | |
| KNNModel=Qwen3-1.7B2026.04 | 61.45 | 37.3 | |
| MLP Router2026.04 | 60.94 | 37 | |
| Best LLM2026.04 | 60.55 | 39.8 | |
| CascadeRouting2026.04 | 60.17 | 39 | |
| AGAR2026.06 | 60.1 | — | |
| DOPAModel=LLaMA3.2-3B2026.04 | 59.97 | 33.2 | |
| RouteLLM2026.04 | 59.93 | 38 | |
| Majority Vote2026.04 | 59.3 | 39.5 | |
| LLM-Blender2026.04 | 59.3 | 37.33 | |
| KNNModel=LLaMA3.2-3B2026.04 | 58.61 | 32 | |
| AGARBackbone=GLM-4.1V-9B-Thinking2026.06 | 58.5 | — | |
| UniformBackbone=GLM-4.1V-9B-Thinking2026.06 | 58.4 | — | |
| VEA-ConBackbone=GLM-4.1V-9B-Thinking2026.06 | 58.4 | — | |
| Random Router2026.04 | 58.11 | 36.2 | |
| Plain VQABackbone=GLM-4.1V-9B-Thinking2026.06 | 58.1 | — | |
| VEA-BrBackbone=GLM-4.1V-9B-Thinking2026.06 | 58 | — | |
| Average2026.04 | 57.97 | 36.1 | |
| RandomBackbone=GLM-4.1V-9B-Thinking2026.06 | 57.9 | — | |
| VARBackbone=GLM-4.1V-9B-Thinking2026.06 | 57.3 | — | |
| KNN Router2026.04 | 57.15 | 34 | |
| Plain VQA2026.06 | 57.1 | — | |
| Uniform2026.06 | 56.3 | — | |
| Random2026.06 | 55.3 | — | |
| VAR2026.06 | 55.3 | — | |
| VEA-Br2026.06 | 54.8 | — | |
| VEA-Con2026.06 | 54.5 | — | |
| HybridLLM2026.04 | 54.19 | 31 | |
| SeCoBackbone=LLaMA-3.2-1B-Instruct, Compression constraint=16x2026.05 | 50.81 | 29.91 | |
| SeCoBackbone=LLaMA-3.2-1B-Instruct, Compression constraint=32x2026.05 | 49.91 | 29.91 | |
| CGRBackbone=GLM-4.1V-9B-Thinking2026.06 | 49.7 | — | |
| AGLA2026.06 | 49.6 | — | |
| AGLABackbone=GLM-4.1V-9B-Thinking2026.06 | 49.5 | — | |
| SeCoBackbone=Qwen3-4B-Instruct, Compression constraint=16x2026.05 | 48.63 | 27.4 | |
| CGR2026.06 | 47 | — | |
| SeCoBackbone=Qwen3-4B-Instruct, Compression constraint=32x2026.05 | 46.57 | 26.23 | |
| SAC (EPL)Backbone=Qwen3-4B-Instruct, Compression constraint=16x2026.05 | 41.08 | 20.25 | |
| 500x (EPL)Backbone=Qwen3-4B-Instruct, Compression constraint=16x2026.05 | 39.7 | 20.16 | |
| RAMBackbone=LLaMA-3.2-1B-Instruct, Compression constraint=16x2026.05 | 35.18 | 19.9 | |
| SAC (EPL)Backbone=Qwen3-4B-Instruct, Compression constraint=32x2026.05 | 33.88 | 15.86 | |
| 500x (EPL)Backbone=Qwen3-4B-Instruct, Compression constraint=32x2026.05 | 32.31 | 14.84 | |
| ICAE (EPL)Backbone=Qwen3-4B-Instruct, Compression constraint=16x2026.05 | 32.2 | 15.08 | |
| 500x (DPL)Backbone=Qwen3-4B-Instruct, Compression constraint=16x2026.05 | 31.06 | 14.58 | |
| 500x (DPL)Backbone=Qwen3-4B-Instruct, Compression constraint=32x2026.05 | 30.5 | 14.34 | |
| RAMBackbone=LLaMA-3.2-1B-Instruct, Compression constraint=32x2026.05 | 30.33 | 17.57 | |
| SAC (EPL)Backbone=LLaMA-3.2-1B-Instruct, Compression constraint=16x2026.05 | 27.93 | 11.89 | |
| ICAE (EPL)Backbone=Qwen3-4B-Instruct, Compression constraint=32x2026.05 | 27.81 | 12.51 | |
| COMIBackbone=LLaMA-3.2-1B-Instruct, Compression constraint=16x2026.05 | 27.13 | 14.62 | |
| 500x (EPL)Backbone=LLaMA-3.2-1B-Instruct, Compression constraint=16x2026.05 | 26.51 | 11.35 | |
| COMIBackbone=LLaMA-3.2-1B-Instruct, Compression constraint=32x2026.05 | 24.73 | 12.89 | |
| ICAE (EPL)Backbone=LLaMA-3.2-1B-Instruct, Compression constraint=16x2026.05 | 24.37 | 10.59 | |
| 500x (DPL)Backbone=LLaMA-3.2-1B-Instruct, Compression constraint=16x2026.05 | 23.06 | 10.02 | |
| SAC (EPL)Backbone=LLaMA-3.2-1B-Instruct, Compression constraint=32x2026.05 | 22.74 | 9.26 | |
| 500x (EPL)Backbone=LLaMA-3.2-1B-Instruct, Compression constraint=32x2026.05 | 22.21 | 9.76 | |
| 500x (DPL)Backbone=LLaMA-3.2-1B-Instruct, Compression constraint=32x2026.05 | 21.82 | 9.43 | |
| ICAE (EPL)Backbone=LLaMA-3.2-1B-Instruct, Compression constraint=32x2026.05 | 20.65 | 8.86 | |
| RAMBackbone=Qwen3-4B-Instruct, Compression constraint=16x2026.05 | 16.87 | 7.48 | |
| COMIBackbone=Qwen3-4B-Instruct, Compression constraint=16x2026.05 | 16.31 | 6.77 | |
| RAMBackbone=Qwen3-4B-Instruct, Compression constraint=32x2026.05 | 13.01 | 5.25 | |
| COMIBackbone=Qwen3-4B-Instruct, Compression constraint=32x2026.05 | 12.51 | 4.77 |