LoRA Routing on PaperQA PMD-BENCH
92.3AccuracyPMDRouter
Evaluation Results
| Method | Links | |
|---|---|---|
| PMDRouterBackbone=Qwen3-8B, Zero-shot=true2026.07 | 92.3 | |
| PMDRouterBackbone=Qwen3-4B-Instruct, Zero-shot=true2026.07 | 89.7 | |
| embeddingBackbone=Llama-3.1-8B2026.07 | 88.4 | |
| embeddingBackbone=Qwen3-4B-Instruct2026.07 | 88.4 | |
| embeddingBackbone=Qwen3-8B2026.07 | 88.4 | |
| bm25Backbone=Llama-3.1-8B2026.07 | 82 | |
| bm25Backbone=Qwen3-4B-Instruct2026.07 | 82 | |
| bm25Backbone=Qwen3-8B2026.07 | 82 | |
| embeddingBackbone=Llama-3.1-8B2026.07 | 76.2 | |
| embeddingBackbone=Qwen3-4B-Instruct2026.07 | 76.2 | |
| embeddingBackbone=Qwen3-8B2026.07 | 76.2 | |
| bm25Backbone=Llama-3.1-8B2026.07 | 73.5 | |
| bm25Backbone=Qwen3-4B-Instruct2026.07 | 73.5 | |
| bm25Backbone=Qwen3-8B2026.07 | 73.5 | |
| PMDRouterBackbone=Llama-3.1-8B, Zero-shot=true2026.07 | 66.3 | |
| PMDRouterBackbone=Qwen3-8B, Zero-shot=true2026.07 | 65.8 | |
| LAGBackbone=Qwen3-8B, Zero-shot=true2026.07 | 63.6 | |
| PMDRouterBackbone=Qwen3-4B-Instruct, Zero-shot=true2026.07 | 61.3 | |
| PMDRouterBackbone=Llama-3.1-8B, Zero-shot=true2026.07 | 60 | |
| LAGBackbone=Qwen3-4B-Instruct, Zero-shot=true2026.07 | 59.8 | |
| LAGBackbone=Llama-3.1-8B, Zero-shot=true2026.07 | 44 | |
| L-AlphaZBackbone=Qwen3-4B-Instruct, Zero-shot=true2026.07 | 40.6 | |
| L-HidL2-LastMedBackbone=Qwen3-4B-Instruct, Zero-shot=true2026.07 | 37.8 | |
| L-HidL2-L4ZBackbone=Qwen3-4B-Instruct, Zero-shot=true2026.07 | 37.7 | |
| B-CtrL2Backbone=Qwen3-4B-Instruct, Zero-shot=true2026.07 | 36.5 | |
| ArrowBackbone=Llama-3.1-8B, Zero-shot=true2026.07 | 36.4 | |
| SPECTRBackbone=Llama-3.1-8B, Zero-shot=true2026.07 | 35.8 | |
| J-GlobalBackbone=Llama-3.1-8B, Zero-shot=true2026.07 | 35.7 | |
| ArrowBackbone=Qwen3-4B-Instruct, Zero-shot=true2026.07 | 34.2 | |
| LAGBackbone=Qwen3-4B-Instruct, Zero-shot=true2026.07 | 32.6 | |
| LAGBackbone=Llama-3.1-8B, Zero-shot=true2026.07 | 31.5 | |
| ArrowBackbone=Qwen3-8B, Zero-shot=true2026.07 | 31.1 | |
| L-AlphaZBackbone=Llama-3.1-8B, Zero-shot=true2026.07 | 30.9 | |
| SPECTRBackbone=Qwen3-4B-Instruct, Zero-shot=true2026.07 | 30.2 | |
| J-GlobalBackbone=Qwen3-4B-Instruct, Zero-shot=true2026.07 | 30.1 | |
| J-MLPGlobalBackbone=Qwen3-4B-Instruct, Zero-shot=true2026.07 | 30 | |
| B-LogitVarBackbone=Qwen3-4B-Instruct, Zero-shot=true2026.07 | 28.8 | |
| B-CtrL2Backbone=Llama-3.1-8B, Zero-shot=true2026.07 | 28.7 | |
| L-AlphaZBackbone=Qwen3-8B, Zero-shot=true2026.07 | 28.5 | |
| L-HidL2-LastMedBackbone=Llama-3.1-8B, Zero-shot=true2026.07 | 28.2 | |
| L-HidL2-L4ZBackbone=Llama-3.1-8B, Zero-shot=true2026.07 | 26.9 | |
| B-LogitVarBackbone=Llama-3.1-8B, Zero-shot=true2026.07 | 26.3 | |
| J-WAllZBackbone=Llama-3.1-8B, Zero-shot=true2026.07 | 25.6 | |
| B-S4aBackbone=Qwen3-4B-Instruct, Zero-shot=true2026.07 | 25.6 | |
| L-AlphaZBackbone=Qwen3-4B-Instruct, Zero-shot=true2026.07 | 22.2 | |
| J-WAllZBackbone=Qwen3-4B-Instruct, Zero-shot=true2026.07 | 21.9 | |
| LAGBackbone=Qwen3-8B, Zero-shot=true2026.07 | 21.7 | |
| SPECTRBackbone=Llama-3.1-8B, Zero-shot=true2026.07 | 18.7 | |
| B-CtrL2Backbone=Qwen3-8B, Zero-shot=true2026.07 | 18.2 | |
| L-HidL2-L4ZBackbone=Qwen3-8B, Zero-shot=true2026.07 | 18 | |
| L-HidL2-L4ZBackbone=Llama-3.1-8B, Zero-shot=true2026.07 | 17.9 | |
| B-S4aBackbone=Qwen3-8B, Zero-shot=true2026.07 | 17.8 | |
| L-HidL2-LastMedBackbone=Qwen3-8B, Zero-shot=true2026.07 | 17.8 | |
| L-HidL2-L4ZBackbone=Qwen3-4B-Instruct, Zero-shot=true2026.07 | 17.1 | |
| B-LogitVarBackbone=Qwen3-8B, Zero-shot=true2026.07 | 17 | |
| L-AlphaZBackbone=Llama-3.1-8B, Zero-shot=true2026.07 | 16.2 | |
| ArrowBackbone=Llama-3.1-8B, Zero-shot=true2026.07 | 15.8 | |
| B-CtrL2Backbone=Qwen3-4B-Instruct, Zero-shot=true2026.07 | 14.2 | |
| ArrowBackbone=Qwen3-4B-Instruct, Zero-shot=true2026.07 | 13.9 | |
| J-GlobalBackbone=Qwen3-8B, Zero-shot=true2026.07 | 13.8 | |
| B-S4aBackbone=Qwen3-4B-Instruct, Zero-shot=true2026.07 | 13.6 | |
| SPECTRBackbone=Qwen3-8B, Zero-shot=true2026.07 | 12.7 | |
| B-S4aBackbone=Llama-3.1-8B, Zero-shot=true2026.07 | 12.6 | |
| J-MLPGlobalBackbone=Llama-3.1-8B, Zero-shot=true2026.07 | 12.5 | |
| L-HidL2-LastMedBackbone=Llama-3.1-8B, Zero-shot=true2026.07 | 12.2 | |
| B-LogitVarBackbone=Llama-3.1-8B, Zero-shot=true2026.07 | 12 | |
| J-MLPGlobalBackbone=Qwen3-4B-Instruct, Zero-shot=true2026.07 | 12 | |
| ArrowBackbone=Qwen3-8B, Zero-shot=true2026.07 | 10.8 | |
| J-MLPGlobalBackbone=Qwen3-8B, Zero-shot=true2026.07 | 10.2 | |
| SPECTRBackbone=Qwen3-4B-Instruct, Zero-shot=true2026.07 | 10 | |
| J-GlobalBackbone=Qwen3-4B-Instruct, Zero-shot=true2026.07 | 9.6 | |
| J-GlobalBackbone=Qwen3-8B, Zero-shot=true2026.07 | 9.1 | |
| L-AlphaZBackbone=Qwen3-8B, Zero-shot=true2026.07 | 8.9 | |
| B-LogitVarBackbone=Qwen3-4B-Instruct, Zero-shot=true2026.07 | 8.2 | |
| J-WAllZBackbone=Llama-3.1-8B, Zero-shot=true2026.07 | 8 | |
| J-WAllZBackbone=Qwen3-4B-Instruct, Zero-shot=true2026.07 | 8 | |
| L-HidL2-L4ZBackbone=Qwen3-8B, Zero-shot=true2026.07 | 8 | |
| B-CtrL2Backbone=Llama-3.1-8B, Zero-shot=true2026.07 | 7.8 | |
| B-LogitVarBackbone=Qwen3-8B, Zero-shot=true2026.07 | 7.8 | |
| J-MLPGlobalBackbone=Llama-3.1-8B, Zero-shot=true2026.07 | 7.3 | |
| L-HidL2-LastMedBackbone=Qwen3-4B-Instruct, Zero-shot=true2026.07 | 7.3 | |
| J-MLPGlobalBackbone=Qwen3-8B, Zero-shot=true2026.07 | 7.1 | |
| L-HidL2-LastMedBackbone=Qwen3-8B, Zero-shot=true2026.07 | 6.9 | |
| B-S4aBackbone=Llama-3.1-8B, Zero-shot=true2026.07 | 6.7 | |
| B-CtrL2Backbone=Qwen3-8B, Zero-shot=true2026.07 | 6.7 | |
| B-S4aBackbone=Qwen3-8B, Zero-shot=true2026.07 | 6.7 | |
| J-WAllZBackbone=Qwen3-8B, Zero-shot=true2026.07 | 6.6 | |
| J-WAllZBackbone=Qwen3-8B, Zero-shot=true2026.07 | 6.4 | |
| J-GlobalBackbone=Llama-3.1-8B, Zero-shot=true2026.07 | 6.2 | |
| SPECTRBackbone=Qwen3-8B, Zero-shot=true2026.07 | 1.7 |