Question Answering on PaperQA
6Judge Scorebm25
Evaluation Results
| Method | Links | |
|---|---|---|
| bm25Backbone=Qwen3 4B, Category=Text retrieval2026.07 | 6 | |
| bm25Backbone=Qwen3 8B, Category=Text retrieval2026.07 | 5.964 | |
| embeddingBackbone=Qwen3 8B, Category=Text retrieval2026.07 | 5.842 | |
| embeddingBackbone=Qwen3 4B, Category=Text retrieval2026.07 | 5.833 | |
| bm25Backbone=Llama 3.1-8B, Category=Text retrieval2026.07 | 5.753 | |
| embeddingBackbone=Llama 3.1-8B, Category=Text retrieval2026.07 | 5.516 | |
| PMDRouterBackbone=Qwen3 8B, Category=Joint Backbone-LoRA2026.07 | 5.309 | |
| PMDRouterBackbone=Llama 3.1-8B, Category=Joint Backbone-LoRA2026.07 | 5.271 | |
| LAGBackbone=Qwen3 8B, Category=Joint Backbone-LoRA2026.07 | 5.176 | |
| PMDRouterBackbone=Qwen3 4B, Category=Joint Backbone-LoRA2026.07 | 5.093 | |
| LAGBackbone=Qwen3 4B, Category=Joint Backbone-LoRA2026.07 | 4.993 | |
| LAGBackbone=Llama 3.1-8B, Category=Joint Backbone-LoRA2026.07 | 4.522 | |
| ArrowBackbone=Llama 3.1-8B, Category=Joint Backbone-LoRA2026.07 | 4.453 | |
| L-HidL2-L4ZBackbone=Llama 3.1-8B, Category=LoRA-only2026.07 | 4.24 | |
| ArrowBackbone=Qwen3 8B, Category=Joint Backbone-LoRA2026.07 | 4.184 | |
| L-AlphaZBackbone=Llama 3.1-8B, Category=LoRA-only2026.07 | 4.164 | |
| ArrowBackbone=Qwen3 4B, Category=Joint Backbone-LoRA2026.07 | 4.08 | |
| J-WAllZBackbone=Llama 3.1-8B, Category=Joint Backbone-LoRA2026.07 | 3.944 | |
| SPECTRBackbone=Llama 3.1-8B, Category=Joint Backbone-LoRA2026.07 | 3.862 | |
| L-HidL2-LastMedBackbone=Llama 3.1-8B, Category=LoRA-only2026.07 | 3.836 | |
| B-CtrL2Backbone=Llama 3.1-8B, Category=Backbone-only2026.07 | 3.727 | |
| J-GlobalBackbone=Llama 3.1-8B, Category=Joint Backbone-LoRA2026.07 | 3.7 | |
| B-LogitVarBackbone=Llama 3.1-8B, Category=Backbone-only2026.07 | 3.696 | |
| B-S4aBackbone=Llama 3.1-8B, Category=Backbone-only2026.07 | 3.691 | |
| B-CtrL2Backbone=Qwen3 4B, Category=Backbone-only2026.07 | 3.604 | |
| J-MLPGlobalBackbone=Llama 3.1-8B, Category=Joint Backbone-LoRA2026.07 | 3.602 | |
| J-GlobalBackbone=Qwen3 8B, Category=Joint Backbone-LoRA2026.07 | 3.596 | |
| L-AlphaZBackbone=Qwen3 4B, Category=LoRA-only2026.07 | 3.564 | |
| L-AlphaZBackbone=Qwen3 8B, Category=LoRA-only2026.07 | 3.507 | |
| L-HidL2-L4ZBackbone=Qwen3 4B, Category=LoRA-only2026.07 | 3.496 | |
| B-S4aBackbone=Qwen3 4B, Category=Backbone-only2026.07 | 3.469 | |
| J-WAllZBackbone=Qwen3 8B, Category=Joint Backbone-LoRA2026.07 | 3.464 | |
| B-S4aBackbone=Qwen3 8B, Category=Backbone-only2026.07 | 3.436 | |
| L-HidL2-L4ZBackbone=Qwen3 8B, Category=LoRA-only2026.07 | 3.413 | |
| L-HidL2-LastMedBackbone=Qwen3 8B, Category=LoRA-only2026.07 | 3.411 | |
| SPECTRBackbone=Qwen3 8B, Category=Joint Backbone-LoRA2026.07 | 3.362 | |
| B-LogitVarBackbone=Qwen3 8B, Category=Backbone-only2026.07 | 3.331 | |
| B-CtrL2Backbone=Qwen3 8B, Category=Backbone-only2026.07 | 3.304 | |
| SPECTRBackbone=Qwen3 4B, Category=Joint Backbone-LoRA2026.07 | 3.3 | |
| B-LogitVarBackbone=Qwen3 4B, Category=Backbone-only2026.07 | 3.26 | |
| J-MLPGlobalBackbone=Qwen3 8B, Category=Joint Backbone-LoRA2026.07 | 3.244 | |
| L-HidL2-LastMedBackbone=Qwen3 4B, Category=LoRA-only2026.07 | 3.231 | |
| J-GlobalBackbone=Qwen3 4B, Category=Joint Backbone-LoRA2026.07 | 3.231 | |
| J-MLPGlobalBackbone=Qwen3 4B, Category=Joint Backbone-LoRA2026.07 | 3.189 | |
| J-WAllZBackbone=Qwen3 4B, Category=Joint Backbone-LoRA2026.07 | 3.096 |