Accuracy on MuSiQue (Question Answering)
71.5Accuracy (MuSiQue QA)BRANE
Evaluation Results
| Method | Links | |
|---|---|---|
| BRANEBudget Goal=50%2026.05 | 71.5 | |
| CARROT-KNN (Extended)Budget Goal=50%2026.05 | 68.9 | |
| Best Static Config. (Murakkab)Budget Goal=50%2026.05 | 68.17 | |
| CARROT-RoBERTa (Extended)Budget Goal=50%2026.05 | 67.5 | |
| CARROT-RoBERTa (LLM-only)Budget Goal=50%2026.05 | 65.6 | |
| CARROT-KNN (LLM-only)Budget Goal=50%2026.05 | 58.3 | |
| Adaptive-RAGBudget Goal=50%2026.05 | 56.5 | |
| METISBudget Goal=50%2026.05 | 48.5 | |
| IRCoTBackbone=GPT-4o2026.03 | 44.2 | |
| Iter-RetGenBackbone=GPT-4o2026.03 | 42 | |
| MARCHBackbone=Llama3.1-8B, Prompting=10-Shots2026.03 | 40.8 | |
| RAG-StarBackbone=GPT-4o2026.03 | 40 | |
| MARCHBackbone=Llama3.1-8B, Prompting=CoT2026.03 | 36.2 | |
| FE-SFT & FC-RLBackbone=Youtu-LLM-2B-Base, Framework=WM-AMT, Optimization Stage=RL stage2026.06 | 34.2 | |
| IWM & RLBackbone=Youtu-LLM-2B-Base, Framework=Standard, Optimization Stage=RL stage2026.06 | 33.2 | |
| FE-SFT & RLBackbone=Youtu-LLM-2B-Base, Framework=WM-AMT, Optimization Stage=RL stage2026.06 | 32.6 | |
| RQ-RAGBackbone=GPT-4o2026.03 | 31.6 | |
| State-SFT & RL†Backbone=Youtu-LLM-2B-Base, Framework=Standard, Optimization Stage=RL stage2026.06 | 31.4 | |
| State-SFT & RL‡Backbone=Youtu-LLM-2B-Base, Framework=WM-AMT, Optimization Stage=RL stage2026.06 | 30.4 | |
| SFT & RL*Backbone=Youtu-LLM-2B-Base, Framework=Standard, Optimization Stage=RL stage2026.06 | 30.2 | |
| RAG-StandardBackbone=GPT-4o2026.03 | 29.8 | |
| FE-SFT & RL*Backbone=Youtu-LLM-2B-Base, Framework=Standard, Optimization Stage=RL stage2026.06 | 29.8 | |
| IRCoTBackbone=Llama3.1-8B2026.03 | 29.2 | |
| CoTBackbone=GPT-4o2026.03 | 26.8 | |
| FE-SFTBackbone=Youtu-LLM-2B-Base, Framework=WM-AMT, Optimization Stage=SFT stage2026.06 | 25.6 | |
| IWMBackbone=Youtu-LLM-2B-Base, Framework=Standard, Optimization Stage=SFT stage2026.06 | 25.4 | |
| SFT & RLBackbone=Youtu-LLM-2B-Base, Framework=WM-AMT, Optimization Stage=RL stage2026.06 | 25.4 | |
| FE-SFT*Backbone=Youtu-LLM-2B-Base, Framework=Standard, Optimization Stage=SFT stage2026.06 | 23.2 | |
| RAFTBackbone=Llama3.1-8B2026.03 | 22 | |
| SFT*Backbone=Youtu-LLM-2B-Base, Framework=Standard, Optimization Stage=SFT stage2026.06 | 21 | |
| SFTBackbone=Youtu-LLM-2B-Base, Framework=WM-AMT, Optimization Stage=SFT stage2026.06 | 20.6 | |
| GenGroundBackbone=7B-class2026.03 | 20.2 | |
| BaselineBackbone=Llama3.1-8B2026.03 | 5.6 |