Accuracy on MedQA (Medical Question Answering)
89.41AccuracySpurious Universal
Evaluation results
| Method | Links | |
|---|---|---|
| Spurious UniversalModel=Qwen/Qwen3.5-27B2026.05 | 89.41 | |
| SpuriousModel=Qwen/Qwen3.5-27B2026.05 | 88.13 | |
| CascadeDebateModel Family=Llama-3.22026.04 | 86.44 | |
| Self-AskModel=Qwen/Qwen3.5-27B2026.05 | 82.88 | |
| PromptWizardModel=Qwen/Qwen3.5-27B2026.05 | 82.34 | |
| Least-to-MostModel=Qwen/Qwen3.5-27B2026.05 | 81.93 | |
| LatentMASModel=Qwen3-14B2026.06 | 80.7 | |
| ReLATModel=Qwen3-14B2026.06 | 80.5 | |
| TextMASModel=Qwen3-14B2026.06 | 80.3 | |
| AnalogicalModel=Qwen/Qwen3.5-27B2026.05 | 80.28 | |
| Plan-and-SolveModel=Qwen/Qwen3.5-27B2026.05 | 77.38 | |
| OriginalBackbone=Llama-3-8B-UltraMedical2026.03 | 77 | |
| TRACE-LFLLM=Qwen3-32B2026.05 | 76.83 | |
| TRACE-CSLLM=Qwen3-32B2026.05 | 76.28 | |
| DMTLLM=Qwen3-32B2026.05 | 75.96 | |
| LatentMASModel=Qwen3-8B2026.06 | 75.3 | |
| CascadeDebateModel Family=Qwen2.52026.04 | 75.22 | |
| TextMASModel=Qwen3-8B2026.06 | 75 | |
| SFT-OracleBackbone=Phi-3.5-mini2026.05 | 74.07 | |
| SafeReActBackbone=Llama-3-8B-UltraMedical2026.03 | 74 | |
| ReLATModel=Qwen3-8B2026.06 | 74 | |
| SFT-OracleBackbone=LLaMA3-8B2026.05 | 73.76 | |
| Zero-Shot CoTModel=Qwen/Qwen3.5-27B2026.05 | 73.45 | |
| Joint Fine-tuningLLM=Qwen3-32B2026.05 | 73.37 | |
| Re-ReadingModel=Qwen/Qwen3.5-27B2026.05 | 71.72 | |
| HSIR-SFTBackbone=Phi-3.5-mini2026.05 | 71.17 | |
| SFT-OracleBackbone=Mistral-7B2026.05 | 70.15 | |
| AceMADBase Model=Qwen3-235B-A22B-Instruct, Iteration Rounds (T)=2, Number of Agents (N)=52026.03 | 69.68 | |
| HSIR-SFTBackbone=LLaMA3-8B2026.05 | 69.68 | |
| TRACE-CSLLM=Qwen2.5-14B2026.05 | 69.29 | |
| Step-BackModel=Qwen/Qwen3.5-27B2026.05 | 68.42 | |
| Standard CascadeModel Family=Llama-3.22026.04 | 68.2 | |
| ReLATModel=Qwen3-4B2026.06 | 67.8 | |
| ReSTEMBackbone=LLaMA3-8B2026.05 | 67.79 | |
| Joint Fine-tuningLLM=Qwen2.5-14B2026.05 | 67.71 | |
| ReSTEMBackbone=Phi-3.5-mini2026.05 | 67.64 | |
| TRACE-LFLLM=Qwen2.5-14B2026.05 | 67.64 | |
| Sequential Fine-tuningLLM=Qwen3-32B2026.05 | 67.4 | |
| DMTLLM=Qwen2.5-14B2026.05 | 66.69 | |
| Sequential LoRALLM=Qwen2.5-14B2026.05 | 66.38 | |
| LatentMASModel=Qwen3-4B2026.06 | 66.3 | |
| SFT-InitialBackbone=Phi-3.5-mini2026.05 | 66.14 | |
| STaRBackbone=Phi-3.5-mini2026.05 | 65.99 | |
| Sequential Fine-tuningLLM=Qwen2.5-14B2026.05 | 65.75 | |
| TextMASModel=Qwen3-4B2026.06 | 65.3 | |
| Citrus-VSize=8B2026.03 | 64.9 | |
| SingleModel=Qwen3-14B2026.06 | 64.7 | |
| SFT-InitialBackbone=LLaMA3-8B2026.05 | 64.57 | |
| S_multi(M_large)Model Family=Llama-3.22026.04 | 64 | |
| MEDIC-ADSize=7B2026.03 | 63.6 | |
| LingshuSize=7B2026.03 | 63.3 | |
| TRACE-CSLLM=LLaMA3-8B2026.05 | 62.21 | |
| AutoAdaptSetting=Template Aware (TA), Technique=SFT2026.03 | 61.27 | |
| HSIR-SFTBackbone=Mistral-7B2026.05 | 61.27 | |
| Sparse MADBase Model=Qwen3-235B-A22B-Instruct, Number of Agents (N)=52026.03 | 60.97 | |
| S_single(M_large)Model Family=Llama-3.22026.04 | 60.89 | |
| DS-AgentSetting=Template Aware (TA), Technique=SFT2026.03 | 59.86 | |
| STaRBackbone=LLaMA3-8B2026.05 | 59.63 | |
| Decentralized MADBase Model=Qwen3-235B-A22B-Instruct, Number of Agents (N)=52026.03 | 59.03 | |
| AutoMLAgentSetting=Template Aware (TA), Technique=SFT2026.03 | 58.84 | |
| TRACE-LFLLM=LLaMA3-8B2026.05 | 58.37 | |
| HF DefaultsSetting=Template Aware (TA), Technique=SFT2026.03 | 58.13 | |
| ReSTEMBackbone=Mistral-7B2026.05 | 58.13 | |
| Joint Fine-tuningLLM=LLaMA3-8B2026.05 | 57.66 | |
| MLCopilotSetting=Template Aware (TA), Technique=SFT2026.03 | 57.5 | |
| evaluation-instructed prompt optimizationBackbone=GPT-4o, Optimization Type=Query-dependent2025.11 | 57 | |
| Majority VotingBase Model=Qwen3-235B-A22B-Instruct, Number of Agents (N)=52026.03 | 56.45 | |
| SFT-InitialBackbone=Mistral-7B2026.05 | 55.93 | |
| Single AgentBase Model=Qwen3-235B-A22B-Instruct, Number of Agents (N)=52026.03 | 55.81 | |
| SFT-OracleBackbone=Qwen3-1.7B2026.05 | 55.77 | |
| HSIR-SFTBackbone=Qwen3-1.7B2026.05 | 55.3 | |
| DMTLLM=LLaMA3-8B2026.05 | 55.3 | |
| Centralized MADBase Model=Qwen3-235B-A22B-Instruct, Number of Agents (N)=52026.03 | 54.84 | |
| ReSTEMBackbone=Qwen3-1.7B2026.05 | 54.67 | |
| Sequential LoRALLM=LLaMA3-8B2026.05 | 54.52 | |
| Self-RefineBackbone=GPT-4o, Optimization Type=Query-dependent2025.11 | 54 | |
| HSIR-DPOIteration=2, Backbone=Qwen2.5-7B, Training Dataset=GSM8K2026.05 | 53.57 | |
| STaRBackbone=Mistral-7B2026.05 | 53.57 | |
| HSIR-DPOIteration=1, Backbone=Qwen2.5-7B, Training Dataset=GSM8K2026.05 | 53.49 | |
| HSIR-DPOIteration=3, Backbone=Qwen2.5-7B, Training Dataset=GSM8K2026.05 | 53.47 | |
| SingleModel=Qwen3-8B2026.06 | 53 | |
| STaRBackbone=Qwen3-1.7B2026.05 | 52.87 | |
| Standard CascadeModel Family=Qwen2.52026.04 | 52.7 | |
| IRPOIteration=1, Backbone=Qwen2.5-7B, Training Dataset=GSM8K2026.05 | 52.63 | |
| evaluation-instructed prompt optimizationBackbone=LLaMA-3, Optimization Type=Query-dependent2025.11 | 52 | |
| SFT-InitialBackbone=Qwen3-1.7B2026.05 | 51.61 | |
| Pro-RefineBackbone=LLaMA-3, Optimization Type=Query-dependent2025.11 | 51 | |
| evaluation-instructed prompt optimizationBackbone=LLaMA-3.1, Optimization Type=Query-dependent2025.11 | 51 | |
| Pro-RefineBackbone=LLaMA-3.1, Optimization Type=Query-dependent2025.11 | 51 | |
| LLM onlyBackbone=GPT-4o, Optimization Type=None2025.11 | 51 | |
| APEBackbone=GPT-4o, Optimization Type=Template2025.11 | 51 | |
| Self-RefineBackbone=LLaMA-3, Optimization Type=Query-dependent2025.11 | 50 | |
| TextGradBackbone=LLaMA-3, Optimization Type=Template2025.11 | 50 | |
| Sequential LoRALLM=Qwen3-32B2026.05 | 49.8 | |
| Spurious UniversalModel=allenai/Olmo-3-7B-Instruct2026.05 | 49.39 | |
| APEBackbone=LLaMA-3, Optimization Type=Template2025.11 | 49 | |
| Self-RefineBackbone=LLaMA-3.1, Optimization Type=Query-dependent2025.11 | 49 | |
| TextGradBackbone=GPT-4o, Optimization Type=Template2025.11 | 49 | |
| SpuriousModel=allenai/Olmo-3-7B-Instruct2026.05 | 48.99 | |
| IRPOIteration=2, Backbone=Qwen2.5-7B, Training Dataset=GSM8K2026.05 | 48.39 |