Science Question Answering on GPQA (test)
78.18AccuracySePO-Generalist
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| SePO-GeneralistTask Agent=DeepSeek-V3.2, Prompt Agent=Gemini 3.1 Pro Preview2026.06 | 78.18 | — | — | |
| SePO-SpecialistTask Agent=DeepSeek-V3.2, Prompt Agent=Gemini 3.1 Pro Preview2026.06 | 76.72 | — | — | |
| Manual-CoTTask Agent=DeepSeek-V3.2, Prompt Agent=Gemini 3.1 Pro Preview2026.06 | 76.46 | — | — | |
| MetaSPOTask Agent=DeepSeek-V3.2, Prompt Agent=Gemini 3.1 Pro Preview2026.06 | 75.51 | — | — | |
| TextGradTask Agent=DeepSeek-V3.2, Prompt Agent=Gemini 3.1 Pro Preview2026.06 | 74.44 | — | — | |
| ExpRAGModel=Qwen3-8B2026.02 | 65.7 | — | — | |
| Ours (theory-guided context selection strategy)Model=Qwen3-8B2026.02 | 65.7 | — | — | |
| BM25Model=Qwen3-8B2026.02 | 65.2 | — | — | |
| DCModel=Qwen3-8B2026.02 | 65.2 | — | — | |
| ReMemModel=Qwen3-8B2026.02 | 65.2 | — | — | |
| ZeroModel=Qwen3-8B2026.02 | 64.6 | — | — | |
| DebateOCRBackbone Model=Pixtral-12B, Debate Rounds=5, Agents=32026.01 | 38.3 | 4.9 | 119 | |
| TS-MADBackbone Model=Pixtral-12B, Debate Rounds=5, Agents=32026.01 | 37.9 | 21.9 | 283.7 | |
| T-MADBackbone Model=Pixtral-12B, Debate Rounds=5, Agents=32026.01 | 36.4 | 73.9 | 268.1 | |
| DebateOCRBackbone Model=Qwen2.5-VL-7B, Debate Rounds=5, Agents=32026.01 | 34.8 | 4.9 | 66.9 | |
| DebateOCRBackbone Model=Llama-3.2-11B, Debate Rounds=5, Agents=32026.01 | 34.4 | 4.9 | 111.7 | |
| TS-MADBackbone Model=Llama-3.2-11B, Debate Rounds=5, Agents=32026.01 | 33.9 | 23.4 | 265.8 | |
| TS-MADBackbone Model=Qwen2.5-VL-7B, Debate Rounds=5, Agents=32026.01 | 33.8 | 22.5 | 159.6 | |
| T-MADBackbone Model=Llama-3.2-11B, Debate Rounds=5, Agents=32026.01 | 33 | 78.9 | 251.4 | |
| Ours (theory-guided context selection strategy)Model=Llama-3.1-8B2026.02 | 32.8 | — | — | |
| T-MADBackbone Model=Qwen2.5-VL-7B, Debate Rounds=5, Agents=32026.01 | 32.6 | 75.9 | 150.8 | |
| DCModel=Llama-3.1-8B2026.02 | 32.3 | — | — | |
| ReMemModel=Llama-3.1-8B2026.02 | 32.3 | — | — | |
| BM25Model=Llama-3.1-8B2026.02 | 31.8 | — | — | |
| ExpRAGModel=Llama-3.1-8B2026.02 | 31.8 | — | — | |
| ZeroModel=Llama-3.1-8B2026.02 | 31.3 | — | — | |
| TS-MADBackbone Model=InternVL-8B, Debate Rounds=5, Agents=32026.01 | 23.3 | 20.7 | 177.3 | |
| DebateOCRBackbone Model=InternVL-8B, Debate Rounds=5, Agents=32026.01 | 22.8 | 4.9 | 74.4 | |
| T-MADBackbone Model=InternVL-8B, Debate Rounds=5, Agents=32026.01 | 22.1 | 69.9 | 167.6 |