Math Word Problem Solving on SVAMP (Performance & Cost Metrics)
98.68AccuracySIGMA
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| SIGMABase LLM=Qwen3-8B, Multi-agent execution=Full, Communication-topology design=Full, Skill-based composition=Full2026.06 | 98.68 | — | — | — | |
| G-DesignerBase LLM=Qwen3-8B, Multi-agent execution=Full, Communication-topology design=Full, Skill-based composition=No2026.06 | 98.45 | — | — | — | |
| ARG-DesignerBase LLM=Qwen3-8B, Multi-agent execution=Full, Communication-topology design=Full, Skill-based composition=No2026.06 | 98.21 | — | — | — | |
| CARDBase LLM=Qwen3-8B, Multi-agent execution=Full, Communication-topology design=Full, Skill-based composition=No2026.06 | 98.13 | — | — | — | |
| GPTSwarmBase LLM=Qwen3-8B, Multi-agent execution=Full, Communication-topology design=Partial, Skill-based composition=No2026.06 | 97.85 | — | — | — | |
| LLM-DebateBase LLM=Qwen3-8B, Multi-agent execution=Full, Communication-topology design=No, Skill-based composition=No2026.06 | 97.62 | — | — | — | |
| CoTBase LLM=Qwen3-8B, Multi-agent execution=No, Communication-topology design=No, Skill-based composition=No2026.06 | 97.01 | — | — | — | |
| VanillaBase LLM=Qwen3-8B, Multi-agent execution=No, Communication-topology design=No, Skill-based composition=No2026.06 | 96.68 | — | — | — | |
| SIGMABase LLM=GPT-OSS-120B, Multi-agent execution=Full, Communication-topology design=Full, Skill-based composition=Full2026.06 | 95.94 | — | — | — | |
| CARDBase LLM=GPT-OSS-120B, Multi-agent execution=Full, Communication-topology design=Full, Skill-based composition=No2026.06 | 95.48 | — | — | — | |
| G-DesignerBase LLM=GPT-OSS-120B, Multi-agent execution=Full, Communication-topology design=Full, Skill-based composition=No2026.06 | 95.21 | — | — | — | |
| CoTBase LLM=GPT-OSS-120B, Multi-agent execution=No, Communication-topology design=No, Skill-based composition=No2026.06 | 95.19 | — | — | — | |
| VanillaBase LLM=GPT-OSS-120B, Multi-agent execution=No, Communication-topology design=No, Skill-based composition=No2026.06 | 95.07 | — | — | — | |
| LLM-DebateBase LLM=GPT-OSS-120B, Multi-agent execution=Full, Communication-topology design=No, Skill-based composition=No2026.06 | 94.89 | — | — | — | |
| ARG-DesignerBase LLM=GPT-OSS-120B, Multi-agent execution=Full, Communication-topology design=Full, Skill-based composition=No2026.06 | 94.81 | — | — | — | |
| GPTSwarmBase LLM=GPT-OSS-120B, Multi-agent execution=Full, Communication-topology design=Partial, Skill-based composition=No2026.06 | 94.77 | — | — | — | |
| SIGMABase LLM=GPT-4o-mini, Multi-agent execution=Full, Communication-topology design=Full, Skill-based composition=Full2026.06 | 93.85 | — | — | — | |
| CARDBase LLM=GPT-4o-mini, Multi-agent execution=Full, Communication-topology design=Full, Skill-based composition=No2026.06 | 91.62 | — | — | — | |
| ARG-DesignerBase LLM=GPT-4o-mini, Multi-agent execution=Full, Communication-topology design=Full, Skill-based composition=No2026.06 | 91.53 | — | — | — | |
| G-DesignerBase LLM=GPT-4o-mini, Multi-agent execution=Full, Communication-topology design=Full, Skill-based composition=No2026.06 | 90.37 | — | — | — | |
| GPTSwarmBase LLM=GPT-4o-mini, Multi-agent execution=Full, Communication-topology design=Partial, Skill-based composition=No2026.06 | 89.21 | — | — | — | |
| LLM-DebateBase LLM=GPT-4o-mini, Multi-agent execution=Full, Communication-topology design=No, Skill-based composition=No2026.06 | 87.67 | — | — | — | |
| CoTBase LLM=GPT-4o-mini, Multi-agent execution=No, Communication-topology design=No, Skill-based composition=No2026.06 | 86.89 | — | — | — | |
| VanillaBase LLM=GPT-4o-mini, Multi-agent execution=No, Communication-topology design=No, Skill-based composition=No2026.06 | 86.12 | — | — | — | |
| LLM w/o cascadeCascade System=Qwen2.5-3B + DistilBERT-base + Phi-3.5-MoE 16x3.8B2026.05 | 76.5 | 92.71 | 9.02 | 100 | |
| InitialCascade System=Qwen2.5-3B + DistilBERT-base + Phi-3.5-MoE 16x3.8B2026.05 | 75 | 54.17 | 6.09 | 57.5 | |
| Joint-target attackCascade System=Qwen2.5-3B + DistilBERT-base + Phi-3.5-MoE 16x3.8B, Setting=P=12026.05 | 69 | 84.21 | 8.24 | 87 | |
| Random noiseCascade System=Qwen2.5-3B + DistilBERT-base + Phi-3.5-MoE 16x3.8B2026.05 | 67.5 | 69.55 | 7.04 | 71.5 | |
| Joint-target attackCascade System=Qwen2.5-3B + DistilBERT-base + Phi-3.5-MoE 16x3.8B, Setting=P=0.82026.05 | 61.5 | 70.54 | 7.11 | 72.5 | |
| Single-target attackCascade System=Qwen2.5-3B + DistilBERT-base + Phi-3.5-MoE 16x3.8B, Setting=SLM only2026.05 | 61 | 67.47 | 6.83 | 68 | |
| Single-target attackCascade System=Qwen2.5-3B + DistilBERT-base + Phi-3.5-MoE 16x3.8B, Setting=DM only2026.05 | 54.5 | 49.86 | 5.29 | 50.5 | |
| Joint-target attackCascade System=Qwen2.5-3B + DistilBERT-base + Phi-3.5-MoE 16x3.8B, Setting=P=0.62026.05 | 53 | 59.73 | 6.31 | 61 | |
| Joint-target attackCascade System=Qwen2.5-3B + DistilBERT-base + Phi-3.5-MoE 16x3.8B, Setting=P=0.42026.05 | 44.5 | 48.17 | 5.07 | 49.5 | |
| Joint-target attackCascade System=Qwen2.5-3B + DistilBERT-base + Phi-3.5-MoE 16x3.8B, Setting=P=0.22026.05 | 37.5 | 39.62 | 4.33 | 40 | |
| Joint-target attackCascade System=Qwen2.5-3B + DistilBERT-base + Phi-3.5-MoE 16x3.8B, Setting=P=02026.05 | 25.5 | 26.49 | 2.94 | 23 |