Massive Multitask Language Understanding on MMLU (Accuracy and Latency)
82.57AccuracyAgentDropout
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| AgentDropoutBackbone=Qwen2.5-72B-Instruct, Topology=Star, Training-Free (TF)=false2026.05 | 82.57 | 180.62 | |
| AgentDropoutBackbone=Qwen2.5-72B-Instruct, Topology=FullConnected, Training-Free (TF)=false2026.05 | 81.92 | 184.71 | |
| Vanilla MASBackbone=Qwen2.5-72B-Instruct, Topology=Star, Training-Free (TF)=true2026.05 | 81.7 | 235.17 | |
| Vanilla MASBackbone=Qwen2.5-72B-Instruct, Topology=Random, Training-Free (TF)=true2026.05 | 81.48 | 148.03 | |
| AgentDropoutBackbone=Qwen2.5-72B-Instruct, Topology=Layered, Training-Free (TF)=false2026.05 | 81.48 | 121.07 | |
| Vanilla MASBackbone=Qwen2.5-72B-Instruct, Topology=Chain, Training-Free (TF)=true2026.05 | 81.05 | 217.53 | |
| AgentDropoutBackbone=Qwen2.5-72B-Instruct, Topology=Random, Training-Free (TF)=false2026.05 | 80.83 | 116.22 | |
| Vanilla MASBackbone=Qwen2.5-72B-Instruct, Topology=Layered, Training-Free (TF)=true2026.05 | 80.39 | 156.18 | |
| SC-CoTBackbone=Qwen2.5-72B-Instruct, Topology=-, Training-Free (TF)=true2026.05 | 79.74 | 84.06 | |
| LLM-DebateBackbone=Qwen2.5-72B-Instruct, Topology=Debate, Training-Free (TF)=true2026.05 | 79.52 | 149.85 | |
| CONCATBackbone=Qwen2.5-72B-Instruct, Topology=Hybrid, Training-Free (TF)=true2026.05 | 79.52 | 107.59 | |
| Vanilla MASBackbone=Qwen2.5-72B-Instruct, Topology=FullConnected, Training-Free (TF)=true2026.05 | 79.3 | 196.93 | |
| AgentDropoutBackbone=Qwen2.5-72B-Instruct, Topology=Chain, Training-Free (TF)=false2026.05 | 78.43 | 176.02 | |
| CONCATBackbone=Qwen2.5-14B-Instruct, Topology=Hybrid, Training-Free (TF)=true2026.05 | 77.78 | 25.76 | |
| CoTBackbone=Qwen2.5-72B-Instruct, Topology=-, Training-Free (TF)=true2026.05 | 77.56 | 23.85 | |
| AgentDropoutBackbone=Qwen2.5-14B-Instruct, Topology=Chain, Training-Free (TF)=false2026.05 | 76.91 | 34.17 | |
| Vanilla MASBackbone=Qwen2.5-14B-Instruct, Topology=Star, Training-Free (TF)=true2026.05 | 76.25 | 51.32 | |
| Vanilla MASBackbone=Qwen2.5-14B-Instruct, Topology=Random, Training-Free (TF)=true2026.05 | 76.03 | 48.65 | |
| AgentDropoutBackbone=Qwen2.5-14B-Instruct, Topology=FullConnected, Training-Free (TF)=false2026.05 | 75.82 | 39.77 | |
| LLM-DebateBackbone=Qwen2.5-14B-Instruct, Topology=Debate, Training-Free (TF)=true2026.05 | 75.6 | 50 | |
| Vanilla MASBackbone=Qwen2.5-14B-Instruct, Topology=Chain, Training-Free (TF)=true2026.05 | 75.16 | 41.82 | |
| AgentDropoutBackbone=Qwen2.5-14B-Instruct, Topology=Star, Training-Free (TF)=false2026.05 | 74.95 | 37.05 | |
| Vanilla MASBackbone=Qwen2.5-14B-Instruct, Topology=FullConnected, Training-Free (TF)=true2026.05 | 74.73 | 55.12 | |
| AgentDropoutBackbone=Qwen2.5-14B-Instruct, Topology=Random, Training-Free (TF)=false2026.05 | 74.51 | 36.57 | |
| AgentDropoutBackbone=Qwen2.5-14B-Instruct, Topology=Layered, Training-Free (TF)=false2026.05 | 74.29 | 37.71 | |
| SC-CoTBackbone=Qwen2.5-14B-Instruct, Topology=-, Training-Free (TF)=true2026.05 | 73.86 | 20.78 | |
| Vanilla MASBackbone=Qwen2.5-14B-Instruct, Topology=Layered, Training-Free (TF)=true2026.05 | 73.42 | 51.87 | |
| CoTBackbone=Qwen2.5-14B-Instruct, Topology=-, Training-Free (TF)=true2026.05 | 73.2 | 6.07 | |
| AgentDropoutBackbone=Llama3-8B-Instruct, Topology=Random, Training-Free (TF)=false2026.05 | 60.35 | 50.33 | |
| Vanilla MASBackbone=Llama3-8B-Instruct, Topology=Layered, Training-Free (TF)=true2026.05 | 59.26 | 71.17 | |
| Vanilla MASBackbone=Llama3-8B-Instruct, Topology=FullConnected, Training-Free (TF)=true2026.05 | 57.3 | 71.81 | |
| AgentDropoutBackbone=Llama3-8B-Instruct, Topology=Chain, Training-Free (TF)=false2026.05 | 56.86 | 39.62 | |
| LLM-DebateBackbone=Llama3-8B-Instruct, Topology=Debate, Training-Free (TF)=true2026.05 | 56.43 | 63.32 | |
| AgentDropoutBackbone=Llama3-8B-Instruct, Topology=Star, Training-Free (TF)=false2026.05 | 55.99 | 48.67 | |
| CONCATBackbone=Llama3-8B-Instruct, Topology=Hybrid, Training-Free (TF)=true2026.05 | 55.56 | 29.39 | |
| AgentDropoutBackbone=Llama3-8B-Instruct, Topology=Layered, Training-Free (TF)=false2026.05 | 54.9 | 48.52 | |
| Vanilla MASBackbone=Llama3-8B-Instruct, Topology=Chain, Training-Free (TF)=true2026.05 | 54.68 | 56.95 | |
| AgentDropoutBackbone=Llama3-8B-Instruct, Topology=FullConnected, Training-Free (TF)=false2026.05 | 54.25 | 52.92 | |
| Vanilla MASBackbone=Llama3-8B-Instruct, Topology=Star, Training-Free (TF)=true2026.05 | 53.59 | 65.9 | |
| Vanilla MASBackbone=Llama3-8B-Instruct, Topology=Random, Training-Free (TF)=true2026.05 | 51.63 | 62.62 | |
| SC-CoTBackbone=Llama3-8B-Instruct, Topology=-, Training-Free (TF)=true2026.05 | 50.54 | 17.66 | |
| CoTBackbone=Llama3-8B-Instruct, Topology=-, Training-Free (TF)=true2026.05 | 48.15 | 5.69 |