List your company
ResearchFormerly WizWand

Multitask Language Understanding on MMLU (Accuracy)

91.5AccuracyGoAgent
69.379275.122180.86586.6079Dec 8, 2021Sep 7, 2022Jun 8, 2023Mar 8, 2024Dec 7, 2024Sep 7, 2025Jun 8, 2026
Updated 4mo ago

Evaluation results

MethodLinks
GoAgent2026.0391.5
Average human expert performance2021.1289.8
ARG-Designer2026.0389.54
EIB-LEARNER2026.0388.9
G-Designer2026.0386.92
AgentReviveVar. NS=True, Flex. State=True, Base model=Qwen2.5-72B-Instruct2026.0586.09
DenseSparsity=0, Model Scale=72B, Model Family=Qwen 2.52025.1086.06
AgentDropout2026.0385.62
AgentPrune2026.0385.07
AgentDropoutVar. NS=True, Flex. State=False, Base model=Qwen2.5-72B-Instruct2026.0584.97
LLM-Debate2026.0384.96
CoFiCotBackbone=Qwen2.5-Math-7B2026.0384.9
Random2026.0384.31
MASround=TVar. NS=False, Flex. State=False, Base model=Qwen2.5-72B-Instruct2026.0584.31
ARG-DesignerVar. NS=True, Flex. State=False, Base model=Qwen2.5-72B-Instruct2026.0584.15
SC (CoT)Var. NS=False, Flex. State=False, Base model=Qwen2.5-72B-Instruct2026.0583.7
SC2026.0383.66
CoTVar. NS=False, Flex. State=False, Base model=Qwen2.5-72B-Instruct2026.0583.66
AgentPruneVar. NS=False, Flex. State=False, Base model=Qwen2.5-72B-Instruct2026.0583.66
DenseSparsity=0, Model Scale=32B, Model Family=Qwen 2.52025.1083.24
Chain2026.0383.01
Best-of-kBackbone=Qwen2.5-Math-7B, k=1202026.0382.6
ARMORSparsity=2:4+2.4%, Model Scale=72B, Model Family=Qwen 2.52025.1082.4
Complete2026.0382.35
VanillaVar. NS=False, Flex. State=False, Base model=Qwen2.5-72B-Instruct2026.0582.35
MASround=1Var. NS=False, Flex. State=False, Base model=Qwen2.5-72B-Instruct2026.0582.35
AutogenVar. NS=False, Flex. State=False, Base model=Qwen2.5-72B-Instruct2026.0582.34
k-way SCBackbone=Qwen2.5-Math-7B, k=1202026.0382
CoT2026.0381.69
AgentVerseVar. NS=False, Flex. State=False, Base model=Qwen2.5-72B-Instruct2026.0581.57
k-way SCBackbone=Qwen2.5-Math-7B, k=402026.0381.3
Tree2026.0381.04
G-DesignerVar. NS=False, Flex. State=False, Base model=Qwen2.5-72B-Instruct2026.0581.02
Vanilla2026.0380.39
DenseSparsity=0, Model Scale=14B, Model Family=Qwen 2.52025.1079.8
WandaSparsity=2:4, Model Scale=72B, Model Family=Qwen 2.52025.1079.61
NoWag-PSparsity=2:4, Model Scale=72B, Model Family=Qwen 2.52025.1078.93
SparseGPTSparsity=2:4, Model Scale=72B, Model Family=Qwen 2.52025.1078.71
ARMORSparsity=2:4+3.44%, Model Scale=32B, Model Family=Qwen 2.52025.1078.18
Qwen3-30B-A3BPruning ratio=0%, Backbone=Qwen3-30B-A3B, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.0777.82
Qwen3-30B-A3BPruning ratio=0%, Base Model=Qwen3-30B-A3B, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.0777.82
Qwen3 14B BaseClassifier=Self-labeled2026.0177
Qwen3 14B BaseClassifier=Majority-labeled2026.0176.9
Qwen3 14BClassifier=Self-labeled2026.0176.5
Qwen3 14BClassifier=Majority-labeled2026.0176.2
WandaSparsity=2:4, Model Scale=32B, Model Family=Qwen 2.52025.1075.45
SparseGPTSparsity=2:4, Model Scale=32B, Model Family=Qwen 2.52025.1075.26
NoWag-PSparsity=2:4, Model Scale=32B, Model Family=Qwen 2.52025.1074.89
RSPruning ratio=25%, Backbone=Qwen3-30B-A3B, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.0774.6
C32A2Base Model=Qwen2.5-7B, #P/B=184.42, #AP/B=13.332026.0374.6
PTBase Model=Qwen2.5-7B, #P/B=7.62, #AP/B=7.622026.0374.28
CASTBase Model=Qwen 2.5 (7B)2025.1274.27
Qwen2.5-7BModel=Qwen2.5-7B2026.0674.2
None (original model)Base Model=Qwen 2.5 (7B)2025.1274.19
DenseSparsity=0, Model Scale=7B, Model Family=Qwen 2.52025.1074.19
LinEAS+E2E-DSAS (R)Base Model=Qwen 2.5 (7B)2025.1274.16
Qwen2-57B-A14BPruning ratio=0%, Backbone=Qwen2-57B-A14B, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.0774.06
Qwen2-57B-A14BPruning ratio=0%, Base Model=Qwen2-57B-A14B, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.0774.06
MoNEPruning ratio=25%, Backbone=Qwen3-30B-A3B, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.0774.04
Qwen2.5-Math-7BBackbone=Qwen2.5-Math-7B2026.0373.9
LinEAS+DSASBase Model=Qwen 2.5 (7B)2025.1273.81
LinEAS+E2E-DSAS (S)Base Model=Qwen 2.5 (7B)2025.1273.61
LinEASBase Model=Qwen 2.5 (7B)2025.1273.59
Qwen3 8B BaseClassifier=Self-labeled2026.0173.5
Qwen3 8B BaseClassifier=Majority-labeled2026.0173.5
CoFiCotBase Model=GPT-3.5-Turbo2026.0373.5
Dream BaseModel=Dream Base, TPS=9.10, Speed=1.00×, Memory=15.642025.0573.49
FLAMODEL=4B, Precision=W4A162026.0673.34
Multiplication-Only Matrix Inversion ApproximationMODEL=4B, Precision=W4A162026.0673.29
dLLM-CacheModel=Dream Base, TPS=31.07, Speed=3.41×, Memory=16.372025.0573.2
FineRMoEBase Model=Qwen2.5-7B, #P/B=26.65, #AP/B=7.942026.0373.08
CAABase Model=Qwen 2.5 (7B)2025.1273.05
CAA+DSASBase Model=Qwen 2.5 (7B)2025.1273.02
CTBase Model=Qwen2.5-7B, #P/B=7.62, #AP/B=7.622026.0372.97
I-DPO + MaPPOModel=Qwen2.5-7B-Instruct2025.0772.9
dKV-CacheModel=Dream Base, TPS=12.80, Speed=1.41×, Memory=15.922025.0572.77
ITI+DSASBase Model=Qwen 2.5 (7B)2025.1272.74
ITIBase Model=Qwen 2.5 (7B)2025.1272.72
Fast-dLLMModel=Dream Base, TPS=23.69, Speed=2.60×, Memory=18.322025.0572.69
DIRBackbone=Llama3.1-8B-Instruct2025.1272.64
ALBMBackbone=Llama3.1-8B-Instruct2025.1272.55
SKBackbone=Llama3.1-8B-Instruct2025.1272.33
BaseBackbone=Llama3.1-8B-Instruct2025.1272.31
MC-SMoEPruning ratio=25%, Backbone=Qwen3-30B-A3B, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.0772.25
InfoRMBackbone=Llama3.1-8B-Instruct2025.1272.22
MERABase Model=Qwen 2.5 (7B)2025.1272.06
DPO + MaPPOModel=Qwen2.5-7B-Instruct2025.0772
PoEBackbone=Llama3.1-8B-Instruct2025.1271.97
DPOModel=Qwen2.5-7B-Instruct2025.0771.9
I-DPOModel=Qwen2.5-7B-Instruct2025.0771.9
Qwen2.5-7B-InstructBackbone=Qwen2.5-7B, Stage=Instruct2025.0671.85
ITModel=Qwen2.5-7B-Instruct2025.0771.8
MoNEPruning ratio=25%, Backbone=Qwen2-57B-A14B, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.0771.64
Qwen3 8BClassifier=Self-labeled2026.0171.6
Qwen3 8BClassifier=Majority-labeled2026.0171.4
RSPruning ratio=25%, Backbone=Qwen2-57B-A14B, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.0770.99
ARMORSparsity=2:4+4.17%, Model Scale=14B, Model Family=Qwen 2.52025.1070.55
k-way SCBase Model=GPT-3.5-Turbo, k=1202026.0370.4
Flash Linear AttentionModel scale=9B2026.0670.26
Multiplication-Only Matrix Inversion ApproximationModel scale=9B2026.0670.23