List your company
ResearchFormerly WizWand

Language Understanding on MMLU (MMLU Accuracy)

90MMLU AccuracyGD
66.39272.52178.6584.779Apr 12, 2026Apr 24, 2026May 6, 2026May 18, 2026May 30, 2026Jun 11, 2026Jun 23, 2026
Updated 4mo ago

Evaluation results

MethodLinks
GDBackbone=Gemma2026.0490
MOCBackbone=Qwen2.5-32B-Instruct, Edge Density=0.72026.0685.26
MOCBackbone=Qwen2.5-32B-Instruct, Edge Density=0.52026.0683.86
MOCBackbone=Qwen2.5-32B-Instruct, Edge Density=0.32026.0683.51
Vanilla MASBackbone=Qwen2.5-32B-Instruct, Edge Density=0.52026.0682.81
Vanilla MASBackbone=Qwen2.5-32B-Instruct, Edge Density=0.72026.0682.11
No DefenseModel=Qwen2.5-7B, Defense=No Defense, Evaluation Protocol (0-shot)=0-shot2026.0682
THRDModel=Qwen2.5-7B, Defense=Ours, Evaluation Protocol (0-shot)=0-shot2026.0681.7
Vanilla MASBackbone=Qwen2.5-32B-Instruct, Edge Density=0.32026.0681.4
Qwen3-30B-A3B-Instruct-2507# Bits=BF162026.0581.12
Single LLMBackbone=Qwen2.5-32B-Instruct2026.0680.7
FlexRound+LFQ# Bits=W4g1282026.0580.25
FlexRound# Bits=W4g1282026.0580.24
SE-GPTBackbone=Qwen2.5-14B-Instruct2026.0580.2
GPTQ# Bits=W4g1282026.0580.15
MetaEvoBackbone=Qwen2.5-14B-Instruct2026.0579.9
MOCEdge Density=0.3, Backbone=Gemma-2-27B-Instruct2026.0679.3
Vanilla MASEdge Density=0.3, Backbone=Gemma-2-27B-Instruct2026.0678.95
MOCEdge Density=0.5, Backbone=Gemma-2-27B-Instruct2026.0678.95
MOCEdge Density=0.7, Backbone=Gemma-2-27B-Instruct2026.0678.95
Qwen2.5-14B-InstructBase Model=Qwen2.5-14B-Instruct, # Bits=BF162026.0578.82
PROACTModel=Qwen2.5-7B, Defense=PROACT, Evaluation Protocol (0-shot)=0-shot2026.0678.6
Vanilla MASEdge Density=0.7, Backbone=Gemma-2-27B-Instruct2026.0678.6
MOCEdge Density=1.0, Backbone=Gemma-2-27B-Instruct2026.0678.6
Phi-4 14B BaseModel=Phi-4 14B, Variant=Base2026.0677.9
Vanilla MASEdge Density=0.5, Backbone=Gemma-2-27B-Instruct2026.0677.89
Phi-4 14B Chapter IModel=Phi-4 14B, Variant=Chapter I2026.0677.6
MetaEvo - w/o CDABackbone=Qwen2.5-14B-Instruct2026.0577.4
FlexRoundBase Model=Qwen2.5-14B-Instruct, # Bits=W42026.0577.33
FlexRound+LFQBase Model=Qwen2.5-14B-Instruct, # Bits=W42026.0577.31
Phi-4 14B Chapter IIModel=Phi-4 14B, Variant=Chapter II2026.0677.2
Vanilla MASEdge Density=1.0, Backbone=Gemma-2-27B-Instruct2026.0677.19
Qwen3 14B BaseModel=Qwen3 14B, Variant=Base2026.0677.1
Qwen3-8BType=AR2026.0676.9
Block-AP+LFQBase Model=Qwen2.5-14B-Instruct, # Bits=W42026.0576.86
Block-APBase Model=Qwen2.5-14B-Instruct, # Bits=W42026.0576.84
Self-DiscoverBackbone=Qwen2.5-14B-Instruct2026.0576.7
OmniQuant+LFQBase Model=Qwen2.5-14B-Instruct, # Bits=W42026.0576.66
OmniQuantBase Model=Qwen2.5-14B-Instruct, # Bits=W42026.0576.64
Base ModelBackbone=Qwen2.5-14B-Instruct2026.0576.5
Uniform scheduleCompression Step=Step 12026.0676.46
MetaEvo - w/o MOBackbone=Qwen2.5-14B-Instruct2026.0576.2
FlexRound+LFQBase Model=Qwen2.5-14B-Instruct, # Bits=W3g1282026.0575.85
FlexRoundBase Model=Qwen2.5-14B-Instruct, # Bits=W3g1282026.0575.84
Self-RefineBackbone=Qwen2.5-14B-Instruct2026.0575.8
OmniQuant+LFQBase Model=Qwen2.5-14B-Instruct, # Bits=W3g1282026.0575.73
OmniQuantBase Model=Qwen2.5-14B-Instruct, # Bits=W3g1282026.0575.62
DreamReasoner-8BType=Block Diffusion2026.0675.4
BaselineBit-Precision=BF162026.0674.96
FTBit-Precision=BF162026.0674.84
Self-ICLBackbone=Qwen2.5-14B-Instruct2026.0574.6
Block-APBase Model=Qwen2.5-14B-Instruct, # Bits=W3g1282026.0574.58
Block-AP+LFQBase Model=Qwen2.5-14B-Instruct, # Bits=W3g1282026.0574.58
Single LLMBackbone=Gemma-2-27B-Instruct2026.0674.39
Qwen3 14B Chapter IModel=Qwen3 14B, Variant=Chapter I2026.0674.3
Qwen3 14B Chapter IIModel=Qwen3 14B, Variant=Chapter II2026.0674.3
ReQATBit-Precision=W4A16 MXFP42026.0674.14
PTQBit-Precision=W4A16 MXFP42026.0673.6
Qwen2.5-7B-Instructbits=BF162026.0573.49
Qwen2.5-7B-InstructBase Model=Qwen2.5-7B-Instruct, # Bits=BF162026.0573.49
ReQATBit-Precision=W4A4KV4 NVFP42026.0673.14
Qwen3 8B BaseModel=Qwen3 8B, Variant=Base2026.0673
FlexRoundbits=W42026.0572.5
FlexRoundBase Model=Qwen2.5-7B-Instruct, # Bits=W42026.0572.5
FlexRound + LFQbits=W42026.0572.48
FlexRound+LFQBase Model=Qwen2.5-7B-Instruct, # Bits=W42026.0572.48
No DefenseModel=Llama3-8B, Defense=No Defense, Evaluation Protocol (0-shot)=0-shot2026.0672.4
PTQBit-Precision=W4A4KV4 NVFP42026.0672.12
Linear scheduleCompression Step=Step 12026.0671.96
Qwen2.5-7BType=AR2026.0671.9
PROACTModel=Llama3-8B, Defense=PROACT, Evaluation Protocol (0-shot)=0-shot2026.0671.4
Qwen3 8B Chapter IModel=Qwen3 8B, Variant=Chapter I2026.0671.3
OmniQuantbits=W42026.0571
OmniQuantBase Model=Qwen2.5-7B-Instruct, # Bits=W42026.0571
OmniQuant + LFQbits=W42026.0570.99
OmniQuant+LFQBase Model=Qwen2.5-7B-Instruct, # Bits=W42026.0570.99
THRDModel=Llama3-8B, Defense=Ours, Evaluation Protocol (0-shot)=0-shot2026.0670.9
Qwen3 8B Chapter IIModel=Qwen3 8B, Variant=Chapter II2026.0670.4
Cosine scheduleCompression Step=Step 12026.0670.34
FlexRound + LFQbits=W3g1282026.0570.26
FlexRound+LFQBase Model=Qwen2.5-7B-Instruct, # Bits=W3g1282026.0570.26
FlexRoundbits=W3g1282026.0570.13
FlexRoundBase Model=Qwen2.5-7B-Instruct, # Bits=W3g1282026.0570.13
Block-AP + LFQbits=W42026.0569.94
Block-AP+LFQBase Model=Qwen2.5-7B-Instruct, # Bits=W42026.0569.94
MiMo-7BType=AR2026.0669.8
Block-APbits=W42026.0569.6
Block-APBase Model=Qwen2.5-7B-Instruct, # Bits=W42026.0569.6
Dream-v0-7BType=Diffusion2026.0669.5
MetaEvoBackbone=LLaMA3.1-8B-Instruct2026.0569.1
SE-GPTBackbone=LLaMA3.1-8B-Instruct2026.0568.5
OmniQuantbits=W3g1282026.0568.43
OmniQuantBase Model=Qwen2.5-7B-Instruct, # Bits=W3g1282026.0568.43
Llama 3.1 8B BaseModel=Llama 3.1 8B, Variant=Base2026.0668.4
OmniQuant + LFQbits=W3g1282026.0568.39
OmniQuant+LFQBase Model=Qwen2.5-7B-Instruct, # Bits=W3g1282026.0568.39
SAGEModel=Qwen2.5-7B, Defense=SAGE, Evaluation Protocol (0-shot)=0-shot2026.0668.3
MetaEvo - w/o CDABackbone=LLaMA3.1-8B-Instruct2026.0567.9
Self-DiscoverBackbone=LLaMA3.1-8B-Instruct2026.0567.4
Self-RefineBackbone=LLaMA3.1-8B-Instruct2026.0567.3