Language Understanding on MMLU (MMLU Accuracy)
90MMLU AccuracyGD
Evaluation Results
| Method | Links | |
|---|---|---|
| GDBackbone=Gemma2026.04 | 90 | |
| MOCBackbone=Qwen2.5-32B-Instruct, Edge Density=0.72026.06 | 85.26 | |
| MOCBackbone=Qwen2.5-32B-Instruct, Edge Density=0.52026.06 | 83.86 | |
| MOCBackbone=Qwen2.5-32B-Instruct, Edge Density=0.32026.06 | 83.51 | |
| Vanilla MASBackbone=Qwen2.5-32B-Instruct, Edge Density=0.52026.06 | 82.81 | |
| Vanilla MASBackbone=Qwen2.5-32B-Instruct, Edge Density=0.72026.06 | 82.11 | |
| No DefenseModel=Qwen2.5-7B, Defense=No Defense, Evaluation Protocol (0-shot)=0-shot2026.06 | 82 | |
| THRDModel=Qwen2.5-7B, Defense=Ours, Evaluation Protocol (0-shot)=0-shot2026.06 | 81.7 | |
| Vanilla MASBackbone=Qwen2.5-32B-Instruct, Edge Density=0.32026.06 | 81.4 | |
| Qwen3-30B-A3B-Instruct-2507# Bits=BF162026.05 | 81.12 | |
| Single LLMBackbone=Qwen2.5-32B-Instruct2026.06 | 80.7 | |
| FlexRound+LFQ# Bits=W4g1282026.05 | 80.25 | |
| FlexRound# Bits=W4g1282026.05 | 80.24 | |
| SE-GPTBackbone=Qwen2.5-14B-Instruct2026.05 | 80.2 | |
| GPTQ# Bits=W4g1282026.05 | 80.15 | |
| MetaEvoBackbone=Qwen2.5-14B-Instruct2026.05 | 79.9 | |
| MOCEdge Density=0.3, Backbone=Gemma-2-27B-Instruct2026.06 | 79.3 | |
| Vanilla MASEdge Density=0.3, Backbone=Gemma-2-27B-Instruct2026.06 | 78.95 | |
| MOCEdge Density=0.5, Backbone=Gemma-2-27B-Instruct2026.06 | 78.95 | |
| MOCEdge Density=0.7, Backbone=Gemma-2-27B-Instruct2026.06 | 78.95 | |
| Qwen2.5-14B-InstructBase Model=Qwen2.5-14B-Instruct, # Bits=BF162026.05 | 78.82 | |
| PROACTModel=Qwen2.5-7B, Defense=PROACT, Evaluation Protocol (0-shot)=0-shot2026.06 | 78.6 | |
| Vanilla MASEdge Density=0.7, Backbone=Gemma-2-27B-Instruct2026.06 | 78.6 | |
| MOCEdge Density=1.0, Backbone=Gemma-2-27B-Instruct2026.06 | 78.6 | |
| Phi-4 14B BaseModel=Phi-4 14B, Variant=Base2026.06 | 77.9 | |
| Vanilla MASEdge Density=0.5, Backbone=Gemma-2-27B-Instruct2026.06 | 77.89 | |
| Phi-4 14B Chapter IModel=Phi-4 14B, Variant=Chapter I2026.06 | 77.6 | |
| MetaEvo - w/o CDABackbone=Qwen2.5-14B-Instruct2026.05 | 77.4 | |
| FlexRoundBase Model=Qwen2.5-14B-Instruct, # Bits=W42026.05 | 77.33 | |
| FlexRound+LFQBase Model=Qwen2.5-14B-Instruct, # Bits=W42026.05 | 77.31 | |
| Phi-4 14B Chapter IIModel=Phi-4 14B, Variant=Chapter II2026.06 | 77.2 | |
| Vanilla MASEdge Density=1.0, Backbone=Gemma-2-27B-Instruct2026.06 | 77.19 | |
| Qwen3 14B BaseModel=Qwen3 14B, Variant=Base2026.06 | 77.1 | |
| Qwen3-8BType=AR2026.06 | 76.9 | |
| Block-AP+LFQBase Model=Qwen2.5-14B-Instruct, # Bits=W42026.05 | 76.86 | |
| Block-APBase Model=Qwen2.5-14B-Instruct, # Bits=W42026.05 | 76.84 | |
| Self-DiscoverBackbone=Qwen2.5-14B-Instruct2026.05 | 76.7 | |
| OmniQuant+LFQBase Model=Qwen2.5-14B-Instruct, # Bits=W42026.05 | 76.66 | |
| OmniQuantBase Model=Qwen2.5-14B-Instruct, # Bits=W42026.05 | 76.64 | |
| Base ModelBackbone=Qwen2.5-14B-Instruct2026.05 | 76.5 | |
| Uniform scheduleCompression Step=Step 12026.06 | 76.46 | |
| MetaEvo - w/o MOBackbone=Qwen2.5-14B-Instruct2026.05 | 76.2 | |
| FlexRound+LFQBase Model=Qwen2.5-14B-Instruct, # Bits=W3g1282026.05 | 75.85 | |
| FlexRoundBase Model=Qwen2.5-14B-Instruct, # Bits=W3g1282026.05 | 75.84 | |
| Self-RefineBackbone=Qwen2.5-14B-Instruct2026.05 | 75.8 | |
| OmniQuant+LFQBase Model=Qwen2.5-14B-Instruct, # Bits=W3g1282026.05 | 75.73 | |
| OmniQuantBase Model=Qwen2.5-14B-Instruct, # Bits=W3g1282026.05 | 75.62 | |
| DreamReasoner-8BType=Block Diffusion2026.06 | 75.4 | |
| BaselineBit-Precision=BF162026.06 | 74.96 | |
| FTBit-Precision=BF162026.06 | 74.84 | |
| Self-ICLBackbone=Qwen2.5-14B-Instruct2026.05 | 74.6 | |
| Block-APBase Model=Qwen2.5-14B-Instruct, # Bits=W3g1282026.05 | 74.58 | |
| Block-AP+LFQBase Model=Qwen2.5-14B-Instruct, # Bits=W3g1282026.05 | 74.58 | |
| Single LLMBackbone=Gemma-2-27B-Instruct2026.06 | 74.39 | |
| Qwen3 14B Chapter IModel=Qwen3 14B, Variant=Chapter I2026.06 | 74.3 | |
| Qwen3 14B Chapter IIModel=Qwen3 14B, Variant=Chapter II2026.06 | 74.3 | |
| ReQATBit-Precision=W4A16 MXFP42026.06 | 74.14 | |
| PTQBit-Precision=W4A16 MXFP42026.06 | 73.6 | |
| Qwen2.5-7B-Instructbits=BF162026.05 | 73.49 | |
| Qwen2.5-7B-InstructBase Model=Qwen2.5-7B-Instruct, # Bits=BF162026.05 | 73.49 | |
| ReQATBit-Precision=W4A4KV4 NVFP42026.06 | 73.14 | |
| Qwen3 8B BaseModel=Qwen3 8B, Variant=Base2026.06 | 73 | |
| FlexRoundbits=W42026.05 | 72.5 | |
| FlexRoundBase Model=Qwen2.5-7B-Instruct, # Bits=W42026.05 | 72.5 | |
| FlexRound + LFQbits=W42026.05 | 72.48 | |
| FlexRound+LFQBase Model=Qwen2.5-7B-Instruct, # Bits=W42026.05 | 72.48 | |
| No DefenseModel=Llama3-8B, Defense=No Defense, Evaluation Protocol (0-shot)=0-shot2026.06 | 72.4 | |
| PTQBit-Precision=W4A4KV4 NVFP42026.06 | 72.12 | |
| Linear scheduleCompression Step=Step 12026.06 | 71.96 | |
| Qwen2.5-7BType=AR2026.06 | 71.9 | |
| PROACTModel=Llama3-8B, Defense=PROACT, Evaluation Protocol (0-shot)=0-shot2026.06 | 71.4 | |
| Qwen3 8B Chapter IModel=Qwen3 8B, Variant=Chapter I2026.06 | 71.3 | |
| OmniQuantbits=W42026.05 | 71 | |
| OmniQuantBase Model=Qwen2.5-7B-Instruct, # Bits=W42026.05 | 71 | |
| OmniQuant + LFQbits=W42026.05 | 70.99 | |
| OmniQuant+LFQBase Model=Qwen2.5-7B-Instruct, # Bits=W42026.05 | 70.99 | |
| THRDModel=Llama3-8B, Defense=Ours, Evaluation Protocol (0-shot)=0-shot2026.06 | 70.9 | |
| Qwen3 8B Chapter IIModel=Qwen3 8B, Variant=Chapter II2026.06 | 70.4 | |
| Cosine scheduleCompression Step=Step 12026.06 | 70.34 | |
| FlexRound + LFQbits=W3g1282026.05 | 70.26 | |
| FlexRound+LFQBase Model=Qwen2.5-7B-Instruct, # Bits=W3g1282026.05 | 70.26 | |
| FlexRoundbits=W3g1282026.05 | 70.13 | |
| FlexRoundBase Model=Qwen2.5-7B-Instruct, # Bits=W3g1282026.05 | 70.13 | |
| Block-AP + LFQbits=W42026.05 | 69.94 | |
| Block-AP+LFQBase Model=Qwen2.5-7B-Instruct, # Bits=W42026.05 | 69.94 | |
| MiMo-7BType=AR2026.06 | 69.8 | |
| Block-APbits=W42026.05 | 69.6 | |
| Block-APBase Model=Qwen2.5-7B-Instruct, # Bits=W42026.05 | 69.6 | |
| Dream-v0-7BType=Diffusion2026.06 | 69.5 | |
| MetaEvoBackbone=LLaMA3.1-8B-Instruct2026.05 | 69.1 | |
| SE-GPTBackbone=LLaMA3.1-8B-Instruct2026.05 | 68.5 | |
| OmniQuantbits=W3g1282026.05 | 68.43 | |
| OmniQuantBase Model=Qwen2.5-7B-Instruct, # Bits=W3g1282026.05 | 68.43 | |
| Llama 3.1 8B BaseModel=Llama 3.1 8B, Variant=Base2026.06 | 68.4 | |
| OmniQuant + LFQbits=W3g1282026.05 | 68.39 | |
| OmniQuant+LFQBase Model=Qwen2.5-7B-Instruct, # Bits=W3g1282026.05 | 68.39 | |
| SAGEModel=Qwen2.5-7B, Defense=SAGE, Evaluation Protocol (0-shot)=0-shot2026.06 | 68.3 | |
| MetaEvo - w/o CDABackbone=LLaMA3.1-8B-Instruct2026.05 | 67.9 | |
| Self-DiscoverBackbone=LLaMA3.1-8B-Instruct2026.05 | 67.4 | |
| Self-RefineBackbone=LLaMA3.1-8B-Instruct2026.05 | 67.3 |