List your company
ResearchFormerly WizWand

Code Generation on HumanEval (Generic Score)

91.56ScoreEAPO
9.025630.452851.8873.3072May 17, 2025Jul 21, 2025Sep 25, 2025Nov 30, 2025Feb 4, 2026Apr 11, 2026Jun 16, 2026
Updated 4mo ago

Evaluation results

MethodLinks
EAPO2025.0991.56
Self-Exploratory RL2025.0989.34
Base Model2025.0986.79
Entropy-Cut MHModel=Qwen3-8B-Base2026.0579.3
SMCModel=Qwen3-8B-Base2026.0576.7
Uniform-Cut MHModel=Qwen3-8B-Base2026.0575.8
TMCModel=Qwen3-8B-Base2026.0575.2
Low-TemperatureModel=Qwen3-8B-Base2026.0573.7
Entropy-Cut MHModel=Qwen2.5-7B2026.0568.9
Entropy-Cut MHModel=Phi-4-mini-instruct2026.0568.4
Uniform-Cut MHModel=Phi-4-mini-instruct2026.0567.9
Uniform-Cut MHModel=Qwen2.5-7B2026.0566.8
SMCModel=Phi-4-mini-instruct2026.0566.7
TMCModel=Qwen2.5-7B2026.0566.2
TMCModel=Phi-4-mini-instruct2026.0566.1
TMCModel=Phi-3.5-mini-instruct2026.0565.6
Entropy-Cut MHModel=Phi-3.5-mini-instruct2026.0565.6
Low-TemperatureModel=Phi-4-mini-instruct2026.0565.5
Task ArithmeticModel Scale=Llama-3.1-8B, Quantization Bit-width=Full-precision, Merging Algorithm=Task Arithmetic2026.0565.24
SMCModel=Phi-3.5-mini-instruct2026.0565.2
SMCModel=Qwen2.5-7B2026.0564.9
Uniform-Cut MHModel=Phi-3.5-mini-instruct2026.0564.9
Low-TemperatureModel=Qwen2.5-7B2026.0564.3
OriginalBlock length B=642026.0662.8
RainbowPaddingBlock length B=642026.0662.8
Low-TemperatureModel=Phi-3.5-mini-instruct2026.0562.2
DreamEnsemble Type=Single Model2026.0661.59
TRACE-CSBackbone=LLaMA3-8B2026.0561
E-PMQModel Scale=Llama-3.1-8B, Quantization Bit-width=4-bit, Merging Algorithm=Task Arithmetic2026.0560.98
OriginalBlock length B=1282026.0660.37
RainbowPaddingBlock length B=1282026.0660.37
VoidPaddingBlock length B=5122026.0660.37
Entropy-Cut MHModel=Qwen2.5-Math-7B2026.0559.9
Uniform-Cut MHModel=Qwen2.5-Math-7B2026.0559.8
VoidPaddingBlock length B=642026.0659.76
VoidPaddingMean=Mean2026.0659.76
VoidPaddingBlock length B=1282026.0659.15
AWQModel Scale=Llama-3.1-8B, Quantization Bit-width=4-bit, Merging Algorithm=Task Arithmetic2026.0558.54
GPTQModel Scale=Llama-3.1-8B, Quantization Bit-width=4-bit, Merging Algorithm=Task Arithmetic2026.0558.54
LLaDA + DreamEnsemble Type=Intermediate-generation Ensemble, Scoring Function=Entropy2026.0657.93
RainbowPaddingMean=Mean2026.0657.72
LLaDA + DreamEnsemble Type=Intermediate-generation Ensemble, Scoring Function=Top-1 probability2026.0657.32
LLaDA + DreamEnsemble Type=Intermediate-generation Ensemble, Scoring Function=Probability margin2026.0657.32
Low-TemperatureModel=Qwen2.5-Math-7B2026.0555.8
LLaDA + DreamEnsemble Type=Post-generation Ensemble2026.0655.49
SMCModel=Qwen2.5-Math-7B2026.0555.2
LLaDA + DreamEnsemble Type=Intermediate-generation Ensemble, Scoring Function=Token change count2026.0654.27
TRACE-LFBackbone=LLaMA3-8B2026.0553.7
Task ArithmeticModel Scale=Llama-3.1-3B, Quantization Bit-width=Full-precision, Merging Algorithm=Task Arithmetic2026.0553.66
GPTQModel Scale=Llama-3.1-3B, Quantization Bit-width=4-bit, Merging Algorithm=Task Arithmetic2026.0551.83
E-PMQModel Scale=Llama-3.1-3B, Quantization Bit-width=4-bit, Merging Algorithm=Task Arithmetic2026.0551.83
AWQModel Scale=Llama-3.1-3B, Quantization Bit-width=4-bit, Merging Algorithm=Task Arithmetic2026.0551.22
TMCModel=Qwen2.5-Math-7B2026.0551.2
RainbowPaddingBlock length B=5122026.0650
TIDE-SharedPipeline=Cross-Tokenizer2026.0449.39
TIDE-SharedPipeline=Shared-Tokenizer2026.0448.78
TIDE-CrossPipeline=Cross-Tokenizer2026.0448.17
StandardModel=Qwen3-8B-Base2026.0547.6
OriginalMean=Mean2026.0647.36
Sequential LoRABackbone=LLaMA3-8B2026.0547
BD3LMPipeline=Qwen3-0.6B2026.0446.34
LLaDAEnsemble Type=Single Model2026.0645.73
CALMPipeline=Cross-Tokenizer2026.0443.9
TIDE-CrossPipeline=Shared-Tokenizer2026.0442.68
KLPipeline=Shared-Tokenizer2026.0441.46
StandardModel=Phi-3.5-mini-instruct2026.0540.5
StandardModel=Phi-4-mini-instruct2026.0539.9
dLLM-CacheModel=LLaDA Instruct, TPS=44.77, Speed=4.24×, Memory=16.652025.0539.02
LLaDA InstructModel=LLaDA Instruct, TPS=10.57, Speed=1.00×, Memory=15.392025.0538.71
dKV-CacheModel=LLaDA Instruct, TPS=14.40, Speed=1.36×, Memory=17.172025.0537.2
Fast-dLLMModel=LLaDA Instruct, TPS=21.50, Speed=2.03×, Memory=16.602025.0536.59
StandardModel=Qwen2.5-7B2026.0533
ARPipeline=Qwen3-0.6B2026.0432.3
StandardModel=Qwen2.5-Math-7B2026.0531
OriginalBlock length B=5122026.0618.9
Joint Fine-tuningBackbone=LLaMA3-8B2026.0518.3
DMTBackbone=LLaMA3-8B2026.0514
Sequential Fine-tuningBackbone=LLaMA3-8B2026.0512.2