General Knowledge on MMLU-Pro (maj@2, maj@4, maj@8)
35.88maj@4 AccuracyDLE (ε-sampling)-PROBFIRST
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| DLE (ε-sampling)-PROBFIRSTModel=Llama3.2-3B-Instruct, Sampling method=ε-sampling, Branching strategy=PROBFIRST2026.04 | 35.88 | 33.18 | 37.53 | |
| DLE (top-p+top-k)Model=Llama3.2-3B-Instruct, Sampling method=top-p+top-k2026.04 | 35.73 | 33.22 | 37.28 | |
| DLE (min-p)Model=Llama3.2-3B-Instruct, Sampling method=min-p2026.04 | 35.71 | 33.23 | 36.91 | |
| DLE (ε-sampling)-DIVFIRSTModel=Llama3.2-3B-Instruct, Sampling method=ε-sampling, Branching strategy=DIVFIRST2026.04 | 35.65 | 33.19 | 37.23 | |
| DLE (ε-sampling)-RANDBRANCHModel=Llama3.2-3B-Instruct, Sampling method=ε-sampling, Branching strategy=RANDBRANCH2026.04 | 35.6 | 33.22 | 37.49 | |
| Self-consistency (ε-sampling)Model=Llama3.2-3B-Instruct, Sampling method=ε-sampling2026.04 | 35.37 | 30.98 | 37.36 | |
| Self-consistency (min-p)Model=Llama3.2-3B-Instruct, Sampling method=min-p2026.04 | 34.73 | 30.68 | 36.08 | |
| Self-consistency (top-p+top-k)Model=Llama3.2-3B-Instruct, Sampling method=top-p+top-k2026.04 | 33.64 | 29.74 | 36.17 | |
| Self-consistencyModel=Llama3.2-3B-Instruct2026.04 | 29.31 | 24.88 | 33.04 | |
| DLE (min-p)Algorithm=DLE, Sampling Strategy=min-p, Model=Qwen2.5-0.5B-Instruct2026.04 | 17.4 | 16.45 | 17.89 | |
| DLE (ε-sampling)Algorithm=DLE, Sampling Strategy=ε-sampling, Model=Qwen2.5-0.5B-Instruct2026.04 | 17.37 | 16.64 | 17.97 | |
| DLE (top-p&top-k)Algorithm=DLE, Sampling Strategy=top-p&top-k, Model=Qwen2.5-0.5B-Instruct2026.04 | 17.3 | 16.36 | 18.19 | |
| Self-consistency (top-p&top-k)Algorithm=Self-consistency, Sampling Strategy=top-p&top-k, Model=Qwen2.5-0.5B-Instruct2026.04 | 16.66 | 14.96 | 17.35 | |
| Self-consistencyAlgorithm=Self-consistency, Temperature (tau)=0.6, Model=Qwen2.5-0.5B-Instruct2026.04 | 16.43 | 16.15 | 17.84 | |
| Self-consistency (min-p)Algorithm=Self-consistency, Sampling Strategy=min-p, Model=Qwen2.5-0.5B-Instruct2026.04 | 16.36 | 14.99 | 17.26 | |
| Diverse beam searchAlgorithm=Diverse beam search, Model=Qwen2.5-0.5B-Instruct2026.04 | 16.2 | — | 16.58 | |
| Self-consistency (ε-sampling)Algorithm=Self-consistency, Sampling Strategy=ε-sampling, Model=Qwen2.5-0.5B-Instruct2026.04 | 16.07 | 14.89 | 17.2 | |
| Self-certaintyAlgorithm=Self-certainty, Model=Qwen2.5-0.5B-Instruct2026.04 | 15.92 | 14.51 | 14.68 | |
| DeepConfAlgorithm=DeepConf, Model=Qwen2.5-0.5B-Instruct2026.04 | 15.58 | 14.41 | 16.76 | |
| Beam searchAlgorithm=Beam search, Model=Qwen2.5-0.5B-Instruct2026.04 | 15.29 | 15.49 | 15.01 | |
| Self-consistencyAlgorithm=Self-consistency, Temperature (tau)=1, Model=Qwen2.5-0.5B-Instruct2026.04 | 15.15 | 13.61 | 16.24 |