STEM Reasoning on GPQA-Diamond 5-shot
57.3AccuracyERNIE 5.0-Base
Evaluation Results
| Method | Links | |
|---|---|---|
| ERNIE 5.0-BaseNumber of shots=5-shot2026.02 | 57.3 | |
| DS V3.2-Exp-BaseNumber of shots=5-shot2026.02 | 53.01 | |
| Kimi K2-BaseNumber of shots=5-shot2026.02 | 48.1 | |
| WeDLM-8BInstruct Model=true, Model Category=WeDLM, Shot Configuration=5-shot, Inference Framework=vLLM, Sampling Temperature=0.1, Max Generation Length=512, Window Size (W)=6, Penalty Coefficient (λ)=0.12025.12 | 44.95 | |
| Qwen3-8BInstruct Model=true, Model Category=AR Baseline, Shot Configuration=5-shot, Inference Framework=vLLM, Sampling Temperature=0.1, Max Generation Length=5122025.12 | 41.41 | |
| SDAR-8BInstruct Model=true, Model Category=DLLM Baseline, Shot Configuration=5-shot, Inference Framework=JetEngine, Sampling Temperature=0.1, Max Generation Length=512, Window Size (W)=6, Penalty Coefficient (λ)=0.12025.12 | 38.38 | |
| Dream-7BInstruct Model=true, Model Category=DLLM Baseline, Shot Configuration=5-shot, Inference Framework=dInfer, Sampling Temperature=0.1, Max Generation Length=512, Window Size (W)=6, Penalty Coefficient (λ)=0.12025.12 | 35.86 | |
| WeDLM-7BInstruct Model=true, Model Category=WeDLM, Shot Configuration=5-shot, Inference Framework=vLLM, Sampling Temperature=0.1, Max Generation Length=512, Window Size (W)=6, Penalty Coefficient (λ)=0.12025.12 | 33.84 | |
| Qwen2.5-7BInstruct Model=true, Model Category=AR Baseline, Shot Configuration=5-shot, Inference Framework=vLLM, Sampling Temperature=0.1, Max Generation Length=5122025.12 | 27.1 | |
| LLaDA-8BInstruct Model=true, Model Category=DLLM Baseline, Shot Configuration=5-shot, Inference Framework=dInfer, Sampling Temperature=0.1, Max Generation Length=512, Window Size (W)=6, Penalty Coefficient (λ)=0.12025.12 | 25.25 |