Multi-task Language Understanding Performance on MMLU (TPS & Speedup)
17.3TPSCreditDecoding
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| CreditDecodingBackbone=LLaDA-8B-Instruct, Gen Length=256, Block Size=642025.10 | 17.3 | 249 | 38.4 | — | — | |
| Fast-dLLMBackbone=LLaDA-8B-Instruct, Gen Length=256, Block Size=642025.10 | 12.49 | — | — | — | — | |
| BaselineBackbone=LLaDA-8B-Instruct, Gen Length=256, Block Size=642025.10 | 4.95 | — | — | — | — | |
| BaselineBackbone=LLaMA 3.1 8B, Shots=5-shot2025.10 | — | — | — | 59.2 | — | |
| Best ModelBackbone=LLaMA 3.1 8B, Shots=5-shot, #D=42025.10 | — | — | — | 62.38 | 1.14 | |
| BSBABackbone=LLaMA 3.1 8B, Shots=5-shot, #D=72025.10 | — | — | — | 59.57 | 1.26 |