Question Answering on CMMLU
88.1AccuracyEvo 8B
Evaluation results
| Method | Links | |
|---|---|---|
| Evo 8BShots=5, E2E Latency (s)=8.6, Inference Speed (tokens/s)=522026.02 | 88.1 | |
| BD3-LM 7BShots=5, E2E Latency (s)=14.2, Inference Speed (tokens/s)=282026.02 | 84.2 | |
| LLaDA 8BShots=5, E2E Latency (s)=21.8, Inference Speed (tokens/s)=162026.02 | 70.2 | |
| LLaMA3 8BShots=5, E2E Latency (s)=7.4, Inference Speed (tokens/s)=582026.02 | 50.4 | |
| MDLM 7BShots=5, E2E Latency (s)=18.9, Inference Speed (tokens/s)=222026.02 | 47 | |
| ARD 7BShots=5, E2E Latency (s)=32.5, Inference Speed (tokens/s)=122026.02 | 33.1 |