Inference Latency on MMLU-Pro
3.3Latency (s)T2
Evaluation results
| Method | Links | |
|---|---|---|
| T2Backbone=Qwen2.5-7B2026.01 | 3.3 | |
| T2Backbone=Llama3.1-8B2026.01 | 3.7 | |
| Std-PTBackbone=Qwen2.5-7B2026.01 | 8.5 | |
| Std-PTBackbone=Llama3.1-8B2026.01 | 8.9 | |
| T2Backbone=Qwen3-4B2026.01 | 12.3 | |
| ESCBackbone=Qwen2.5-7B2026.01 | 15.1 | |
| ACBackbone=Qwen2.5-7B2026.01 | 17.3 | |
| ESCBackbone=Llama3.1-8B2026.01 | 19.2 | |
| DSCBackbone=Llama3.1-8B2026.01 | 25.2 | |
| DeepConfBackbone=Qwen2.5-7B2026.01 | 27.3 | |
| ACBackbone=Llama3.1-8B2026.01 | 28.2 | |
| Std-PTBackbone=Qwen3-4B2026.01 | 30.7 | |
| T2Backbone=Deepseek-R1-Distill-Qwen-7B2026.01 | 31.7 | |
| DSCBackbone=Qwen2.5-7B2026.01 | 31.8 | |
| DeepConfBackbone=Llama3.1-8B2026.01 | 37 | |
| Std-PTBackbone=Deepseek-R1-Distill-Qwen-7B2026.01 | 64.7 | |
| ESCBackbone=Qwen3-4B2026.01 | 68.5 | |
| ACBackbone=Qwen3-4B2026.01 | 72.5 | |
| DSCBackbone=Qwen3-4B2026.01 | 91.2 | |
| DSCBackbone=Deepseek-R1-Distill-Qwen-7B2026.01 | 115.8 | |
| ESCBackbone=Deepseek-R1-Distill-Qwen-7B2026.01 | 124.2 | |
| ACBackbone=Deepseek-R1-Distill-Qwen-7B2026.01 | 150.1 | |
| DeepConfBackbone=Qwen3-4B2026.01 | 153.2 | |
| DeepConfBackbone=Deepseek-R1-Distill-Qwen-7B2026.01 | 251 |