Reasoning on MMLU-Pro 233-sample stratified
86.71AccuracyDPA
Evaluation Results
| Method | Links | |
|---|---|---|
| DPAModel=Qwen3.5-9B, Prep.=✗, Latency=1.009×, Peak GPU Mem.=16.75 GB2026.05 | 86.71 | |
| ITIModel=Qwen3.5-9B, Prep.=✓, Latency=1.018×, Peak GPU Mem.=16.75 GB2026.05 | 79.81 | |
| InstructionModel=Qwen3.5-9B, Prep.=✗, Latency=1.000×, Peak GPU Mem.=16.75 GB2026.05 | 78.87 | |
| OriginalModel=Qwen3.5-9B, Prep.=✗, Latency=1.000×, Peak GPU Mem.=16.75 GB2026.05 | 77.62 | |
| DoLaModel=Qwen3.5-9B, Prep.=✗, Latency=2.262×, Peak GPU Mem.=38.80 GB2026.05 | 75.94 | |
| TAEModel=Qwen3.5-9B, Prep.=✓, Latency=1.410×, Peak GPU Mem.=16.75 GB2026.05 | 75 | |
| CDSModel=Qwen3.5-9B, Prep.=✓, Latency=1.002×, Peak GPU Mem.=16.75 GB2026.05 | 60.99 | |
| CDSModel=LLaMA3.1-8B, Prep.=✓, Latency=1.000×, Peak GPU Mem.=15.05 GB2026.05 | 34.01 | |
| DPAModel=LLaMA3.1-8B, Prep.=✗, Latency=1.022×, Peak GPU Mem.=15.05 GB2026.05 | 28.78 | |
| DoLaModel=LLaMA3.1-8B, Prep.=✗, Latency=1.951×, Peak GPU Mem.=41.64 GB2026.05 | 24.62 | |
| TAEModel=LLaMA3.1-8B, Prep.=✓, Latency=1.379×, Peak GPU Mem.=15.05 GB2026.05 | 23.61 | |
| OriginalModel=LLaMA3.1-8B, Prep.=✗, Latency=1.000×, Peak GPU Mem.=15.05 GB2026.05 | 21.43 | |
| InstructionModel=LLaMA3.1-8B, Prep.=✗, Latency=1.000×, Peak GPU Mem.=15.05 GB2026.05 | 20.59 | |
| ITIModel=LLaMA3.1-8B, Prep.=✓, Latency=1.132×, Peak GPU Mem.=15.05 GB2026.05 | 9.95 |