List your company
ResearchFormerly WizWand

Reasoning on MMLU-Pro 233-sample stratified

86.71AccuracyDPA
6.879627.604848.3369.0552May 7, 2026
Updated 5mo ago

Evaluation results

MethodLinks
DPAModel=Qwen3.5-9B, Prep.=✗, Latency=1.009×, Peak GPU Mem.=16.75 GB2026.0586.71
ITIModel=Qwen3.5-9B, Prep.=✓, Latency=1.018×, Peak GPU Mem.=16.75 GB2026.0579.81
InstructionModel=Qwen3.5-9B, Prep.=✗, Latency=1.000×, Peak GPU Mem.=16.75 GB2026.0578.87
OriginalModel=Qwen3.5-9B, Prep.=✗, Latency=1.000×, Peak GPU Mem.=16.75 GB2026.0577.62
DoLaModel=Qwen3.5-9B, Prep.=✗, Latency=2.262×, Peak GPU Mem.=38.80 GB2026.0575.94
TAEModel=Qwen3.5-9B, Prep.=✓, Latency=1.410×, Peak GPU Mem.=16.75 GB2026.0575
CDSModel=Qwen3.5-9B, Prep.=✓, Latency=1.002×, Peak GPU Mem.=16.75 GB2026.0560.99
CDSModel=LLaMA3.1-8B, Prep.=✓, Latency=1.000×, Peak GPU Mem.=15.05 GB2026.0534.01
DPAModel=LLaMA3.1-8B, Prep.=✗, Latency=1.022×, Peak GPU Mem.=15.05 GB2026.0528.78
DoLaModel=LLaMA3.1-8B, Prep.=✗, Latency=1.951×, Peak GPU Mem.=41.64 GB2026.0524.62
TAEModel=LLaMA3.1-8B, Prep.=✓, Latency=1.379×, Peak GPU Mem.=15.05 GB2026.0523.61
OriginalModel=LLaMA3.1-8B, Prep.=✗, Latency=1.000×, Peak GPU Mem.=15.05 GB2026.0521.43
InstructionModel=LLaMA3.1-8B, Prep.=✗, Latency=1.000×, Peak GPU Mem.=15.05 GB2026.0520.59
ITIModel=LLaMA3.1-8B, Prep.=✓, Latency=1.132×, Peak GPU Mem.=15.05 GB2026.059.95