Multi-turn Conversation on MT-Bench (Throughput and Difference Metrics)
177.54ThroughputOur approach
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Our approachVocabulary size=13264, Hardware=Single A100-80G GPU, Target Model=Llama-3.1-8B-Instruct, Inference Engine=SGLang, Framework=SpecForge2026.03 | 177.54 | 3 | |
| 128k-vocabVocabulary size=128k, Hardware=Single A100-80G GPU, Target Model=Llama-3.1-8B-Instruct, Inference Engine=SGLang, Framework=SpecForge2026.03 | 172.38 | — |