Summarization on Spec-Bench Sum.
4.73CR ScoreSpecBound
Evaluation results
| Method | Links | ||
|---|---|---|---|
| SpecBoundBase Model=Vicuna-13B2026.04 | 4.73 | 2.18 | |
| SpecBoundBase Model=Vicuna-7B2026.04 | 3.97 | 1.95 | |
| SpecBoundBase Model=Codellama-13B-Instruct2026.04 | 3 | 2.13 | |
| SpecBoundBase Model=Codellama-7B-Instruct2026.04 | 2.98 | 1.78 | |
| AdaDecodeBase Model=Codellama-13B-Instruct2026.04 | 2.35 | 1.48 | |
| SPACEBase Model=Vicuna-7B2026.04 | 2.19 | 1.47 | |
| AdaDecodeBase Model=Codellama-7B-Instruct2026.04 | 2.15 | 1.38 | |
| KangarooBase Model=Vicuna-13B2026.04 | 2 | 1.41 | |
| KangarooBase Model=Vicuna-7B2026.04 | 1.87 | 1.5 | |
| RESTBase Model=Codellama-13B-Instruct2026.04 | 1.73 | 1.41 | |
| RESTBase Model=Vicuna-13B2026.04 | 1.62 | 1.35 | |
| RESTBase Model=Vicuna-7B2026.04 | 1.6 | 1.34 | |
| LookaheadBase Model=Vicuna-7B2026.04 | 1.57 | 1.38 | |
| MedusaBase Model=Vicuna-13B2026.04 | 1.57 | 1.64 | |
| MedusaBase Model=Codellama-7B-Instruct2026.04 | 1.57 | 1.61 | |
| RESTBase Model=Codellama-7B-Instruct2026.04 | 1.57 | 1.32 | |
| LookaheadBase Model=Codellama-13B-Instruct2026.04 | 1.57 | 1.32 | |
| LookaheadBase Model=Vicuna-13B2026.04 | 1.54 | 1.3 | |
| MedusaBase Model=Vicuna-7B2026.04 | 1.53 | 1.59 | |
| MedusaBase Model=Codellama-13B-Instruct2026.04 | 1.47 | 1.58 | |
| LookaheadBase Model=Codellama-7B-Instruct2026.04 | 1.45 | 1.32 |