Question Answering on Spec-Bench QA
4.54CRSpecBound
Evaluation results
| Method | Links | ||
|---|---|---|---|
| SpecBoundBase Model=Vicuna-13B2026.04 | 4.54 | 2.18 | |
| SpecBoundBase Model=Vicuna-7B2026.04 | 4.23 | 2.16 | |
| SpecBoundBase Model=Codellama-7B-Instruct2026.04 | 3.13 | 1.84 | |
| SpecBoundBase Model=Codellama-13B-Instruct2026.04 | 2.97 | 1.97 | |
| AdaDecodeBase Model=Codellama-13B-Instruct2026.04 | 2.5 | 1.6 | |
| AdaDecodeBase Model=Codellama-7B-Instruct2026.04 | 2.3 | 1.5 | |
| RESTBase Model=Codellama-13B-Instruct2026.04 | 1.99 | 1.57 | |
| RESTBase Model=Vicuna-13B2026.04 | 1.96 | 1.55 | |
| RESTBase Model=Vicuna-7B2026.04 | 1.87 | 1.64 | |
| KangarooBase Model=Vicuna-7B2026.04 | 1.87 | 1.43 | |
| RESTBase Model=Codellama-7B-Instruct2026.04 | 1.84 | 1.62 | |
| KangarooBase Model=Vicuna-13B2026.04 | 1.79 | 1.34 | |
| SPACEBase Model=Vicuna-7B2026.04 | 1.72 | 1.33 | |
| MedusaBase Model=Codellama-7B-Instruct2026.04 | 1.58 | 1.69 | |
| MedusaBase Model=Vicuna-7B2026.04 | 1.54 | 1.67 | |
| MedusaBase Model=Vicuna-13B2026.04 | 1.53 | 1.71 | |
| LookaheadBase Model=Vicuna-7B2026.04 | 1.52 | 1.23 | |
| LookaheadBase Model=Codellama-7B-Instruct2026.04 | 1.49 | 1.21 | |
| LookaheadBase Model=Codellama-13B-Instruct2026.04 | 1.46 | 1.22 | |
| LookaheadBase Model=Vicuna-13B2026.04 | 1.43 | 1.2 | |
| MedusaBase Model=Codellama-13B-Instruct2026.04 | 1.42 | 1.65 |