Output Equivalence on Vicuna
97.3Exact MatchEQSPEC
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| EQSPECBatch Size=12025.10 | 97.3 | 98.6 | |
| EXSPECBatch Size=12025.10 | 97.3 | 98.6 | |
| Spec-1Batch Size=12025.10 | 97.1 | 98.4 | |
| vLLM + SpecSpeculative Decoding=enabled2025.10 | 96.9 | 98 | |
| EQSPECBatch Size=42025.10 | 92.1 | 98.6 | |
| EXSPECBatch Size=42025.10 | 90.8 | 97.6 | |
| SGLang + EAGLE + DetDrafter=EAGLE, Deterministic=true2025.10 | 85 | 90 | |
| SGLang + EAGLEDrafter=EAGLE2025.10 | 69.8 | 79.5 | |
| Non-Spec-BatchBatch Size=42025.10 | 53.8 | 98.2 | |
| BSPBatch Size=12025.10 | 1.9 | 39.7 | |
| BSPBatch Size=42025.10 | 0.2 | 31.3 | |
| DSDBatch Size=12025.10 | 0 | 8.1 | |
| DSDBatch Size=42025.10 | 0 | 2.2 |