Text Summarization on XSum (Accept Rate, ROUGE-2)
8.38ROUGE-2SD (Lossy)
Evaluation results
| Method | Links | ||
|---|---|---|---|
| SD (Lossy)Target Model=Gemma-3-12B, Draft Model=Gemma-3-4B, Temperature=1, Draft Length (N)=52026.04 | 8.38 | 39.48 | |
| AutoregressiveTarget Model=Gemma-3-12B, Draft Model=Gemma-3-4B, Temperature=1, Draft Length (N)=52026.04 | 8.31 | — | |
| SDTarget Model=Gemma-3-12B, Draft Model=Gemma-3-4B, Temperature=1, Draft Length (N)=52026.04 | 8.27 | 35.76 | |
| SpecCascadeTarget Model=Gemma-3-12B, Draft Model=Gemma-3-4B, Temperature=1, Draft Length (N)=52026.04 | 8.27 | 39.42 | |
| Static EnsembleTarget Model=Gemma-3-12B, Draft Model=Gemma-3-4B, Temperature=1, Draft Length (N)=52026.04 | 7.9 | 61.42 | |
| DIVERSEDTarget Model=Gemma-3-12B, Draft Model=Gemma-3-4B, Temperature=1, Draft Length (N)=52026.04 | 7.22 | 63.38 | |
| DiversedTarget Model=Qwen3-8B, Draft Model=Qwen3-0.6B, Temperature=0, Draft length (N)=52026.04 | 5.54 | 38.12 | |
| Static EnsembleTarget Model=Qwen3-8B, Draft Model=Qwen3-0.6B, Temperature=0, Draft length (N)=52026.04 | 5.39 | 34.14 | |
| AutoregressiveTarget Model=Qwen3-8B, Draft Model=Qwen3-0.6B, Temperature=0, Draft length (N)=52026.04 | 5.38 | — | |
| SDTarget Model=Qwen3-8B, Draft Model=Qwen3-0.6B, Temperature=0, Draft length (N)=52026.04 | 5.35 | 19.56 |