Abstractive Summarization on XSum (Accept Rate, ROUGE-2)
8.79ROUGE-2SpecCascade
Evaluation results
| Method | Links | ||
|---|---|---|---|
| SpecCascadeTarget Model=Gemma-3-12B, Draft Model=Gemma-3-4B, Temperature=0.1, Draft Length (N)=52026.04 | 8.79 | 39.42 | |
| SD (Lossy)Target Model=Gemma-3-12B, Draft Model=Gemma-3-4B, Temperature=0.1, Draft Length (N)=52026.04 | 8.37 | 40.08 | |
| Static EnsembleTarget Model=Gemma-3-12B, Draft Model=Gemma-3-4B, Temperature=0.1, Draft Length (N)=52026.04 | 8.31 | 62.65 | |
| AutoregressiveTarget Model=Gemma-3-12B, Draft Model=Gemma-3-4B, Temperature=0.1, Draft Length (N)=52026.04 | 8.27 | — | |
| SDTarget Model=Gemma-3-12B, Draft Model=Gemma-3-4B, Temperature=0.1, Draft Length (N)=52026.04 | 8.27 | 39.63 | |
| DiversedTarget Model=Gemma-3-12B, Draft Model=Gemma-3-4B, Temperature=0.1, Draft Length (N)=52026.04 | 7.84 | 71.22 |