Long-context language generation on QMSum
3.15Average Acceptance Length (τ)LongSpec
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| LongSpecBackbone=LC-13B, Temperature=02025.02 | 3.15 | 64.41 | 2.37 | |
| LongSpecBackbone=V-7B, Temperature=02025.02 | 3.14 | 88.87 | 2.04 | |
| LongSpecBackbone=LC-7B, Temperature=02025.02 | 3.06 | 85.23 | 2.31 | |
| LongSpecBackbone=L-8B, Temperature=02025.02 | 2.99 | 75.68 | 1.48 | |
| MagicDecBackbone=V-13B, Temperature=02025.02 | 2.87 | 37.15 | 1.35 | |
| LongSpecBackbone=V-13B, Temperature=02025.02 | 2.76 | 57.15 | 2.08 | |
| MagicDecBackbone=LC-13B, Temperature=02025.02 | 2.38 | 30.84 | 1.13 | |
| MagicDecBackbone=V-7B, Temperature=02025.02 | 2.29 | 42.91 | 0.98 | |
| MagicDecBackbone=LC-7B, Temperature=02025.02 | 2.2 | 40.82 | 1.11 | |
| MagicDecBackbone=L-8B, Temperature=02025.02 | 2 | 35.78 | 0.7 | |
| PLDBackbone=L-8B, Temperature=02025.02 | 1.52 | 45.76 | 0.89 | |
| PLDBackbone=V-13B, Temperature=02025.02 | 1.28 | 28.29 | 1.03 | |
| PLDBackbone=LC-7B, Temperature=02025.02 | 1.24 | 36.58 | 0.99 | |
| PLDBackbone=V-7B, Temperature=02025.02 | 1.22 | 39.08 | 0.89 | |
| PLDBackbone=LC-13B, Temperature=02025.02 | 1.18 | 24.1 | 0.89 | |
| Vanilla HFBackbone=V-7B, Temperature=02025.02 | 1 | 18.12 | — | |
| Vanilla FABackbone=V-7B, Temperature=02025.02 | 1 | 43.68 | 1 | |
| Vanilla HFBackbone=V-13B, Temperature=02025.02 | 1 | 11.86 | — | |
| Vanilla FABackbone=V-13B, Temperature=02025.02 | 1 | 27.43 | 1 | |
| Vanilla HFBackbone=LC-7B, Temperature=02025.02 | 1 | 14.11 | — | |
| Vanilla FABackbone=LC-7B, Temperature=02025.02 | 1 | 36.87 | 1 | |
| Vanilla HFBackbone=LC-13B, Temperature=02025.02 | 1 | 12.08 | — | |
| Vanilla FABackbone=LC-13B, Temperature=02025.02 | 1 | 27.18 | 1 | |
| Vanilla HFBackbone=L-8B, Temperature=02025.02 | 1 | 18.67 | — | |
| Vanilla FABackbone=L-8B, Temperature=02025.02 | 1 | 51.22 | 1 |