Long-context Input Summarization on PG19
1.26SpeedupKnapSpec
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| KnapSpecBackbone=Llama3.1-8B, Layers (L)=322026.02 | 1.26 | 23.91 | 94.6 | |
| SSSDTarget Model=Llama-3.3-70B, Context Length=up to 40k tokens, System Configuration=PD disaggregation2024.11 | 1.26 | — | — | |
| SSSDTarget Model=Llama-3.1-8B, Context Length=up to 40k tokens, System Configuration=PD disaggregation2024.11 | 1.23 | — | — | |
| KnapSpecBackbone=Llama3.1-70B, Layers (L)=802026.02 | 1.21 | 10.67 | 89.5 | |
| KnapSpecBackbone=Llama3.2-3B, Layers (L)=282026.02 | 1.16 | 34.04 | 92.1 | |
| LookaheadTarget Model=Llama-3.3-70B, Context Length=up to 40k tokens, System Configuration=PD disaggregation2024.11 | 1.15 | — | — | |
| CLaSpBackbone=Llama3.1-70B, Layers (L)=802026.02 | 1.1 | 6.52 | 87.8 | |
| CLaSpBackbone=Llama3.1-8B, Layers (L)=322026.02 | 1.1 | 13.94 | 91.4 | |
| EAGLE-3Target Model=Llama-3.3-70B, Context Length=up to 40k tokens, System Configuration=PD disaggregation2024.11 | 1.09 | — | — | |
| CLaSpBackbone=Llama3.2-3B, Layers (L)=282026.02 | 1.08 | 21.1 | 91 | |
| LookaheadTarget Model=Llama-3.1-8B, Context Length=up to 40k tokens, System Configuration=PD disaggregation2024.11 | 1.08 | — | — | |
| KnapSpecBackbone=Llama3.2-1B, Layers (L)=162026.02 | 1.06 | 58.17 | 91.7 | |
| CLaSpBackbone=Llama3.2-1B, Layers (L)=162026.02 | 1.01 | 33.61 | 81.7 | |
| ARBackbone=Llama3.1-70B, Layers (L)=802026.02 | 1 | 6.9 | — | |
| ARBackbone=Llama3.1-8B, Layers (L)=322026.02 | 1 | 16.11 | — | |
| ARBackbone=Llama3.2-3B, Layers (L)=282026.02 | 1 | 23.54 | — | |
| ARBackbone=Llama3.2-1B, Layers (L)=162026.02 | 1 | 46.68 | — | |
| SWIFTBackbone=Llama3.1-70B, Layers (L)=802026.02 | 0.98 | 6.12 | 61.9 | |
| SWIFTBackbone=Llama3.2-1B, Layers (L)=162026.02 | 0.91 | 31.34 | 71.5 | |
| DELBackbone=Llama3.1-70B, Layers (L)=802026.02 | 0.87 | 6.99 | 1.3 | |
| SWIFTBackbone=Llama3.1-8B, Layers (L)=322026.02 | 0.83 | 12.43 | 47.7 | |
| SWIFTBackbone=Llama3.2-3B, Layers (L)=282026.02 | 0.83 | 18.82 | 53.6 | |
| EAGLE-3Target Model=Llama-3.1-8B, Context Length=up to 40k tokens, System Configuration=PD disaggregation2024.11 | 0.8 | — | — | |
| DELBackbone=Llama3.2-1B, Layers (L)=162026.02 | 0.77 | 49.98 | 32.9 | |
| DELBackbone=Llama3.1-8B, Layers (L)=322026.02 | 0.71 | 16.12 | 0.1 | |
| DELBackbone=Llama3.2-3B, Layers (L)=282026.02 | 0.68 | 25.62 | 15.5 |