Document Summarization on GovReport (G-mean and PPL)
18.43G-meanE2LLM
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| E2LLMTrainable Parameters=11M, Context Window=400K, Extension Method=Soft Comp.2024.09 | 18.43 | 2.81 | |
| LongLoRATrainable Parameters=140M, Context Window=100K, Extension Method=Sparse Attn.2024.09 | 16.35 | 2.88 | |
| RAGTrainable Parameters=0M, Context Window=+∞, Extension Method=Hard Comp.2024.09 | 14.45 | 5.88 | |
| YaRNTrainable Parameters=17M, Context Window=64K, Extension Method=Len. Exten.2024.09 | 6.72 | 2.94 | |
| LLoCOTrainable Parameters=17M, Context Window=128K, Extension Method=Soft Comp.2024.09 | 5.73 | 6.42 | |
| Llama2-7B-chatTrainable Parameters=0M, Context Window=4K, Extension Method=-2024.09 | 5.5 | 9.04 | |
| CEPETrainable Parameters=1.31B, Context Window=128k, Extension Method=Len. Exten.2024.09 | 4.82 | 52.32 | |
| LongLLMLinguaTrainable Parameters=0M, Context Window=40K, Extension Method=Hard Comp.2024.09 | 4.56 | 23.53 | |
| StreamingLLMTrainable Parameters=0M, Context Window=4M, Extension Method=Sparse Attn.2024.09 | 4.51 | 330.54 |