LooGLE ArXiv Paper Summarization
29.15BLEU-1VIST2-8B
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| VIST2-8BContext=8k×42026.01 | 29.15 | 2.15 | 33.24 | 11.45 | 29.75 | 22.88 | 85.12 | 88.42 | |
| GPT4-8kContext=8k2026.01 | 29.02 | 2.09 | 32.08 | 11.11 | 28.85 | 22.64 | 84.92 | 85.42 | |
| GPT3.5-turbo-16kContext=16k2026.01 | 28.7 | 1.59 | 32.04 | 10.69 | 28.89 | 22.34 | 84.82 | 86.84 | |
| VIST2-4BContext=8k×42026.01 | 28.45 | 1.88 | 31.8 | 10.42 | 28.52 | 22.1 | 84.7 | 86.15 | |
| VIST2-0.6BContext=8k×42026.01 | 26.12 | 1.1 | 29.45 | 8.54 | 26.15 | 20.12 | 83.95 | 81.2 | |
| GPT4-32kContext=32k2026.01 | 24.5 | 0.73 | 27.15 | 7.1 | 24.25 | 19.03 | 84.04 | 82.84 | |
| LlamaIndexContext=-2026.01 | 22.53 | 0.63 | 26.28 | 6.97 | 23.73 | 21.07 | 83.09 | 76.35 | |
| RWKV-4-14B-ravenContext=8k2026.01 | 15.51 | 0.09 | 18.98 | 3.32 | 17.12 | 14.78 | 80.05 | 34.79 | |
| LLaMA2-7B-32K-InstructContext=32k2026.01 | 0.09 | 0 | 0.2 | 0 | 0.2 | 2.44 | 72.41 | 6.14 | |
| ChatGLM2-6BContext=32k2026.01 | 0.04 | 0 | 5.97 | 0 | 5.82 | 6.4 | 73.25 | 13.23 | |
| LongLLaMa-3B-InstructContext=256k2026.01 | 0 | 0 | 0.07 | 0 | 0.07 | 0.33 | 26.01 | 5.57 |