Long-context understanding on ZeroSCROLLS (val)
50.5Qasper F1GPT-4
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| GPT-42024.07 | 50.5 | 95.2 | 13.2 | — | — | — | — | — | — | |
| Llama 3 405B2024.07 | 49.8 | 95.2 | 15.4 | — | — | — | — | — | — | |
| GPT-4o2024.07 | 49.2 | 90.5 | 18.8 | — | — | — | — | — | — | |
| Llama 3 70B2024.07 | 49 | 90.5 | 16.4 | — | — | — | — | — | — | |
| Llama 3 8B2024.07 | 39.3 | 81 | 15.3 | — | — | — | — | — | — | |
| LPESModel=Qwen2.5-7B, Context length=16k2026.06 | 24.63 | — | 19.06 | 35.77 | 15.91 | 20.84 | 8.02 | 53.77 | 25.43 | |
| BaselineModel=Qwen2.5-7B, Context length=16k2026.06 | 23.33 | — | 17.6 | 25.17 | 14.23 | 17.36 | 6 | 51.12 | 22.12 | |
| MoICEModel=Qwen2.5-7B, Context length=16k2026.06 | 18.83 | — | 18.06 | 38.77 | 15.21 | 17.84 | 6.14 | 53.63 | 24.07 | |
| Claude 3.5 Sonnet2024.07 | 18.5 | 90.5 | 13.4 | — | — | — | — | — | — |