Abstract Retrieval on Synthetic Tasks (EM)
98.5Exact MatchOur model
Evaluation results
| Method | Links | |
|---|---|---|
| Our modelContext Window Size=8K2023.11 | 98.5 | |
| ChatGLM3-6B-32KContext Window Size=32K, Backbone=6B2023.11 | 94 | |
| Baichuan2-Turbo-192KContext Window Size=192K2023.11 | 90 | |
| GPT3.5-Turbo-16KContext Window Size=16K2023.11 | 77.5 | |
| ChatGLM2-6B-32KContext Window Size=32K, Backbone=6B2023.11 | 64.5 | |
| Qwen-14B-ChatContext Window Size=8K, Backbone=14B2023.11 | 40 | |
| Longchat-v1.5-7B-32KContext Window Size=32K, Backbone=7B2023.11 | 7.6 | |
| Vicuna-v1.5-7B-16KContext Window Size=16K, Backbone=7B2023.11 | 5 | |
| Xgen-7B-8KContext Window Size=8K, Backbone=7B2023.11 | 3.5 | |
| InternLM-7B-8KContext Window Size=8K, Backbone=7B2023.11 | 0.9 |