Key-Value Retrieval on Key-value retrieval 140 pairs (test)
100P@1 PrecisionGPT-3.5-Turbo-16k
Evaluation Results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| GPT-3.5-Turbo-16kContext Window=16k2024.05 | 100 | 97 | 72.6 | 91.8 | 99.8 | 92.2 | |
| ProLong-13bParameters=13B2024.05 | 100 | 96.2 | 96.6 | 94.8 | 89.4 | 95.4 | |
| ProLong-7bParameters=7B2024.05 | 99.6 | 95.2 | 91.8 | 91.6 | 88.8 | 93.4 | |
| GPT-4-32kContext Window=32k2024.05 | 98.2 | 98.2 | 93.6 | 84 | 100 | 94.8 | |
| Vicuna-v1.5-13B-16kParameters=13B, Context Window=16k2024.05 | 97.8 | 79.2 | 84.6 | 78.6 | 28.2 | 73.7 | |
| Yarn-7b-64kParameters=7B, Context Window=64k2024.05 | 97.6 | 19.2 | 11 | 29 | 92.8 | 49.9 | |
| Yarn-13b-64kParameters=13B, Context Window=64k2024.05 | 74 | 23.6 | 49 | 12.4 | 89.2 | 49.6 |