Key-Value Retrieval on Key-value retrieval 300 pairs (test)
100Precision@1 (P1)GPT-3.5-Turbo-16k
Evaluation Results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| GPT-3.5-Turbo-16kContext Window=16k2024.05 | 100 | 85 | 75.8 | 50 | 29.4 | 55.2 | 99.4 | 70.7 | |
| ProLong-13bParameters=13B2024.05 | 99.8 | 86 | 73.6 | 59.4 | 88.4 | 94.2 | 87.6 | 84.1 | |
| GPT-4-32kContext Window=32k2024.05 | 99 | 94 | 52.2 | 37.8 | 20.8 | 20 | 99.8 | 60.5 | |
| ProLong-7bParameters=7B2024.05 | 98 | 61.8 | 82.4 | 88.2 | 86.4 | 93.4 | 92 | 86 | |
| Yarn-13b-64kParameters=13B, Context Window=64k2024.05 | 72.2 | 9.2 | 3.6 | 1.2 | 8.4 | 7 | 86.4 | 26.9 | |
| Yarn-7b-64kParameters=7B, Context Window=64k2024.05 | 71.6 | 10.8 | 1.6 | 1 | 4.6 | 3.8 | 73.2 | 23.8 |