Long-context retrieval on Ruler (1K to 32K Context Sweep)
100Retrieval Accuracy @ 1024 ContextBAM
Evaluation results
| Method | Links | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| BAMPE=SSMax, Model Parameters=1B2025.05 | 100 | 100 | 100 | 100 | 100 | 100 | 100 | 100 | 100 | 100 | 100 | |
| RoPEPE=SSMax, Model Parameters=1B2025.05 | 100 | 88 | 68 | 40 | 30 | 10 | 2 | 0 | 0 | 0 | 0 | |
| BAMPE=SSMax, Model Parameters=1B2025.05 | 100 | 100 | 100 | 98 | 100 | 88 | 82 | 46 | 18 | 6 | 2 | |
| RoPEPE=SSMax, Model Parameters=1B2025.05 | 100 | 82 | 62 | 32 | 18 | 4 | 0 | 0 | 0 | 0 | 0 | |
| BAMPE=SSMax, Model Parameters=1B2025.05 | 88 | 92 | 88 | 86 | 80 | 62 | 30 | 10 | 2 | 0 | 0 | |
| RoPEPE=SSMax, Model Parameters=1B2025.05 | 76 | 30 | 16 | 4 | 2 | 0 | 0 | 0 | 0 | 0 | 0 | |
| RoPEPE=SSMax, Model Parameters=1B2025.05 | 26 | 14 | 4 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | |
| BAMPE=SSMax, Model Parameters=1B2025.05 | 22 | 16 | 12 | 4 | 4 | 0 | 2 | 2 | 0 | 0 | 0 |