Single-Needle-In-A-Haystack retrieval on RULER S-NIAH-3
99.8Accuracy (512 Context)PKDA
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| PKDANumber of Parameters=340M, Training Tokens=15B, Recurrence Parameter Range (beta_t)=[0, 1]2026.04 | 99.8 | 83.2 | 53.8 | 19 | 7 | |
| GDNNumber of Parameters=340M, Training Tokens=15B, Recurrence Parameter Range (beta_t)=[-1, 1]2026.04 | 99.6 | 78.8 | 62.2 | 24.8 | 13.6 | |
| PGDNNumber of Parameters=340M, Training Tokens=15B, Recurrence Parameter Range (beta_t)=[0, 1]2026.04 | 99.2 | 89 | 66.8 | 63.8 | 12.8 | |
| KDANumber of Parameters=340M, Training Tokens=15B, Recurrence Parameter Range (beta_t)=[-1, 1]2026.04 | 99.2 | 94.6 | 4.6 | 1.4 | 8 | |
| PGDNNumber of Parameters=1B, Training Tokens=50B, Recurrence Parameter Range (beta_t)=[0, 1]2026.04 | 98.2 | 90.8 | 76.8 | 63.8 | 12.8 | |
| GDNNumber of Parameters=1B, Training Tokens=50B, Recurrence Parameter Range (beta_t)=[0, 1]2026.04 | 97.6 | 58.6 | 60.4 | 64.2 | 9.4 | |
| PKDANumber of Parameters=1B, Training Tokens=50B, Recurrence Parameter Range (beta_t)=[0, 1]2026.04 | 97.4 | 92.4 | 78.2 | 47.6 | 8.8 | |
| GDNNumber of Parameters=340M, Training Tokens=15B, Recurrence Parameter Range (beta_t)=[0, 1]2026.04 | 96.2 | 10.4 | 35.4 | 20 | 3 | |
| KDANumber of Parameters=1B, Training Tokens=50B, Recurrence Parameter Range (beta_t)=[0, 1]2026.04 | 95.2 | 98 | 80.2 | 40.8 | 5.4 | |
| DNNumber of Parameters=340M, Training Tokens=15B, Recurrence Parameter Range (beta_t)=[-1, 1]2026.04 | 94.4 | 76.8 | 51.2 | 12 | 8 | |
| KDANumber of Parameters=340M, Training Tokens=15B, Recurrence Parameter Range (beta_t)=[0, 1]2026.04 | 93.6 | 77 | 58.4 | 24.6 | 3.2 | |
| PDNNumber of Parameters=340M, Training Tokens=15B, Recurrence Parameter Range (beta_t)=[0, 1]2026.04 | 90.6 | 82.8 | 59.4 | 22 | 2 | |
| DNNumber of Parameters=340M, Training Tokens=15B, Recurrence Parameter Range (beta_t)=[0, 1]2026.04 | 79.2 | 18 | 33.4 | 19 | 3 |