Single-Needle-In-A-Haystack retrieval on RULER S-NIAH-2
100Accuracy (512 Context)DN
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| DNNumber of Parameters=340M, Training Tokens=15B, Recurrence Parameter Range (beta_t)=[0, 1]2026.04 | 100 | 91.8 | 95.4 | 29.2 | 8.4 | |
| PDNNumber of Parameters=340M, Training Tokens=15B, Recurrence Parameter Range (beta_t)=[0, 1]2026.04 | 100 | 100 | 99.2 | 28.4 | 14.2 | |
| DNNumber of Parameters=340M, Training Tokens=15B, Recurrence Parameter Range (beta_t)=[-1, 1]2026.04 | 100 | 99.8 | 98.8 | 31.2 | 14.4 | |
| GDNNumber of Parameters=340M, Training Tokens=15B, Recurrence Parameter Range (beta_t)=[0, 1]2026.04 | 100 | 100 | 97.6 | 29.4 | 1.4 | |
| PGDNNumber of Parameters=340M, Training Tokens=15B, Recurrence Parameter Range (beta_t)=[0, 1]2026.04 | 100 | 100 | 93.8 | 52.8 | 31.4 | |
| GDNNumber of Parameters=340M, Training Tokens=15B, Recurrence Parameter Range (beta_t)=[-1, 1]2026.04 | 100 | 100 | 88.4 | 31.6 | 15.2 | |
| KDANumber of Parameters=340M, Training Tokens=15B, Recurrence Parameter Range (beta_t)=[0, 1]2026.04 | 100 | 100 | 98.8 | 37.6 | 11.4 | |
| PKDANumber of Parameters=340M, Training Tokens=15B, Recurrence Parameter Range (beta_t)=[0, 1]2026.04 | 100 | 100 | 98.2 | 44.2 | 12 | |
| KDANumber of Parameters=340M, Training Tokens=15B, Recurrence Parameter Range (beta_t)=[-1, 1]2026.04 | 100 | 100 | 99.4 | 36 | 3.4 | |
| GDNNumber of Parameters=1B, Training Tokens=50B, Recurrence Parameter Range (beta_t)=[0, 1]2026.04 | 100 | 100 | 100 | 67.4 | 24.8 | |
| PGDNNumber of Parameters=1B, Training Tokens=50B, Recurrence Parameter Range (beta_t)=[0, 1]2026.04 | 100 | 100 | 99 | 90.2 | 11.3 | |
| KDANumber of Parameters=1B, Training Tokens=50B, Recurrence Parameter Range (beta_t)=[0, 1]2026.04 | 100 | 100 | 100 | 80.4 | 22.8 | |
| PKDANumber of Parameters=1B, Training Tokens=50B, Recurrence Parameter Range (beta_t)=[0, 1]2026.04 | 100 | 100 | 100 | 79 | 25.8 |