In-context Retrieval on LM-Evaluation-Harness (Zero-Shot)
23.23FDA AccuracyKDA
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| KDAParameters=1B, Input tokens=2K2026.04 | 23.23 | 50.14 | 37 | 42.7 | 19.7 | 34.55 | |
| KDAnumber of parameters=1B, training tokens=50B, batch size=0.5M tokens, beta range=[0, 1]2026.04 | 23.23 | 50.14 | 37 | 42.7 | 19.7 | 34.55 | |
| PKDAParameters=1B, Input tokens=2K2026.04 | 21.69 | 48.79 | 38.61 | 44.6 | 20.7 | 34.88 | |
| PKDAnumber of parameters=1B, training tokens=50B, batch size=0.5M tokens, beta range=[0, 1]2026.04 | 21.69 | 48.79 | 38.61 | 44.6 | 20.7 | 34.88 | |
| GDNParameters=1B, Input tokens=2K2026.04 | 19.69 | 49.86 | 37.7 | 41.6 | 20.5 | 33.87 | |
| GDNnumber of parameters=1B, training tokens=50B, batch size=0.5M tokens, beta range=[0, 1]2026.04 | 19.69 | 49.86 | 37.7 | 41.6 | 20.5 | 33.87 | |
| PGDNParameters=1B, Input tokens=2K2026.04 | 18.33 | 50.32 | 41.19 | 44.3 | 20.8 | 34.99 | |
| PGDNnumber of parameters=1B, training tokens=50B, batch size=0.5M tokens, beta range=[0, 1]2026.04 | 18.33 | 50.32 | 41.19 | 44.3 | 20.8 | 34.99 | |
| GDNnumber of parameters=340M, training tokens=15B, batch size=0.5M tokens, beta range=[-1, 1]2026.04 | 16.97 | 31.41 | 31.03 | 35 | 16.7 | 26.22 | |
| KDAnumber of parameters=340M, training tokens=15B, batch size=0.5M tokens, beta range=[-1, 1]2026.04 | 16.42 | 32.49 | 30.73 | 35 | 15.3 | 25.99 | |
| PGDNParameters=340M, Input tokens=2K2026.04 | 14.25 | 32.24 | 31.76 | 35.2 | 17.4 | 26.17 | |
| PKDAParameters=355M, Input tokens=2K2026.04 | 14.25 | 34.65 | 31.39 | 35.5 | 18 | 26.76 | |
| PGDNnumber of parameters=340M, training tokens=15B, batch size=0.5M tokens, beta range=[0, 1]2026.04 | 14.25 | 32.24 | 31.76 | 35.2 | 17.4 | 26.17 | |
| PKDAnumber of parameters=340M, training tokens=15B, batch size=0.5M tokens, beta range=[0, 1]2026.04 | 14.25 | 34.65 | 31.39 | 35.5 | 18 | 26.76 | |
| PDNParameters=340M, Input tokens=2K2026.04 | 14.16 | 29.17 | 28.89 | 30.6 | 16.4 | 23.84 | |
| PDNnumber of parameters=340M, training tokens=15B, batch size=0.5M tokens, beta range=[0, 1]2026.04 | 14.16 | 29.17 | 28.89 | 30.6 | 16.4 | 23.84 | |
| KDAParameters=355M, Input tokens=2K2026.04 | 13.88 | 34.11 | 29.69 | 34.8 | 16.9 | 25.88 | |
| KDAnumber of parameters=340M, training tokens=15B, batch size=0.5M tokens, beta range=[0, 1]2026.04 | 13.88 | 34.11 | 29.69 | 34.8 | 16.9 | 25.88 | |
| GDNParameters=340M, Input tokens=2K2026.04 | 13.61 | 29.43 | 29.83 | 33.8 | 17.2 | 24.77 | |
| GDNnumber of parameters=340M, training tokens=15B, batch size=0.5M tokens, beta range=[0, 1]2026.04 | 13.61 | 29.43 | 29.83 | 33.8 | 17.2 | 24.77 | |
| DNnumber of parameters=340M, training tokens=15B, batch size=0.5M tokens, beta range=[-1, 1]2026.04 | 9.53 | 30.15 | 28.95 | 29.6 | 15.4 | 22.73 | |
| DNParameters=340M, Input tokens=2K2026.04 | 8.53 | 27.09 | 28.32 | 31.2 | 17.8 | 22.59 | |
| DNnumber of parameters=340M, training tokens=15B, batch size=0.5M tokens, beta range=[0, 1]2026.04 | 8.53 | 27.09 | 28.32 | 31.2 | 17.8 | 22.59 |