In-Context Learning on Garg ICL d=20 2022 (held-out tasks)
0.989R2Transformer (continuous attn)
Evaluation results
| Method | Links | |
|---|---|---|
| Transformer (continuous attn)Params=1.0M2026.07 | 0.989 | |
| DendriCNN (continuous ablation)Params=0.75M2026.07 | 0.833 | |
| DendriCL (single-layer)Params=0.75M, Training budget=50k steps2026.07 | 0.82 | |
| DendriCL (single-layer)Params=0.75M, Training budget=10k steps2026.07 | 0.805 | |
| DendriCL (L = 2)Params=0.75M2026.07 | 0.793 | |
| Spikformer (spike attn)Params=1.0M2026.07 | 0.724 | |
| Pure LIFParams=1.0M2026.07 | 0.086 | |
| Active DendritesParams=1.29M2026.07 | 0.061 | |
| GRUParams=0.79M2026.07 | 0.058 | |
| LSNNParams=0.62M2026.07 | 0.008 |