Throughput Measurement on Synthetic Benchmark (H100, T=1024, B=8)
159.8Throughput (tokens/s)Transformer
Evaluation results
| Method | Links | |
|---|---|---|
| TransformerState update (sketch)=softmax(QK⊤)V, Erase=Exact, Write=Exact, CA=Yes, SKA=Yes, Train cost=O(T²d), Infer/tok=O(Td), Year=20172026.06 | 159.8 | |
| Lin. AttnState update (sketch)=S + vk⊤, Erase=None, Write=+1, CA=No, SKA=No, Train cost=O(Td²), Infer/tok=O(d²), Year=20202026.06 | 148.6 | |
| Delta RuleState update (sketch)=S + (v − Sk)k⊤, Erase=None, Write=Error-corr., CA=No, SKA=No, Train cost=O(Td²), Infer/tok=O(d²), Year=20212026.06 | 142 | |
| CARVE+SWAState update (sketch)=Hybrid, Erase=Key-axis, Write=Scalar, CA=Yes, SKA=Yes, Train cost=O(T(dvdk + Wd)), Infer/tok=O(dvdk)2026.06 | 120 | |
| GDNState update (sketch)=αS + βΔ, Erase=Scalar, Write=Scalar, CA=No, SKA=No, Train cost=O(Td²), Infer/tok=O(d²), Year=20242026.06 | 106 | |
| GDN-2State update (sketch)=(1 − B)⊙S + W⊙Δ, Erase=EW matrix, Write=EW matrix, CA=No, SKA=No, Train cost=O(Tdvdk), Infer/tok=O(dvdk), Year=20262026.06 | 93.8 | |
| CARVEState update (sketch)=Eq. 5, Erase=Key-axis, Write=Scalar-w, CA=Yes, SKA=Yes, Train cost=O(Tdvdk), Infer/tok=O(dvdk)2026.06 | 93.4 | |
| CA-exactState update (sketch)=Eq. 5, Erase=Key-axis, Write=Scalar, CA=Yes, SKA=No, Train cost=O(Tdvdk), Infer/tok=O(dvdk)2026.06 | 24.8 |