Zero-shot Reasoning on PiQA, LAMBDA, ARC (e/c), and HellaSwag
62.69PiQA ScoreNSA
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| NSAModel=Qwen3 0.6B, Operation Category=Token-Level Operation, Latency@128k (Decode)=9.73, Latency@128k (Forward)=20.322026.03 | 62.69 | 35.01 | 45.1 | 20.47 | 34.42 | 39.54 | |
| Dense(full)Model=Qwen3 0.6B, Operation Category=Baseline, Latency@128k (Decode)=80.84, Latency@128k (Forward)=77.652026.03 | 62.47 | 34.82 | 45.41 | 20.35 | 33.95 | 39.4 | |
| MLAModel=Qwen3 0.6B, Operation Category=Feature-Level Operation, Latency@128k (Decode)=8.74, Latency@128k (Forward)=68.922026.03 | 62.39 | 34.71 | 45.41 | 20.17 | 34.21 | 39.38 | |
| QuantModel=Qwen3 0.6B, Operation Category=Feature-Level Operation, Latency@128k (Decode)=72.23, Latency@128k (Forward)=59.732026.03 | 62.29 | 34.33 | 45.39 | 20.02 | 33.91 | 39.19 | |
| SFA (k = 16)Model=Qwen3 0.6B, Operation Category=Feature-Level Operation, Latency@128k (Decode)=66.29, Latency@128k (Forward)=34.202026.03 | 61.73 | 34.05 | 45.62 | 19.27 | 34.03 | 38.94 | |
| MLA + SFAModel=Qwen3 0.6B, Operation Category=Feature-Level Operation, Latency@128k (Decode)=6.72, Latency@128k (Forward)=65.292026.03 | 61.22 | 33.94 | 43.36 | 19.25 | 33.94 | 38.34 | |
| +SFA (k = 16)Model=Qwen3 0.6B, Operation Category=Token-Level Operation, Latency@128k (Decode)=8.85, Latency@128k (Forward)=17.172026.03 | 60.02 | 33.58 | 42.74 | 18.31 | 32.48 | 37.43 | |
| SFA (quant)Model=Qwen3 0.6B, Operation Category=Feature-Level Operation, Latency@128k (Decode)=57.47, Latency@128k (Forward)=30.742026.03 | 59.63 | 33.1 | 44.93 | 15.98 | 33.64 | 37.46 | |
| Low-RankModel=Qwen3 0.6B, Operation Category=Feature-Level Operation, Latency@128k (Decode)=40.58, Latency@128k (Forward)=32.462026.03 | 59.19 | 31.49 | 41.77 | 15.8 | 30.65 | 35.78 | |
| Short (d = 64)Model=Qwen3 0.6B, Operation Category=Feature-Level Operation, Latency@128k (Decode)=38.68, Latency@128k (Forward)=30.842026.03 | 58.43 | 31.27 | 41.58 | 15.83 | 28.29 | 35.08 | |
| MLAModel=GPT2 124M, Operation Category=Feature-Level Operation, Latency@128k (Decode)=5.04, Latency@128k (Forward)=15.392026.03 | 57.83 | 22.29 | 28.37 | 13.92 | 19.66 | 28.41 | |
| Dense (full)Model=GPT2 124M, Operation Category=Baseline, Latency@128k (Decode)=17.08, Latency@128k (Forward)=16.862026.03 | 56.34 | 22.78 | 28.35 | 14.32 | 19.61 | 28.28 | |
| QuantModel=GPT2 124M, Operation Category=Feature-Level Operation, Latency@128k (Decode)=14.26, Latency@128k (Forward)=12.972026.03 | 56.18 | 21.03 | 28.09 | 13.58 | 19.05 | 27.59 | |
| SFA (k = 8)Model=GPT2 124M, Operation Category=Feature-Level Operation, Latency@128k (Decode)=14.12, Latency@128k (Forward)=9.412026.03 | 54.92 | 21.03 | 28.41 | 13.41 | 19.26 | 27.4 | |
| SFA (quant)Model=GPT2 124M, Operation Category=Feature-Level Operation, Latency@128k (Decode)=12.28, Latency@128k (Forward)=8.722026.03 | 54.53 | 20.81 | 28.39 | 13.27 | 18.97 | 27.12 | |
| MLA + SFAModel=GPT2 124M, Operation Category=Feature-Level Operation, Latency@128k (Decode)=3.98, Latency@128k (Forward)=15.052026.03 | 54.33 | 21.92 | 27.88 | 13.1 | 19.01 | 27.25 | |
| LongformerModel=GPT2 124M, Operation Category=Token-Level Operation, Latency@128k (Decode)=6.75, Latency@128k (Forward)=7.932026.03 | 54.25 | 21.27 | 28.02 | 13.01 | 18.92 | 28.1 | |
| +SFA (k = 8)Model=GPT2 124M, Operation Category=Token-Level Operation, Latency@128k (Decode)=5.23, Latency@128k (Forward)=6.182026.03 | 52.81 | 20.54 | 26.39 | 12.59 | 17.24 | 25.91 | |
| Low-RankModel=GPT2 124M, Operation Category=Feature-Level Operation, Latency@128k (Decode)=8.93, Latency@128k (Forward)=7.992026.03 | 51.79 | 20.04 | 26.47 | 12.92 | 14.99 | 25.24 | |
| Short (d = 32)Model=GPT2 124M, Operation Category=Feature-Level Operation, Latency@128k (Decode)=8.37, Latency@128k (Forward)=7.862026.03 | 51.3 | 19.39 | 25.72 | 12.47 | 14.26 | 24.63 |