Kernel-level Attention Speed and Memory Analysis on Qwen3-8B (GQA 4:1) on A100 (test)
27.1Forward Pass Time (ms)DualKV
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| DualKVN (rollout factor)=28, P (prompt length)=4K, R (response length)=2048, Hardware=NVIDIA A100 GPU, Precision=fp16, Gradient computation=enabled2026.05 | 27.1 | 77.1 | 104.2 | 1.61 | 1.67 | 1.65 | 63 | |
| FA2N (rollout factor)=28, P (prompt length)=4K, R (response length)=2048, Hardware=NVIDIA A100 GPU, Precision=fp16, Gradient computation=enabled2026.05 | 43.6 | 128.6 | 172.2 | — | — | — | — | |
| DualKVN (rollout factor)=28, P (prompt length)=16K, R (response length)=2048, Hardware=NVIDIA A100 GPU, Precision=fp16, Gradient computation=enabled2026.05 | 98.7 | 273 | 371.7 | 3.86 | 3.88 | 3.88 | 85 | |
| DualKVN (rollout factor)=16, P (prompt length)=32K, R (response length)=2048, Hardware=NVIDIA A100 GPU, Precision=fp16, Gradient computation=enabled2026.05 | 139.5 | 386.2 | 525.8 | 5.53 | 5.47 | 5.48 | 86 | |
| DualKVN (rollout factor)=16, P (prompt length)=64K, R (response length)=2048, Hardware=NVIDIA A100 GPU, Precision=fp16, Gradient computation=enabled2026.05 | 363.3 | 994.5 | 1,357.8 | — | — | — | — | |
| FA2N (rollout factor)=28, P (prompt length)=16K, R (response length)=2048, Hardware=NVIDIA A100 GPU, Precision=fp16, Gradient computation=enabled2026.05 | 381.3 | 1,059.6 | 1,441 | — | — | — | — | |
| FA2N (rollout factor)=16, P (prompt length)=32K, R (response length)=2048, Hardware=NVIDIA A100 GPU, Precision=fp16, Gradient computation=enabled2026.05 | 771.5 | 2,111.8 | 2,883.4 | — | — | — | — |