End-to-end attention latency measurement on RK3588S2
2.95Attention Latency (ms)IntAttention
Evaluation Results
| Method | Links | |
|---|---|---|
| IntAttentionSequence Length=1K2025.11 | 2.95 | |
| Quant-OnlySequence Length=1K2025.11 | 5.96 | |
| FP16Sequence Length=1K2025.11 | 10.96 | |
| IntAttentionSequence Length=2K2025.11 | 11.29 | |
| FP32Sequence Length=1K2025.11 | 12.57 | |
| Quant-OnlySequence Length=2K2025.11 | 22.24 | |
| FP16Sequence Length=2K2025.11 | 36.04 | |
| FP32Sequence Length=2K2025.11 | 42.09 | |
| IntAttentionSequence Length=4K2025.11 | 46.96 | |
| Quant-OnlySequence Length=4K2025.11 | 84.85 | |
| FP16Sequence Length=4K2025.11 | 134.64 | |
| FP32Sequence Length=4K2025.11 | 160.11 | |
| IntAttentionSequence Length=8K2025.11 | 200.94 | |
| Quant-OnlySequence Length=8K2025.11 | 325.48 | |
| FP16Sequence Length=8K2025.11 | 440.3 | |
| FP32Sequence Length=8K2025.11 | 586.86 | |
| IntAttentionSequence Length=16K2025.11 | 794.36 | |
| Quant-OnlySequence Length=16K2025.11 | 1,279.94 | |
| FP16Sequence Length=16K2025.11 | 1,677.49 | |
| FP32Sequence Length=16K2025.11 | 2,279.96 |