End-to-end attention latency measurement on Apple M2
0.87Latency (ms)IntAttention
Evaluation Results
| Method | Links | |
|---|---|---|
| IntAttentionSequence Length=1K2025.11 | 0.87 | |
| Quant-OnlySequence Length=1K2025.11 | 1.61 | |
| FP16Sequence Length=1K2025.11 | 2.08 | |
| IntAttentionSequence Length=2K2025.11 | 2.62 | |
| FP32Sequence Length=1K2025.11 | 2.81 | |
| Quant-OnlySequence Length=2K2025.11 | 5.85 | |
| FP16Sequence Length=2K2025.11 | 6.72 | |
| FP32Sequence Length=2K2025.11 | 9.04 | |
| IntAttentionSequence Length=4K2025.11 | 10.64 | |
| Quant-OnlySequence Length=4K2025.11 | 22.41 | |
| FP16Sequence Length=4K2025.11 | 25.42 | |
| FP32Sequence Length=4K2025.11 | 32.64 | |
| IntAttentionSequence Length=8K2025.11 | 38.86 | |
| Quant-OnlySequence Length=8K2025.11 | 84.6 | |
| FP16Sequence Length=8K2025.11 | 100.36 | |
| FP32Sequence Length=8K2025.11 | 125.13 | |
| IntAttentionSequence Length=16K2025.11 | 142.5 | |
| Quant-OnlySequence Length=16K2025.11 | 341.7 | |
| FP16Sequence Length=16K2025.11 | 398.39 | |
| FP32Sequence Length=16K2025.11 | 529.4 |