Single-stream decode on 622.9 MB 4-layer model
1.08Latency (ms/token)CUDA-graph eager
Evaluation results
| Method | Links | ||
|---|---|---|---|
| CUDA-graph eagerGPU=RTX 5090, Decode type=single-token, Correctness-gated=true2026.06 | 1.08 | 1.13 | |
| AMKGPU=RTX 5090, Comparison Baseline=eager per-op, Decode type=single-token, Correctness-gated=true2026.06 | 1.23 | 3.6 | |
| AMKGPU=RTX 5090, Comparison Baseline=CUDA-graph eager, Decode type=single-token, Correctness-gated=true2026.06 | 1.23 | — | |
| eager per-opGPU=RTX 5090, Decode type=single-token, Correctness-gated=true2026.06 | 4.44 | — |