Generative Recommendation on KuaiRand 1K
14.4Latency (ms)MTServe
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| MTServeBatch Size=12026.04 | 14.4 | 1.47 | 63.52 | 94.66 | |
| GPU-OnlyBatch Size=12026.04 | 17.4 | 1.22 | 63.52 | 63.52 | |
| Recomp.Batch Size=12026.04 | 21.2 | 1 | — | — | |
| MTServeBatch Size=42026.04 | 33.5 | 2.17 | 63.92 | 98.57 | |
| GPU-OnlyBatch Size=42026.04 | 42.1 | 1.73 | 63.92 | 63.92 | |
| MTServeBatch Size=82026.04 | 47.3 | 3.04 | 64.36 | 98.59 | |
| Recomp.Batch Size=42026.04 | 72.8 | 1 | — | — | |
| GPU-OnlyBatch Size=82026.04 | 72.9 | 1.97 | 64.36 | 64.36 | |
| Recomp.Batch Size=82026.04 | 143.6 | 1 | — | — |