Generative Recommendation on MT
11.7Latency (ms)MTServe
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| MTServeBatch Size=12026.04 | 11.7 | 1.21 | 60.26 | 96.73 | |
| GPU-OnlyBatch Size=12026.04 | 12.8 | 1.1 | 60.26 | 60.26 | |
| Recomp.Batch Size=12026.04 | 14.1 | 1 | — | — | |
| MTServeBatch Size=42026.04 | 17.3 | 2.52 | 60.69 | 97.44 | |
| MTServeBatch Size=82026.04 | 26.6 | 3.1 | 61.73 | 98.71 | |
| GPU-OnlyBatch Size=42026.04 | 28.9 | 1.51 | 60.69 | 60.69 | |
| Recomp.Batch Size=42026.04 | 43.6 | 1 | — | — | |
| GPU-OnlyBatch Size=82026.04 | 48.6 | 1.7 | 61.73 | 61.73 | |
| Recomp.Batch Size=82026.04 | 82.4 | 1 | — | — |