Hybrid Mamba-Transformer Inference on ShareGPT (trace replay)
2.33OOM Ratefixed_dual_mr05
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| fixed_dual_mr05Variant=fixed_dual_mr052026.05 | 2.33 | 244 | 155 | — | — | |
| avmp_static_mr05Variant=avmp_static_mr052026.05 | 2.33 | 606 | 155 | 0 | 2.48 | |
| avmp_dynamic_b128Variant=avmp_dynamic_b1282026.05 | 2.33 | 578 | 155 | 0 | 2.36 | |
| fixed_dual_mr09Variant=fixed_dual_mr092026.05 | 67.8 | 270 | 155 | 65.5 | 1.1 | |
| padded_unifiedVariant=padded_unified2026.05 | 78.4 | 1,129 | 155 | 76.1 | 4.62 |