Inference Throughput on SAMSum
6,581.55Prefill (tokens/s)BUDDY
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| BUDDYRM Blocks=16(50%)2026.06 | 6,581.55 | 1.63 | 99.12 | 1.63 | |
| BUDDYRM Blocks=16(50%)2026.06 | 6,581.55 | 1.63 | 99.12 | 1.63 | |
| BUDDYRM Blocks=12(37.5%)2026.06 | 5,476.04 | 1.36 | 79.11 | 1.3 | |
| BUDDYRM Blocks=12(37.5%)2026.06 | 5,476.04 | 1.36 | 79.11 | 1.3 | |
| PuDDingRM Blocks=16(50%)2026.06 | 4,836.26 | 1.2 | 124.85 | 2.06 | |
| BUDDYRM Blocks=8(25%)2026.06 | 4,777.07 | 1.18 | 71.91 | 1.18 | |
| BUDDYRM Blocks=8(25%)2026.06 | 4,777.07 | 1.18 | 71.91 | 1.18 | |
| FiRSTRM Blocks=12(37.5%)2026.06 | 4,082.99 | 1.01 | 51.63 | 0.85 | |
| FiRSTRM Blocks=4(12.5%)2026.06 | 4,075.96 | 1.01 | 51.58 | 0.85 | |
| FiRSTRM Blocks=16(50%)2026.06 | 4,041.14 | 1 | 51.2 | 0.84 | |
| BUDDYRM Blocks=4(12.5%)2026.06 | 4,035.68 | 1 | 61.28 | 1.01 | |
| BUDDYRM Blocks=4(12.5%)2026.06 | 4,035.68 | 1 | 61.28 | 1.01 | |
| DenseRM Blocks=0(0.0%)2026.06 | 4,033.83 | 1 | 60.7 | 1 | |
| DenseRM Blocks=0(0.0%)2026.06 | 4,033.83 | 1 | 60.7 | 1 | |
| FiRSTRM Blocks=8(25%)2026.06 | 3,972.77 | 0.98 | 51.05 | 0.84 | |
| PuDDingRM Blocks=12(37.5%)2026.06 | 3,951.91 | 0.98 | 95.53 | 1.57 | |
| PuDDingRM Blocks=8(25%)2026.06 | 3,662.78 | 0.91 | 84.29 | 1.39 | |
| PuDDingRM Blocks=4(12.5%)2026.06 | 3,255.28 | 0.81 | 72.84 | 1.2 |