Prefill Throughput on 2WikiMultihopQA (test)
48.01Throughput (Samples/s)MiniPIC
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| MiniPICStrategy=ISPS, max_num_seqs=10242026.06 | 48.01 | 1.49 | |
| MiniPICStrategy=ISPS, max_num_seqs=20482026.06 | 47.67 | 1.48 | |
| MiniPICStrategy=ISPS, max_num_seqs=642026.06 | 44.51 | 1.38 | |
| MiniPICStrategy=batched, max_num_seqs=10242026.06 | 44.07 | 1.37 | |
| SPNL (CIDRA)Strategy=batched, max_num_seqs=10242026.06 | 36.14 | 1.12 | |
| Baseline vLLMStrategy=full-batch, max_num_seqs=10242026.06 | 32.21 | 1 | |
| MiniPICStrategy=PIC disabled, max_num_seqs=10242026.06 | 30.37 | 0.94 |