KV cache serving on 3774-token document N=80M agents largest measured point
2Serving Cost (USD)ship int8
Evaluation results
| Method | Links | ||
|---|---|---|---|
| ship int8Precision=int8, Compression=2×, Exactness=lossy2026.06 | 2 | 0.8 | |
| ship raw fp16Precision=fp16, Exactness=lossless, Transport=egress2026.06 | 3.9 | 0.4 | |
| host, physicalCompute ratio (Creuse/Cprefill)=1/49.7, Status=measured2026.06 | 30,370 | 49.7 | |
| host, pricedTariff=0.1pin2026.06 | 150,960 | 10 | |
| re-prefill (today)Strategy=re-prefill2026.06 | 1,509,600 | 1 |