LLM Inference Performance on Context Length 60K
2.59Prefill Time (s)IndexCache
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| IndexCacheRetention Ratio=1/42026.03 | 2.59 | 89.5 | 840 | |
| IndexCacheRetention Ratio=1/22026.03 | 2.86 | 80 | 750 | |
| DSARetention Ratio=Full2026.03 | 3.38 | 67 | 613 |