LLM Inference Performance on Context Length 120K
5.66Prefill Time (s)IndexCache
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| IndexCacheRetention Ratio=1/42026.03 | 5.66 | 88 | 498 | |
| IndexCacheRetention Ratio=1/22026.03 | 6.57 | 77 | 431 | |
| DSARetention Ratio=Full2026.03 | 8.57 | 63 | 341 |