LLM Inference Performance on Context Length 200K
10.7Prefill Time (s)IndexCache
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| IndexCacheRetention Ratio=1/42026.03 | 10.7 | 86 | 297 | |
| IndexCacheRetention Ratio=1/22026.03 | 13.7 | 73 | 253 | |
| DSARetention Ratio=Full2026.03 | 19.5 | 58 | 197 |