ResearchDatasetsLong-context memory-constrained settingLong-context memory-constrained settingFollowBenchmarksTaskDatasetSOTA methodSortMost resultsRecently updatedMost papersApplyTaskDatasetSOTA resultTrendResultsLong-context inferenceLong-context memory-constrained setting 16 docs x 8K tokens, 10 GB KV pool7.2Time To First Token (TTFT) (s)2
Long-context inferenceLong-context memory-constrained setting 16 docs x 8K tokens, 10 GB KV pool7.2Time To First Token (TTFT) (s)2