# Inference Efficiency on LLaMA 8B 8K context length 3.1 Source: [Inference Efficiency on LLaMA 8B 8K context length 3.1]() Collection index: [llms.txt]() Data updated: 2026-03-12T09:25:43.146Z Evaluation of the inference efficiency, including theoretical and empirical Time To First Token (TTFT) metrics, compute, and memory traffic for the LLaMA 8B model operating with an 8K context length (Llama 3.1 variant). A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Inference Efficiency | | Dataset (including any stated split) | LLaMA 8B 8K context length 3.1 | | Metric | Theoretical Compute (TFLOPs) | | Unit | SIZE | | Direction | Higher is better | | Results loaded | 10 of 10 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Theoretical Compute (TFLOPs)](): Higher is better; unit: SIZE. - [Theoretical Memory Traffic (GB)](): Lower is better; unit: SIZE. - [Theoretical TTFT (ms)](): Lower is better; unit: TIME. - [Theoretical TTFT Overhead (ms)](): Lower is better; unit: TIME. - [Empirical TTFT (ms)](): Lower is better; unit: TIME. - [Empirical TTFT Overhead (ms)](): Lower is better; unit: TIME. ## Recorded results | Method | Theoretical Compute (TFLOPs) | Result date | Report paper | | --- | --- | --- | --- | | SpecKV | 159 | Not recorded | 69b2488fd030084c22e2f8ea | | SpecKV | 159 | Not recorded | 69b2488fd030084c22e2f8ea | | LOOKAHEADKV | 137 | Not recorded | 69b2488fd030084c22e2f8ea | | LAQ | 137 | Not recorded | 69b2488fd030084c22e2f8ea | | LOOKAHEADKV | 137 | Not recorded | 69b2488fd030084c22e2f8ea | | LAQ | 137 | Not recorded | 69b2488fd030084c22e2f8ea | | Forward Pass Only | 136 | Not recorded | 69b2488fd030084c22e2f8ea | | SnapKV | 136 | Not recorded | 69b2488fd030084c22e2f8ea | | Forward Pass Only | 136 | Not recorded | 69b2488fd030084c22e2f8ea | | SnapKV | 136 | Not recorded | 69b2488fd030084c22e2f8ea | ### SpecKV Theoretical Compute (TFLOPs): 159. Result ID: 69b2867a6dbb7075b95d7600. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Cache Budget (C)":"128"}. This record does not assert independent reproduction. ### SpecKV Theoretical Compute (TFLOPs): 159. Result ID: 69b2867a6dbb7075b95d7695. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### LOOKAHEADKV Theoretical Compute (TFLOPs): 137. Result ID: 69b2867a6dbb7075b95d75fe. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Cache Budget (C)":"128"}. This record does not assert independent reproduction. ### LAQ Theoretical Compute (TFLOPs): 137. Result ID: 69b2867a6dbb7075b95d7601. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Cache Budget (C)":"128"}. This record does not assert independent reproduction. ### LOOKAHEADKV Theoretical Compute (TFLOPs): 137. Result ID: 69b2867a6dbb7075b95d7693. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### LAQ Theoretical Compute (TFLOPs): 137. Result ID: 69b2867a6dbb7075b95d7696. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### Forward Pass Only Theoretical Compute (TFLOPs): 136. Result ID: 69b2867a6dbb7075b95d75fd. [Reporting paper]() Recorded evaluation configuration: {"Cache Budget (C)":"128"}. This record does not assert independent reproduction. ### SnapKV Theoretical Compute (TFLOPs): 136. Result ID: 69b2867a6dbb7075b95d75ff. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Cache Budget (C)":"128"}. This record does not assert independent reproduction. ### Forward Pass Only Theoretical Compute (TFLOPs): 136. Result ID: 69b2867a6dbb7075b95d7692. [Reporting paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### SnapKV Theoretical Compute (TFLOPs): 136. Result ID: 69b2867a6dbb7075b95d7694. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.