# LLM Decoding on Llama 70B (H100 GPU Cluster) 3.1 Source: [LLM Decoding on Llama 70B (H100 GPU Cluster) 3.1]() Collection index: [llms.txt]() Data updated: 2026-02-26T06:06:23.897Z Evaluation of the throughput performance for the Llama 70B model running on an H100 GPU cluster, version 3.1. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | LLM Decoding | | Dataset (including any stated split) | Llama 70B (H100 GPU Cluster) 3.1 | | Metric | Throughput | | Unit | SCALAR | | Direction | Higher is better | | Results loaded | 27 of 27 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Throughput](): Higher is better; unit: SCALAR. ## Recorded results | Method | Throughput | Result date | Report paper | | --- | --- | --- | --- | | DeepFusionKernel | 894.32 | Not recorded | 698eb00ac97a0b3e3b5e6734 | | SGLang | 878.08 | Not recorded | 698eb00ac97a0b3e3b5e6734 | | vLLM | 865.37 | Not recorded | 698eb00ac97a0b3e3b5e6734 | | DeepFusionKernel | 863.4 | Not recorded | 698eb00ac97a0b3e3b5e6734 | | vLLM | 850.33 | Not recorded | 698eb00ac97a0b3e3b5e6734 | | SGLang | 815.4 | Not recorded | 698eb00ac97a0b3e3b5e6734 | | DeepFusionKernel | 764.57 | Not recorded | 698eb00ac97a0b3e3b5e6734 | | vLLM | 746.42 | Not recorded | 698eb00ac97a0b3e3b5e6734 | | SGLang | 732.18 | Not recorded | 698eb00ac97a0b3e3b5e6734 | | DeepFusionKernel | 234.61 | Not recorded | 698eb00ac97a0b3e3b5e6734 | | DeepFusionKernel | 230.98 | Not recorded | 698eb00ac97a0b3e3b5e6734 | | SGLang | 228.9 | Not recorded | 698eb00ac97a0b3e3b5e6734 | | DeepFusionKernel | 227.73 | Not recorded | 698eb00ac97a0b3e3b5e6734 | | vLLM | 227.7 | Not recorded | 698eb00ac97a0b3e3b5e6734 | | vLLM | 227.69 | Not recorded | 698eb00ac97a0b3e3b5e6734 | | SGLang | 219.48 | Not recorded | 698eb00ac97a0b3e3b5e6734 | | vLLM | 218.73 | Not recorded | 698eb00ac97a0b3e3b5e6734 | | SGLang | 213.94 | Not recorded | 698eb00ac97a0b3e3b5e6734 | | DeepFusionKernel | 61.22 | Not recorded | 698eb00ac97a0b3e3b5e6734 | | DeepFusionKernel | 61.08 | Not recorded | 698eb00ac97a0b3e3b5e6734 | | SGLang | 60.77 | Not recorded | 698eb00ac97a0b3e3b5e6734 | | DeepFusionKernel | 60.72 | Not recorded | 698eb00ac97a0b3e3b5e6734 | | SGLang | 60.2 | Not recorded | 698eb00ac97a0b3e3b5e6734 | | SGLang | 60.18 | Not recorded | 698eb00ac97a0b3e3b5e6734 | | vLLM | 58.7 | Not recorded | 698eb00ac97a0b3e3b5e6734 | | vLLM | 58.37 | Not recorded | 698eb00ac97a0b3e3b5e6734 | | vLLM | 57.98 | Not recorded | 698eb00ac97a0b3e3b5e6734 | ### DeepFusionKernel Throughput: 894.32. Result ID: 6998bccf5698665ec41feab8. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Batch size":16,"Output len":1024,"Kernel scheduler":true}. This record does not assert independent reproduction. ### SGLang Throughput: 878.08. Result ID: 6998bccf5698665ec41feab6. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Batch size":16,"Output len":1024,"Kernel scheduler":true}. This record does not assert independent reproduction. ### vLLM Throughput: 865.37. Result ID: 6998bccf5698665ec41feab7. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Batch size":16,"Output len":1024,"Kernel scheduler":true}. This record does not assert independent reproduction. ### DeepFusionKernel Throughput: 863.4. Result ID: 6998bccf5698665ec41feabb. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Batch size":16,"Output len":4096,"Kernel scheduler":true}. This record does not assert independent reproduction. ### vLLM Throughput: 850.33. Result ID: 6998bccf5698665ec41feaba. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Batch size":16,"Output len":4096,"Kernel scheduler":true}. This record does not assert independent reproduction. ### SGLang Throughput: 815.4. Result ID: 6998bccf5698665ec41feab9. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Batch size":16,"Output len":4096,"Kernel scheduler":true}. This record does not assert independent reproduction. ### DeepFusionKernel Throughput: 764.57. Result ID: 6998bccf5698665ec41feabe. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Batch size":16,"Output len":16384,"Kernel scheduler":true}. This record does not assert independent reproduction. ### vLLM Throughput: 746.42. Result ID: 6998bccf5698665ec41feabd. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Batch size":16,"Output len":16384,"Kernel scheduler":true}. This record does not assert independent reproduction. ### SGLang Throughput: 732.18. Result ID: 6998bccf5698665ec41feabc. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Batch size":16,"Output len":16384,"Kernel scheduler":true}. This record does not assert independent reproduction. ### DeepFusionKernel Throughput: 234.61. Result ID: 6998bccf5698665ec41feab2. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Batch size":4,"Output len":4096,"Kernel scheduler":true}. This record does not assert independent reproduction. ### DeepFusionKernel Throughput: 230.98. Result ID: 6998bccf5698665ec41feaaf. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Batch size":4,"Output len":1024,"Kernel scheduler":true}. This record does not assert independent reproduction. ### SGLang Throughput: 228.9. Result ID: 6998bccf5698665ec41feaad. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Batch size":4,"Output len":1024,"Kernel scheduler":true}. This record does not assert independent reproduction. ### DeepFusionKernel Throughput: 227.73. Result ID: 6998bccf5698665ec41feab5. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Batch size":4,"Output len":16384,"Kernel scheduler":true}. This record does not assert independent reproduction. ### vLLM Throughput: 227.7. Result ID: 6998bccf5698665ec41feab1. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Batch size":4,"Output len":4096,"Kernel scheduler":true}. This record does not assert independent reproduction. ### vLLM Throughput: 227.69. Result ID: 6998bccf5698665ec41feaae. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Batch size":4,"Output len":1024,"Kernel scheduler":true}. This record does not assert independent reproduction. ### SGLang Throughput: 219.48. Result ID: 6998bccf5698665ec41feab0. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Batch size":4,"Output len":4096,"Kernel scheduler":true}. This record does not assert independent reproduction. ### vLLM Throughput: 218.73. Result ID: 6998bccf5698665ec41feab4. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Batch size":4,"Output len":16384,"Kernel scheduler":true}. This record does not assert independent reproduction. ### SGLang Throughput: 213.94. Result ID: 6998bccf5698665ec41feab3. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Batch size":4,"Output len":16384,"Kernel scheduler":true}. This record does not assert independent reproduction. ### DeepFusionKernel Throughput: 61.22. Result ID: 6998bccf5698665ec41feaa6. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Batch size":1,"Output len":1024,"Kernel scheduler":true}. This record does not assert independent reproduction. ### DeepFusionKernel Throughput: 61.08. Result ID: 6998bccf5698665ec41feaa9. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Batch size":1,"Output len":4096,"Kernel scheduler":true}. This record does not assert independent reproduction. ### SGLang Throughput: 60.77. Result ID: 6998bccf5698665ec41feaa7. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Batch size":1,"Output len":4096,"Kernel scheduler":true}. This record does not assert independent reproduction. ### DeepFusionKernel Throughput: 60.72. Result ID: 6998bccf5698665ec41feaac. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Batch size":1,"Output len":16384,"Kernel scheduler":true}. This record does not assert independent reproduction. ### SGLang Throughput: 60.2. Result ID: 6998bccf5698665ec41feaa4. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Batch size":1,"Output len":1024,"Kernel scheduler":true}. This record does not assert independent reproduction. ### SGLang Throughput: 60.18. Result ID: 6998bccf5698665ec41feaaa. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Batch size":1,"Output len":16384,"Kernel scheduler":true}. This record does not assert independent reproduction. ### vLLM Throughput: 58.7. Result ID: 6998bccf5698665ec41feaa8. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Batch size":1,"Output len":4096,"Kernel scheduler":true}. This record does not assert independent reproduction. ### vLLM Throughput: 58.37. Result ID: 6998bccf5698665ec41feaa5. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Batch size":1,"Output len":1024,"Kernel scheduler":true}. This record does not assert independent reproduction. ### vLLM Throughput: 57.98. Result ID: 6998bccf5698665ec41feaab. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Batch size":1,"Output len":16384,"Kernel scheduler":true}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.