ResearchTasksInference Latency MeasurementInference Latency MeasurementFollowBenchmarksDatasetSOTA methodSortMost resultsRecently updatedMost papersApplyDatasetSOTA methodMetricTrendResultsLast updated16k input points on single H100 GPU (test)Lemon0.052Latency (s)5Feb 26, 2026LLaMA-13B (13824 × 5120 linear layer)MBOK0.05Latency (ms)4Mar 6, 2026LLaMA 5120 × 13824 linear layer 13BMBOK0.051Latency (ms)4Mar 6, 2026LLaMA-13B 5120 × 5120 linear layerMBOK0.051Latency (ms)4Mar 6, 2026