ResearchBenchmarksLanguage Modeling Inference on Qwen2.5-7B (64K Context Length)Language Modeling Inference on Qwen2.5-7B (64K Context Length)Follow13.6Decode Latency (ms/token)FastMKA12.57619.48826.433.312Mar 21, 2026Evaluation resultsMethodMethodLinksDecode Latency (ms/token)Speedup vs MLAFastMKABatch size=1, Precisio...Batch size=1, Precision=bf162026.0313.61.68MLABatch size=1, Precisio...Batch size=1, Precision=bf162026.0322.8—GQABatch size=1, Precisio...Batch size=1, Precision=bf162026.0333.4—MHABatch size=1, Precisio...Batch size=1, Precision=bf162026.0339.2—