ResearchBenchmarksDownstream Task Evaluation on MNLI, SCIQ, LAMBADA, HellaSwag, ARC, and MMLUDownstream Task Evaluation on MNLI, SCIQ, LAMBADA, HellaSwag, ARC, and MMLUFollow0.3852MNLI AccFusedKV0.37220.3755750.378950.382325Dec 3, 2025Evaluation resultsMethodMethodLinksMNLI AccSCIQ AccLAMBADA AccHellaSwag AccARC-E AccARC-C AccMMLU AccAverage AccuracyFusedKVModel Parameters=4BModel Parameters=4B2025.120.38520.9520.50180.69940.77780.48630.39830.6001VanillaModel Parameters=4BModel Parameters=4B2025.120.37270.960.4960.68920.76430.46590.38710.5907