ResearchBenchmarksLanguage Modeling on 50B tokens (train)Language Modeling on 50B tokens (train)Follow0.85Relative Loss ErrorHiFloat40.81241.06621.321.5738Apr 9, 2026Evaluation resultsMethodMethodLinksRelative Loss ErrorHiFloat4Model=Llama3-8BModel=Llama3-8B2026.040.85HiFloat4Model=Qwen3-MoE-30BModel=Qwen3-MoE-30B2026.040.88HiFloat4Model=OpenPangu-1BModel=OpenPangu-1B2026.041.19MXFP4Model=Llama3-8BModel=Llama3-8B2026.041.44MXFP4Model=Qwen3-MoE-30BModel=Qwen3-MoE-30B2026.041.55MXFP4Model=OpenPangu-1BModel=OpenPangu-1B2026.041.79