List your company
ResearchFormerly WizWand

Multitask Language Understanding on MMLU CoT

72.8MMLU (CoT)FP16
54.350459.140263.9368.7198Sep 27, 2025
Updated 8mo ago

Evaluation results

MethodLinks
FP16Format=FP162025.0972.8
RTNFormat=INT82025.0972.5
GPTQFormat=INT82025.0972.4
RTNFormat=FP82025.0972.4
GPTQFormat=FP82025.0971.8
MR-GPTQFormat=NVINT42025.0969.71
GPTQ+HadFormat=NVFP2025.0969.4
GPTQ+Had128Format=NVFP2025.0968.9
QAT+HadFormat=NVFP2025.0968.9
RTNFormat=NVFP2025.0968.7
GPTQFormat=NVINT42025.0968.69
GPTQFormat=NVFP2025.0968.6
RTN + HTFormat=NVINT42025.0968.59
RTNFormat=NVINT42025.0968.56
QATFormat=NVFP2025.0968.2
QAT+HadFormat=MXFP2025.0967.6
GPTQ+HadFormat=MXFP2025.0967.2
RTN+HadFormat=NVFP2025.0967
GPTQ+Had128Format=MXFP2025.0966.8
RTN+Had128Format=NVFP2025.0966.6
MR-GPTQFormat=MXINT42025.0965.48
QATFormat=MXFP2025.0965
RTN+Had128Format=MXFP2025.0964.5
GPTQFormat=MXFP2025.0963.74
RTN+HadFormat=MXFP2025.0962.6
RTNFormat=MXFP2025.0962.2
GPTQFormat=MXINT42025.0961.22
RTN + HTFormat=MXINT42025.0958.44
RTNFormat=MXINT42025.0955.06