List your company
ResearchFormerly WizWand

Zero-shot Language Modeling and Reasoning on MMLU-CoT, GSM8k, HellaSwag, and WinoGrande

72.76MMLU-CoT AccuracyFP16
48.94455.12761.3167.493Sep 27, 2025
Updated 8mo ago

Evaluation results

MethodLinks
FP16Format=Baseline2025.0972.7685.0680.0177.978.93100
RTNFormat=INT82025.0972.584.880.277.478.7399.74
GPTQFormat=INT82025.0972.484.48077.378.5399.48
RTNFormat=FP82025.0972.484.779.877.778.6599.64
GPTQFormat=FP82025.0971.884.579.978.178.5899.55
MR-GPTQFormat=NVINT42025.0969.7182.2679.1475.5376.6697.12
MR-GPTQFormat=NVFP42025.0969.1280.878.1775.2475.8496.08
SmoothQuantFormat=NVFP42025.0968.979.579.574.775.795.9
GPTQFormat=NVFP42025.0968.8582.678.2674.5175.7295.92
GPTQFormat=NVINT42025.0968.6981.5877.5973.475.3295.42
RTN + HTFormat=NVINT42025.0968.5981.7378.3874.2775.7495.96
RTNFormat=NVINT42025.0968.5678.1778.6475.1475.1395.18
RTN+HTFormat=INT42025.0968.379.6177.673.4874.7594.71
RTNFormat=NVFP42025.0968.2678.3978.1574.1174.7394.67
RTN + HTFormat=NVFP42025.0967.4178.0177.3173.4874.0593.82
MR-GPTQFormat=MXFP42025.0967.1975.776.9174.873.6593.31
QuaRotFormat=NVFP42025.0966.577.477.2575.1474.193.8
SpinQuantFormat=NVFP42025.0966.576.176.9675.3273.793.4
GPTQFormat=INT42025.0966.3676.6577.3872.4873.2192.75
RTNFormat=INT42025.0965.9674.6877.6274.1973.1192.63
MR-GPTQFormat=MXINT42025.0965.4874.8376.6373.0972.5191.86
SmoothQuantFormat=MXFP42025.0963.9368.5475.173.5670.389.06
GPTQFormat=MXFP42025.0963.4968.4676.0174.5170.6289.47
RTN + HTFormat=MXFP42025.0962.3872.4875.2971.6770.4589.26
RTNFormat=MXFP42025.0962.2167.8573.9973.2469.3287.83
SpinQuantFormat=MXFP42025.0961.868.1674.8772.9369.488
GPTQFormat=MXINT42025.0961.2267.775.0471.6768.9187.3
RTN + HTFormat=MXINT42025.0958.4461.6473.9471.1966.384
RTNFormat=MXINT42025.0955.0656.7972.0668.2763.0579.87
QuaRotFormat=MXFP42025.0949.8656.9473.571.4362.979.7