List your company
ResearchFormerly WizWand

Mathematical Reasoning on GSM8K @MetaMath (test)

72.3AccuracyBF16
3.930421.680239.4357.1798Feb 16, 2024
Updated 8mo ago

Evaluation results

MethodLinks
BF16Model Size=13B, Bits=16, Backbone=MetaMath2024.0272.3
BitDistillerModel Size=13B, Bits=3, Group Size=128, Backbone=MetaMath2024.0269.69
GPTQModel Size=13B, Bits=3, Group Size=128, Backbone=MetaMath2024.0268.75
AWQModel Size=13B, Bits=3, Group Size=128, Backbone=MetaMath2024.0268.67
RTNModel Size=13B, Bits=3, Group Size=128, Backbone=MetaMath2024.0268.51
OmniQuantModel Size=13B, Bits=3, Group Size=128, Backbone=MetaMath2024.0268.28
LLM-QATModel Size=13B, Bits=3, Group Size=128, Backbone=MetaMath2024.0266.62
BF16Model Size=7B, Bits=16, Backbone=MetaMath2024.0266.41
BitDistillerModel Size=7B, Bits=3, Group Size=128, Backbone=MetaMath2024.0264.38
AWQModel Size=7B, Bits=3, Group Size=128, Backbone=MetaMath2024.0262.34
GPTQModel Size=7B, Bits=3, Group Size=128, Backbone=MetaMath2024.0262.11
OmniQuantModel Size=7B, Bits=3, Group Size=128, Backbone=MetaMath2024.0261.7
BitDistillerModel Size=13B, Bits=2, Group Size=128, Backbone=MetaMath2024.0261.33
LLM-QATModel Size=7B, Bits=3, Group Size=128, Backbone=MetaMath2024.0260.78
RTNModel Size=7B, Bits=3, Group Size=128, Backbone=MetaMath2024.0259.3
BitDistillerModel Size=7B, Bits=2, Group Size=128, Backbone=MetaMath2024.0251.02
LLM-QATModel Size=13B, Bits=2, Group Size=128, Backbone=MetaMath2024.0236.64
BF16Model Size=3B, Bits=16, Backbone=MetaMath2024.0236.4
BitDistillerModel Size=3B, Bits=3, Group Size=128, Backbone=MetaMath2024.0232.5
LLM-QATModel Size=3B, Bits=3, Group Size=128, Backbone=MetaMath2024.0226.25
OmniQuantModel Size=3B, Bits=3, Group Size=128, Backbone=MetaMath2024.0223.67
LLM-QATModel Size=7B, Bits=2, Group Size=128, Backbone=MetaMath2024.0223.13
AWQModel Size=3B, Bits=3, Group Size=128, Backbone=MetaMath2024.0221.87
RTNModel Size=3B, Bits=3, Group Size=128, Backbone=MetaMath2024.0217.5
BitDistillerModel Size=3B, Bits=2, Group Size=128, Backbone=MetaMath2024.0216.09
GPTQModel Size=13B, Bits=2, Group Size=128, Backbone=MetaMath2024.0211.43
OmniQuantModel Size=13B, Bits=2, Group Size=128, Backbone=MetaMath2024.029.45
RTNModel Size=13B, Bits=2, Group Size=128, Backbone=MetaMath2024.027.89
AWQModel Size=13B, Bits=2, Group Size=128, Backbone=MetaMath2024.027.89
GPTQModel Size=3B, Bits=3, Group Size=128, Backbone=MetaMath2024.026.72
LLM-QATModel Size=3B, Bits=2, Group Size=128, Backbone=MetaMath2024.026.56