List your company
ResearchFormerly WizWand

Mathematical Reasoning on MATH 500 (Acc., TPS)

74.8AccuracySpec (K=3)
-2.78417.35837.557.642May 12, 2026May 20, 2026May 28, 2026Jun 6, 2026Jun 14, 2026Jun 22, 2026Jul 1, 2026
Updated 4mo ago

Evaluation results

MethodLinks
Spec (K=3)Model scale=8B, K=32026.0574.81.35
Base (r=1)Model scale=8B, r=12026.0572.2114
Base (r=1)Model scale=4B, r=12026.0571.4165.4
Direct (K=1)Model scale=8B, K=12026.0571.41.6
Direct (K=2)Model scale=8B, K=22026.0570.81.84
Base (r=2)Model scale=8B, r=22026.0568.41.04
Spec (K=3)Model scale=4B, K=32026.05681.3
Direct (K=1)Model scale=4B, K=12026.0564.41.6
Base (r=2)Model scale=4B, r=22026.05641.02
Direct (K=2)Model scale=4B, K=22026.0560.41.79
Spec (K=3)Model scale=1.7B, K=32026.0558.41.23
Base (r=1)Model scale=1.7B, r=12026.0555277.6
Direct (K=1)Model scale=1.7B, K=12026.0554.21.5
Direct (K=2)Model scale=1.7B, K=22026.0550.61.67
Base (r=2)Model scale=1.7B, r=22026.0545.80.92
FP16Backbone=LLaMA-3-8B-Instruct, BPA=162026.0720.4—
GSRQBackbone=LLaMA-3-8B-Instruct, BPA=22026.0719—
KIVIBackbone=LLaMA-3-8B-Instruct, BPA=22026.0715.8—
GSRQBackbone=LLaMA-3-8B-Instruct, BPA=12026.0714.2—
VQLLMBackbone=LLaMA-3-8B-Instruct, BPA=22026.0713.4—
KIVIBackbone=LLaMA-3-8B-Instruct, BPA=12026.070.8—
VQLLMBackbone=LLaMA-3-8B-Instruct, BPA=12026.070.2—