List your company
ResearchFormerly WizWand

Mathematical Problem Solving on MATH (Accuracy)

95.86AccuracyINT4
19.638439.426759.21579.0033Aug 21, 2025Oct 9, 2025Nov 28, 2025Jan 17, 2026Mar 8, 2026Apr 27, 2026Jun 16, 2026
Updated 4mo ago

Evaluation results

MethodLinks
INT4Model=GLM-4.7 (358B)2026.0495.86
BF16Model=GLM-4.7 (358B)2026.0495.66
BDR-64Model=GLM-4.7 (358B)2026.0495.59
BDR-16Model=GLM-4.7 (358B)2026.0495.46
BDR-128Model=GLM-4.7 (358B)2026.0495.32
Llama-4-MaverickCategory=Candidate Models2026.0395.2
Llama-3.3-70BCategory=Candidate Models2026.0394
FineRouterCategory=Routers2026.0393.8
IPRCategory=Routers2026.0393.6
Qwen3-235B-A22BCategory=Candidate Models2026.0393.3
BDR-64 (K)Model=Qwen3-8B2026.0492.71
BDR-128Model=Qwen3-8B2026.0492.63
BF16Model=Qwen3-8B2026.0492.59
kNNCategory=Routers2026.0392.2
BDR-64Model=Qwen3-8B2026.0492.18
BDR-16Model=Qwen3-8B2026.0492.06
QuratingBase Model=OpenR1-Distill-7B2026.0691.88
LESSBase Model=OpenR1-Distill-7B2026.0691.76
DeepSeek-R1Category=Candidate Models2026.0391.6
IF (Impl. w. GraSS)Base Model=OpenR1-Distill-7B2026.0691.52
BM25Base Model=OpenR1-Distill-7B2026.0691.2
Claude-Sonnet-4.5Category=Candidate Models2026.0391.1
Random BaselineBase Model=OpenR1-Distill-7B2026.0691.05
RDSBase Model=OpenR1-Distill-7B2026.0691
OpenR1-Distill-7BBase Model=OpenR1-Distill-7B2026.0690.98
DSIRBase Model=OpenR1-Distill-7B2026.0690.85
DRIFTBase Model=OpenR1-Distill-7B2026.0690.82
GraphRouterCategory=Routers2026.0389.6
DeepSeek-v3Category=Candidate Models2026.0389
GPT-OSS-120BCategory=Candidate Models2026.0388.4
OriginalVocabulary size=32,0002025.0888.4
VocabTailorVocabulary size=5,135 + [14], % Original Vocabulary=(16.09%)2025.0888.4
Claude-Haiku-4.5Category=Candidate Models2026.0388.1
RouterDCCategory=Routers2026.0388
VPVocabulary size=10,300, % Original Vocabulary=(32.19%)2025.0887.8
RouteLLMCategory=Routers2026.0387.5
Self-DistillationBase Model=OpenR1-Distill-7B2026.0683.78
ToolTreeBackbone Model=GPT-4o2026.0378.19
Qwen3-32BCategory=Candidate Models2026.0375.5
TokenBuncherModel=Qwen2.5-7B-Instruct2025.0872.8
Mistral-LargeCategory=Candidate Models2026.0371
BaseModel=Qwen2.5-7B-Instruct2025.0870.4
Qwen-2.5-7B-InstructDecoding Method=Medusa-style tree-based greedy decoding, Decoding Head=multi-token head2025.1169.9
ToolTreeBackbone Model=GPT-4o-mini2026.0369.42
OctoToolsBackbone Model=GPT-4o2026.0368.57
BM25Base Model=Olmo3-7B-Instruct-SFT2026.0667.86
DRIFTBase Model=Olmo3-7B-Instruct-SFT2026.0667.52
Self-DistillationBase Model=Olmo3-7B-Instruct-SFT2026.0667.48
QuratingBase Model=Olmo3-7B-Instruct-SFT2026.0667.29
RepNoiseModel=Qwen2.5-7B-Instruct2025.0866.8
Olmo3-7B-Instruct-SFTBase Model=Olmo3-7B-Instruct-SFT2026.0666.16
RDSBase Model=Olmo3-7B-Instruct-SFT2026.0666.16
Random BaselineBase Model=Olmo3-7B-Instruct-SFT2026.0666.09
IF (Impl. w. GraSS)Base Model=Olmo3-7B-Instruct-SFT2026.0665.42
BaseModel=Qwen2.5-3B-Instruct2025.0865.4
LESSBase Model=Olmo3-7B-Instruct-SFT2026.0664.22
TokenBuncherModel=Qwen2.5-3B-Instruct2025.0862.8
Few-ShotBackbone Model=GPT-4o2026.0361.45
MLPCategory=Routers2026.0359.5
OctoToolsBackbone Model=GPT-4o-mini2026.0358.43
DSIRBase Model=Olmo3-7B-Instruct-SFT2026.0654.63
HuggingGPTBackbone Model=GPT-4o2026.0353.51
Few-ShotBackbone Model=GPT-4o-mini2026.0353.26
TokenBuncherModel=Ministral-8B-Instruct2025.0853
BaseModel=Ministral-8B-Instruct2025.0852
RepNoiseModel=Qwen2.5-3B-Instruct2025.0851.8
BaselineN=16, setting=English occurrence as language confusion, aggregation=Averaged across four models and four target languages2026.0449.59
OriginalModel Family=LLaMA2026.0548.34
TLPON=16, setting=English occurrence as language confusion, aggregation=Averaged across four models and four target languages2026.0447.73
HuggingGPTBackbone Model=GPT-4o-mini2026.0345.14
CTRAPModel=Qwen2.5-7B-Instruct2025.0845
CTRAPModel=Qwen2.5-3B-Instruct2025.0843.8
ORPON=16, setting=English occurrence as language confusion, aggregation=Averaged across four models and four target languages2026.0443.78
Tulu 3 8BDecoding Method=Medusa-style tree-based greedy decoding, Decoding Head=multi-token head2025.1143.7
Llama-3.1-8B-InstructDecoding Method=Medusa-style tree-based greedy decoding, Decoding Head=multi-token head2025.1142.5
DPON=16, setting=English occurrence as language confusion, aggregation=Averaged across four models and four target languages2026.0442
SOMAModel Family=LLaMA2026.0541.62
SFTN=16, setting=English occurrence as language confusion, aggregation=Averaged across four models and four target languages2026.0441.35
Claude 3.5 Haiku (target)Model=Claude 3.5 Haiku2026.0441.3
Ministral-8B-InstructDecoding Method=Medusa-style tree-based greedy decoding, Decoding Head=multi-token head2025.1140
RepNoiseModel=Ministral-8B-Instruct2025.0839.8
OriginalModel Family=Qwen2026.0536.48
CTRAPModel=Ministral-8B-Instruct2025.0834.4
RouteLLMModel Family=LLaMA2026.0533.88
MAP-Neo-7B-InstructDecoding Method=Medusa-style tree-based greedy decoding, Decoding Head=multi-token head2025.1131.5
Llama 3.1 8B (baseline)Model=Llama 3.1 8B2026.0431.46
History-FTModel Family=LLaMA2026.0531.46
OLMo-2-7B-1124-InstructDecoding Method=Medusa-style tree-based greedy decoding, Decoding Head=multi-token head2025.1131.3
SOMAModel Family=Qwen2026.0531.14
Gemma-2-9B-itDecoding Method=Medusa-style tree-based greedy decoding, Decoding Head=multi-token head2025.1129.8
EvaByte-SFTDecoding Method=Medusa-style tree-based greedy decoding, Decoding Head=multi-token head2025.1129.8
GARFA (Exp A)Optimization Strategy=L*+L*2026.0428.95
Exp C: Lr+L*Optimization Strategy=Lr+L*2026.0428.12
Mistral-SmallCategory=Candidate Models2026.0325.1
OLMo-2-7B-SFTDecoding Method=Medusa-style tree-based greedy decoding, Decoding Head=multi-token head2025.1125.1
RouteLLMModel Family=Qwen2026.0525.08
Exp B: L*+LrOptimization Strategy=L*+Lr2026.0425.04
History-PrefixModel Family=LLaMA2026.0525.03
Exp D: Lr+Lr (control)Optimization Strategy=Lr+Lr2026.0422.87
History-FTModel Family=Qwen2026.0522.57