List your company
ResearchFormerly WizWand

Mathematical Reasoning on GSM8K (8-shot)

95.1AccuracyAuto-Judge
20.2239.6659.178.54Feb 11, 2026Feb 25, 2026Mar 11, 2026Mar 26, 2026Apr 9, 2026Apr 23, 2026May 8, 2026
Updated 5mo ago

Evaluation results

MethodLinks
Auto-JudgeThreshold=0.014, L=32, Model=Llama-3.1-8B/70B-Instruct2026.0295.157.4——
Auto-JudgeThreshold=0.022, L=32, Model=Llama-3.1-8B/70B-Instruct2026.0294.362.2——
Auto-Judge + DMThreshold=0.014, L=32, Model=Llama-3.1-8B/70B-Instruct, Dropout probability p_drop=0.3, K=52026.0294.173.91.29—
Auto-Judge + DMThreshold=0.022, L=32, Model=Llama-3.1-8B/70B-Instruct, Dropout probability p_drop=0.3, K=52026.0293.977.61.21—
Auto-Judge + DMThreshold=0.112, L=32, Model=Llama-3.1-8B/70B-Instruct, Dropout probability p_drop=0.3, K=52026.029390.41.07—
BF16Model=Qwen3-30B-A3B, Quantization=BF162026.0492.4———
Auto-JudgeThreshold=0.112, L=32, Model=Llama-3.1-8B/70B-Instruct2026.0291.484.8——
Auto-Judge + DMThreshold=0.281, L=32, Model=Llama-3.1-8B/70B-Instruct, Dropout probability p_drop=0.3, K=52026.0290.795.91.06—
Auto-JudgeThreshold=0.281, L=32, Model=Llama-3.1-8B/70B-Instruct2026.0288.290.1——
CodeQuantModel=Qwen3-30B-A3B, Quantization Setting=A4W4 Embedding-wise2026.0486.7———
Baseline+ParallelModel=LLaDA-8B-Instruct2026.0381.4312.44——
d2CacheModel=LLaDA-8B-Instruct2026.0381.3547.03——
Elastic-CacheModel=LLaDA-8B-Instruct2026.0381.232.89——
EntropyCacheModel=LLaDA-8B-Instruct2026.0381.262.31——
BaselineModel=LLaDA-8B-Instruct2026.0381.122.07——
Fast-dLLM (Dual)Model=LLaDA-8B-Instruct2026.0380.4441.68——
BaselineModel=Dream-v0-Instruct-7B2026.0378.012.51——
Baseline+ParallelModel=Dream-v0-Instruct-7B2026.0377.4113.95——
EntropyCacheModel=Dream-v0-Instruct-7B2026.0375.7460.01——
Fast-dLLM (Dual)Model=Dream-v0-Instruct-7B2026.0374.2255.5——
d2CacheModel=Dream-v0-Instruct-7B2026.0373.6949.59——
Elastic-CacheModel=Dream-v0-Instruct-7B2026.0369.3736.95——
QuaRotModel=Qwen3-30B-A3B, Quantization Setting=A4W4 Embedding-wise2026.0450.8———
MATRYOSHKALORAAdapter Type=MATRYOSHKALORA, Rank (r)=16, Bottleneck size (R)=322026.0539.1——38.4
MATRYOSHKALORAAdapter Type=MATRYOSHKALORA, Rank (r)=8, Bottleneck size (R)=322026.0538.8——38.4
MATRYOSHKALORAAdapter Type=MATRYOSHKALORA, Rank (r)=32, Bottleneck size (R)=322026.0538.3——38.4
MATRYOSHKALORAAdapter Type=MATRYOSHKALORA, Rank (r)=8, Bottleneck size (R)=162026.0537.7——37.2
MATRYOSHKALORAAdapter Type=MATRYOSHKALORA, Rank (r)=4, Bottleneck size (R)=162026.0537.4——37.2
MATRYOSHKALORAAdapter Type=MATRYOSHKALORA, Rank (r)=8, Bottleneck size (R)=82026.0537.3——36.5
MATRYOSHKALORAAdapter Type=MATRYOSHKALORA, Rank (r)=4, Bottleneck size (R)=322026.0537.2——38.4
MATRYOSHKALORAAdapter Type=MATRYOSHKALORA, Rank (r)=16, Bottleneck size (R)=162026.0537.1——37.2
MATRYOSHKALORAAdapter Type=MATRYOSHKALORA, Rank (r)=2, Bottleneck size (R)=42026.0536.8——36.3
MATRYOSHKALORAAdapter Type=MATRYOSHKALORA, Rank (r)=1, Bottleneck size (R)=42026.0536.5——36.3
BF16Model=DeepSeek-V2-Lite, Quantization=BF162026.0436.4———
MATRYOSHKALORAAdapter Type=MATRYOSHKALORA, Rank (r)=4, Bottleneck size (R)=82026.0536.4——36.5
MATRYOSHKALORAAdapter Type=MATRYOSHKALORA, Rank (r)=2, Bottleneck size (R)=82026.0536——36.5
MATRYOSHKALORAAdapter Type=MATRYOSHKALORA, Rank (r)=1, Bottleneck size (R)=22026.0535.9——35.3
MATRYOSHKALORAAdapter Type=MATRYOSHKALORA, Rank (r)=2, Bottleneck size (R)=162026.0535.9——37.2
MATRYOSHKALORAAdapter Type=MATRYOSHKALORA, Rank (r)=1, Bottleneck size (R)=322026.0535.8——38.4
MATRYOSHKALORAAdapter Type=MATRYOSHKALORA, Rank (r)=2, Bottleneck size (R)=322026.0535.6——38.4
DyLoRAAdapter Type=DyLoRA, Rank (r)=32, Bottleneck size (R)=322026.0535.4——34.9
MATRYOSHKALORAAdapter Type=MATRYOSHKALORA, Rank (r)=4, Bottleneck size (R)=42026.0535.4——36.3
MATRYOSHKALORAAdapter Type=MATRYOSHKALORA, Rank (r)=1, Bottleneck size (R)=162026.0535.3——37.2
MATRYOSHKALORAAdapter Type=MATRYOSHKALORA, Rank (r)=1, Bottleneck size (R)=82026.0535.2——36.5
LoRAAdapter Type=LoRA, Rank (r)=8, Bottleneck size (R)=82026.0534.9——34.5
LoRAAdapter Type=LoRA, Rank (r)=16, Bottleneck size (R)=162026.0534.9——34.6
DyLoRAAdapter Type=DyLoRA, Rank (r)=16, Bottleneck size (R)=322026.0534.9——34.9
LoRAAdapter Type=LoRA, Rank (r)=2, Bottleneck size (R)=42026.0534.8——34.4
LoRAAdapter Type=LoRA, Rank (r)=8, Bottleneck size (R)=162026.0534.8——34.6
DyLoRAAdapter Type=DyLoRA, Rank (r)=8, Bottleneck size (R)=162026.0534.8——34.6
DyLoRAAdapter Type=DyLoRA, Rank (r)=16, Bottleneck size (R)=162026.0534.8——34.6
DyLoRAAdapter Type=DyLoRA, Rank (r)=8, Bottleneck size (R)=82026.0534.7——34.4
DyLoRAAdapter Type=DyLoRA, Rank (r)=8, Bottleneck size (R)=322026.0534.7——34.9
MATRYOSHKALORAAdapter Type=MATRYOSHKALORA, Rank (r)=2, Bottleneck size (R)=22026.0534.7——35.3
LoRAAdapter Type=LoRA, Rank (r)=4, Bottleneck size (R)=82026.0534.6——34.5
LoRAAdapter Type=LoRA, Rank (r)=32, Bottleneck size (R)=322026.0534.6——34.5
DyLoRAAdapter Type=DyLoRA, Rank (r)=4, Bottleneck size (R)=82026.0534.6——34.4
DyLoRAAdapter Type=DyLoRA, Rank (r)=4, Bottleneck size (R)=162026.0534.6——34.6
LoRAAdapter Type=LoRA, Rank (r)=4, Bottleneck size (R)=162026.0534.5——34.6
LoRAAdapter Type=LoRA, Rank (r)=16, Bottleneck size (R)=322026.0534.5——34.5
DyLoRAAdapter Type=DyLoRA, Rank (r)=2, Bottleneck size (R)=322026.0534.5——34.9
DyLoRAAdapter Type=DyLoRA, Rank (r)=1, Bottleneck size (R)=322026.0534.4——34.9
LoRAAdapter Type=LoRA, Rank (r)=4, Bottleneck size (R)=42026.0534.3——34.4
LoRAAdapter Type=LoRA, Rank (r)=2, Bottleneck size (R)=322026.0534.3——34.5
LoRAAdapter Type=LoRA, Rank (r)=4, Bottleneck size (R)=322026.0534.3——34.5
LoRAAdapter Type=LoRA, Rank (r)=8, Bottleneck size (R)=322026.0534.3——34.5
DyLoRAAdapter Type=DyLoRA, Rank (r)=4, Bottleneck size (R)=42026.0534.3——34
LoRAAdapter Type=LoRA, Rank (r)=1, Bottleneck size (R)=82026.0534.2——34.5
LoRAAdapter Type=LoRA, Rank (r)=2, Bottleneck size (R)=82026.0534.1——34.5
DyLoRAAdapter Type=DyLoRA, Rank (r)=4, Bottleneck size (R)=322026.0534.1——34.9
DyLoRAAdapter Type=DyLoRA, Rank (r)=2, Bottleneck size (R)=42026.0534——34
DyLoRAAdapter Type=DyLoRA, Rank (r)=2, Bottleneck size (R)=162026.0533.9——34.6
DyLoRAAdapter Type=DyLoRA, Rank (r)=2, Bottleneck size (R)=82026.0533.8——34.4
LoRAAdapter Type=LoRA, Rank (r)=2, Bottleneck size (R)=22026.0533.6——33
LoRAAdapter Type=LoRA, Rank (r)=2, Bottleneck size (R)=162026.0533.6——34.6
LoRAAdapter Type=LoRA, Rank (r)=1, Bottleneck size (R)=42026.0533.5——34.4
DyLoRAAdapter Type=DyLoRA, Rank (r)=2, Bottleneck size (R)=22026.0533.5——33.2
DyLoRAAdapter Type=DyLoRA, Rank (r)=1, Bottleneck size (R)=42026.0533.4——34
LoRAAdapter Type=LoRA, Rank (r)=1, Bottleneck size (R)=322026.0533.3——34.5
DyLoRAAdapter Type=DyLoRA, Rank (r)=1, Bottleneck size (R)=82026.0533.1——34.4
CodeQuantModel=DeepSeek-V2-Lite, Quantization Setting=A4W4 Embedding-wise2026.0433———
DyLoRAAdapter Type=DyLoRA, Rank (r)=1, Bottleneck size (R)=22026.0532.9——33.2
DyLoRAAdapter Type=DyLoRA, Rank (r)=1, Bottleneck size (R)=162026.0532.9——34.6
LoRAAdapter Type=LoRA, Rank (r)=1, Bottleneck size (R)=162026.0532.5——34.6
LoRAAdapter Type=LoRA, Rank (r)=1, Bottleneck size (R)=12026.0532.4——32.4
LoRAAdapter Type=LoRA, Rank (r)=1, Bottleneck size (R)=22026.0532.4——33
DyLoRAAdapter Type=DyLoRA, Rank (r)=1, Bottleneck size (R)=12026.0532.4——32.4
MATRYOSHKALORAAdapter Type=MATRYOSHKALORA, Rank (r)=1, Bottleneck size (R)=12026.0532.3——32.3
QuaRotModel=DeepSeek-V2-Lite, Quantization Setting=A4W4 Embedding-wise2026.0423.1———