List your company
ResearchFormerly WizWand

Logical Reasoning on Countdown

63.2AccuracyFlowTracer
15.15227.62640.152.574Apr 12, 2026Apr 21, 2026May 1, 2026May 11, 2026May 20, 2026May 30, 2026Jun 9, 2026
Updated 4mo ago

Evaluation results

MethodLinks
FlowTracerBackbone=Qwen3-4B-Base, Context window=1K2026.0663.2
AttentionBackbone=Qwen3-4B-Base, Context window=1K2026.0660.4
High-entropyBackbone=Qwen3-4B-Base, Context window=1K2026.0657.7
RandomBackbone=Qwen3-4B-Base, Context window=1K2026.0655
GRPOBackbone=Qwen3-4B-Base, Context window=1K2026.0652.6
+ EOS Temp.Sampling Method=+ EOS Temp., Timestep Budget (T)=1282026.0452
OracleBackbone=Dream-7B-Base, Shots=4-shot2026.0451.3
+ EOS Temp.Sampling Method=+ EOS Temp., Timestep Budget (T)=642026.0451.2
OracleBackbone=LLaDA-8B-Base, Shots=4-shot2026.0449.6
Auto-ICLBackbone=Dream-7B-Base, Shots=4-shot, Latency (s)=0.702026.0449.5
BothSampling Method=Both, Timestep Budget (T)=1282026.0448.4
VanillaBackbone=Dream-7B-Base, Shots=4-shot, Latency (s)=0.712026.0448.1
+ PlannerSampling Method=+ Planner, Timestep Budget (T)=642026.0446.9
BothSampling Method=Both, Timestep Budget (T)=642026.0446.5
Prob MarginSampling Method=Prob Margin, Timestep Budget (T)=642026.0446.1
+ PlannerSampling Method=+ Planner, Timestep Budget (T)=1282026.0445.7
Prob MarginSampling Method=Prob Margin, Timestep Budget (T)=1282026.0444.5
TemperatureSampling Method=Temperature, Timestep Budget (T)=1282026.0443.8
Auto-ICLBackbone=LLaDA-8B-Base, Shots=4-shot, Latency (s)=0.742026.0442.2
Top1 ProbSampling Method=Top1 Prob, Timestep Budget (T)=1282026.0441.8
Top1 ProbSampling Method=Top1 Prob, Timestep Budget (T)=642026.0441.4
TemperatureSampling Method=Temperature, Timestep Budget (T)=642026.0441.4
Init. PositionSampling Method=Init. Position, Timestep Budget (T)=1282026.0439.5
Init. PositionSampling Method=Init. Position, Timestep Budget (T)=642026.0436.7
PrefixBackbone=Dream-7B-Base, Shots=4-shot, Latency (s)=0.712026.0434.7
PrefixBackbone=LLaDA-8B-Base, Shots=4-shot, Latency (s)=0.752026.0432
VanillaBackbone=LLaDA-8B-Base, Shots=4-shot, Latency (s)=0.742026.0431
AncestralSampling Method=Ancestral, Timestep Budget (T)=642026.0425.4
AncestralSampling Method=Ancestral, Timestep Budget (T)=1282026.0423.8
RandomBackbone=Dream-7B-Base, Shots=4-shot, Latency (s)=0.722026.0421.8
RandomBackbone=LLaDA-8B-Base, Shots=4-shot, Latency (s)=0.742026.0417