List your company
ResearchFormerly WizWand

Multi-step Reasoning on StrategyQA

66.99AccuracyQwen3-8B + SFT + WeMask(TF)
45.961251.420656.8862.3394May 8, 2026May 17, 2026May 27, 2026Jun 6, 2026Jun 16, 2026Jun 26, 2026Jul 6, 2026
Updated 3mo ago

Evaluation results

MethodLinks
Qwen3-8B + SFT + WeMask(TF)shot=0-shot, mask rate=0.1, base model=Qwen3-8B2026.0566.99
Qwen3-8B + SFTshot=0-shot, mask rate=0.1, base model=Qwen3-8B2026.0566.77
Qwen3-8B + WeMask(SFT)shot=0-shot, mask rate=0.1, base model=Qwen3-8B2026.0566.69
Qwen3-4B + WeMask(SFT)shot=0-shot, mask rate=0.1, base model=Qwen3-4B2026.0564.54
Qwen3-4B + SFT + WeMask(TF)shot=0-shot, mask rate=0.1, base model=Qwen3-4B2026.0564.24
Qwen3-4B + SFTshot=0-shot, mask rate=0.1, base model=Qwen3-4B2026.0564.15
Qwen3-4B + Gated Attentionshot=0-shot, mask rate=0.1, base model=Qwen3-4B2026.0562.62
Qwen3-8B + Gated Attentionshot=0-shot, mask rate=0.1, base model=Qwen3-8B2026.0562.53
Cumulative VotingBackbone=Llama 8B2026.0761.2
Majority ConsensusBackbone=Llama 8B2026.0759.9
Unanimity Con.Backbone=Llama 8B2026.0758.8
Approval VotingBackbone=Llama 8B2026.0758.7
Simple VotingBackbone=Llama 8B2026.0758.5
Supermajority Con.Backbone=Llama 8B2026.0756.4
Ranked VotingBackbone=Llama 8B2026.0756.2
Baseline with CoTBackbone=Llama 8B2026.0755.5
JudgeBackbone=Llama 8B2026.0753.7
Qwen3-8Bshot=0-shot, mask rate=0.1, base model=Qwen3-8B2026.0553.32
BaselineBackbone=Llama 8B2026.0751.7
Qwen3-4Bshot=0-shot, mask rate=0.1, base model=Qwen3-4B2026.0546.77