Multi-step Reasoning on StrategyQA
66.99AccuracyQwen3-8B + SFT + WeMask(TF)
Evaluation Results
| Method | Links | |
|---|---|---|
| Qwen3-8B + SFT + WeMask(TF)shot=0-shot, mask rate=0.1, base model=Qwen3-8B2026.05 | 66.99 | |
| Qwen3-8B + SFTshot=0-shot, mask rate=0.1, base model=Qwen3-8B2026.05 | 66.77 | |
| Qwen3-8B + WeMask(SFT)shot=0-shot, mask rate=0.1, base model=Qwen3-8B2026.05 | 66.69 | |
| Qwen3-4B + WeMask(SFT)shot=0-shot, mask rate=0.1, base model=Qwen3-4B2026.05 | 64.54 | |
| Qwen3-4B + SFT + WeMask(TF)shot=0-shot, mask rate=0.1, base model=Qwen3-4B2026.05 | 64.24 | |
| Qwen3-4B + SFTshot=0-shot, mask rate=0.1, base model=Qwen3-4B2026.05 | 64.15 | |
| Qwen3-4B + Gated Attentionshot=0-shot, mask rate=0.1, base model=Qwen3-4B2026.05 | 62.62 | |
| Qwen3-8B + Gated Attentionshot=0-shot, mask rate=0.1, base model=Qwen3-8B2026.05 | 62.53 | |
| Cumulative VotingBackbone=Llama 8B2026.07 | 61.2 | |
| Majority ConsensusBackbone=Llama 8B2026.07 | 59.9 | |
| Unanimity Con.Backbone=Llama 8B2026.07 | 58.8 | |
| Approval VotingBackbone=Llama 8B2026.07 | 58.7 | |
| Simple VotingBackbone=Llama 8B2026.07 | 58.5 | |
| Supermajority Con.Backbone=Llama 8B2026.07 | 56.4 | |
| Ranked VotingBackbone=Llama 8B2026.07 | 56.2 | |
| Baseline with CoTBackbone=Llama 8B2026.07 | 55.5 | |
| JudgeBackbone=Llama 8B2026.07 | 53.7 | |
| Qwen3-8Bshot=0-shot, mask rate=0.1, base model=Qwen3-8B2026.05 | 53.32 | |
| BaselineBackbone=Llama 8B2026.07 | 51.7 | |
| Qwen3-4Bshot=0-shot, mask rate=0.1, base model=Qwen3-4B2026.05 | 46.77 |