Closed-book Question Answering on SimpleQA (train)
18.8AccuracySFT
Evaluation Results
| Method | Links | |
|---|---|---|
| SFTModel=Qwen3-4B-It2026.05 | 18.8 | |
| SFTModel=Qwen3-8B2026.05 | 16.8 | |
| MIXSDModel=Qwen3-4B-It, lambda=0.32026.05 | 10.5 | |
| MIXSDModel=Qwen3-4B-It, lambda=0.52026.05 | 10.4 | |
| MIXSDModel=Qwen3-8B, lambda=0.52026.05 | 9.3 | |
| MIXSDModel=Qwen3-8B, lambda=0.32026.05 | 9 | |
| SFTModel=Qwen3-1.7B2026.05 | 8.5 | |
| OPSDModel=Qwen3-4B-It, n=1, T=02026.05 | 8.5 | |
| OPSDModel=Qwen3-8B2026.05 | 8.3 | |
| OPSD-NLLModel=Qwen3-4B-It, n=1, T=02026.05 | 7.8 | |
| OPSDModel=Qwen3-4B-It2026.05 | 7.5 | |
| MIXSDModel=Qwen3-1.7B, lambda=0.32026.05 | 7.1 | |
| OPSDModel=Qwen3-8B, n=1, T=02026.05 | 6.8 | |
| MIXSDModel=Qwen3-1.7B, lambda=0.52026.05 | 6.3 | |
| OPSD-NLLModel=Qwen3-8B, n=1, T=02026.05 | 6 | |
| OPSDModel=Qwen3-1.7B, n=1, T=02026.05 | 3.9 | |
| OPSD-NLLModel=Qwen3-1.7B, n=1, T=02026.05 | 3.4 | |
| BaseModel=Qwen3-4B-It2026.05 | 3.4 | |
| OPSDModel=Qwen3-1.7B2026.05 | 2.9 | |
| BaseModel=Qwen3-8B2026.05 | 2.5 | |
| BaseModel=Qwen3-1.7B2026.05 | 0.5 |