Twenty Questions on Twenty Questions held-out (test)
35.65Mean@8CIA
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| CIABase Model=Qwen3-4B, Training Strategy=CIA2026.02 | 35.65 | 2.04 | 66.05 | |
| CIABase Model=Qwen3-1.7B, Training Strategy=CIA2026.02 | 24.8 | 1.1 | 53.1 | |
| Qwen3-4B + SFT & GRPO (STARPO)Base Model=Qwen3-4B, Training Strategy=SFT & GRPO (STARPO)2026.02 | 24.36 | 1.18 | 59.12 | |
| Qwen3-1.7B + SFT & GRPO (STARPO)Base Model=Qwen3-1.7B, Training Strategy=SFT & GRPO (STARPO)2026.02 | 16.54 | 1.32 | 45.73 | |
| DeepSeek-V3.2Base Model=DeepSeek-V3.22026.02 | 14.35 | 0.87 | 47.34 | |
| Qwen3-4B + SFTBase Model=Qwen3-4B, Training Strategy=SFT2026.02 | 13.34 | 1.05 | 36.87 | |
| Qwen3-1.7B + SFTBase Model=Qwen3-1.7B, Training Strategy=SFT2026.02 | 9.97 | 1.04 | 32.03 | |
| Qwen3-235B-A22B-Instruct 2507Base Model=Qwen3-235B-A22B-Instruct 25072026.02 | 8.83 | 0.87 | 27.71 | |
| Qwen3-4BBase Model=Qwen3-4B, Training Strategy=Base2026.02 | 5.02 | 0.6 | 15.94 | |
| Qwen3-1.7BBase Model=Qwen3-1.7B, Training Strategy=Base2026.02 | 0 | 0 | 0 |