Search-based Reasoning on Search-based QA suite
48.8NQ AccuracyOPID
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| OPIDBackbone=Qwen2.5-7B-Instruct, Validation with skills=false2026.06 | 48.8 | 65.6 | 46.8 | 46.1 | 42.7 | 21.7 | 72.6 | 49.2 | |
| GRPO+OPSDBackbone=Qwen2.5-7B-Instruct, Validation with skills=false2026.06 | 47.3 | 64.5 | 46.9 | 43.8 | 39.3 | 18 | 69.4 | 47 | |
| Skill-SDBackbone=Qwen2.5-7B-Instruct, Validation with skills=false2026.06 | 47.1 | 64.5 | 47.8 | 44.2 | 42.1 | 20.2 | 69 | 47.8 | |
| RLSDBackbone=Qwen2.5-7B-Instruct, Validation with skills=false2026.06 | 46.8 | 63 | 44.4 | 45.5 | 48.9 | 21.5 | 73 | 49 | |
| SDARBackbone=Qwen2.5-7B-Instruct, Validation with skills=false2026.06 | 46.3 | 63.5 | 48.2 | 43.8 | 48.4 | 19.6 | 73 | 49 | |
| OPIDBackbone=Qwen2.5-3B-Instruct, Validation with skills=false2026.06 | 45.9 | 61.4 | 45.7 | 40.7 | 38.8 | 16.4 | 66.1 | 45 | |
| Skill-GRPOBackbone=Qwen2.5-7B-Instruct, Validation with skills=false2026.06 | 45.2 | 63.7 | 45.7 | 43.1 | 43.3 | 19.6 | 21.4 | 40.3 | |
| GRPOBackbone=Qwen2.5-7B-Instruct, Validation with skills=false2026.06 | 45.1 | 63.7 | 44 | 43.6 | 43.2 | 16.8 | 37.6 | 42 | |
| GRPO+OPSDBackbone=Qwen2.5-3B-Instruct, Validation with skills=false2026.06 | 44.9 | 61.2 | 45.2 | 40.4 | 38.5 | 16 | 66.1 | 44.6 | |
| SDARBackbone=Qwen2.5-3B-Instruct, Validation with skills=false2026.06 | 44.8 | 58.1 | 44.3 | 38.6 | 36.2 | 15.7 | 66.1 | 43.4 | |
| Skill-GRPOBackbone=Qwen2.5-7B-Instruct, Validation with skills=true2026.06 | 44.8 | 63 | 45.1 | 43.7 | 43.7 | 20.5 | 71.4 | 47.5 | |
| Skill-SDBackbone=Qwen2.5-3B-Instruct, Validation with skills=false2026.06 | 44.4 | 60.4 | 44 | 39.5 | 40.4 | 15.4 | 64.9 | 44.1 | |
| Skill-GRPOBackbone=Qwen2.5-3B-Instruct, Validation with skills=true2026.06 | 44.3 | 59.6 | 44.3 | 39 | 36.1 | 14.5 | 14.9 | 36.1 | |
| Skill-GRPOBackbone=Qwen2.5-3B-Instruct, Validation with skills=false2026.06 | 43.5 | 58.8 | 43 | 36.8 | 32.2 | 11.7 | 12.5 | 34.1 | |
| RLSDBackbone=Qwen2.5-3B-Instruct, Validation with skills=false2026.06 | 41.5 | 58.6 | 42.3 | 40.4 | 40.2 | 16.8 | 66.9 | 43.8 | |
| GRPO+OPSDBackbone=Qwen3-1.7B-Instruct, Validation with skills=false2026.06 | 40.7 | 58.9 | 45 | 37 | 34.6 | 13.3 | 65.7 | 42.2 | |
| GRPOBackbone=Qwen3-1.7B-Instruct, Validation with skills=false2026.06 | 40 | 58.9 | 43.5 | 35.4 | 30.3 | 12 | 65.7 | 40.8 | |
| SDARBackbone=Qwen3-1.7B-Instruct, Validation with skills=false2026.06 | 39.7 | 58.9 | 45.3 | 35.9 | 35.5 | 12.6 | 65.3 | 41.9 | |
| GRPOBackbone=Qwen2.5-3B-Instruct, Validation with skills=false2026.06 | 39.3 | 60.6 | 41.1 | 37.4 | 34.6 | 15.4 | 26.4 | 36.4 | |
| Skill-GRPOBackbone=Qwen3-1.7B-Instruct, Validation with skills=false2026.06 | 39.2 | 58.6 | 43.9 | 35.2 | 28.2 | 11.5 | 66.1 | 40.4 | |
| Skill-SDBackbone=Qwen3-1.7B-Instruct, Validation with skills=false2026.06 | 39.1 | 57.5 | 45.4 | 34.8 | 34.1 | 10.7 | 64.1 | 40.8 | |
| RLSDBackbone=Qwen3-1.7B-Instruct, Validation with skills=false2026.06 | 38.6 | 57.3 | 43 | 34.5 | 34.1 | 11.5 | 65.3 | 40.6 | |
| OPIDBackbone=Qwen3-1.7B-Instruct, Validation with skills=false2026.06 | 38.1 | 58.1 | 43.4 | 35.5 | 31.7 | 11.7 | 64.5 | 40.4 | |
| Skill-GRPOBackbone=Qwen3-1.7B-Instruct, Validation with skills=true2026.06 | 38 | 58.4 | 43.9 | 36.3 | 29 | 12.5 | 66.9 | 40.7 | |
| Skill-PromptBackbone=Qwen2.5-7B-Instruct, Validation with skills=true2026.06 | 30.9 | 52.1 | 32.7 | 32.7 | 27.9 | 12.7 | 66.1 | 36.4 | |
| VanillaBackbone=Qwen3-1.7B-Instruct, Validation with skills=false2026.06 | 29.4 | 46.9 | 37 | 23.5 | 19.6 | 6.4 | 10.5 | 24.8 | |
| Skill-PromptBackbone=Qwen3-1.7B-Instruct, Validation with skills=true2026.06 | 29.4 | 46.5 | 36.2 | 22.9 | 20.8 | 4.3 | 10.1 | 24.3 | |
| VanillaBackbone=Qwen2.5-7B-Instruct, Validation with skills=false2026.06 | 25.2 | 50.8 | 29.5 | 29 | 29 | 10.4 | 63.7 | 33.9 | |
| VanillaBackbone=Qwen2.5-3B-Instruct, Validation with skills=false2026.06 | 24.6 | 48.1 | 31 | 26.3 | 25.3 | 7.2 | 59.7 | 31.7 | |
| Skill-PromptBackbone=Qwen2.5-3B-Instruct, Validation with skills=true2026.06 | 23.7 | 46.2 | 30.6 | 24.4 | 22.1 | 7.5 | 12.5 | 23.9 | |
| OPSDBackbone=Qwen2.5-7B-Instruct, Validation with skills=false2026.06 | 8.8 | 8.6 | 17.5 | 2.5 | 4.2 | 0.5 | 1.2 | 6.2 | |
| OPSDBackbone=Qwen3-1.7B-Instruct, Validation with skills=false2026.06 | 4.2 | 8.3 | 4.6 | 6.6 | 15.3 | 0.7 | 1.2 | 5.8 | |
| OPSDBackbone=Qwen2.5-3B-Instruct, Validation with skills=false2026.06 | 0.1 | 0.1 | 0.1 | 0 | 0 | 0 | 0 | 0 |