Software Engineering on SWE-bench Verified (all 500 problems)
55.4Avg@16Random rollout selection
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| Random rollout selectionModel=gpt-oss-120B, Anchor=<|start|>, N (rollouts)=162026.06 | 55.4 | — | — | |
| Random rollout selectionModel=Qwen3.6-35B-A3B, Anchor=</think>, N (rollouts)=162026.06 | 49.2 | — | — | |
| Random rollout selectionModel=gpt-oss-20B, Anchor=<|start|>, N (rollouts)=162026.06 | 41.3 | — | — | |
| Routing Agreement Decoding (RAD)Model=Qwen3.6-35B-A3B, Anchor=</think>, N (rollouts)=162026.06 | — | 51.4 | 2.2 | |
| Routing Agreement Decoding (RAD)Model=gpt-oss-120B, Anchor=<|start|>, N (rollouts)=162026.06 | — | 57.8 | 2.4 | |
| Routing Agreement Decoding (RAD)Model=gpt-oss-20B, Anchor=<|start|>, N (rollouts)=162026.06 | — | 44.2 | 2.9 |