Software Engineering on SWE-Bench Lite (pass@1)
33Pass@1LLM-as-a-Verifier
Evaluation Results
| Method | Links | |
|---|---|---|
| LLM-as-a-VerifierVerification Budget=2N budget, Sampling Strategy=Best-of-N2026.07 | 33 | |
| PairwiseVerifier=V1, Verification Budget=3N budget, Sampling Strategy=Best-of-N2026.07 | 31 | |
| PointwiseMechanism=LM judge, Verification Budget=N budget, Sampling Strategy=Best-of-N2026.07 | 29.7 | |
| Base modelSampling Strategy=Best-of-N2026.07 | 23.5 |