Verus Code Generation on MBPP
44.11Pass@1ANCORA
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| ANCORABackbone=Qwen2.5-Coder-3B-Instruct, Evaluation Setting=Test-Time Training, Seed Configuration=Test Dataset, Prompting=0-shot2026.04 | 44.11 | 46.62 | 47.91 | |
| PSVBackbone=Qwen2.5-Coder-3B-Instruct, Evaluation Setting=Test-Time Training, Seed Configuration=Test Dataset, Prompting=1-shot2026.04 | 36.78 | 51.22 | 53.67 | |
| ANCORABackbone=Qwen2.5-Coder-3B-Instruct, Evaluation Setting=Transfer Learning, Seed Configuration=Dafny2Verus, Prompting=0-shot2026.04 | 36.2 | 38.49 | 39.94 | |
| PSVBackbone=Qwen2.5-Coder-3B-Instruct, Evaluation Setting=Transfer Learning, Seed Configuration=Dafny2Verus, Prompting=1-shot2026.04 | 25.25 | 38.32 | 41.51 | |
| RFTBackbone=Qwen2.5-Coder-3B-Instruct, Evaluation Setting=Transfer Learning, Seed Configuration=Dafny2Verus, Prompting=1-shot2026.04 | 10.99 | 26.03 | 31.19 | |
| AlphaVerusBackbone=Qwen2.5-Coder-3B-Instruct, Evaluation Setting=Transfer Learning, Seed Configuration=Dafny2Verus, Prompting=50-shot2026.04 | 6.48 | 18.36 | 24.57 | |
| AlphaVerusBackbone=Qwen2.5-Coder-3B-Instruct, Evaluation Setting=Test-Time Training, Seed Configuration=Test Dataset, Prompting=50-shot2026.04 | 6.48 | 18.36 | 24.57 | |
| RFTBackbone=Qwen2.5-Coder-3B-Instruct, Evaluation Setting=Test-Time Training, Seed Configuration=Test Dataset, Prompting=1-shot2026.04 | 3.83 | 14.56 | 22.64 |