Code Generation on HumanEval (Pass@k)
19.15Pass@1ANCORA
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| ANCORABackbone=Qwen2.5-Coder-3B-Instruct, Evaluation Setting=Test-Time Training, Seed Configuration=Test Dataset, Prompting=0-shot2026.04 | 19.15 | 21.67 | 23.91 | |
| PSVBackbone=Qwen2.5-Coder-3B-Instruct, Evaluation Setting=Test-Time Training, Seed Configuration=Test Dataset, Prompting=1-shot2026.04 | 19.07 | 23.42 | 25.16 | |
| ANCORABackbone=Qwen2.5-Coder-3B-Instruct, Evaluation Setting=Transfer Learning, Seed Configuration=Dafny2Verus, Prompting=0-shot2026.04 | 17.19 | 20.16 | 21.03 | |
| PSVBackbone=Qwen2.5-Coder-3B-Instruct, Evaluation Setting=Transfer Learning, Seed Configuration=Dafny2Verus, Prompting=1-shot2026.04 | 16.18 | 21.61 | 23.13 | |
| RFTBackbone=Qwen2.5-Coder-3B-Instruct, Evaluation Setting=Transfer Learning, Seed Configuration=Dafny2Verus, Prompting=1-shot2026.04 | 10.99 | 18.07 | 20.02 | |
| AlphaVerusBackbone=Qwen2.5-Coder-3B-Instruct, Evaluation Setting=Transfer Learning, Seed Configuration=Dafny2Verus, Prompting=50-shot2026.04 | 7.24 | 15.38 | 18.02 | |
| AlphaVerusBackbone=Qwen2.5-Coder-3B-Instruct, Evaluation Setting=Test-Time Training, Seed Configuration=Test Dataset, Prompting=50-shot2026.04 | 7.24 | 15.38 | 18.02 | |
| RFTBackbone=Qwen2.5-Coder-3B-Instruct, Evaluation Setting=Test-Time Training, Seed Configuration=Test Dataset, Prompting=1-shot2026.04 | 5.56 | 12.46 | 14.8 |