Test generation on UnLeakedTestBench
36.79Pass@1Code-A1
Evaluation Results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| Code-A1Test LLM=Qwen2.5-Coder-7B-Instruct2026.03 | 36.79 | 37.87 | 37.15 | 35.5 | 36.6 | 53.14 | 19.74 | |
| Self-PlayTest LLM=Qwen2.5-Coder-7B-Instruct2026.03 | 32.98 | 34.74 | 35.13 | 31.42 | 38.54 | 55.57 | 19.52 | |
| SFTTest LLM=Qwen2.5-Coder-7B-Instruct2026.03 | 29.32 | 29.94 | 28.72 | 31.33 | 41.01 | 50.85 | 14.6 | |
| Code-A1Test LLM=Qwen2.5-Coder-3B-Instruct2026.03 | 27.83 | 30.31 | 30.86 | 26.44 | 36.96 | 49.56 | 15.29 | |
| /Test LLM=Qwen2.5-Coder-7B-Instruct2026.03 | 27.71 | 28.64 | 28.73 | 30.36 | 36.07 | 51.25 | 14.72 | |
| Code-A1Test LLM=Qwen2.5-Coder-1.5B-Instruct2026.03 | 27.42 | 25.68 | 27.05 | 17.31 | 23.57 | 26.41 | 7.14 | |
| Self-PlayTest LLM=Qwen2.5-Coder-3B-Instruct2026.03 | 25.84 | 29.65 | 29.64 | 29.66 | 38.88 | 50.92 | 15.09 | |
| SFTTest LLM=Qwen2.5-Coder-3B-Instruct2026.03 | 25.4 | 25.71 | 23.51 | 26.77 | 29.08 | 36.29 | 8.53 | |
| Self-PlayTest LLM=Qwen2.5-Coder-1.5B-Instruct2026.03 | 24.12 | 23.37 | 23.39 | 14.82 | 26.98 | 28.91 | 6.76 | |
| /Test LLM=Qwen2.5-Coder-3B-Instruct2026.03 | 19.77 | 20.5 | 20.93 | 28.35 | 29.99 | 42.55 | 8.91 | |
| /Test LLM=Qwen2.5-Coder-1.5B-Instruct2026.03 | 18.34 | 16.91 | 16.29 | 17.1 | 20.35 | 22.3 | 3.63 | |
| SFTTest LLM=Qwen2.5-Coder-1.5B-Instruct2026.03 | 16.01 | 15.16 | 14.76 | 14.53 | 27.57 | 29.45 | 4.35 |