Code Simulation on CruxEval-I
61.4Pass@1TAC
Evaluation results
| Method | Links | |
|---|---|---|
| TACBackbone=Qwen3-4B-Base, Seeds=12026.06 | 61.4 | |
| RandomBackbone=Qwen3-4B-Base, Seeds=12026.06 | 59.6 | |
| SECBackbone=Qwen3-4B-Base, Seeds=12026.06 | 59.5 | |
| SECBackbone=Qwen3-1.7B2026.06 | 42.9 | |
| M2OBackbone=Qwen3-1.7B2026.06 | 42.3 | |
| TACBackbone=Qwen3-1.7B2026.06 | 42.2 | |
| TACBackbone=Llama3.2-3B2026.06 | 40.6 | |
| RandomBackbone=Llama3.2-3B2026.06 | 39.7 | |
| RandomBackbone=Qwen3-1.7B2026.06 | 37.8 | |
| M2OBackbone=Llama3.2-3B2026.06 | 36.1 | |
| SECBackbone=Llama3.2-3B2026.06 | 35.8 | |
| SECBackbone=Qwen3-0.6B-Base, Seeds=12026.06 | 33.4 | |
| BaseBackbone=Llama3.2-3B2026.06 | 31.4 | |
| TACBackbone=Qwen3-0.6B-Base, Seeds=12026.06 | 29.6 | |
| RandomBackbone=Qwen3-0.6B-Base, Seeds=12026.06 | 26.9 | |
| BaseBackbone=Qwen3-1.7B2026.06 | 15.1 |