Code Simulation on CruxEval-O
62.7Pass@1SEC
Evaluation results
| Method | Links | |
|---|---|---|
| SECBackbone=Qwen3-4B-Base, Seeds=12026.06 | 62.7 | |
| TACBackbone=Qwen3-4B-Base, Seeds=12026.06 | 62.1 | |
| RandomBackbone=Qwen3-4B-Base, Seeds=12026.06 | 47.6 | |
| M2OBackbone=Qwen3-1.7B2026.06 | 41.5 | |
| TACBackbone=Qwen3-1.7B2026.06 | 39.5 | |
| M2OBackbone=Llama3.2-3B2026.06 | 37.4 | |
| RandomBackbone=Llama3.2-3B2026.06 | 36.2 | |
| SECBackbone=Qwen3-1.7B2026.06 | 35.8 | |
| TACBackbone=Qwen3-0.6B-Base, Seeds=12026.06 | 35.6 | |
| TACBackbone=Llama3.2-3B2026.06 | 35.5 | |
| SECBackbone=Qwen3-0.6B-Base, Seeds=12026.06 | 35.2 | |
| RandomBackbone=Qwen3-1.7B2026.06 | 34.8 | |
| SECBackbone=Llama3.2-3B2026.06 | 34 | |
| RandomBackbone=Qwen3-0.6B-Base, Seeds=12026.06 | 28.6 | |
| BaseBackbone=Llama3.2-3B2026.06 | 27.6 | |
| BaseBackbone=Qwen3-1.7B2026.06 | 15 |