Code Simulation on CodeI/O
6.7Pass@1 AccuracyRandom
Evaluation results
| Method | Links | |
|---|---|---|
| RandomBackbone=Llama3.2-3B2026.06 | 6.7 | |
| RandomBackbone=Qwen3-1.7B2026.06 | 5.2 | |
| TACBackbone=Qwen3-1.7B2026.06 | 4.8 | |
| SECBackbone=Qwen3-1.7B2026.06 | 4.4 | |
| M2OBackbone=Qwen3-1.7B2026.06 | 4.2 | |
| TACBackbone=Llama3.2-3B2026.06 | 3.3 | |
| BaseBackbone=Qwen3-1.7B2026.06 | 3.2 | |
| SECBackbone=Llama3.2-3B2026.06 | 2.7 | |
| BaseBackbone=Llama3.2-3B2026.06 | 2.3 | |
| M2OBackbone=Llama3.2-3B2026.06 | 2 |