Code Translation on HumanEval-X (test)
87.19CA@1 (C++ to Java)CTO (Qwen2.5-Coder-7B)
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| CTO (Qwen2.5-Coder-7B)Backbone=Qwen2.5-Coder-7B, Training Strategy=CTO2026.05 | 87.19 | 82.31 | 73.78 | 87.19 | 64.02 | 71.95 | |
| Qwen2.5-Coder-7B-SFTBackbone=Qwen2.5-Coder-7B, Training Strategy=Supervised Fine-Tuning2026.05 | 84.76 | 77.44 | 72.56 | 86.59 | 62.8 | 70.12 | |
| CTO (CodeLlama-7B)Backbone=CodeLlama-7B, Training Strategy=CTO2026.05 | 84.15 | 74.39 | 65.85 | 83.54 | 50.61 | 59.15 | |
| CodeLlama-7B-SFTBackbone=CodeLlama-7B, Training Strategy=Supervised Fine-Tuning2026.05 | 82.93 | 68.29 | 59.15 | 82.32 | 50 | 56.71 | |
| Qwen2.5-Coder-7BBackbone=Qwen2.5-Coder-7B, Training Strategy=Pre-trained2026.05 | 77.44 | 74.39 | 70.12 | 81.71 | 56.71 | 68.29 | |
| CodeLlama-7BBackbone=CodeLlama-7B, Training Strategy=Pre-trained2026.05 | 75.61 | 62.8 | 50 | 78.66 | 42.68 | 51.83 | |
| CTO (CodeT5)Backbone=CodeT5, Training Strategy=CTO2026.05 | 50 | 45.73 | 49.39 | 69.51 | 29.27 | 29.27 | |
| TransCoderBackbone=TransCoder, Training Strategy=Unsupervised2026.05 | 48.17 | 38.41 | 38.41 | 42.68 | 20.73 | 26.83 | |
| CodeT5-SFTBackbone=CodeT5, Training Strategy=Supervised Fine-Tuning2026.05 | 47.56 | 43.9 | 45.73 | 65.24 | 27.44 | 28.66 | |
| TransCoder-STBackbone=TransCoder, Training Strategy=Unsupervised + Unit Tests2026.05 | 46.34 | 40.85 | 40.85 | 64.63 | 21.95 | 23.17 | |
| PPOCoderBackbone=CodeT5, Training Strategy=RL (PPO)2026.05 | 42.68 | 40.85 | 42.68 | 59.76 | 26.83 | 29.27 |