Code Generation on mHumanEval
0.94Pass@1BanglaCodeAct
Evaluation Results
| Method | Links | |
|---|---|---|
| BanglaCodeActLLM Model=Qwen3-8B2025.11 | 0.94 | |
| Self-ConsistencyLLM Model=Qwen3-8B2025.11 | 0.88 | |
| Few-ShotLLM Model=Llama-3.1-8B2025.11 | 0.77 | |
| BanglaCodeActLLM Model=DeepSeek-Coder-V2-Lite2025.11 | 0.738 | |
| Few-ShotLLM Model=DeepSeek-Coder-V2-Lite2025.11 | 0.73 | |
| Zero-ShotLLM Model=DeepSeek-Coder-V2-Lite2025.11 | 0.714 | |
| Majority VotingLLM Model=Qwen3-8B2025.11 | 0.66 | |
| Few-ShotLLM Model=Qwen2.5-Coder-7B2025.11 | 0.51 | |
| Few-ShotLLM Model=Qwen3-8B2025.11 | 0.46 | |
| Zero-ShotLLM Model=Qwen2.5-Coder-7B2025.11 | 0.44 | |
| Zero-ShotLLM Model=Llama-3.1-8B2025.11 | 0.39 | |
| Zero-ShotLLM Model=Qwen3-8B2025.11 | 0.36 | |
| Zero-ShotLLM Model=TigerLLM-1B-it2025.11 | 0.11 |