ResearchBenchmarksCode Generation on xCodeEval RustCode Generation on xCodeEval RustFollow23.6Pass@1CodeSim11.84814.89917.9521.001Feb 8, 2025Evaluation resultsMethodMethodLinksPass@1CodeSimBase LLM=ChatGPTBase LLM=ChatGPT2025.0223.6MapCoderBase LLM=ChatGPTBase LLM=ChatGPT2025.0221.7DirectBase LLM=ChatGPTBase LLM=ChatGPT2025.0212.3