List your company
ResearchFormerly WizWand

Code Completion on HumanEval+

56.7Pass@1NonDPFT
15.51626.20836.947.592Feb 29, 2024Jun 15, 2024Oct 1, 2024Jan 16, 2025May 4, 2025Aug 19, 2025Dec 5, 2025
Updated 8mo ago

Evaluation results

MethodLinks
NonDPFTBackbone=Qwen2.5-Coder-7B, Privacy Epsilon=None2025.1256.7
NonDPFTBackbone=CodeQwen1.5-7B, Privacy Epsilon=None2025.1253.7
DeepSeekCoder-33Bdecoding=greedy2024.0246.3
NonDPFTBackbone=DS-Coder-6.7B, Privacy Epsilon=None2025.1245.7
CodeLlama-34Bdecoding=greedy2024.0244.3
PrivCodeBackbone=DS-Coder-6.7B, Privacy Epsilon=42025.1243.3
JFTBackbone=DS-Coder-6.7B, Privacy Epsilon=42025.1242.7
PrivCodeBackbone=CodeQwen1.5-7B, Privacy Epsilon=42025.1241.5
DP-AdapterBackbone=DS-Coder-6.7B, Privacy Epsilon=42025.1241.3
JFTBackbone=CodeQwen1.5-7B, Privacy Epsilon=42025.1241.1
DP-AdapterBackbone=CodeQwen1.5-7B, Privacy Epsilon=42025.1239.8
DeepSeekCoder-6.7Bdecoding=greedy2024.0239.6
DPFTBackbone=DS-Coder-6.7B, Privacy Epsilon=42025.1239
DPFTBackbone=CodeQwen1.5-7B, Privacy Epsilon=42025.1239
PrivCodeBackbone=Qwen2.5-Coder-7B, Privacy Epsilon=42025.1238.7
StarCoder2-15Bdecoding=greedy2024.0237.8
NonDPFTBackbone=CodeGemma-7B, Privacy Epsilon=None2025.1237.2
JFTBackbone=Qwen2.5-Coder-7B, Privacy Epsilon=42025.1233
CodeLlama-13Bdecoding=greedy2024.0232.3
PrivCodeBackbone=CodeGemma-7B, Privacy Epsilon=42025.1231.7
StarCoder2-7Bdecoding=greedy2024.0229.9
DPFTBackbone=Qwen2.5-Coder-7B, Privacy Epsilon=42025.1229.9
DP-AdapterBackbone=Qwen2.5-Coder-7B, Privacy Epsilon=42025.1228.2
StarCoder2-3Bdecoding=greedy2024.0227.4
JFTBackbone=CodeGemma-7B, Privacy Epsilon=42025.1226.9
CodeLlama-7Bdecoding=greedy2024.0225.6
StarCoderBase-15Bdecoding=greedy2024.0225.6
DPFTBackbone=CodeGemma-7B, Privacy Epsilon=42025.1225.6
DP-AdapterBackbone=CodeGemma-7B, Privacy Epsilon=42025.1225.6
StarCoderBase-7Bdecoding=greedy2024.0225
StableCode-3Bdecoding=greedy2024.0224.4
DeepSeekCoder-1.3Bdecoding=greedy2024.0223.8
StarCoderBase-3Bdecoding=greedy2024.0217.1