List your company
ResearchFormerly WizWand

Code Generation on xCodeEval (sample 500)

0.831Accuracy (Easy)FUNCODER
-0.009320.208840.4270.64516May 30, 2024
Updated 8mo ago

Evaluation results

MethodLinks
FUNCODERModel=GPT-42024.050.8310.580.2640.0340.486
CodeTModel=GPT-42024.050.7640.5180.2180.0340.44
ReflexionModel=GPT-42024.050.7130.4110.1950.0250.386
StandardModel=GPT-42024.050.6850.3930.1950.0170.374
FUNCODERModel=GPT-3.52024.050.6240.2950.11600.314
CodeTModel=GPT-3.52024.050.5060.1610.0800.232
StandardModel=GPT-3.52024.050.4440.1520.04600.202
ReflexionModel=GPT-3.52024.050.4440.170.05700.206
FUNCODERModel=Llama3-8b2024.050.220.009000.08
FUNCODERModel=StableCode-3b2024.050.1350.0450.01100.062
CodeTModel=Llama3-8b2024.050.1240000.044
CodeTModel=StableCode-3b2024.050.1120.018000.046
FUNCODERModel=CodeLlama-34b2024.050.1020000.036
CodeTModel=CodeLlama-34b2024.050.1010000.036
StandardModel=Llama3-8b2024.050.090.018000.036
StandardModel=StableCode-3b2024.050.0730.009000.028
StandardModel=CodeLlama-34b2024.050.0230000.008