List your company
ResearchFormerly WizWand

Code Correctness Prediction on LiveCodeBench Python (AUROC)

86.7AUROCFunc. Equiv.
54.14862.59971.0579.501May 27, 2026
Updated 5mo ago

Evaluation results

MethodLinks
Func. Equiv.Model=Gemini-2.5-Flash, Category=Functional Equivalence Methods2026.0586.7
Func. Equiv.Model=Gemini-2.5-Flash-Lite, Category=Functional Equivalence Methods2026.0585.9
Func. Equiv.Model=GPT-4o-mini, Category=Functional Equivalence Methods2026.0585.9
Func. EntropyModel=GPT-4o-mini, Category=Functional Equivalence Methods2026.0585.9
Func. EntropyModel=Gemini-2.5-Flash, Category=Functional Equivalence Methods2026.0585.7
Func. EntropyModel=Gemini-2.5-Flash-Lite, Category=Functional Equivalence Methods2026.0585.7
Func. Equiv.Model=GPT-4o, Category=Functional Equivalence Methods2026.0585.6
Func. SetsModel=GPT-4o-mini, Category=Functional Equivalence Methods2026.0585.6
Func. EntropyModel=GPT-4o, Category=Functional Equivalence Methods2026.0585.5
Func. SetsModel=Gemini-2.5-Flash, Category=Functional Equivalence Methods2026.0585.5
Func. SetsModel=GPT-4o, Category=Functional Equivalence Methods2026.0585.5
Func. SetsModel=Gemini-2.5-Flash-Lite, Category=Functional Equivalence Methods2026.0585.2
P(True)Model=Gemini-2.5-Flash-Lite, Category=Reflexive Methods2026.0583
Func. EntropyModel=Gemini-2.5-Pro, Category=Functional Equivalence Methods2026.0582.9
Func. Equiv.Model=Gemini-2.5-Pro, Category=Functional Equivalence Methods2026.0582.6
Func. SetsModel=Gemini-2.5-Pro, Category=Functional Equivalence Methods2026.0582.6
P(True)Model=GPT-4o, Category=Reflexive Methods2026.0582.4
P(True)Model=Gemini-2.5-Pro, Category=Reflexive Methods2026.0581.5
CodeBLEUModel=Gemini-2.5-Pro, Category=Similarity-Based Methods2026.0581.3
Cos. Sim.Model=Gemini-2.5-Pro, Category=Similarity-Based Methods2026.0580.4
Min. Tok. Neg.Model=Gemini-2.5-Flash-Lite, Category=Token-Probability Methods2026.0579.7
CodeBLEUModel=Gemini-2.5-Flash, Category=Similarity-Based Methods2026.0579.5
CodeBLEUModel=Gemini-2.5-Flash-Lite, Category=Similarity-Based Methods2026.0579.5
Verb. Conf.Model=Gemini-2.5-Flash, Category=Reflexive Methods2026.0579.5
Min. Prob.Model=Gemini-2.5-Flash-Lite, Category=Token-Probability Methods2026.0578.3
Min. Tok. Neg.Model=GPT-4o-mini, Category=Token-Probability Methods2026.0577.9
Avg. Tok. Neg.Model=GPT-4o-mini, Category=Token-Probability Methods2026.0577.6
Avg. Tok. Neg.Model=Gemini-2.5-Flash-Lite, Category=Token-Probability Methods2026.0577.4
Min. Prob.Model=Gemini-2.5-Flash, Category=Token-Probability Methods2026.0577.3
P(True)Model=Gemini-2.5-Flash, Category=Reflexive Methods2026.0577.1
Min. Tok. Neg.Model=Gemini-2.5-Pro, Category=Token-Probability Methods2026.0577
Min. Tok. Neg.Model=GPT-4o, Category=Token-Probability Methods2026.0576.8
Min. Tok. Neg.Model=Gemini-2.5-Flash, Category=Token-Probability Methods2026.0576.6
CodeBLEUModel=GPT-4o-mini, Category=Similarity-Based Methods2026.0576.6
Min. Prob.Model=Gemini-2.5-Pro, Category=Token-Probability Methods2026.0576.2
Prob. MarginModel=Gemini-2.5-Flash-Lite, Category=Token-Probability Methods2026.0576.2
Seq. Prob.Model=Gemini-2.5-Flash-Lite, Category=Token-Probability Methods2026.0576.1
Avg. Tok. Neg.Model=GPT-4o, Category=Token-Probability Methods2026.0576
Seq. Prob.Model=GPT-4o-mini, Category=Token-Probability Methods2026.0575.7
Cos. Sim.Model=Gemini-2.5-Flash-Lite, Category=Similarity-Based Methods2026.0575.6
Cos. Sim.Model=Gemini-2.5-Flash, Category=Similarity-Based Methods2026.0575.3
Prob. MarginModel=GPT-4o-mini, Category=Token-Probability Methods2026.0575.2
P(True)Model=GPT-4o-mini, Category=Reflexive Methods2026.0575.2
Prob. MarginModel=GPT-4o, Category=Token-Probability Methods2026.0574.9
CodeBLEUModel=GPT-4o, Category=Similarity-Based Methods2026.0574.4
Min. Prob.Model=GPT-4o-mini, Category=Token-Probability Methods2026.0573.8
Cos. Sim.Model=GPT-4o-mini, Category=Similarity-Based Methods2026.0573.8
Seq. Prob.Model=GPT-4o, Category=Token-Probability Methods2026.0572.9
Verb. Conf.Model=GPT-4o-mini, Category=Reflexive Methods2026.0572.5
Cos. Sim.Model=GPT-4o, Category=Similarity-Based Methods2026.0572.4
Verb. Conf.Model=Gemini-2.5-Pro, Category=Reflexive Methods2026.0571.6
Min. Prob.Model=GPT-4o, Category=Token-Probability Methods2026.0570.5
Avg. Tok. Neg.Model=Gemini-2.5-Flash, Category=Token-Probability Methods2026.0569.5
Seq. Prob.Model=Gemini-2.5-Flash, Category=Token-Probability Methods2026.0569.1
Verb. Conf.Model=GPT-4o, Category=Reflexive Methods2026.0568.1
Prob. MarginModel=Gemini-2.5-Flash, Category=Token-Probability Methods2026.0566.8
Verb. Conf.Model=Gemini-2.5-Flash-Lite, Category=Reflexive Methods2026.0559.7
Avg. Tok. Neg.Model=Gemini-2.5-Pro, Category=Token-Probability Methods2026.0558.2
Seq. Prob.Model=Gemini-2.5-Pro, Category=Token-Probability Methods2026.0558
Prob. MarginModel=Gemini-2.5-Pro, Category=Token-Probability Methods2026.0555.4