List your company

Code Generation on HumanEval (Spearman rho, Resolution Gain)

1Spearman rho Agreement (FullPair)FullPair
0.0640.3070.550.793May 28, 2026

Evaluation results

MethodLinks
FullPairProtocol=FullPair2026.0511.88
PointwiseProtocol=Pointwise2026.050.951.18
SEALProtocol=SEAL2026.050.952.1
FlatBrkProtocol=FlatBrk2026.050.931.77
ListwiseProtocol=Listwise2026.050.699.84
OriginalProtocol=Original2026.050.431
FixedRubProtocol=FixedRub2026.050.10.12