Coding on Coding Tasks (test) benchmark leaderboard · SOTA2 Research

Coding on Coding Tasks (test)

98.3Pass@1SALE

Evaluation Results

Method	Links
SALE2026.02		98.3	0.18
FrugalGPT2026.02		97.5	0.39
SALE w/o memory2026.02		97.5	0.22
Best single agent2026.02		95	0.36
CARROT2026.02		95	0.36
TO-Router2026.02		95	0.36
WTP2026.02		93.8	0.16
CARROT2026.02		82.3	0.25
SALE w/o memory2026.02		82.3	0.28
SALE2026.02		82	0.27
Best single agent2026.02		79.7	0.36
TO-Router2026.02		79.7	0.36
WTP2026.02		76	0.15
FrugalGPT2026.02		69.6	0.61
SALE2026.02		69	0.29
SALE w/o memory2026.02		68.8	0.31
Best single agent2026.02		67.5	0.36
TO-Router2026.02		67.5	0.36
SALE2026.02		61.1	0.27
WTP2026.02		60	0.15
CARROT2026.02		60	0.26
SALE w/o memory2026.02		59.9	0.27
Best single agent2026.02		58.4	0.36
TO-Router2026.02		58.4	0.36
CARROT2026.02		57.1	0.31
FrugalGPT2026.02		56.3	0.61
FrugalGPT2026.02		50.4	0.57
WTP2026.02		50.1	0.11
SALE2026.02		30.4	0.3
Best single agent2026.02		27.2	0.36
CARROT2026.02		27.2	0.36
TO-Router2026.02		27.2	0.36
SALE w/o memory2026.02		27.2	0.32
SALE2026.02		26.1	0.29
SALE w/o memory2026.02		24.1	0.31
Best single agent2026.02		22.8	0.36
TO-Router2026.02		22.8	0.36
CARROT2026.02		21.5	0.35
FrugalGPT2026.02		18.5	0.61
WTP2026.02		14.8	0.05
FrugalGPT2026.02		10.1	0.61
WTP2026.02		6.3	0.05