Code Generation on LiveBench (Signal, UT, Code, BoN)
58.7SignalBaseline
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| BaselineModel=DeepSeek-V3.2-685B, Model Category=Base Models2026.05 | 58.7 | 82.1 | 68.5 | 77.3 | |
| CoSPlay (Ours)Model=DeepSeek-V3.2-685B, Model Category=Base Models2026.05 | 57.3 | 74 | 76.9 | 76.7 | |
| + ClusterModel=DeepSeek-V3.2-685B, Model Category=Base Models2026.05 | 57.3 | 74 | 76.9 | 76 | |
| BaselineModel=Gemini-2.0-Flash, Model Category=Base Models2026.05 | 56.7 | 50.3 | 59.8 | 68 | |
| + ClusterModel=AceCoder-RM-7B, Model Category=RL-tuned Models2026.05 | 56.7 | 61.3 | 47 | 56 | |
| CoSPlay (Ours)Model=AceCoder-RM-7B, Model Category=RL-tuned Models2026.05 | 54 | 61.3 | 47 | 56.7 | |
| + ClusterModel=DeepSeek-Coder-V2-Lite-Instruct-16B, Model Category=Base Models2026.05 | 53.3 | 68.7 | 42 | 51.3 | |
| + ClusterModel=Gemini-2.0-Flash, Model Category=Base Models2026.05 | 53.3 | 76.2 | 63.2 | 66 | |
| CoSPlay (Ours)Model=DeepSeek-Coder-V2-Lite-Instruct-16B, Model Category=Base Models2026.05 | 52.7 | 68.7 | 42 | 48 | |
| + ClusterModel=AceCoder-Rule-7B, Model Category=RL-tuned Models2026.05 | 52.7 | 64.6 | 45.2 | 54.7 | |
| + ClusterModel=Seed-Coder-8B-Instruct, Model Category=Base Models2026.05 | 52 | 63.3 | 47.5 | 56 | |
| + ClusterModel=Absolute-Zero-7B, Model Category=RL-tuned Models2026.05 | 52 | 80.4 | 33 | 47.3 | |
| CoSPlay (Ours)Model=Gemini-2.0-Flash, Model Category=Base Models2026.05 | 51.3 | 76.2 | 63.2 | 64.7 | |
| CoSPlay (Ours)Model=AceCoder-Rule-7B, Model Category=RL-tuned Models2026.05 | 51.3 | 64.6 | 45.2 | 54.7 | |
| CoSPlay (Ours)Model=Seed-Coder-8B-Instruct, Model Category=Base Models2026.05 | 50.7 | 63.3 | 47.5 | 56 | |
| BaselineModel=DeepSeek-Coder-V2-Lite-Instruct-16B, Model Category=Base Models2026.05 | 48.7 | 27.4 | 30.8 | 38.7 | |
| BaselineModel=Seed-Coder-8B-Instruct, Model Category=Base Models2026.05 | 48 | 26.8 | 40.7 | 50.7 | |
| BaselineModel=Absolute-Zero-7B, Model Category=RL-tuned Models2026.05 | 44 | 19.3 | 11.6 | 36.7 | |
| BaselineModel=AceCoder-Rule-7B, Model Category=RL-tuned Models2026.05 | 44 | 13.9 | 40.7 | 46 | |
| BaselineModel=AceCoder-RM-7B, Model Category=RL-tuned Models2026.05 | 43.3 | 15.1 | 40.9 | 48.7 | |
| CoSPlay (Ours)Model=Absolute-Zero-7B, Model Category=RL-tuned Models2026.05 | 42.7 | 80.4 | 33 | 40.7 |