Code Generation on CoopHE (test)
75.2Pass RateCoLLM-CC
Evaluation results
| Method | Links | |
|---|---|---|
| CoLLM-CCResponse time (seconds)=2.6, Cost (tokens/agent/turn)=1662026.01 | 75.2 | |
| MAGRPOResponse time (seconds)=2.3, Cost (tokens/agent/turn)=1322026.01 | 74.3 | |
| ACResponse time (seconds)=2.5, Cost (tokens/agent/turn)=912026.01 | 62.5 | |
| PipelineResponse time (seconds)=2.6, Cost (tokens/agent/turn)=1772026.01 | 62.5 | |
| GRPOResponse time (seconds)=2.5, Cost (tokens/agent/turn)=882026.01 | 61.8 | |
| CoLLM-DCResponse time (seconds)=2.5, Cost (tokens/agent/turn)=1612026.01 | 59.1 | |
| Raw ModelResponse time (seconds)=2.5, Cost (tokens/agent/turn)=902026.01 | 56.3 | |
| ParallelResponse time (seconds)=2.3, Cost (tokens/agent/turn)=1382026.01 | 50 | |
| DiscussionResponse time (seconds)=2.9, Cost (tokens/agent/turn)=1912026.01 | 25 |