Language-based Games on HouseBuild (test)
100HPGRPO
Evaluation results
| Method | Links | ||
|---|---|---|---|
| GRPOResponse time (seconds)=22.0, Cost (tokens/agent/turn)=8902026.01 | 100 | 54.6 | |
| ACResponse time (seconds)=22.2, Cost (tokens/agent/turn)=9042026.01 | 100 | 55.9 | |
| Raw ModelResponse time (seconds)=22.6, Cost (tokens/agent/turn)=10162026.01 | 99.6 | 43.2 | |
| CoLLM-CCResponse time (seconds)=19.0, Cost (tokens/agent/turn)=4422026.01 | 86.4 | 52.7 | |
| MAGRPOResponse time (seconds)=19.2, Cost (tokens/agent/turn)=4462026.01 | 80.2 | 50.9 | |
| CoLLM-DCResponse time (seconds)=19.4, Cost (tokens/agent/turn)=4702026.01 | 43.8 | 46.8 | |
| PipelineResponse time (seconds)=20.2, Cost (tokens/agent/turn)=4882026.01 | 30.6 | 41.3 | |
| DiscussionResponse time (seconds)=21.0, Cost (tokens/agent/turn)=5102026.01 | 27.6 | 38.1 | |
| ParallelResponse time (seconds)=19.2, Cost (tokens/agent/turn)=5022026.01 | 21.8 | 46.1 |