Simulation generation on PLE (test)
82Flappy ScoreFactorSmith
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| FactorSmithModel=GPT-4, Mode=Full2026.03 | 82 | 74 | 52 | 81 | 68 | 85 | 70 | 50 | |
| GPT-4 FactorSimModel=GPT-4, Mode=FactorSim2026.03 | 78 | 66 | 44 | 78 | 61 | 81 | 62 | 44 | |
| GPT-4 VanillaModel=GPT-4, Mode=Vanilla2026.03 | 35 | 35 | 42 | 44 | 25 | 34 | 46 | 21 | |
| GPT-4 Self-DebugModel=GPT-4, Mode=Self-Debug2026.03 | 33 | 53 | 43 | 51 | 75 | 41 | 45 | 31 | |
| GPT-4 CoT+DebugModel=GPT-4, Mode=CoT+Debug2026.03 | 30 | 51 | 39 | 53 | 64 | 47 | 50 | 34 | |
| GPT-4 AgentCoderModel=GPT-4, Mode=AgentCoder2026.03 | 18 | 45 | 27 | 43 | 43 | 33 | 20 | 23 |