Physical outcome and stability prediction on TraySim
79.8AccuracyLLMPhy
Evaluation Results
| Method | Links | |
|---|---|---|
| LLMPhyPhase 1=LLMPhy (Phi-4), Phase 2=GT2024.11 | 79.8 | |
| LLMPhyPhase 1=LLMPhy (o1-mini), Phase 2=GT2024.11 | 77.5 | |
| LLMPhyPhase 1=LLMPhy (Gemma3), Phase 2=GT2024.11 | 77.3 | |
| CMA-ESPhase 1=CMA-ES, Phase 2=GT2024.11 | 75.8 | |
| Best-of-KPhase 1=Best-of-K, Phase 2=GT2024.11 | 72 | |
| GPT-4.1Phase 1=GT, Phase 2=GPT-4.12024.11 | 68.3 | |
| LLMPhyPhase 1=LLMPhy (o3-mini), Phase 2=LLMPhy (GPT-4.1)2024.11 | 66.3 | |
| LLMPhyPhase 1=LLMPhy (o1-mini), Phase 2=LLMPhy (GPT-4.1)2024.11 | 66 | |
| GPT-4oPhase 1=GT, Phase 2=GPT-4o2024.11 | 65.1 | |
| LLMPhyPhase 1=LLMPhy (o1-mini), Phase 2=LLMPhy (GPT-4o)2024.11 | 62 | |
| CMA-ESPhase 1=CMA-ES, Phase 2=LLMPhy (GPT-4o)2024.11 | 59.7 | |
| Bayes Opt.Phase 1=Bayes Opt., Phase 2=LLMPhy (GPT-4o)2024.11 | 59.6 | |
| HumanPhase 1=Human, Phase 2=Human2024.11 | 54.2 | |
| RandomPhase 1=Random, Phase 2=GT2024.11 | 50.8 | |
| Intuitive PhysicsPhase 1=Intuitive Physics, Phase 2=GPT-4o2024.11 | 32.1 |