Simulator selection on Supply Chain
0.24Top-1 RegretMechSim
Evaluation results
| Method | Links | ||
|---|---|---|---|
| MechSimLLM Backbone=Qwen3-235B2026.06 | 0.24 | 0.51 | |
| MechSimLLM=Qwen3-235B2026.06 | 0.24 | 0.513 | |
| MechSimLLM=DeepSeek V32026.06 | 0.467 | 0.389 | |
| MechSimLLM Backbone=DeepSeek V3.22026.06 | 0.47 | 0.39 | |
| Logic-LMLLM Backbone=DeepSeek V3.22026.06 | 0.54 | 0.44 | |
| Logic-LMLLM=DeepSeek V32026.06 | 0.542 | 0.438 | |
| Graph of ThoughtsLLM Backbone=DeepSeek V3.22026.06 | 0.59 | 0.51 | |
| Causal-CopilotLLM Backbone=Qwen3-235B2026.06 | 0.59 | 0.56 | |
| Graph of ThoughtsLLM Backbone=Qwen3-235B2026.06 | 0.59 | 0.56 | |
| GoTLLM=Qwen3-235B2026.06 | 0.591 | 0.556 | |
| GoTLLM=DeepSeek V32026.06 | 0.592 | 0.514 | |
| Causal-CopilotLLM=Qwen3-235B2026.06 | 0.593 | 0.556 | |
| Causal-CopilotLLM=DeepSeek V32026.06 | 0.617 | 0.535 | |
| Causal-CopilotLLM Backbone=DeepSeek V3.22026.06 | 0.62 | 0.54 | |
| Logic-LMLLM Backbone=Qwen3-235B2026.06 | 0.62 | 0.51 | |
| Logic-LMLLM=Qwen3-235B2026.06 | 0.621 | 0.514 | |
| MechSimLLM Backbone=Claude Sonnet 4.62026.06 | 4.25 | 3.47 | |
| MechSimLLM=Claude Sonnet 4.62026.06 | 4.25 | 3.47 | |
| Logic-LMLLM Backbone=Claude Sonnet 4.62026.06 | 5.42 | 4.38 | |
| Logic-LMLLM=Claude Sonnet 4.62026.06 | 5.42 | 4.38 | |
| Graph of ThoughtsLLM Backbone=Claude Sonnet 4.62026.06 | 5.92 | 5.14 | |
| GoTLLM=Claude Sonnet 4.62026.06 | 5.92 | 5.14 | |
| Causal-CopilotLLM Backbone=Claude Sonnet 4.62026.06 | 6.17 | 5.35 | |
| Causal-CopilotLLM=Claude Sonnet 4.62026.06 | 6.17 | 5.35 |