LLM Planning on SafeRun 1.0 (test)
100Success RatePE
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| PEModel=GPT-52026.06 | 100 | 99.8 | 85.2 | |
| CodeActModel=GPT-52026.06 | 100 | 100 | 84.3 | |
| SafeRunModel=GPT-52026.06 | 100 | 100 | 87.3 | |
| PEModel=GPT-4.12026.06 | 100 | 89.5 | 81.5 | |
| CodeActModel=GPT-4.12026.06 | 100 | 96 | 78.7 | |
| SafeRunModel=GPT-4.12026.06 | 100 | 100 | 79.6 | |
| PEModel=GPT-4.1-mini2026.06 | 100 | 67.8 | 54.4 | |
| CodeActModel=GPT-4.1-mini2026.06 | 100 | 96.2 | 76.4 | |
| SafeRunModel=GPT-4.1-mini2026.06 | 100 | 100 | 79.9 | |
| PEModel=DeepSeek-V3.22026.06 | 100 | 76.5 | 66.2 | |
| CodeActModel=DeepSeek-V3.22026.06 | 100 | 96.5 | 79.5 | |
| SafeRunModel=DeepSeek-V3.22026.06 | 100 | 100 | 78.7 | |
| PEModel=Qwen3.5-397B-A17B2026.06 | 100 | 62 | 58.8 | |
| CodeActModel=Qwen3.5-397B-A17B2026.06 | 100 | 99.5 | 82.4 | |
| SafeRunModel=Qwen3.5-397B-A17B2026.06 | 100 | 100 | 82.1 |