Robotic Manipulation Planning on ALRM Benchmark
54Score 0Granite-3.3-8B
Evaluation results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Granite-3.3-8BMode=Tool-as-Policy2026.01 | 54 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 19.07 | |
| Mistral-7BMode=Tool-as-Policy2026.01 | 54 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 10.81 | |
| Llama-3.1-8BMode=Tool-as-Policy2026.01 | 50 | 4 | 0 | 5.6 | 0 | 0 | 5.6 | 5.6 | 3.7 | 36.07 | |
| Mistral-7BMode=Code-as-Policy2026.01 | 45 | 9 | 0 | 11.1 | 11.1 | 5.6 | 5.6 | 8.3 | 8.3 | 18.96 | |
| Falcon-H1-7BMode=Tool-as-Policy2026.01 | 38 | 16 | 0 | 27.8 | 16.7 | 16.7 | 0 | 13.9 | 14.8 | 161.34 | |
| DeepSeek-R1-7BMode=Code-as-Policy2026.01 | 31 | 23 | 0 | 33.3 | 5.6 | 5.6 | 27.8 | 27.8 | 21.3 | 53.55 | |
| Qwen3-8BMode=Code-as-Policy2026.01 | 7 | 24 | 23 | 66.7 | 72.2 | 88.9 | 61.1 | 50 | 64.8 | 343.33 | |
| Granite-3.3-8BMode=Code-as-Policy2026.01 | 6 | 38 | 10 | 61.1 | 56.6 | 56.6 | 44.4 | 52.8 | 53.7 | 24.48 | |
| Qwen3-8BMode=Tool-as-Policy2026.01 | 5 | 40 | 9 | 55.6 | 61.1 | 55.6 | 55.6 | 47.2 | 53.7 | 392.08 | |
| Llama-3.1-8BMode=Code-as-Policy2026.01 | 2 | 28 | 24 | 72.2 | 77.8 | 77.8 | 77.8 | 58.3 | 68.5 | 20.13 | |
| Claude-4.1-OpusMode=Code-as-Policy2026.01 | 0 | 8 | 46 | 94.4 | 94.4 | 94.4 | 88.9 | 91.7 | 92.6 | 33.44 | |
| GPT-5Mode=Code-as-Policy2026.01 | 0 | 10 | 44 | 88.9 | 94.4 | 100 | 88.9 | 86.1 | 90.7 | 145.59 | |
| DeepSeek-V3.1Mode=Code-as-Policy2026.01 | 0 | 17 | 37 | 88.9 | 88.9 | 83.3 | 83.3 | 80.6 | 84.3 | 69.83 | |
| Gemini-2.5-ProMode=Code-as-Policy2026.01 | 0 | 29 | 25 | 72.2 | 77.8 | 66.7 | 77.8 | 72.2 | 73.1 | 52.62 | |
| Falcon-H1-7BMode=Code-as-Policy2026.01 | 0 | 17 | 37 | 83.3 | 88.9 | 88.9 | 88.9 | 77.8 | 84.3 | 24.89 | |
| Claude-4.1-OpusMode=Tool-as-Policy2026.01 | 0 | 7 | 47 | 94.4 | 94.4 | 94.4 | 94.4 | 91.7 | 93.5 | 82.6 | |
| GPT-5Mode=Tool-as-Policy2026.01 | 0 | 16 | 38 | 94.4 | 100 | 72.2 | 72.2 | 86.1 | 85.2 | 113.88 | |
| DeepSeek-V3.1Mode=Tool-as-Policy2026.01 | 0 | 16 | 38 | 88.9 | 94.4 | 83.3 | 83.3 | 80.6 | 85.2 | 161.73 | |
| Gemini-2.5-ProMode=Tool-as-Policy2026.01 | 0 | 14 | 40 | 88.9 | 94.4 | 77.8 | 88.9 | 86.1 | 87 | 117.43 |