# Robotic Manipulation Planning on ALRM Benchmark Source: [Robotic Manipulation Planning on ALRM Benchmark]() Collection index: [llms.txt]() Data updated: 2026-02-26T06:04:07.235Z Evaluation of robotic manipulation planning performance on the ALRM Benchmark, reporting success rates across specific criteria (CAN, LEX, SYN, SEM, HLR) and associated latency. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Robotic Manipulation Planning | | Dataset (including any stated split) | ALRM Benchmark | | Metric | Score 0 | | Unit | SCALAR | | Direction | Higher is better | | Results loaded | 19 of 19 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Score 0](): Higher is better; unit: SCALAR. - [Score 1](): Higher is better; unit: SCALAR. - [Score 2](): Higher is better; unit: SCALAR. - [Success Rate (CAN)](): Higher is better; unit: PERCENT. - [Success Rate (LEX)](): Higher is better; unit: PERCENT. - [Success Rate (SYN)](): Higher is better; unit: PERCENT. - [Success Rate (SEM)](): Higher is better; unit: PERCENT. - [Success Rate (HLR)](): Higher is better; unit: PERCENT. - [Success Rate (AVG)](): Higher is better; unit: PERCENT. - [Latency (s)](): Lower is better; unit: TIME. ## Recorded results | Method | Score 0 | Result date | Report paper | | --- | --- | --- | --- | | Granite-3.3-8B | 54 | Not recorded | 6979984ca2b28e761d87cd14 | | Mistral-7B | 54 | Not recorded | 6979984ca2b28e761d87cd14 | | Llama-3.1-8B | 50 | Not recorded | 6979984ca2b28e761d87cd14 | | Mistral-7B | 45 | Not recorded | 6979984ca2b28e761d87cd14 | | Falcon-H1-7B | 38 | Not recorded | 6979984ca2b28e761d87cd14 | | DeepSeek-R1-7B | 31 | Not recorded | 6979984ca2b28e761d87cd14 | | Qwen3-8B | 7 | Not recorded | 6979984ca2b28e761d87cd14 | | Granite-3.3-8B | 6 | Not recorded | 6979984ca2b28e761d87cd14 | | Qwen3-8B | 5 | Not recorded | 6979984ca2b28e761d87cd14 | | Llama-3.1-8B | 2 | Not recorded | 6979984ca2b28e761d87cd14 | | Claude-4.1-Opus | 0 | Not recorded | 6979984ca2b28e761d87cd14 | | GPT-5 | 0 | Not recorded | 6979984ca2b28e761d87cd14 | | DeepSeek-V3.1 | 0 | Not recorded | 6979984ca2b28e761d87cd14 | | Gemini-2.5-Pro | 0 | Not recorded | 6979984ca2b28e761d87cd14 | | Falcon-H1-7B | 0 | Not recorded | 6979984ca2b28e761d87cd14 | | Claude-4.1-Opus | 0 | Not recorded | 6979984ca2b28e761d87cd14 | | GPT-5 | 0 | Not recorded | 6979984ca2b28e761d87cd14 | | DeepSeek-V3.1 | 0 | Not recorded | 6979984ca2b28e761d87cd14 | | Gemini-2.5-Pro | 0 | Not recorded | 6979984ca2b28e761d87cd14 | ### Granite-3.3-8B Score 0: 54. Result ID: 6999d0ef5698665ec44f504a. [Reporting paper]() Recorded evaluation configuration: {"Mode":"Tool-as-Policy"}. This record does not assert independent reproduction. ### Mistral-7B Score 0: 54. Result ID: 6999d0ef5698665ec44f504b. [Reporting paper]() Recorded evaluation configuration: {"Mode":"Tool-as-Policy"}. This record does not assert independent reproduction. ### Llama-3.1-8B Score 0: 50. Result ID: 6999d0ef5698665ec44f5049. [Reporting paper]() Recorded evaluation configuration: {"Mode":"Tool-as-Policy"}. This record does not assert independent reproduction. ### Mistral-7B Score 0: 45. Result ID: 6999d0ef5698665ec44f5042. [Reporting paper]() Recorded evaluation configuration: {"Mode":"Code-as-Policy"}. This record does not assert independent reproduction. ### Falcon-H1-7B Score 0: 38. Result ID: 6999d0ef5698665ec44f5047. [Reporting paper]() Recorded evaluation configuration: {"Mode":"Tool-as-Policy"}. This record does not assert independent reproduction. ### DeepSeek-R1-7B Score 0: 31. Result ID: 6999d0ef5698665ec44f5041. [Reporting paper]() Recorded evaluation configuration: {"Mode":"Code-as-Policy"}. This record does not assert independent reproduction. ### Qwen3-8B Score 0: 7. Result ID: 6999d0ef5698665ec44f503e. [Reporting paper]() Recorded evaluation configuration: {"Mode":"Code-as-Policy"}. This record does not assert independent reproduction. ### Granite-3.3-8B Score 0: 6. Result ID: 6999d0ef5698665ec44f5040. [Reporting paper]() Recorded evaluation configuration: {"Mode":"Code-as-Policy"}. This record does not assert independent reproduction. ### Qwen3-8B Score 0: 5. Result ID: 6999d0ef5698665ec44f5048. [Reporting paper]() Recorded evaluation configuration: {"Mode":"Tool-as-Policy"}. This record does not assert independent reproduction. ### Llama-3.1-8B Score 0: 2. Result ID: 6999d0ef5698665ec44f503f. [Reporting paper]() Recorded evaluation configuration: {"Mode":"Code-as-Policy"}. This record does not assert independent reproduction. ### Claude-4.1-Opus Score 0: 0. Result ID: 6999d0ef5698665ec44f5039. [Reporting paper]() Recorded evaluation configuration: {"Mode":"Code-as-Policy"}. This record does not assert independent reproduction. ### GPT-5 Score 0: 0. Result ID: 6999d0ef5698665ec44f503a. [Reporting paper]() Recorded evaluation configuration: {"Mode":"Code-as-Policy"}. This record does not assert independent reproduction. ### DeepSeek-V3.1 Score 0: 0. Result ID: 6999d0ef5698665ec44f503b. [Reporting paper]() Recorded evaluation configuration: {"Mode":"Code-as-Policy"}. This record does not assert independent reproduction. ### Gemini-2.5-Pro Score 0: 0. Result ID: 6999d0ef5698665ec44f503c. [Reporting paper]() Recorded evaluation configuration: {"Mode":"Code-as-Policy"}. This record does not assert independent reproduction. ### Falcon-H1-7B Score 0: 0. Result ID: 6999d0ef5698665ec44f503d. [Reporting paper]() Recorded evaluation configuration: {"Mode":"Code-as-Policy"}. This record does not assert independent reproduction. ### Claude-4.1-Opus Score 0: 0. Result ID: 6999d0ef5698665ec44f5043. [Reporting paper]() Recorded evaluation configuration: {"Mode":"Tool-as-Policy"}. This record does not assert independent reproduction. ### GPT-5 Score 0: 0. Result ID: 6999d0ef5698665ec44f5044. [Reporting paper]() Recorded evaluation configuration: {"Mode":"Tool-as-Policy"}. This record does not assert independent reproduction. ### DeepSeek-V3.1 Score 0: 0. Result ID: 6999d0ef5698665ec44f5045. [Reporting paper]() Recorded evaluation configuration: {"Mode":"Tool-as-Policy"}. This record does not assert independent reproduction. ### Gemini-2.5-Pro Score 0: 0. Result ID: 6999d0ef5698665ec44f5046. [Reporting paper]() Recorded evaluation configuration: {"Mode":"Tool-as-Policy"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.