Hand-eye Calibration on Hand-eye calibration task
96.67Code Gen Success RateMCS + Debugger w/ Probing
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| MCS + Debugger w/ ProbingLLM Backbone=GPT-4o, Code Generation Strategy=MCS, Feedback Strategy=Debugger, Probing=true2026.06 | 96.67 | 86.67 | 0.048 | 0.011 | 0.17 | 0.012 | |
| CaP with Full APICode Generation Strategy=CaP, API Configuration=Full API2026.06 | 90 | — | — | — | 0.038 | 0.039 | |
| MCS + Debugger w/ ProbingLLM Backbone=Gemini, Code Generation Strategy=MCS, Feedback Strategy=Debugger, Probing=true2026.06 | 90 | 73.33 | 0.064 | 0.064 | 0.091 | 0.065 | |
| MCS + DebuggerLLM Backbone=GPT-4o, Code Generation Strategy=MCS, Feedback Strategy=Debugger, Probing=false2026.06 | 73.33 | 60 | 0.174 | 0.019 | 0.074 | 0.014 | |
| MCS + Debugger w/ ProbingLLM Backbone=GPT-4.1 mini, Code Generation Strategy=MCS, Feedback Strategy=Debugger, Probing=true2026.06 | 73.33 | 40 | 0.085 | 0.068 | 0.337 | 0.068 | |
| ProgPromptCode Generation Strategy=ProgPrompt2026.06 | 66.67 | — | — | — | 0.026 | 0.026 | |
| MCSLLM Backbone=GPT-4o, Code Generation Strategy=MCS2026.06 | 23.33 | 6.67 | 0.029 | 0.029 | 0.308 | 0.283 | |
| SPG + Debugger w/ ProbingLLM Backbone=GPT-4o, Code Generation Strategy=SPG, Feedback Strategy=Debugger, Probing=true2026.06 | 10 | — | — | — | 0.485 | 0.062 | |
| SPG + DebuggerLLM Backbone=GPT-4o, Code Generation Strategy=SPG, Feedback Strategy=Debugger, Probing=false2026.06 | 6.67 | — | — | — | 0.802 | 0.802 |