Mission-level query rule evaluation on Fixed-protocol simulation Hard/ambiguous regime
39Clean Success Rate@1mrule-based LLM
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| rule-based LLMRuns=402026.06 | 39 | 97.5 | 1.2 | 683 | 1 | |
| learned-CVIRuns=402026.06 | 38 | 95 | 1 | 483.4 | 1 | |
| one-shot LLMRuns=402026.06 | 30 | 75 | 1 | 443.4 | 2 | |
| periodic LLMRuns=402026.06 | 25 | 62.5 | 3 | 1,194.8 | 9 | |
| local-onlyRuns=402026.06 | 2 | 5 | 0 | 0 | 0 |