Constrained navigation on FlightQA
32.2AccuracySiRA
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| SiRAReasoning System=System II, Policy=Simulative Reasoning2025.07 | 32.2 | 36.7 | 32.2 | 38.9 | 1.1 | 40 | 18.9 | 1.1 | |
| SiRAReasoning System=System I, Policy=Reactive Policy2025.07 | 14.4 | 15.6 | 14.4 | 16.7 | 0 | 37.8 | 44.4 | 1.1 | |
| Unconstrained CoTPlanner=o3-mini2025.07 | 3.3 | 4.4 | 3.3 | 4.4 | 3.3 | 51.1 | 32.2 | 8.9 | |
| Unconstrained CoTPlanner=o12025.07 | 1.1 | 1.1 | 1.1 | 1.1 | 11.1 | 40 | 37.8 | 10 | |
| OpenHands BrowsingAgentMethod=OpenHands BrowsingAgent2025.07 | 0 | 0 | 0 | 0 | 3.3 | 3.3 | 0 | 93.3 |