Factual Question Answering on AURATown environment-grounded queries N=50, 3 seeds 1.0
76.6Factual AccuracyFixed-Probe
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| Fixed-Probe2026.06 | 76.6 | 8 | 56 | 3.2 | 10 | 0.006 | |
| Plan-and-Solve2026.06 | 76.4 | 4.64 | 58.7 | 5.2 | — | 0.004 | |
| AURAProbe Mode=GapRouted2026.06 | 69.6 | 1.4 | 75.3 | 4.3 | 10 | 0.098 | |
| AURAProbe Mode=Full, B=22026.06 | 64 | 0 | 66.7 | 3.9 | 0.001 | — | |
| AURAProbe Mode=No Probe2026.06 | 60.3 | 0 | 75.3 | 2.2 | 0.0005 | 0.06 | |
| ReAct Agent2026.06 | 55 | — | 51.3 | 6 | 0.1 | 0.033 | |
| Static Context2026.06 | 45 | 0 | 70 | 1.8 | 0.032 | 10 | |
| Reflexion2026.06 | 17.4 | 2.15 | 55.9 | 20.6 | — | 10 | |
| Vanilla LLM2026.06 | 7 | 0 | 93.3 | 2.3 | — | 10 |