Travel Planning on Travel Planning
96.2Composite ScorePIVOT
Evaluation results
| Method | Links | |
|---|---|---|
| PIVOTBackbone=Opus 4.6, Feedback regime=w/ HITL2026.05 | 96.2 | |
| PIVOTBackbone=Opus 4.6, Feedback regime=w/o HITL2026.05 | 89.3 | |
| Self-CritiqueBackbone=Opus 4.62026.05 | 87.3 | |
| AgentDebugBackbone=Opus 4.62026.05 | 86.1 | |
| ReACTBackbone=Opus 4.62026.05 | 86 | |
| SE-AgentBackbone=Opus 4.62026.05 | 82.8 | |
| PIVOTBackbone=Opus 4.5, Feedback regime=w/ HITL2026.05 | 82.7 | |
| PIVOTBackbone=Opus 4.5, Feedback regime=w/o HITL2026.05 | 76.5 | |
| Self-CritiqueBackbone=Opus 4.52026.05 | 75.2 | |
| PIVOTBackbone=Sonnet 4.5, Feedback regime=w/ HITL2026.05 | 67.1 | |
| SE-AgentBackbone=Opus 4.52026.05 | 63.9 | |
| ReACTBackbone=Opus 4.52026.05 | 62.9 | |
| PIVOTBackbone=Sonnet 4.5, Feedback regime=w/o HITL2026.05 | 59.8 | |
| AgentDebugBackbone=Opus 4.52026.05 | 59.3 | |
| PIVOTBackbone=Haiku 4.5, Feedback regime=w/ HITL2026.05 | 57.3 | |
| PIVOTBackbone=Haiku 4.5, Feedback regime=w/o HITL2026.05 | 53.7 | |
| ReACTBackbone=Sonnet 4.52026.05 | 53.1 | |
| Self-CritiqueBackbone=Sonnet 4.52026.05 | 52.6 | |
| SE-AgentBackbone=Sonnet 4.52026.05 | 52.6 | |
| AgentDebugBackbone=Sonnet 4.52026.05 | 46.2 | |
| Self-CritiqueBackbone=Haiku 4.52026.05 | 45.1 | |
| AgentDebugBackbone=Haiku 4.52026.05 | 42.7 | |
| SE-AgentBackbone=Haiku 4.52026.05 | 41.9 | |
| PIVOTBackbone=Q3 235B, Feedback regime=w/ HITL2026.05 | 39.9 | |
| PIVOTBackbone=Q3 235B, Feedback regime=w/o HITL2026.05 | 39.8 | |
| ReACTBackbone=Haiku 4.52026.05 | 35.2 | |
| Self-CritiqueBackbone=Q3 235B2026.05 | 30.1 | |
| SE-AgentBackbone=Q3 235B2026.05 | 29.6 | |
| ReACTBackbone=Q3 235B2026.05 | 28.3 | |
| AgentDebugBackbone=Q3 235B2026.05 | 25.1 |