Shopping Planning on Shopping Planning
98.3Case AccuracyPIVOT
Evaluation results
| Method | Links | |
|---|---|---|
| PIVOTBackbone=Opus 4.6, Feedback regime=w/ HITL2026.05 | 98.3 | |
| PIVOTBackbone=Opus 4.5, Feedback regime=w/ HITL2026.05 | 94.2 | |
| PIVOTBackbone=Haiku 4.5, Feedback regime=w/ HITL2026.05 | 89.2 | |
| PIVOTBackbone=Sonnet 4.5, Feedback regime=w/ HITL2026.05 | 89.2 | |
| PIVOTBackbone=Q3 235B, Feedback regime=w/ HITL2026.05 | 76.7 | |
| Self-CritiqueBackbone=Opus 4.62026.05 | 70 | |
| Self-CritiqueBackbone=Sonnet 4.52026.05 | 55.8 | |
| PIVOTBackbone=Opus 4.6, Feedback regime=w/o HITL2026.05 | 55.8 | |
| SE-AgentBackbone=Opus 4.62026.05 | 53.7 | |
| Self-CritiqueBackbone=Opus 4.52026.05 | 53.3 | |
| AgentDebugBackbone=Opus 4.62026.05 | 53.3 | |
| ReACTBackbone=Opus 4.62026.05 | 50.8 | |
| PIVOTBackbone=Opus 4.5, Feedback regime=w/o HITL2026.05 | 48.3 | |
| AgentDebugBackbone=Sonnet 4.52026.05 | 47.5 | |
| SE-AgentBackbone=Sonnet 4.52026.05 | 45.8 | |
| AgentDebugBackbone=Opus 4.52026.05 | 45.8 | |
| SE-AgentBackbone=Opus 4.52026.05 | 43.7 | |
| ReACTBackbone=Opus 4.52026.05 | 43.3 | |
| PIVOTBackbone=Sonnet 4.5, Feedback regime=w/o HITL2026.05 | 43.3 | |
| ReACTBackbone=Sonnet 4.52026.05 | 40.8 | |
| PIVOTBackbone=Haiku 4.5, Feedback regime=w/o HITL2026.05 | 37.5 | |
| Self-CritiqueBackbone=Haiku 4.52026.05 | 35.8 | |
| AgentDebugBackbone=Haiku 4.52026.05 | 33.3 | |
| SE-AgentBackbone=Haiku 4.52026.05 | 31.7 | |
| ReACTBackbone=Haiku 4.52026.05 | 25.8 | |
| PIVOTBackbone=Q3 235B, Feedback regime=w/o HITL2026.05 | 20 | |
| Self-CritiqueBackbone=Q3 235B2026.05 | 16.7 | |
| SE-AgentBackbone=Q3 235B2026.05 | 15.8 | |
| AgentDebugBackbone=Q3 235B2026.05 | 15 | |
| ReACTBackbone=Q3 235B2026.05 | 10.8 |