Web UI Navigation on VisualWebArena 72 easy tasks 1.0 (test)
78.4ASRClaude 3.5 Sonnet
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Claude 3.5 SonnetObservation Modality=SoM, Action Step Limit=30, Decoding Temperature=02024.11 | 78.4 | 47.2 | 54.2 | |
| GPT-4-TurboObservation Modality=SoM, Action Step Limit=30, Decoding Temperature=02024.11 | 78 | 43.1 | 50 | |
| Claude 3.5 Sonnet v2Observation Modality=SoM, Action Step Limit=30, Decoding Temperature=02024.11 | 76.8 | 48.6 | 50 | |
| Gemini 1.5Observation Modality=SoM, Action Step Limit=30, Decoding Temperature=02024.11 | 70.1 | 44.4 | 48.6 | |
| GPT-4oObservation Modality=SoM, Action Step Limit=30, Decoding Temperature=02024.11 | 62.1 | 45.8 | 63.9 |