Graph algorithm reasoning on IsoBench Graph
96.8MaxflowGPT-4 Turbo + Sketchpad
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| GPT-4 Turbo + SketchpadVisual Input=true, Sketchpad=true2024.06 | 96.8 | 97.6 | 97.6 | |
| GPT-4o + SketchpadVisual Input=true, Sketchpad=true2024.06 | 66.3 | 65.3 | 98.4 | |
| Claude 3 OPUSVisual Input=false2024.06 | 50 | 82 | 93 | |
| GPT-4 TurboVisual Input=true, Sketchpad=false2024.06 | 32.8 | 62.5 | 66 | |
| GPT-4oVisual Input=true, Sketchpad=false2024.06 | 25 | 50.8 | 96.1 | |
| LLaMA-2-70BVisual Input=false2024.06 | 18 | 50 | 50 | |
| Gemini-ProVisual Input=false2024.06 | 15.6 | 47.7 | 50 | |
| Mixtral 8x7BVisual Input=false2024.06 | 8.6 | 50 | 62.5 |