Multimodal Agent Evaluation on OSWorld
0.73Pearson rADARUBRIC-DA (visual)
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| ADARUBRIC-DA (visual)Modality=visual, Fine-tuning Base Model=Llama-3.1-8B-Instruct2026.03 | 0.73 | 22.8 | |
| ADARUBRIC-WM (visual)Modality=visual, Fine-tuning Base Model=Llama-3.1-8B-Instruct2026.03 | 0.7 | 21.1 | |
| ADARUBRIC-WM (text)Modality=text, Fine-tuning Base Model=Llama-3.1-8B-Instruct2026.03 | 0.65 | 19.4 | |
| GPT-4V Direct2026.03 | 0.59 | 17.6 | |
| Prometheus2026.03 | 0.53 | 16.3 | |
| G-Eval (text only)Modality=text only2026.03 | 0.44 | 14.8 |