Website Navigation on WebLINX OOD 1.0 (test)
84IMS-LLaMA
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| S-LLaMAProtocol=Finetuned, Size=2.7B, Modality=Text-only, Representation=Dense Markup Ranking (DMR)2024.02 | 84 | 22.6 | 27.2 | 25 | |
| Llama-2Protocol=Finetuned, Size=13B, Modality=Text-only, Representation=Dense Markup Ranking (DMR)2024.02 | 83 | 22.8 | 26.6 | 25.2 | |
| Pix2ActProtocol=Finetuned, Size=1.3B, Modality=Image-to-text2024.02 | 81.8 | 8.3 | 25.2 | 16.9 | |
| Flan-T5Protocol=Finetuned, Size=3B, Modality=Text-only, Representation=Dense Markup Ranking (DMR)2024.02 | 81.1 | 20.3 | 25.8 | 23.8 | |
| FuyuProtocol=Finetuned, Size=8B, Modality=Multimodal2024.02 | 80.1 | 15.7 | 22.3 | 20 | |
| MindActProtocol=Finetuned, Size=3B, Modality=Text-only2024.02 | 79.9 | 16.5 | 23.2 | 20.9 | |
| GPT-3.5FProtocol=Finetuned, Modality=Text-only2024.02 | 77.6 | 18.6 | 22.4 | 21.2 | |
| Llama-2Protocol=Zero-shot, Size=13B, Modality=Text-only2024.02 | 43.7 | 4.8 | 1.3 | 5.2 | |
| GPT-3.5TProtocol=Zero-shot, Modality=Text-only2024.02 | 42.8 | 8.6 | 3.5 | 8.5 | |
| GPT-4VProtocol=Zero-shot, Modality=Multimodal2024.02 | 42.4 | 10.9 | 6.2 | 10.4 | |
| GPT-4TProtocol=Zero-shot, Modality=Text-only2024.02 | 41.7 | 10.9 | 6.8 | 10.7 |