Website Navigation on WebLINX IID 1.0 (test)
37.4Overall ScoreS-LLaMA
Evaluation results
| Method | Links | |
|---|---|---|
| S-LLaMAProtocol=Finetuned, Size=2.7B, Modality=Text-only, Representation=Dense Markup Ranking (DMR)2024.02 | 37.4 | |
| Llama-2Protocol=Finetuned, Size=13B, Modality=Text-only, Representation=Dense Markup Ranking (DMR)2024.02 | 37 | |
| Flan-T5Protocol=Finetuned, Size=3B, Modality=Text-only, Representation=Dense Markup Ranking (DMR)2024.02 | 31.1 | |
| FuyuProtocol=Finetuned, Size=8B, Modality=Multimodal2024.02 | 30.9 | |
| GPT-3.5FProtocol=Finetuned, Modality=Text-only2024.02 | 30.8 | |
| MindActProtocol=Finetuned, Size=3B, Modality=Text-only2024.02 | 25.7 | |
| Pix2ActProtocol=Finetuned, Size=1.3B, Modality=Image-to-text2024.02 | 23.9 | |
| GPT-4VProtocol=Zero-shot, Modality=Multimodal2024.02 | 12.9 | |
| GPT-4TProtocol=Zero-shot, Modality=Text-only2024.02 | 12.2 | |
| GPT-3.5TProtocol=Zero-shot, Modality=Text-only2024.02 | 10.3 | |
| Llama-2Protocol=Zero-shot, Size=13B, Modality=Text-only2024.02 | 5.6 |