Action Prediction on MIND2WEB Cross-Website 1.0
47.6Element AccuracyHTML-T5-XL
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| HTML-T5-XLTrain=SL2023.07 | 47.6 | 71.9 | 42.9 | — | |
| MindActBackbone=Flan-T5-XL2023.06 | 42 | 65.2 | 38.9 | 5.1 | |
| MindAct (Flan-T5-XL)Train=SL, LLM=Flan-T5-XL2023.07 | 42 | 65.2 | 38.9 | — | |
| MindActBackbone=Flan-T5-Large2023.06 | 39.2 | 67.1 | 35.3 | 1.1 | |
| MindActBackbone=GPT-4, Tasks=50, Candidates=Top-102023.06 | 35.8 | 51.1 | 30.1 | 2 | |
| MindAct (GPT-4)Train=ICL, LLM=GPT-42023.07 | 35.8 | 51.1 | 30.1 | — | |
| MindActBackbone=Flan-T5-Base2023.06 | 32.1 | 67.6 | 29.5 | 1.7 | |
| Synapse (GPT-3.5)Train=ICL, LLM=GPT-3.52023.07 | 28.8 | — | 23.4 | — | |
| ClassificationBackbone=DeBERTa-Base2023.06 | 21.6 | — | — | — | |
| MindActBackbone=GPT-3.52023.06 | 19.3 | 48.8 | 16.2 | 0.6 | |
| GenerationBackbone=Flan-T5-Base2023.06 | 13.9 | 44.7 | 11 | 0 |