Widget Captioning on Widget Captioning (test)
159.8CIDErPaLI-3
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| PaLI-3OCR Usage=Without OCR2024.02 | 159.8 | — | — | — | — | — | — | |
| ScreenAIOCR Usage=Without OCR2024.02 | 156.4 | — | — | — | — | — | — | |
| Pix2Struct LargePretraining=Screenshot parsing, Pixel only=true2022.10 | 136.7 | — | — | — | — | — | — | |
| Pix2Struct BasePretraining=Screenshot parsing, Pixel only=true2022.10 | 133.1 | — | — | — | — | — | — | |
| DonutPretraining=OCR, Pixel only=true2022.10 | 127.4 | — | — | — | — | — | — | |
| Learnable-Fourier+MLPdimension_split=4/12021.06 | 100.7 | 46.8 | 33.4 | 46.1 | 32.4 | 17.8 | — | |
| Fixed-Fourier+MLPdimension_split=2/22021.06 | 100.2 | 46.1 | 32.5 | 45.8 | 32.5 | 18.4 | — | |
| Learnable-Fourier+MLPdimension_split=1/42021.06 | 99.1 | 45.6 | 32.7 | 45.2 | 32.2 | 17.1 | — | |
| Learnable-Fourier+MLPdimension_split=2/22021.06 | 98 | 46.1 | 32.7 | 45.9 | 32.6 | 17.9 | — | |
| Fixed-Fourier+MLPdimension_split=4/12021.06 | 97.2 | 45.5 | 32.1 | 45.1 | 31.7 | 17.6 | — | |
| SOTA2021.06 | 97 | 44.9 | 32.2 | 44.7 | 31.7 | 17.6 | — | |
| Embed-4Ddescription=reproduction of SOTA [20]2021.06 | 97 | 45.2 | 31.9 | 45 | 31.7 | 17.3 | — | |
| VUTtype=State of the art w/ pipelines2022.10 | 97 | — | — | — | — | — | — | |
| Fixed-Fourier+MLPdimension_split=1/42021.06 | 95.4 | 45 | 32.1 | 44.2 | 31.2 | 17.1 | — | |
| Learnable-Fourierdimension_split=2/22021.06 | 95.3 | 44.9 | 31.6 | 44.3 | 31.6 | 17.7 | — | |
| Sine-4D2021.06 | 94.9 | 44.9 | 31.9 | 43.9 | 31 | 16.7 | — | |
| MLP2021.06 | 70.3 | 34 | 23.5 | 33.7 | 23.7 | 10.2 | — | |
| Ferret-UI-13B2024.09 | — | — | — | — | — | — | 142 | |
| MM1.5-UI-1B2024.09 | — | — | — | — | — | — | 144.4 | |
| MM1.5-UI-30B2024.09 | — | — | — | — | — | — | 145.9 | |
| MM1.5-UI-3B2024.09 | — | — | — | — | — | — | 145 | |
| MM1.5-UI-3BTraining epochs=1 ep.2024.09 | — | — | — | — | — | — | 145.2 | |
| MM1.5-UI-3BTraining epochs=1 ep., SFT strategy=w/o MM1.5 SFT2024.09 | — | — | — | — | — | — | 139.5 | |
| MM1.5-UI-7B2024.09 | — | — | — | — | — | — | 149.7 | |
| PaliGemma-3BFine-tuning protocol=per-task fine-tuning2024.09 | — | — | — | — | — | — | 148.4 | |
| Spotlight2024.09 | — | — | — | — | — | — | 141.8 |