Web Question Answering on WebSRC
19.65ScoreMMICL
Evaluation results
| Method | Links | ||
|---|---|---|---|
| MMICLIn-Context Learning (ICL) setting=w/ ICL examples (4), Base Language Model / Backbone=Instruct-FLAN-T5-XXL2023.09 | 19.65 | — | |
| MMICLIn-Context Learning (ICL) setting=w/o ICL example, Base Language Model / Backbone=FLAN-T5-XXL2023.09 | 18.85 | — | |
| MMICLIn-Context Learning (ICL) setting=w/ ICL examples (4), Base Language Model / Backbone=FLAN-T5-XXL2023.09 | 18.7 | — | |
| MMICLIn-Context Learning (ICL) setting=w/o ICL example, Base Language Model / Backbone=Instruct-FLAN-T5-XXL2023.09 | 17.05 | — | |
| MMICLIn-Context Learning (ICL) setting=w/ ICL examples (4), Base Language Model / Backbone=Instruct-FLAN-T5-XL2023.09 | 14.8 | — | |
| MMICLIn-Context Learning (ICL) setting=w/o ICL example, Base Language Model / Backbone=Instruct-FLAN-T5-XL2023.09 | 14.75 | — | |
| MMICLIn-Context Learning (ICL) setting=w/o ICL example, Base Language Model / Backbone=FLAN-T5-XL2023.09 | 12.55 | — | |
| MMICLIn-Context Learning (ICL) setting=w/ ICL examples (4), Base Language Model / Backbone=FLAN-T5-XL2023.09 | 12.3 | — | |
| BLIP-2In-Context Learning (ICL) setting=w/o ICL example, Base Language Model / Backbone=FLANT5-XL2023.09 | 12.25 | — | |
| InstructBLIPIn-Context Learning (ICL) setting=w/o ICL example, Base Language Model / Backbone=FLANT5-XXL2023.09 | 11.5 | — | |
| InstructBLIPIn-Context Learning (ICL) setting=w/o ICL example, Base Language Model / Backbone=FLANT5-XL2023.09 | 10.8 | — | |
| BLIP-2In-Context Learning (ICL) setting=w/o ICL example, Base Language Model / Backbone=FLANT5-XXL2023.09 | 10.1 | — | |
| KOSMOS-1In-Context Learning (ICL) setting=w/o ICL example2023.09 | 3.8 | — | |
| MarkupLMPre-train (W)=true2022.10 | — | 74.5 | |
| XDocPre-training Steps=100K, Pre-train (P)=true2022.10 | — | 58.5 | |
| XDocPre-training Steps=100K, Pre-train (D)=true2022.10 | — | 58.6 | |
| XDocPre-training Steps=100K, Pre-train (W)=true2022.10 | — | 64.8 | |
| XDocPre-training Steps=100K, Pre-train (P)=true, Pre-train (D)=true2022.10 | — | 57.5 | |
| XDocPre-training Steps=100K, Pre-train (D)=true, Pre-train (W)=true2022.10 | — | 65 | |
| XDocPre-training Steps=100K, Pre-train (P)=true, Pre-train (W)=true2022.10 | — | 64.8 | |
| XDocPre-training Steps=100K, Pre-train (P)=true, Pre-train (D)=true, Pre-train (W)=true2022.10 | — | 67 | |
| XDocPre-training Steps=500K, Pre-train (P)=true, Pre-train (D)=true, Pre-train (W)=true2022.10 | — | 73.3 | |
| XDocPre-training Steps=1M, Pre-train (P)=true, Pre-train (D)=true, Pre-train (W)=true2022.10 | — | 74.8 |