Single-hop Question Answering on Complex-TR ODQA 1.0 (test)
49Set AccuracyFiD-PIT + Refine
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| FiD-PIT + RefineModel Backbone=T5-large, Architecture=FiD, Reasoning Enhancement=PIT, Context Refinement=true2023.11 | 49 | 49.7 | |
| FiD-PITModel Backbone=T5-large, Architecture=FiD, Reasoning Enhancement=PIT2023.11 | 46.9 | 47.8 | |
| FiDModel Backbone=T5-large, Architecture=FiD2023.11 | 46.2 | 46.6 | |
| FiD-PIT + RefineModel Backbone=T5-base, Architecture=FiD, Reasoning Enhancement=PIT, Context Refinement=true2023.11 | 42.7 | 44.3 | |
| PIT-SFTModel Backbone=T5-base, Architecture=SFT, Reasoning Enhancement=PIT2023.11 | 39.9 | 40.8 | |
| FiD-PITModel Backbone=T5-base, Architecture=FiD, Reasoning Enhancement=PIT2023.11 | 39.2 | 40.1 | |
| PIT-SFTModel Backbone=T5-large, Architecture=SFT, Reasoning Enhancement=PIT2023.11 | 39.2 | 40.1 | |
| SFTModel Backbone=T5-large, Architecture=SFT2023.11 | 35 | 35.7 | |
| SFTModel Backbone=T5-base, Architecture=SFT2023.11 | 33.6 | 35.6 | |
| FiDModel Backbone=T5-base, Architecture=FiD2023.11 | 33.6 | 35.6 | |
| FLAN-T5-XLModel Backbone=FLAN-T5-XL2023.11 | 30.1 | 31.5 | |
| GPT-4Model Backbone=GPT-42023.11 | 19.6 | 35.1 | |
| GPT-3.5Model Backbone=GPT-3.52023.11 | 17.5 | 26.3 |