Multi-hop Question Answering on Complex-TR ODQA 1.0 (test)
0.312Set AccuracyFiD-PIT + Refine
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| FiD-PIT + RefineModel Backbone=T5-large, Architecture=FiD, Reasoning Enhancement=PIT, Context Refinement=true2023.11 | 0.312 | 0.391 | |
| FiD-PITModel Backbone=T5-large, Architecture=FiD, Reasoning Enhancement=PIT2023.11 | 0.29 | 0.375 | |
| FiDModel Backbone=T5-large, Architecture=FiD2023.11 | 0.274 | 0.373 | |
| PIT-SFTModel Backbone=T5-large, Architecture=SFT, Reasoning Enhancement=PIT2023.11 | 0.253 | 0.328 | |
| FiD-PIT + RefineModel Backbone=T5-base, Architecture=FiD, Reasoning Enhancement=PIT, Context Refinement=true2023.11 | 0.247 | 0.312 | |
| FiD-PITModel Backbone=T5-base, Architecture=FiD, Reasoning Enhancement=PIT2023.11 | 0.231 | 0.302 | |
| SFTModel Backbone=T5-large, Architecture=SFT2023.11 | 0.231 | 0.321 | |
| PIT-SFTModel Backbone=T5-base, Architecture=SFT, Reasoning Enhancement=PIT2023.11 | 0.226 | 0.301 | |
| SFTModel Backbone=T5-base, Architecture=SFT2023.11 | 0.177 | 0.266 | |
| FiDModel Backbone=T5-base, Architecture=FiD2023.11 | 0.177 | 0.26 | |
| FLAN-T5-XLModel Backbone=FLAN-T5-XL2023.11 | 0.145 | 0.203 | |
| GPT-4Model Backbone=GPT-42023.11 | 0.14 | 0.372 | |
| GPT-3.5Model Backbone=GPT-3.52023.11 | 0.097 | 0.23 |