Event-Event Temporal Reasoning (L3) on TEMPREASON 1.0 (test)
8,110EMTempT5
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| TempT5Setting=ReasonQA2023.06 | 8,110 | 86.1 | 3.1 | |
| T5-SFTSetting=ReasonQA2023.06 | 7,820 | 83 | — | |
| ChatGPTSetting=ReasonQA2023.06 | 4,950 | 52.3 | — | |
| FLAN-T5-LSetting=ReasonQA2023.06 | 3,630 | 47.5 | — | |
| TempT5Setting=OBQA2023.06 | 2,110 | 32.4 | 1.2 | |
| T5-SFTSetting=OBQA2023.06 | 1,970 | 31.2 | — | |
| ChatGPTSetting=OBQA2023.06 | 1,700 | 25.3 | — | |
| TempT5Setting=CBQA2023.06 | 1,230 | 25.4 | 0.1 | |
| T5-SFTSetting=CBQA2023.06 | 1,210 | 25.3 | — | |
| ChatGPTSetting=CBQA2023.06 | 1,200 | 21.8 | — | |
| FLAN-T5-LSetting=OBQA2023.06 | 810 | 19.2 | — | |
| FLAN-T5-LSetting=CBQA2023.06 | 40 | 10.5 | — |