Time-Event Temporal Reasoning (L2) on TEMPREASON 1.0 (test)
8,480EMTempT5
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| TempT5Setting=ReasonQA2023.06 | 8,480 | 88.9 | 1.8 | |
| T5-SFTSetting=ReasonQA2023.06 | 8,260 | 87.1 | — | |
| FLAN-T5-LSetting=ReasonQA2023.06 | 5,730 | 66.3 | — | |
| ChatGPTSetting=ReasonQA2023.06 | 4,750 | 51 | — | |
| TempT5Setting=OBQA2023.06 | 1,540 | 36.3 | 1.1 | |
| T5-SFTSetting=OBQA2023.06 | 1,480 | 35.2 | — | |
| FLAN-T5-LSetting=OBQA2023.06 | 940 | 22.5 | — | |
| ChatGPTSetting=OBQA2023.06 | 850 | 16.1 | — | |
| ChatGPTSetting=CBQA2023.06 | 650 | 11.5 | — | |
| TempT5Setting=CBQA2023.06 | 150 | 23.4 | 0.2 | |
| T5-SFTSetting=CBQA2023.06 | 140 | 23.2 | — | |
| FLAN-T5-LSetting=CBQA2023.06 | 50 | 9.2 | — |