Binary temporal relation classification on TRACIE, MATRES, and TODAY (test)
79.9TRACIE AccuracyPatternTime (all)
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| PatternTime (all)Train Data=MATRES + TRACIE + TODAY + GPT-3.5, Loss=CE + MR2022.12 | 79.9 | 86.3 | 62.9 | 82.3 | 76.4 | |
| PatternTime (M+T+O)Train Data=MATRES + TRACIE + TODAY, Loss=CE + MR2022.12 | 79.8 | 85.8 | 60.9 | 82.2 | 75.5 | |
| PatternTime (M+T)Train Data=MATRES + TRACIE, Loss=CE2022.12 | 79.7 | 85 | 56.3 | 66.5 | 73.7 | |
| PatternTimeTrain Data=Distant, Loss=Distant2022.12 | 77 | 73 | 54.1 | 67.7 | 68 | |
| T5 (T+O+G)Train Data=TRACIE + TODAY + GPT-3.5, Loss=CE + MR2022.12 | 73.5 | 68.8 | 62.1 | 82 | 68.1 | |
| T5 (M+T+O+G)Train Data=MATRES + TRACIE + TODAY + GPT-3.5, Loss=CE + MR2022.12 | 73.3 | 83.9 | 63.2 | 81.6 | 73.5 | |
| T5 (M+T+O)Train Data=MATRES + TRACIE + TODAY, Loss=CE + MR2022.12 | 73 | 83.5 | 57.9 | 77.8 | 71.5 | |
| T5 (T+O)Train Data=TRACIE + TODAY, Loss=CE + MR2022.12 | 72.9 | 69.4 | 59.9 | 81.6 | 67.4 | |
| T5 (in-domain)Loss=CE/MR2022.12 | 66.2 | 81.2 | 52.9 | 55.7 | 66.8 | |
| T5 (T)Train Data=TRACIE, Loss=CE2022.12 | 66.2 | 63.2 | 52.3 | 56 | 60.7 | |
| T5 (M+T)Train Data=MATRES + TRACIE, Loss=CE2022.12 | 66.2 | 82 | 52.5 | 58.5 | 66.9 | |
| GPT-3.5 text-davinci-002Train Data=FewShot, Loss=FewShot2022.12 | 56.1 | 49 | 57.9 | 68.7 | 54.3 | |
| T5 (O+G)Train Data=TODAY + GPT-3.5-generated incidental supervision, Loss=MR2022.12 | 55.4 | 52.3 | 55 | 66.5 | 54.2 | |
| T5 (M)Train Data=MATRES, Loss=CE2022.12 | 52.7 | 81.2 | 52.5 | 57.5 | 62.1 | |
| GPT-3.5 text-davinci-003Train Data=FewShot, Loss=FewShot2022.12 | 52.3 | 50.1 | 59 | 70 | 53.8 | |
| T5 (M+O)Train Data=MATRES + TODAY, Loss=CE + MR2022.12 | 51.5 | 81.7 | 57.4 | 82.7 | 63.5 | |
| T5 (O)Train Data=TODAY (ours), Loss=MR2022.12 | 50.6 | 49.8 | 52.9 | 55.7 | 51.1 | |
| T5 (M+O+G)Train Data=MATRES + TODAY + GPT-3.5, Loss=CE + MR2022.12 | 49.9 | 82.9 | 61.4 | 82.9 | 64.8 |