Event Relation Extraction on Causal-TimeBank 1.0 (test)
31Micro F1Davinci
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| DavinciBackbone=text-davinci-003, prompting=w. all logical constraints2023.10 | 31 | 21.8 | |
| TurboBackbone=gpt-3.5-turbo, prompting=CoT w. logical constraints2023.10 | 27 | 12.8 | |
| GPT-4prompting=w. all logical constraints2023.10 | 26 | 20 | |
| GPT-4prompting=CoT w. logical constraints2023.10 | 24.5 | 24 | |
| GPT-4prompting=w. retrieved logical constraints2023.10 | 24 | 13.5 | |
| DavinciBackbone=text-davinci-003, prompting=CoT w. logical constraints2023.10 | 23 | 39.2 | |
| GPT-4prompting=vanilla CoT2023.10 | 23 | 35 | |
| GPT-4prompting=vanilla ICL2023.10 | 22.5 | 30.5 | |
| ROBERTa-Largemode=fully fine-tuned2023.10 | 22.2 | 36.2 | |
| TurboBackbone=gpt-3.5-turbo, prompting=w. retrieved logical constraints2023.10 | 22 | 11.3 | |
| DavinciBackbone=text-davinci-003, prompting=w. retrieved logical constraints2023.10 | 22 | 40.5 | |
| DavinciBackbone=text-davinci-003, prompting=vanilla CoT2023.10 | 21 | 64.7 | |
| TurboBackbone=gpt-3.5-turbo, prompting=w. all logical constraints2023.10 | 20 | 36.8 | |
| TurboBackbone=gpt-3.5-turbo, prompting=vanilla ICL2023.10 | 19 | 54 | |
| DavinciBackbone=text-davinci-003, prompting=w. post-processing2023.10 | 19 | 0 | |
| GPT-4prompting=w. post-processing2023.10 | 19 | 0 | |
| DavinciBackbone=text-davinci-003, prompting=vanilla ICL2023.10 | 18 | 58.8 | |
| TurboBackbone=gpt-3.5-turbo, prompting=vanilla CoT2023.10 | 17 | 30.3 | |
| TurboBackbone=gpt-3.5-turbo, prompting=w. post-processing2023.10 | 15 | 0 | |
| Llama2-FTBackbone=Llama2-13B, Finetuning=LLM-ERL, prompting=CoT w. logical constraints2023.10 | 13.3 | 13 | |
| Llama2Backbone=Llama2-13B, prompting=CoT w. logical constraints2023.10 | 13 | 18.1 | |
| Vicuna-FTBackbone=Vicuna-13B, Finetuning=LLM-ERL, prompting=w. all logical constraints2023.10 | 12.1 | 0 | |
| Llama2-FTBackbone=Llama2-13B, Finetuning=LLM-ERL, prompting=vanilla ICL2023.10 | 12 | 22.7 | |
| Llama2-FTBackbone=Llama2-13B, Finetuning=LLM-ERL, prompting=w. all logical constraints2023.10 | 12 | 23 | |
| Llama2Backbone=Llama2-13B, prompting=vanilla ICL2023.10 | 11.5 | 26.7 | |
| Llama2-FTBackbone=Llama2-13B, Finetuning=LLM-ERL, prompting=vanilla CoT2023.10 | 11.5 | 3 | |
| VicunaBackbone=Vicuna-13B, prompting=w. all logical constraints2023.10 | 11 | 23.5 | |
| Llama2-FTBackbone=Llama2-13B, Finetuning=LLM-ERL, prompting=w. retrieved logical constraints2023.10 | 11 | 19.4 | |
| Llama2-FTBackbone=Llama2-13B, Finetuning=LLM-ERL, prompting=w. post-processing2023.10 | 11 | 0 | |
| Vicuna-FTBackbone=Vicuna-13B, Finetuning=LLM-ERL, prompting=w. retrieved logical constraints2023.10 | 10.7 | 9.5 | |
| Llama2Backbone=Llama2-13B, prompting=vanilla CoT2023.10 | 10.5 | 33.6 | |
| VicunaBackbone=Vicuna-13B, prompting=w. retrieved logical constraints2023.10 | 10 | 26.7 | |
| Llama2Backbone=Llama2-13B, prompting=w. all logical constraints2023.10 | 10 | 23.5 | |
| Llama2Backbone=Llama2-13B, prompting=w. retrieved logical constraints2023.10 | 9.5 | 26.7 | |
| Llama2Backbone=Llama2-13B, prompting=w. post-processing2023.10 | 9.5 | 0 | |
| VicunaBackbone=Vicuna-13B, prompting=w. post-processing2023.10 | 9 | 0 | |
| Vicuna-FTBackbone=Vicuna-13B, Finetuning=LLM-ERL, prompting=CoT w. logical constraints2023.10 | 8.5 | 2 | |
| VicunaBackbone=Vicuna-13B, prompting=CoT w. logical constraints2023.10 | 8 | 33.1 | |
| Vicuna-FTBackbone=Vicuna-13B, Finetuning=LLM-ERL, prompting=vanilla ICL2023.10 | 8 | 35.5 | |
| Vicuna-FTBackbone=Vicuna-13B, Finetuning=LLM-ERL, prompting=w. post-processing2023.10 | 8 | 0 | |
| Vicuna-FTBackbone=Vicuna-13B, Finetuning=LLM-ERL, prompting=vanilla CoT2023.10 | 7.5 | 52.5 | |
| VicunaBackbone=Vicuna-13B, prompting=vanilla CoT2023.10 | 6 | 57.6 | |
| VicunaBackbone=Vicuna-13B, prompting=vanilla ICL2023.10 | 4.5 | 84.1 |