List your company
ResearchFormerly WizWand

Event Relation Extraction on MAVEN-ERE 1.0 (test)

56.8Micro F1ROBERTa-Large
7.9220.6133.345.99Oct 13, 2023
Updated 8mo ago

Evaluation results

MethodLinks
ROBERTa-Largemode=fully fine-tuned2023.1056.86.4
GPT-4prompting=w. all logical constraints2023.1037.38.3
GPT-4prompting=w. retrieved logical constraints2023.1033.528.8
GPT-4prompting=CoT w. logical constraints2023.1032.313.7
GPT-4prompting=vanilla CoT2023.1030.336.7
GPT-4prompting=vanilla ICL2023.1029.350.7
DavinciBackbone=text-davinci-003, prompting=w. retrieved logical constraints2023.1027.830.8
DavinciBackbone=text-davinci-003, prompting=w. all logical constraints2023.102725.6
Llama2-FTBackbone=Llama2-13B, Finetuning=LLM-ERL, prompting=CoT w. logical constraints2023.1026.415.7
TurboBackbone=gpt-3.5-turbo, prompting=CoT w. logical constraints2023.1025.337.9
DavinciBackbone=text-davinci-003, prompting=CoT w. logical constraints2023.1024.85.5
TurboBackbone=gpt-3.5-turbo, prompting=w. retrieved logical constraints2023.1022.330.2
Llama2-FTBackbone=Llama2-13B, Finetuning=LLM-ERL, prompting=vanilla CoT2023.1022.142.9
DavinciBackbone=text-davinci-003, prompting=vanilla ICL2023.1021.649.1
Llama2Backbone=Llama2-13B, prompting=CoT w. logical constraints2023.1021.518.9
TurboBackbone=gpt-3.5-turbo, prompting=w. all logical constraints2023.1020.830.9
DavinciBackbone=text-davinci-003, prompting=vanilla CoT2023.1020.560.5
Llama2-FTBackbone=Llama2-13B, Finetuning=LLM-ERL, prompting=w. all logical constraints2023.1020.228.7
Llama2Backbone=Llama2-13B, prompting=w. all logical constraints2023.1019.534.6
Llama2-FTBackbone=Llama2-13B, Finetuning=LLM-ERL, prompting=vanilla ICL2023.101945.8
TurboBackbone=gpt-3.5-turbo, prompting=vanilla CoT2023.1018.849.3
Llama2-FTBackbone=Llama2-13B, Finetuning=LLM-ERL, prompting=w. retrieved logical constraints2023.1018.734.2
Llama2Backbone=Llama2-13B, prompting=w. retrieved logical constraints2023.1018.338.2
TurboBackbone=gpt-3.5-turbo, prompting=vanilla ICL2023.101853.3
Vicuna-FTBackbone=Vicuna-13B, Finetuning=LLM-ERL, prompting=CoT w. logical constraints2023.10186
Llama2Backbone=Llama2-13B, prompting=vanilla CoT2023.1017.858.4
ROBERTa-Largemode=one-shot2023.1017.454.8
GPT-4prompting=w. post-processing2023.10170
Llama2Backbone=Llama2-13B, prompting=vanilla ICL2023.101754.6
Vicuna-FTBackbone=Vicuna-13B, Finetuning=LLM-ERL, prompting=w. all logical constraints2023.1016.38.7
Vicuna-FTBackbone=Vicuna-13B, Finetuning=LLM-ERL, prompting=w. retrieved logical constraints2023.1016.119
Vicuna-FTBackbone=Vicuna-13B, Finetuning=LLM-ERL, prompting=vanilla CoT2023.1015.817.8
VicunaBackbone=Vicuna-13B, prompting=w. retrieved logical constraints2023.1015.733.2
Vicuna-FTBackbone=Vicuna-13B, Finetuning=LLM-ERL, prompting=vanilla ICL2023.1015.321.2
VicunaBackbone=Vicuna-13B, prompting=w. all logical constraints2023.1015.237.6
VicunaBackbone=Vicuna-13B, prompting=CoT w. logical constraints2023.1014.921.7
DavinciBackbone=text-davinci-003, prompting=w. post-processing2023.1014.80
TurboBackbone=gpt-3.5-turbo, prompting=w. post-processing2023.10140
VicunaBackbone=Vicuna-13B, prompting=vanilla ICL2023.1013.825.4
Llama2Backbone=Llama2-13B, prompting=w. post-processing2023.10120
VicunaBackbone=Vicuna-13B, prompting=vanilla CoT2023.1011.647.4
Vicuna-FTBackbone=Vicuna-13B, Finetuning=LLM-ERL, prompting=w. post-processing2023.10110
Llama2-FTBackbone=Llama2-13B, Finetuning=LLM-ERL, prompting=w. post-processing2023.10110
VicunaBackbone=Vicuna-13B, prompting=w. post-processing2023.109.80