Relation Extraction on TACREV (test)
88.3F1 ScoreRAG4RE
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| RAG4REbackbone=Flan T5-XL2024.04 | 88.3 | — | — | |
| KLG2024.04 | 84.1 | — | — | |
| GAP2024.04 | 82.7 | — | — | |
| SimpleREBackbone=BERT2020.12 | 80.7 | — | — | |
| GDPNetBackbone=SpanBERT2020.12 | 80.2 | — | — | |
| KnowBERTKnowledge Source=External Knowledge Base2020.12 | 79.3 | — | — | |
| SpanBERT2020.12 | 78 | — | — | |
| C-AGGCN2020.12 | 75.5 | — | — | |
| PA-LSTM2020.12 | 74.3 | — | — | |
| LSTM2020.12 | 70.6 | — | — | |
| text-003Model=text-davinci-003, Framework=QA4RE, Evaluation Protocol=Zero-shot2023.05 | 59.4 | 46 | 83.6 | |
| FLAN-T5 XLargeModel Size=3B, Framework=Vanilla RE, Evaluation Protocol=Zero-shot2023.05 | 57.5 | 56 | 59.1 | |
| NLIDEBERTAEvaluation Protocol=Zero-shot2023.05 | 57.2 | 43.3 | 84.6 | |
| NLIBARTEvaluation Protocol=Zero-shot2023.05 | 55.3 | 44 | 74.6 | |
| FLAN-T5 XLargeModel Size=3B, Framework=QA4RE, Evaluation Protocol=Zero-shot2023.05 | 55.3 | 40.7 | 85.9 | |
| FLAN-T5 XXLargeModel Size=11B, Framework=QA4RE, Evaluation Protocol=Zero-shot2023.05 | 54.3 | 39.6 | 86.4 | |
| FLAN-T5 XXLargeModel Size=11B, Framework=Vanilla RE, Evaluation Protocol=Zero-shot2023.05 | 51.7 | 52.6 | 50.9 | |
| NLIROBERTAEvaluation Protocol=Zero-shot2023.05 | 51.4 | 37.1 | 83.6 | |
| text-003Model=text-davinci-003, Framework=Vanilla RE, Evaluation Protocol=Zero-shot2023.05 | 51 | 38.2 | 76.8 | |
| text-002Model=text-davinci-002, Framework=QA4RE, Evaluation Protocol=Zero-shot2023.05 | 48.8 | 35.7 | 76.8 | |
| ChatGPTModel=gpt-3.5-turbo-0301, Framework=QA4RE, Evaluation Protocol=Zero-shot2023.05 | 48.2 | 34.7 | 79.1 | |
| ChatGPTModel=gpt-3.5-turbo-0301, Framework=Vanilla RE, Evaluation Protocol=Zero-shot2023.05 | 43.9 | 30.3 | 79.6 | |
| text-002Model=text-davinci-002, Framework=Vanilla RE, Evaluation Protocol=Zero-shot2023.05 | 43.3 | 30.2 | 76.8 | |
| code-002Model=code-davinci-002, Framework=QA4RE, Evaluation Protocol=Zero-shot2023.05 | 42.7 | 31.7 | 65.5 | |
| code-002Model=code-davinci-002, Framework=Vanilla RE, Evaluation Protocol=Zero-shot2023.05 | 40.6 | 27.5 | 77.7 | |
| SUREBARTEvaluation Protocol=Zero-shot2023.05 | 22.2 | 14.1 | 52.3 | |
| SUREPEGASUSEvaluation Protocol=Zero-shot2023.05 | 21.6 | 13.5 | 54.1 |