Relation Extraction on RETACRED (test)
71.7PrecisionNLIDEBERTA
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| NLIDEBERTAEvaluation Protocol=Zero-shot2023.05 | 71.7 | 58.3 | 64.3 | |
| NLIBARTEvaluation Protocol=Zero-shot2023.05 | 59.5 | 34.9 | 44 | |
| FLAN-T5 XLargeModel Size=3B, Framework=QA4RE, Evaluation Protocol=Zero-shot2023.05 | 57.1 | 79.7 | 66.5 | |
| FLAN-T5 XXLargeModel Size=11B, Framework=Vanilla RE, Evaluation Protocol=Zero-shot2023.05 | 56.6 | 54 | 55.2 | |
| FLAN-T5 XXLargeModel Size=11B, Framework=QA4RE, Evaluation Protocol=Zero-shot2023.05 | 56.6 | 82.9 | 67.3 | |
| text-003Model=text-davinci-003, Framework=QA4RE, Evaluation Protocol=Zero-shot2023.05 | 56.2 | 67.2 | 61.2 | |
| FLAN-T5 XLargeModel Size=3B, Framework=Vanilla RE, Evaluation Protocol=Zero-shot2023.05 | 54.3 | 40.3 | 46.3 | |
| NLIROBERTAEvaluation Protocol=Zero-shot2023.05 | 52.3 | 67 | 58.7 | |
| text-003Model=text-davinci-003, Framework=Vanilla RE, Evaluation Protocol=Zero-shot2023.05 | 49.7 | 62.2 | 55.3 | |
| ChatGPTModel=gpt-3.5-turbo-0301, Framework=QA4RE, Evaluation Protocol=Zero-shot2023.05 | 48.3 | 76.8 | 59.3 | |
| code-002Model=code-davinci-002, Framework=QA4RE, Evaluation Protocol=Zero-shot2023.05 | 48 | 74 | 58.2 | |
| text-002Model=text-davinci-002, Framework=QA4RE, Evaluation Protocol=Zero-shot2023.05 | 46.4 | 72.4 | 56.5 | |
| ChatGPTModel=gpt-3.5-turbo-0301, Framework=Vanilla RE, Evaluation Protocol=Zero-shot2023.05 | 45.4 | 61.3 | 52.1 | |
| text-002Model=text-davinci-002, Framework=Vanilla RE, Evaluation Protocol=Zero-shot2023.05 | 44.1 | 76.3 | 55.9 | |
| code-002Model=code-davinci-002, Framework=Vanilla RE, Evaluation Protocol=Zero-shot2023.05 | 42.7 | 70.4 | 53.1 | |
| SUREBARTEvaluation Protocol=Zero-shot2023.05 | 17.9 | 34.6 | 23.6 | |
| SUREPEGASUSEvaluation Protocol=Zero-shot2023.05 | 16.6 | 34.6 | 22.4 |