Relation Extraction on Re-DocRED (test)
83.11Ignored F1TTM-RE
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| TTM-RETraining Setting=Human + Distant2024.06 | 83.11 | 84.01 | 86.03 | 82.09 | — | |
| TTM-RETraining Setting=Human + Distant, Variant=PU2024.06 | 80.99 | 79.24 | 78.9 | 80.49 | — | |
| KD-DocRETraining Setting=Human + Distant2024.06 | 80.32 | 80.62 | 83.58 | 75.06 | — | |
| DREEAMTraining Setting=Human + Distant2024.06 | 78.95 | 81.67 | 75.72 | 84.88 | — | |
| SSR-PUTraining Setting=Human + Distant2024.06 | 78.84 | 80.52 | 74.24 | 87.96 | — | |
| SSR-PUTraining Setting=Human Annotation Only2024.06 | 78.61 | 80.18 | 69.43 | 90.5 | — | |
| DREEAMTraining Setting=Human Annotation Only2024.06 | 78.56 | 80.2 | 75.74 | 83.89 | — | |
| TTM-RETraining Setting=Human Annotation Only2024.06 | 78.2 | 79.95 | 85.81 | 76.68 | — | |
| KD-DocRETraining Setting=Human Annotation Only2024.06 | 77.6 | 78.28 | 89.76 | 69.4 | — | |
| ATLOPTraining Setting=Human Annotation Only2024.06 | 76.13 | 77.81 | 67.76 | 85.35 | — | |
| ATLOPTraining Setting=Human + Distant2024.06 | 75.72 | 77.31 | 69.92 | 84.1 | — | |
| MTB + CEEncoder=MTB, Objective=CE, Training Data=100%2022.12 | 69.9 | 71.2 | — | — | — | |
| PLM + CEEncoder=PLM, Objective=CE, Training Data=100%2022.12 | 69.4 | 70.9 | — | — | — | |
| MTB + MCCLEncoder=MTB, Objective=MCCL, Training Data=100%2022.12 | 67.9 | 69.3 | — | — | — | |
| PLM + MCCLEncoder=PLM, Objective=MCCL, Training Data=100%2022.12 | 66.7 | 68.2 | — | — | — | |
| MTB + MCCLEncoder=MTB, Objective=MCCL, Training Data=10%2022.12 | 62.1 | 63.8 | — | — | — | |
| PLM + MCCLEncoder=PLM, Objective=MCCL, Training Data=10%2022.12 | 61.8 | 63.2 | — | — | — | |
| TTM-RETraining Setting=Distant Only2024.06 | 61.55 | 63 | 67.56 | 59.02 | — | |
| MTB + CEEncoder=MTB, Objective=CE, Training Data=10%2022.12 | 61.4 | 62.9 | — | — | — | |
| PLM + CEEncoder=PLM, Objective=CE, Training Data=10%2022.12 | 60.3 | 61.3 | — | — | — | |
| SSR-PU+ATLOP+RoBERTaBackbone=RoBERTa-Large2022.10 | 58.68 | 59.5 | 74.21 | 49.67 | — | |
| MTB + MCCLEncoder=MTB, Objective=MCCL, Training Data=5%2022.12 | 58.2 | 59.7 | — | — | — | |
| S-PU+ATLOP+RoBERTaBackbone=RoBERTa-Large2022.10 | 58.19 | 58.95 | 75.68 | 48.29 | — | |
| SR-PU+ATLOP+RoBERTaBackbone=RoBERTa-Large2022.10 | 57.64 | 58.77 | 66.39 | 52.72 | — | |
| PLM + MCCLEncoder=PLM, Objective=MCCL, Training Data=5%2022.12 | 57.5 | 59.1 | — | — | — | |
| PU+ATLOP+RoBERTaBackbone=RoBERTa-Large2022.10 | 56.97 | 58.04 | 67.39 | 50.98 | — | |
| MTB + CEEncoder=MTB, Objective=CE, Training Data=5%2022.12 | 56.4 | 57.9 | — | — | — | |
| SR-PN+ATLOP+RoBERTaBackbone=RoBERTa-Large2022.10 | 56.06 | 56.39 | 87.47 | 41.61 | — | |
| SSR-PU+ATLOP+BERTBackbone=BERT-Base2022.10 | 55.21 | 56.14 | 70.42 | 46.67 | — | |
| TTM-RETraining Setting=Distant Only, Variant=PU2024.06 | 54.63 | 57.48 | 44.56 | 81.71 | — | |
| PN+ATLOP+RoBERTaBackbone=RoBERTa-Large2022.10 | 54.21 | 54.47 | 89.22 | 39.2 | — | |
| SR-PU+ATLOP+BERTBackbone=BERT-Base2022.10 | 53.87 | 55.06 | 63.42 | 48.66 | — | |
| S-PU+ATLOP+BERTBackbone=BERT-Base2022.10 | 53.36 | 54.44 | 65.95 | 46.38 | — | |
| SSR-PUTraining Setting=Distant Only2024.06 | 53.26 | 54.46 | 48.02 | 62.89 | — | |
| MTB + LazyEncoder=MTB, Objective=Lazy, Training Data=100%2022.12 | 52.9 | 54.1 | — | — | — | |
| SR-PN+ATLOP+BERTBackbone=BERT-Base2022.10 | 52.7 | 53.1 | 83.76 | 38.87 | — | |
| PLM + CEEncoder=PLM, Objective=CE, Training Data=5%2022.12 | 52.6 | 54.1 | — | — | — | |
| PU+ATLOP+BERTBackbone=BERT-Base2022.10 | 51.8 | 53.14 | 58.81 | 48.15 | — | |
| ATLOPTraining Setting=Distant Only2024.06 | 51.14 | 53.42 | 51.11 | 55.95 | — | |
| PN+ATLOP+BERTBackbone=BERT-Base2022.10 | 51.11 | 51.68 | 77.55 | 38.79 | — | |
| MTB + LazyEncoder=MTB, Objective=Lazy, Training Data=10%2022.12 | 46.2 | 47.3 | — | — | — | |
| DocuNET+RoBERTaBackbone=RoBERTa-Large2022.10 | 45.88 | 45.99 | 94.16 | 30.42 | — | |
| GAIN+BERTBackbone=BERT-Base2022.10 | 45.57 | 45.82 | 88.11 | 30.98 | — | |
| ATLOP+RoBERTaBackbone=RoBERTa-Large2022.10 | 45.09 | 45.19 | 94.75 | 29.67 | — | |
| MTB + MCCLEncoder=MTB, Objective=MCCL, Training Data=1%2022.12 | 44.5 | 46.4 | — | — | — | |
| MTB + LazyEncoder=MTB, Objective=Lazy, Training Data=5%2022.12 | 43.4 | 44.7 | — | — | — | |
| ATLOP+BERTBackbone=BERT-Base, Implementation=Re-implemented2022.10 | 43.12 | 43.25 | 92.49 | 28.23 | — | |
| PLM + MCCLEncoder=PLM, Objective=MCCL, Training Data=1%2022.12 | 43.1 | 44.7 | — | — | — | |
| MTB + CEEncoder=MTB, Objective=CE, Training Data=1%2022.12 | 40.7 | 42.3 | — | — | — | |
| PLM + CEEncoder=PLM, Objective=CE, Training Data=1%2022.12 | 38.9 | 40.3 | — | — | — | |
| MTB + LazyEncoder=MTB, Objective=Lazy, Training Data=1%2022.12 | 34.4 | 35.2 | — | — | — | |
| BiLSTMImplementation=Re-implemented2022.10 | 32.57 | 32.86 | 77.04 | 20.89 | — | |
| PLM + LazyEncoder=PLM, Objective=Lazy, Training Data=100%2022.12 | 28 | 28.7 | — | — | — | |
| PLM + LazyEncoder=PLM, Objective=Lazy, Training Data=5%2022.12 | 19.4 | 20.1 | — | — | — | |
| PLM + LazyEncoder=PLM, Objective=Lazy, Training Data=10%2022.12 | 19.2 | 21.6 | — | — | — | |
| PLM + LazyEncoder=PLM, Objective=Lazy, Training Data=1%2022.12 | 14.9 | 15.6 | — | — | — | |
| Claude Sonnet 4.6Shots=0-shot2026.06 | — | — | — | — | 52.3 | |
| GPT-5.4Shots=0-shot2026.06 | — | — | — | — | 55.9 | |
| Llama-3.2-3B GenTuneModel=Llama-3.2-3B, Shots=2s2026.06 | — | — | — | — | 74.3 | |
| Llama-3.2-3B GenTuneModel=Llama-3.2-3B, Shots=0s2026.06 | — | — | — | — | 72.7 | |
| Qwen2.5-0.5B GenTuneModel=Qwen2.5-0.5B, Shots=2s2026.06 | — | — | — | — | 72.4 |