Relation Extraction on FewRel
87.34Micro-F1Human Labeled
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Human LabeledNum. of Samples=35840, Cost (USD)=6400 - 10,000, Time (Mins)=325822022.12 | 87.34 | 87.55 | 87.43 | — | |
| Human LabeledNum. of Samples=12800, Cost (USD)=1828 - 3584, Time (Mins)=116362022.12 | 84.95 | 85.19 | 85.07 | — | |
| PGDGNum. of Samples=12800, Cost (USD)=99.35, Time (Mins)=340†, Evaluation Protocol=5-shot2022.12 | 67.71 | 70.59 | 67.99 | — | |
| DADGNum. of Samples=12800, Cost (USD)=88.91, Time (Mins)=265†, Evaluation Protocol=5-shot2022.12 | 57.98 | 59.76 | 60.85 | — | |
| PGDGNum. of Samples=12800, Cost (USD)=30.58, Time (Mins)=285†, Evaluation Protocol=1-shot2022.12 | 44.11 | 47.82 | 45.58 | — | |
| KnowCoder-XEvaluation Mode=Zero-shot2024.11 | 42.18 | — | — | — | |
| IEPILEEvaluation Mode=Zero-shot2024.11 | 41.28 | — | — | — | |
| DADGNum. of Samples=12800, Cost (USD)=17.16, Time (Mins)=220†, Evaluation Protocol=1-shot2022.12 | 40.02 | 45.41 | 42.41 | — | |
| YAYI-UIEEvaluation Mode=Zero-shot2024.11 | 36.09 | — | — | — | |
| Human LabeledNum. of Samples=704, Cost (USD)=101 - 200, Time (Mins)=6402022.12 | 34.22 | 41.92 | 41.45 | — | |
| PGINum. of Samples=4480, Cost (USD)=33.30, Time (Mins)=160†2022.12 | 25.85 | 29.86 | 29.82 | — | |
| GLiRELEntities=gold2026.05 | 24 | — | — | 8.1 | |
| GPT-4Evaluation Mode=Zero-shot2024.11 | 22.43 | — | — | — | |
| GLiNER2Entities=predicted2026.05 | 20.8 | — | — | 17.8 | |
| GPT-5-miniEntities=prompted2026.05 | 15 | — | — | 22.1 | |
| GLiNER-RelexEntities=predicted2026.05 | 12.5 | — | — | 25.6 | |
| chatgptSetting=zero-shot2023.04 | 9.96 | — | — | — | |
| PGDA2Num. of Samples=384, Cost (USD)=25.40, Time (Mins)=10†, Evaluation Protocol=1-shot2022.12 | 0.18 | 0.14 | 1.7 | — | |
| PGDA3Num. of Samples=384, Cost (USD)=25.19, Time (Mins)=11†, Evaluation Protocol=1-shot2022.12 | 0.13 | 0.09 | 1.65 | — | |
| PGDA1Num. of Samples=384, Cost (USD)=28.55, Time (Mins)=13†, Evaluation Protocol=1-shot2022.12 | 0.05 | 0.03 | 1.56 | — | |
| PGDA4Num. of Samples=384, Cost (USD)=25.57, Time (Mins)=10†, Evaluation Protocol=1-shot2022.12 | 0.05 | 0.02 | 1.56 | — | |
| PGDA5Num. of Samples=384, Cost (USD)=25.56, Time (Mins)=11†, Evaluation Protocol=1-shot2022.12 | 0.05 | 0.02 | 1.56 | — | |
| davinciSetting=zero-shot2023.04 | 0 | — | — | — |