Open Information Extraction on CaRB (test)
56.8F1 ScoreIMOJIE
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| IMOJIEType=Generation-based2023.09 | 56.8 | — | 64.7 | 45.6 | — | — | — | — | — | |
| CIGL-OIESpeed (sent./sec)=142.02022.06 | 54 | 35.7 | — | — | 42.8 | 24.6 | 59.2 | 40 | 36.8 | |
| CIGL-OIE + IGL-CAGraph Type=None2022.12 | 54 | 35.7 | — | — | — | — | — | — | — | |
| SMiLe-OIEGraph Type=Dependency & Constituency2022.12 | 53.76 | 34.92 | — | — | — | — | — | — | — | |
| IMOJIESpeed (sent./sec)=2.62022.06 | 53.5 | 33.3 | — | — | 41.4 | 22.2 | 56.8 | 39.6 | 36 | |
| IMOJIEGraph Type=None2022.12 | 53.5 | 33.3 | — | — | — | — | — | — | — | |
| BERT + Const-GCN → Dep-GCNGraph Type=Dependency & Constituency, Aggregation=Sequential (Const to Dep)2022.12 | 53.28 | 33.51 | — | — | — | — | — | — | — | |
| BERT + Dep-GCN → Const-GCNGraph Type=Dependency & Constituency, Aggregation=Sequential (Dep to Const)2022.12 | 52.85 | 33.37 | — | — | — | — | — | — | — | |
| BERT + Const-GCNGraph Type=Constituency2022.12 | 52.83 | 33.1 | — | — | — | — | — | — | — | |
| OpenIE6Speed (sent./sec)=31.72022.06 | 52.7 | 33.7 | — | — | 46.4 | 26.8 | 65.6 | 48.4 | 40 | |
| BERT + Dep-GCN ⊕ Const-GCNGraph Type=Dependency & Constituency, Aggregation=Parallel2022.12 | 52.7 | 33.28 | — | — | — | — | — | — | — | |
| BERT + Dep-GCNGraph Type=Dependency2022.12 | 52.49 | 32.85 | — | — | — | — | — | — | — | |
| IGL-OIESpeed (sent./sec)=142.02022.06 | 52.4 | 33.7 | — | — | 41.1 | 22.9 | 55 | 36 | 34.9 | |
| Multi²OIEBackbone=BERT, Attention=Multi-head2020.09 | 52.3 | 32.6 | 60.9 | 45.8 | — | — | — | — | — | |
| DetIEIMOJIEBackbone=IMOJIE, Speed (sent./sec)=708.62022.06 | 52.1 | 36.7 | — | — | 40.1 | 24 | 56 | 38.7 | 36 | |
| GPT-3.5-TURBOStrategy=Selected Demo & Uncertainty2023.09 | 52.1 | — | 54.3 | 50.1 | — | — | — | — | — | |
| GPT-3.5-TURBOStrategy=Selected Demo2023.09 | 52 | — | 54.7 | 49.5 | — | — | — | — | — | |
| OpenIE4Speed (sent./sec)=20.12022.06 | 51.6 | 29.5 | — | — | 40.5 | 20.1 | 54.3 | 37.1 | 34.4 | |
| NeuralOIESpeed (sent./sec)=11.52022.06 | 51.6 | 32.8 | — | — | 38.7 | 19.8 | 53.5 | 37 | 33.3 | |
| CopyAttentionGraph Type=None2022.12 | 51.6 | 32.8 | — | — | — | — | — | — | — | |
| LLAMA-2-70BStrategy=Selected Demo & Uncertainty2023.09 | 51.5 | — | 56.3 | 47.5 | — | — | — | — | — | |
| BERTGraph Type=None2022.12 | 51.45 | 30.62 | — | — | — | — | — | — | — | |
| GPT-3.5-TURBOStrategy=Fixed Demo2023.09 | 51.1 | — | 53.8 | 48.6 | — | — | — | — | — | |
| LLAMA-2-70BStrategy=Selected Demo2023.09 | 50.9 | — | 55.4 | 47.1 | — | — | — | — | — | |
| BERT+BiLSTMBackbone=BERT, Decoder=BiLSTM2020.09 | 50.6 | 30.6 | 61.3 | 43.1 | — | — | — | — | — | |
| SpanOIE+MHTagging=Span selection, Attention=Multi-head2020.09 | 50 | 30.2 | 62.2 | 41.8 | — | — | — | — | — | |
| SpanOIETagging=Span selection2020.09 | 49.4 | 30 | 60.9 | 41.6 | — | — | — | — | — | |
| SpanOIEType=Tagging-based2023.09 | 49.4 | — | 60.9 | 41.6 | — | — | — | — | — | |
| RnnOIESpeed (sent./sec)=149.22022.06 | 49 | 26 | — | — | 39.5 | 18.3 | 56 | 32 | 26.4 | |
| OpenIE42020.09 | 48.8 | 27.2 | — | — | — | — | — | — | — | |
| GloVe + bi-LSTM + CRFGraph Type=None2022.12 | 48.72 | 27.51 | — | — | — | — | — | — | — | |
| GloVe + bi-LSTMGraph Type=None2022.12 | 48.52 | 27.1 | — | — | — | — | — | — | — | |
| SpanOIESpeed (sent./sec)=19.42022.06 | 48.5 | — | — | — | 37.9 | — | 54 | — | 31.9 | |
| OpenIE5Speed (sent./sec)=3.12022.06 | 48 | 25 | — | — | 42.7 | 20.6 | 59.9 | 39.9 | 35.4 | |
| OpenIE4Type=Rule-based2023.09 | 48 | — | — | — | — | — | — | — | — | |
| BIO+MHTagging=BIO tagging, Attention=Multi-head2020.09 | 47.5 | 27.3 | 57.2 | 40.7 | — | — | — | — | — | |
| DetIEIMOJIEBackbone=IMOJIE, Coordination Analysis (IGL-CA)=true, Speed (sent./sec)=112.22022.06 | 47.3 | 35.1 | — | — | 43.1 | 29.3 | 67.7 | 54 | 37.8 | |
| LLAMA-2-70BStrategy=Fixed Demo2023.09 | 47.1 | — | 43.7 | 51 | — | — | — | — | — | |
| RnnOIE2020.09 | 46.7 | 26.8 | 55.6 | 40.2 | — | — | — | — | — | |
| BIOTagging=BIO tagging2020.09 | 46.6 | 27.7 | 55.1 | 40.4 | — | — | — | — | — | |
| ClausIESpeed (sent./sec)=4.02022.06 | 45 | 22 | — | — | 40.2 | 17.7 | 61 | 38 | 33.2 | |
| ClausIE2020.09 | 44.9 | 22.4 | — | — | — | — | — | — | — | |
| DetIELSOIEBackbone=LSOIE, Speed (sent./sec)=708.62022.06 | 43 | 27.2 | — | — | 33.1 | 18.3 | 49.7 | 32.7 | 31.2 | |
| MinIESpeed (sent./sec)=8.92022.06 | 41.9 | — | — | — | 38.4 | — | 52.3 | — | 28.5 | |
| OLLIE2020.09 | 41.1 | 22.4 | — | — | — | — | — | — | — | |
| DetIELSOIEBackbone=LSOIE, Coordination Analysis (IGL-CA)=true, Speed (sent./sec)=112.22022.06 | 39.6 | 26.7 | — | — | 36.3 | 22.7 | 63.3 | 47.9 | 33.5 | |
| GPT-3.5-TURBOStrategy=Vanilla2023.09 | 39.1 | — | 41 | 37.5 | — | — | — | — | — | |
| LLAMA-2-13BStrategy=Selected Demo & Uncertainty2023.09 | 36.2 | — | 44.4 | 30.6 | — | — | — | — | — | |
| LLAMA-2-13BStrategy=Selected Demo2023.09 | 35.7 | — | 43.2 | 30.4 | — | — | — | — | — | |
| LLAMA-2-13BStrategy=Fixed Demo2023.09 | 33.4 | — | 41.6 | 28 | — | — | — | — | — | |
| PropS2020.09 | 31.9 | 12.6 | — | — | — | — | — | — | — | |
| SenseOIE2022.06 | 28.2 | — | — | — | 23.9 | — | 31.1 | — | 10.7 | |
| Stanford2020.09 | 23 | 13.4 | — | — | — | — | — | — | — |