Event Argument Extraction on DocEE (test)
56.4EM PrecisionMODEE-Base
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| MODEE-BaseModel Version=Base2026.04 | 56.4 | 35 | 43.1 | 88.9 | 55.1 | 68.1 | |
| Ontology_QA2026.04 | 51.3 | 34.2 | 41 | 80.3 | 53.6 | 64.3 | |
| BERT_QA2026.04 | 41.9 | 28.1 | 33.5 | 75.8 | 50.6 | 60.7 | |
| Doc2EDAG2026.04 | 37.1 | 36.1 | 36.6 | 54.2 | 53.7 | 53.9 | |
| Ontology QACategory=Supervised ML, Training Cost=$$$, Inference Cost=~02024.01 | 36.6 | 25.2 | 29.8 | 69.7 | 48 | 56.9 | |
| Flan-UL2 (Aligned instruction)Category=Flan-UL2, Training Cost=$, Inference Cost=~02024.01 | 36.1 | 20.7 | 26.3 | 76.6 | 52 | 62 | |
| ChatGPTCategory=Closed LLM, Training Cost=0, Inference Cost=$-$$2024.01 | 35.6 | 18 | 23.9 | 74.4 | 58 | 65.2 | |
| BERT_Seq2026.04 | 35.3 | 35.9 | 35.6 | 20.3 | 25 | 22.4 | |
| ULTRA-longCategory=ULTRA (Ours), Training Cost=$, Inference Cost=~0, Backbone=Flan-UL22024.01 | 32.3 | 30.5 | 31.4 | 68.4 | 65.9 | 67.1 | |
| CoT-ChatGPTCategory=Closed LLM, Training Cost=0, Inference Cost=$-$$2024.01 | 31.2 | 16.2 | 21.3 | 71 | 55.2 | 62.1 | |
| ChatGPT (single question)Category=Closed LLM, Training Cost=0, Inference Cost=$-$$2024.01 | 30.9 | 22.7 | 26.2 | 63.5 | 65.3 | 64.4 | |
| MG-Reader2026.04 | 30.3 | 35.9 | 32.9 | 45.6 | 50.8 | 48.1 | |
| ULTRA+Category=ULTRA (Ours), Base Method=ULTRA-long, Training Cost=$, Inference Cost=~0, Ensemble=true2024.01 | 30.2 | 35.5 | 32.6 | 68.6 | 71.5 | 70.1 | |
| EEQACategory=Supervised ML, Training Cost=$$$, Inference Cost=~02024.01 | 29.4 | 20.3 | 24 | 68.1 | 46.9 | 55.5 | |
| ULTRA-baseCategory=ULTRA (Ours), Training Cost=$, Inference Cost=~0, Backbone=Flan-UL22024.01 | 29 | 34.5 | 31.5 | 61.8 | 70.3 | 65.8 | |
| ULTRA+Category=ULTRA (Ours), Base Method=ULTRA-base, Training Cost=$, Inference Cost=~0, Ensemble=true2024.01 | 28 | 39.4 | 32.7 | 63.7 | 75.3 | 69 | |
| Flan-UL2 (Custom instructions)Category=Flan-UL2, Training Cost=$, Inference Cost=~0, Details=Average results of 5 instructions2024.01 | 27.6 | 17.8 | 21.6 | 69.2 | 45.2 | 54.6 |