Open-domain event extraction on News reports dataset (test)
58.7Exact Match PrecisionMODEE-Base
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| MODEE-BaseModel Scale=Base2026.04 | 58.7 | 56.7 | 57.7 | 75.1 | 75.9 | 73.7 | 94.7 | 94.8 | 94.7 | |
| Llama 1BEvaluation Protocol=Fine-tuned, Model Scale=1B2026.04 | 58.2 | 55.3 | 56.8 | 73.6 | 72.5 | 71.3 | 94.4 | 94.4 | 94.4 | |
| T5-LargeEvaluation Protocol=Fine-tuned, Model Scale=Large2026.04 | 57.2 | 54.4 | 55.8 | 75.2 | 75.1 | 73 | 94.5 | 94.5 | 94.5 | |
| Qwen 0.6BEvaluation Protocol=Fine-tuned, Model Scale=0.6B2026.04 | 56 | 57.7 | 56.9 | 72.8 | 73.2 | 71.1 | 94.3 | 94.4 | 94.3 | |
| T5-BaseEvaluation Protocol=Fine-tuned, Model Scale=Base2026.04 | 54.9 | 51.8 | 53.3 | 71.7 | 72.8 | 70.5 | 94.1 | 94.2 | 94.1 | |
| MODEE-SmallModel Scale=Small2026.04 | 54.7 | 51.7 | 53.2 | 71.2 | 72.8 | 70.1 | 93.9 | 94.2 | 94 | |
| T5-SmallEvaluation Protocol=Fine-tuned, Model Scale=Small2026.04 | 52.2 | 47.5 | 49.8 | 69.8 | 69.7 | 67.8 | 93.5 | 93.6 | 93.5 | |
| Qwen 8BEvaluation Protocol=0-shot, Model Scale=8B2026.04 | 15.8 | 19.2 | 17.3 | 37.1 | 55.9 | 40.5 | 86.7 | 89.4 | 88 | |
| Qwen 8BEvaluation Protocol=5-shot, Model Scale=8B2026.04 | 15.8 | 19.3 | 17.4 | 37.2 | 56.4 | 40.5 | 86.7 | 89.4 | 88 | |
| Giveme5W1H2026.04 | 15.6 | 16.6 | 16.1 | 31.1 | 31.4 | 28.8 | 86.6 | 86.3 | 86.3 | |
| Mistral 7BEvaluation Protocol=0-shot, Model Scale=7B2026.04 | 15.5 | 18.8 | 17 | 36.9 | 51.4 | 38.6 | 86.8 | 89.2 | 87.9 | |
| Mistral 7BEvaluation Protocol=5-shot, Model Scale=7B2026.04 | 15.4 | 18.6 | 16.8 | 36.6 | 51.2 | 38.5 | 86.8 | 89.2 | 87.9 | |
| Llama 70BEvaluation Protocol=5-shot, Model Scale=70B2026.04 | 14.4 | 13.7 | 14 | 31.3 | 45.2 | 31.5 | 85.3 | 87.6 | 86.3 | |
| Llama 70BEvaluation Protocol=0-shot, Model Scale=70B2026.04 | 14.1 | 13.4 | 13.8 | 31.2 | 45.2 | 31.4 | 85.2 | 87.5 | 86.2 | |
| Qwen 32BEvaluation Protocol=0-shot, Model Scale=32B2026.04 | 9.9 | 11.6 | 10.7 | 30.3 | 68 | 37.2 | 84.6 | 89.9 | 87.1 | |
| Llama 8BEvaluation Protocol=0-shot, Model Scale=8B2026.04 | 9.8 | 10.6 | 10.2 | 27.7 | 52.7 | 30.8 | 84.1 | 87.9 | 85.8 | |
| Llama 8BEvaluation Protocol=5-shot, Model Scale=8B2026.04 | 9.6 | 10.3 | 10 | 27.1 | 51.8 | 30.1 | 83.9 | 87.8 | 85.7 | |
| Qwen 32BEvaluation Protocol=5-shot, Model Scale=32B2026.04 | 9 | 10.5 | 9.7 | 29.4 | 67.7 | 36.5 | 84.5 | 89.8 | 87 |