# Open-domain event extraction on News reports dataset (test) Source: [Open-domain event extraction on News reports dataset (test)]() Collection index: [llms.txt]() Data updated: 2026-04-24T08:51:06.500Z Evaluation of Open-domain Event Extraction performance on the test split of the News reports dataset, measured using Exact Match, ROUGE-L, and BERTScore metrics. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Open-domain event extraction | | Dataset (including any stated split) | News reports dataset (test) | | Metric | Exact Match Precision | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 18 of 18 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Exact Match Precision](): Higher is better; unit: PERCENT. - [Exact Match Recall](): Higher is better; unit: PERCENT. - [Exact Match F1 Score](): Higher is better; unit: PERCENT. - [ROUGE-L Precision](): Higher is better; unit: PERCENT. - [ROUGE-L Recall](): Higher is better; unit: PERCENT. - [ROUGE-L F1 Score](): Higher is better; unit: PERCENT. - [BERTScore Precision](): Higher is better; unit: PERCENT. - [BERTScore Recall](): Higher is better; unit: PERCENT. - [BERTScore F1 Score](): Higher is better; unit: PERCENT. ## Recorded results | Method | Exact Match Precision | Result date | Report paper | | --- | --- | --- | --- | | MODEE-Base | 58.7 | Not recorded | 69eaf91755a3c32af89f898b | | Llama 1B | 58.2 | Not recorded | 69eaf91755a3c32af89f898b | | T5-Large | 57.2 | Not recorded | 69eaf91755a3c32af89f898b | | Qwen 0.6B | 56 | Not recorded | 69eaf91755a3c32af89f898b | | T5-Base | 54.9 | Not recorded | 69eaf91755a3c32af89f898b | | MODEE-Small | 54.7 | Not recorded | 69eaf91755a3c32af89f898b | | T5-Small | 52.2 | Not recorded | 69eaf91755a3c32af89f898b | | Qwen 8B | 15.8 | Not recorded | 69eaf91755a3c32af89f898b | | Qwen 8B | 15.8 | Not recorded | 69eaf91755a3c32af89f898b | | Giveme5W1H | 15.6 | Not recorded | 69eaf91755a3c32af89f898b | | Mistral 7B | 15.5 | Not recorded | 69eaf91755a3c32af89f898b | | Mistral 7B | 15.4 | Not recorded | 69eaf91755a3c32af89f898b | | Llama 70B | 14.4 | Not recorded | 69eaf91755a3c32af89f898b | | Llama 70B | 14.1 | Not recorded | 69eaf91755a3c32af89f898b | | Qwen 32B | 9.9 | Not recorded | 69eaf91755a3c32af89f898b | | Llama 8B | 9.8 | Not recorded | 69eaf91755a3c32af89f898b | | Llama 8B | 9.6 | Not recorded | 69eaf91755a3c32af89f898b | | Qwen 32B | 9 | Not recorded | 69eaf91755a3c32af89f898b | ### MODEE-Base Exact Match Precision: 58.7. Result ID: 69eb2ee647f34ce924d393d2. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Scale":"Base"}. This record does not assert independent reproduction. ### Llama 1B Exact Match Precision: 58.2. Result ID: 69eb2ee647f34ce924d393cf. [Reporting paper]() Recorded evaluation configuration: {"Evaluation Protocol":"Fine-tuned","Model Scale":"1B"}. This record does not assert independent reproduction. ### T5-Large Exact Match Precision: 57.2. Result ID: 69eb2ee647f34ce924d393c3. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Evaluation Protocol":"Fine-tuned","Model Scale":"Large"}. This record does not assert independent reproduction. ### Qwen 0.6B Exact Match Precision: 56. Result ID: 69eb2ee647f34ce924d393d0. [Reporting paper]() Recorded evaluation configuration: {"Evaluation Protocol":"Fine-tuned","Model Scale":"0.6B"}. This record does not assert independent reproduction. ### T5-Base Exact Match Precision: 54.9. Result ID: 69eb2ee647f34ce924d393c2. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Evaluation Protocol":"Fine-tuned","Model Scale":"Base"}. This record does not assert independent reproduction. ### MODEE-Small Exact Match Precision: 54.7. Result ID: 69eb2ee647f34ce924d393d1. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Scale":"Small"}. This record does not assert independent reproduction. ### T5-Small Exact Match Precision: 52.2. Result ID: 69eb2ee647f34ce924d393c1. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Evaluation Protocol":"Fine-tuned","Model Scale":"Small"}. This record does not assert independent reproduction. ### Qwen 8B Exact Match Precision: 15.8. Result ID: 69eb2ee647f34ce924d393c9. [Reporting paper]() Recorded evaluation configuration: {"Evaluation Protocol":"0-shot","Model Scale":"8B"}. This record does not assert independent reproduction. ### Qwen 8B Exact Match Precision: 15.8. Result ID: 69eb2ee647f34ce924d393ca. [Reporting paper]() Recorded evaluation configuration: {"Evaluation Protocol":"5-shot","Model Scale":"8B"}. This record does not assert independent reproduction. ### Giveme5W1H Exact Match Precision: 15.6. Result ID: 69eb2ee647f34ce924d393c4. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### Mistral 7B Exact Match Precision: 15.5. Result ID: 69eb2ee647f34ce924d393cd. [Reporting paper]() Recorded evaluation configuration: {"Evaluation Protocol":"0-shot","Model Scale":"7B"}. This record does not assert independent reproduction. ### Mistral 7B Exact Match Precision: 15.4. Result ID: 69eb2ee647f34ce924d393ce. [Reporting paper]() Recorded evaluation configuration: {"Evaluation Protocol":"5-shot","Model Scale":"7B"}. This record does not assert independent reproduction. ### Llama 70B Exact Match Precision: 14.4. Result ID: 69eb2ee647f34ce924d393c8. [Reporting paper]() Recorded evaluation configuration: {"Evaluation Protocol":"5-shot","Model Scale":"70B"}. This record does not assert independent reproduction. ### Llama 70B Exact Match Precision: 14.1. Result ID: 69eb2ee647f34ce924d393c7. [Reporting paper]() Recorded evaluation configuration: {"Evaluation Protocol":"0-shot","Model Scale":"70B"}. This record does not assert independent reproduction. ### Qwen 32B Exact Match Precision: 9.9. Result ID: 69eb2ee647f34ce924d393cb. [Reporting paper]() Recorded evaluation configuration: {"Evaluation Protocol":"0-shot","Model Scale":"32B"}. This record does not assert independent reproduction. ### Llama 8B Exact Match Precision: 9.8. Result ID: 69eb2ee647f34ce924d393c5. [Reporting paper]() Recorded evaluation configuration: {"Evaluation Protocol":"0-shot","Model Scale":"8B"}. This record does not assert independent reproduction. ### Llama 8B Exact Match Precision: 9.6. Result ID: 69eb2ee647f34ce924d393c6. [Reporting paper]() Recorded evaluation configuration: {"Evaluation Protocol":"5-shot","Model Scale":"8B"}. This record does not assert independent reproduction. ### Qwen 32B Exact Match Precision: 9. Result ID: 69eb2ee647f34ce924d393cc. [Reporting paper]() Recorded evaluation configuration: {"Evaluation Protocol":"5-shot","Model Scale":"32B"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.