# Hypernym Discovery on SemEval Task 9 English general-purpose subtask 2018 Source: [Hypernym Discovery on SemEval Task 9 English general-purpose subtask 2018]() Collection index: [llms.txt]() Data updated: 2026-02-26T05:50:26.076Z Evaluation of Hypernym Discovery performance on the English general-purpose subtask of SemEval Task 9 2018, measured using MAP, MRR, and Precision@K metrics. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Hypernym Discovery | | Dataset (including any stated split) | SemEval Task 9 English general-purpose subtask 2018 (gold standard evaluation) | | Metric | MRR | | Unit | SCALAR | | Direction | Higher is better | | Results loaded | 18 of 18 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [MRR](): Higher is better; unit: SCALAR. - [MAP](): Higher is better; unit: SCALAR. - [Precision@1](): Higher is better; unit: SCALAR. - [Precision@3](): Higher is better; unit: SCALAR. - [Precision@5](): Higher is better; unit: SCALAR. - [Precision@15](): Higher is better; unit: SCALAR. ## Recorded results | Method | MRR | Result date | Report paper | | --- | --- | --- | --- | | TaxoLLaMA | 0.5439 | Not recorded | 68fa75ad1e209131cea693ce | | TaxoLLaMA-bench | 0.5159 | Not recorded | 68fa75ad1e209131cea693ce | | T5 | 0.4522 | Not recorded | 68fa75ad1e209131cea693ce | | RMM | 0.3907 | Not recorded | 68fa75ad1e209131cea693ce | | TaxoLLaMA | 0.3805 | Not recorded | 68fa75ad1e209131cea693ce | | TaxoLLaMA-bench | 0.3766 | Not recorded | 68fa75ad1e209131cea693ce | | CRIM | 0.361 | Not recorded | 68fa75ad1e209131cea693ce | | Hybrid | 0.3407 | Not recorded | 68fa75ad1e209131cea693ce | | GRU | 0.1622 | Not recorded | 68fa646b1e209131cea0498d | | CNN | 0.1333 | Not recorded | 68fa646b1e209131cea0498d | | RCNN | 0.1283 | Not recorded | 68fa646b1e209131cea0498d | | TEA | 0.1113 | Not recorded | 68fa646b1e209131cea0498d | | CNN | 0.1092 | Not recorded | 68fa646b1e209131cea0498d | | LSTM | 0.0997 | Not recorded | 68fa646b1e209131cea0498d | | GRU | 0.0944 | Not recorded | 68fa646b1e209131cea0498d | | RCNN | 0.0924 | Not recorded | 68fa646b1e209131cea0498d | | TEA | 0.0871 | Not recorded | 68fa646b1e209131cea0498d | | LSTM | 0.0752 | Not recorded | 68fa646b1e209131cea0498d | ### TaxoLLaMA MRR: 0.5439. Result ID: 6993f4ed3a6c31484919c9dd. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Evaluation Protocol":"fine-tuned","Training Data":"Target dataset"}. This record does not assert independent reproduction. ### TaxoLLaMA-bench MRR: 0.5159. Result ID: 6993f4ed3a6c31484919c9de. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Evaluation Protocol":"fine-tuned","Training Data":"Target dataset"}. This record does not assert independent reproduction. ### T5 MRR: 0.4522. Result ID: 6993f4ed3a6c31484919c9da. [Reporting paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### RMM MRR: 0.3907. Result ID: 6993f4ed3a6c31484919c9d9. [Reporting paper]() Recorded evaluation configuration: {"LLM pre-training":"No"}. This record does not assert independent reproduction. ### TaxoLLaMA MRR: 0.3805. Result ID: 6993f4ed3a6c31484919c9db. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Evaluation Protocol":"zero-shot","Training Data":"WordNet only"}. This record does not assert independent reproduction. ### TaxoLLaMA-bench MRR: 0.3766. Result ID: 6993f4ed3a6c31484919c9dc. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Evaluation Protocol":"zero-shot","Training Data":"WordNet only"}. This record does not assert independent reproduction. ### CRIM MRR: 0.361. Result ID: 6993f4ed3a6c31484919c9d7. [Reporting paper]() Recorded evaluation configuration: {"LLM pre-training":"No"}. This record does not assert independent reproduction. ### Hybrid MRR: 0.3407. Result ID: 6993f4ed3a6c31484919c9d8. [Reporting paper]() Recorded evaluation configuration: {"LLM pre-training":"No"}. This record does not assert independent reproduction. ### GRU MRR: 0.1622. Result ID: 698c3211be3fcfa82d014d36. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Embedding":"Word"}. This record does not assert independent reproduction. ### CNN MRR: 0.1333. Result ID: 698c3211be3fcfa82d014d38. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Embedding":"Word"}. This record does not assert independent reproduction. ### RCNN MRR: 0.1283. Result ID: 698c3211be3fcfa82d014d39. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Embedding":"Word"}. This record does not assert independent reproduction. ### TEA MRR: 0.1113. Result ID: 698c3211be3fcfa82d014d35. [Reporting paper]() Recorded evaluation configuration: {"Embedding":"Word"}. This record does not assert independent reproduction. ### CNN MRR: 0.1092. Result ID: 698c3211be3fcfa82d014d3d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Embedding":"Sense"}. This record does not assert independent reproduction. ### LSTM MRR: 0.0997. Result ID: 698c3211be3fcfa82d014d3c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Embedding":"Sense"}. This record does not assert independent reproduction. ### GRU MRR: 0.0944. Result ID: 698c3211be3fcfa82d014d3b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Embedding":"Sense"}. This record does not assert independent reproduction. ### RCNN MRR: 0.0924. Result ID: 698c3211be3fcfa82d014d3e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Embedding":"Sense"}. This record does not assert independent reproduction. ### TEA MRR: 0.0871. Result ID: 698c3211be3fcfa82d014d3a. [Reporting paper]() Recorded evaluation configuration: {"Embedding":"Sense"}. This record does not assert independent reproduction. ### LSTM MRR: 0.0752. Result ID: 698c3211be3fcfa82d014d37. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Embedding":"Word"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.