# Theme Detection on DSTC Travel domain 12 (test) Source: [Theme Detection on DSTC Travel domain 12 (test)]() Collection index: [llms.txt]() Data updated: 2026-02-26T06:04:13.872Z Evaluation of Theme Detection performance on the test split of the DSTC Travel domain 12 dataset, reporting metrics related to semantic relevance, utility, and quality. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Theme Detection | | Dataset (including any stated split) | DSTC Travel domain 12 (test) | | Metric | Semantic Relevance (SR) | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 6 of 6 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Semantic Relevance (SR)](): Higher is better; unit: PERCENT. - [Analytical Utility (AU)](): Higher is better; unit: PERCENT. - [Granularity (GR)](): Higher is better; unit: PERCENT. - [Actionability (ACT)](): Higher is better; unit: PERCENT. - [Domain Relevance (DR)](): Higher is better; unit: PERCENT. - [Conciseness & Word Choice (CWC)](): Higher is better; unit: PERCENT. - [Grammatical Structure (GS)](): Higher is better; unit: PERCENT. - [Thematic Distinctiveness (TD)](): Higher is better; unit: PERCENT. - [Overall Average Score](): Higher is better; unit: PERCENT. ## Recorded results | Method | Semantic Relevance (SR) | Result date | Report paper | | --- | --- | --- | --- | | Team C | 89.7 | Not recorded | 69520ad653fdd5c36173d2dd | | CATCH | 86.2 | Not recorded | 69520ad653fdd5c36173d2dd | | Team A | 77.3 | Not recorded | 69520ad653fdd5c36173d2dd | | Team D | 68.8 | Not recorded | 69520ad653fdd5c36173d2dd | | Team B | 65 | Not recorded | 69520ad653fdd5c36173d2dd | | Team F | 45.2 | Not recorded | 69520ad653fdd5c36173d2dd | ### Team C Semantic Relevance (SR): 89.7. Result ID: 6996af8f5698665ec4f9bff7. [Reporting paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### CATCH Semantic Relevance (SR): 86.2. Result ID: 6996af8f5698665ec4f9bff8. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Team ID":"Team E"}. This record does not assert independent reproduction. ### Team A Semantic Relevance (SR): 77.3. Result ID: 6996af8f5698665ec4f9bffa. [Reporting paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### Team D Semantic Relevance (SR): 68.8. Result ID: 6996af8f5698665ec4f9bff9. [Reporting paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### Team B Semantic Relevance (SR): 65. Result ID: 6996af8f5698665ec4f9bffc. [Reporting paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### Team F Semantic Relevance (SR): 45.2. Result ID: 6996af8f5698665ec4f9bffb. [Reporting paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.