# Conversation-level Topic Discovery and Labeling on DSTC-12 Travel domain (blind test)

Source: [Conversation-level Topic Discovery and Labeling on DSTC-12 Travel domain (blind test)](<https://www.sota2.com/research/sota/conversation-level-topic-discovery-and-labeling-on-dstc-12-travel-domain-blind-test>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-02-26T05:31:07.644Z

Evaluation of conversation-level topic discovery and labeling performance on the blind test set of the DSTC-12 Travel domain dataset.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Conversation-level Topic Discovery and Labeling |
| Dataset (including any stated split) | DSTC-12 Travel domain (blind test) |
| Metric | Accuracy |
| Unit | PERCENT |
| Direction | Higher is better |
| Results loaded | 6 of 6 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [Accuracy](<https://www.sota2.com/research/sota/conversation-level-topic-discovery-and-labeling-on-dstc-12-travel-domain-blind-test.md?metric=Accuracy>): Higher is better; unit: PERCENT.

- [NMI](<https://www.sota2.com/research/sota/conversation-level-topic-discovery-and-labeling-on-dstc-12-travel-domain-blind-test.md?metric=NMI>): Higher is better; unit: PERCENT.

- [ROUGE-1](<https://www.sota2.com/research/sota/conversation-level-topic-discovery-and-labeling-on-dstc-12-travel-domain-blind-test.md?metric=ROUGE-1>): Higher is better; unit: PERCENT.

- [ROUGE-2](<https://www.sota2.com/research/sota/conversation-level-topic-discovery-and-labeling-on-dstc-12-travel-domain-blind-test.md?metric=ROUGE-2>): Higher is better; unit: PERCENT.

- [ROUGE-L](<https://www.sota2.com/research/sota/conversation-level-topic-discovery-and-labeling-on-dstc-12-travel-domain-blind-test.md?metric=ROUGE-L>): Higher is better; unit: PERCENT.

- [CosSim](<https://www.sota2.com/research/sota/conversation-level-topic-discovery-and-labeling-on-dstc-12-travel-domain-blind-test.md?metric=CosSim>): Higher is better; unit: PERCENT.

- [BERTScore Precision](<https://www.sota2.com/research/sota/conversation-level-topic-discovery-and-labeling-on-dstc-12-travel-domain-blind-test.md?metric=BERTScore%20Precision>): Higher is better; unit: PERCENT.

- [BERTScore Recall](<https://www.sota2.com/research/sota/conversation-level-topic-discovery-and-labeling-on-dstc-12-travel-domain-blind-test.md?metric=BERTScore%20Recall>): Higher is better; unit: PERCENT.

- [BERTScore F1](<https://www.sota2.com/research/sota/conversation-level-topic-discovery-and-labeling-on-dstc-12-travel-domain-blind-test.md?metric=BERTScore%20F1>): Higher is better; unit: PERCENT.

- [Success Rate (Sec 1)](<https://www.sota2.com/research/sota/conversation-level-topic-discovery-and-labeling-on-dstc-12-travel-domain-blind-test.md?metric=Success%20Rate%20(Sec%201)>): Higher is better; unit: PERCENT.

- [Success Rate (Sec 2)](<https://www.sota2.com/research/sota/conversation-level-topic-discovery-and-labeling-on-dstc-12-travel-domain-blind-test.md?metric=Success%20Rate%20(Sec%202)>): Higher is better; unit: PERCENT.

- [Average Success Rate](<https://www.sota2.com/research/sota/conversation-level-topic-discovery-and-labeling-on-dstc-12-travel-domain-blind-test.md?metric=Average%20Success%20Rate>): Higher is better; unit: PERCENT.

- [Overall Score](<https://www.sota2.com/research/sota/conversation-level-topic-discovery-and-labeling-on-dstc-12-travel-domain-blind-test.md?metric=Overall%20Score>): Higher is better; unit: PERCENT.

## Recorded results

| Method | Accuracy | Result date | Report paper |
| --- | --- | --- | --- |
| Team C | 0.68 | Not recorded | 69520ad653fdd5c36173d2dd |
| Team D | 0.518 | Not recorded | 69520ad653fdd5c36173d2dd |
| Team A | 0.484 | Not recorded | 69520ad653fdd5c36173d2dd |
| CATCH (Team E) | 0.358 | Not recorded | 69520ad653fdd5c36173d2dd |
| Team F | 0.267 | Not recorded | 69520ad653fdd5c36173d2dd |
| Team B | 0.179 | Not recorded | 69520ad653fdd5c36173d2dd |

### Team C

Accuracy: 0.68. Result ID: 6996af8f5698665ec4f9bff1.

[Reporting paper](<https://www.sota2.com/research/paper/69520ad653fdd5c36173d2dd.md>)

Recorded evaluation configuration: {"LLM":"API"}. This record does not assert independent reproduction.

### Team D

Accuracy: 0.518. Result ID: 6996af8f5698665ec4f9bff3.

[Reporting paper](<https://www.sota2.com/research/paper/69520ad653fdd5c36173d2dd.md>)

Recorded evaluation configuration: {"LLM":"\<30B"}. This record does not assert independent reproduction.

### Team A

Accuracy: 0.484. Result ID: 6996af8f5698665ec4f9bff4.

[Reporting paper](<https://www.sota2.com/research/paper/69520ad653fdd5c36173d2dd.md>)

Recorded evaluation configuration: {"LLM":"API"}. This record does not assert independent reproduction.

### CATCH (Team E)

Accuracy: 0.358. Result ID: 6996af8f5698665ec4f9bff2.

[Reporting paper](<https://www.sota2.com/research/paper/69520ad653fdd5c36173d2dd.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69520ad653fdd5c36173d2dd.md>)

Recorded evaluation configuration: {"LLM":"\<30B"}. This record does not assert independent reproduction.

### Team F

Accuracy: 0.267. Result ID: 6996af8f5698665ec4f9bff5.

[Reporting paper](<https://www.sota2.com/research/paper/69520ad653fdd5c36173d2dd.md>)

Recorded evaluation configuration: {"LLM":"\<30B"}. This record does not assert independent reproduction.

### Team B

Accuracy: 0.179. Result ID: 6996af8f5698665ec4f9bff6.

[Reporting paper](<https://www.sota2.com/research/paper/69520ad653fdd5c36173d2dd.md>)

Recorded evaluation configuration: {"LLM":"API"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.