Topic Relatedness Assessment on Literary passage sets 50 (Evaluation)
70Very RelatedRetell-paraphrase
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Retell-paraphraseBackbone Model=Phi-3.5-mini, k (number of topics)=502025.05 | 70 | 22 | 8 | |
| Retell-describeBackbone Model=GPT-4o mini, k (number of topics)=502025.05 | 68 | 20 | 12 | |
| Retell-summarizeBackbone Model=Llama 3.1 8B, k (number of topics)=2062025.05 | 67 | 20 | 13 | |
| Retell-describeBackbone Model=GPT-4o mini, k (number of topics)=892025.05 | 67 | 22 | 12 | |
| Retell-summarizeBackbone Model=GPT-4o mini, k (number of topics)=892025.05 | 65 | 22 | 13 | |
| Retell-paraphraseBackbone Model=Llama 3.1 8B, k (number of topics)=502025.05 | 63 | 20 | 17 | |
| Retell-describeBackbone Model=Phi-3.5-mini, k (number of topics)=5382025.05 | 63 | 30 | 7 | |
| Retell-describeBackbone Model=Phi-3.5-mini, k (number of topics)=502025.05 | 63 | 32 | 5 | |
| Retell-summarizeBackbone Model=Phi-3.5-mini, k (number of topics)=5382025.05 | 62 | 23 | 15 | |
| Retell-summarizeBackbone Model=Gemma 2 2B, k (number of topics)=1572025.05 | 62 | 28 | 10 | |
| Retell-describeBackbone Model=Gemma 2 2B, k (number of topics)=1572025.05 | 62 | 23 | 15 | |
| Retell-summarizeBackbone Model=GPT-4o mini, k (number of topics)=502025.05 | 60 | 30 | 10 | |
| Retell-paraphraseBackbone Model=GPT-4o mini, k (number of topics)=502025.05 | 58 | 37 | 5 | |
| Retell-summarizeBackbone Model=Llama 3.1 8B, k (number of topics)=502025.05 | 58 | 33 | 8 | |
| Retell-describeBackbone Model=Llama 3.1 8B, k (number of topics)=2062025.05 | 55 | 35 | 10 | |
| Retell-describeBackbone Model=Llama 3.1 8B, k (number of topics)=502025.05 | 53 | 40 | 7 | |
| Retell-summarizeBackbone Model=Phi-3.5-mini, k (number of topics)=502025.05 | 50 | 42 | 8 | |
| Retell-summarizeBackbone Model=Gemma 2 2B, k (number of topics)=502025.05 | 50 | 37 | 13 | |
| TopicGPT-liteBackbone Model=Llama 3.1 8B, k (number of topics)=2062025.05 | 48 | 42 | 10 | |
| Retell-paraphraseBackbone Model=Phi-3.5-mini, k (number of topics)=5382025.05 | 47 | 35 | 18 | |
| Retell-paraphraseBackbone Model=Gemma 2 2B, k (number of topics)=1572025.05 | 47 | 32 | 22 | |
| Retell-paraphraseBackbone Model=Gemma 2 2B, k (number of topics)=502025.05 | 47 | 32 | 22 | |
| Retell-describeBackbone Model=Gemma 2 2B, k (number of topics)=502025.05 | 47 | 42 | 12 | |
| TopicGPT-liteBackbone Model=Phi-3.5-mini, k (number of topics)=5382025.05 | 42 | 30 | 28 | |
| Retell-paraphraseBackbone Model=GPT-4o mini, k (number of topics)=892025.05 | 38 | 35 | 27 | |
| Retell-paraphraseBackbone Model=Llama 3.1 8B, k (number of topics)=2062025.05 | 38 | 35 | 27 | |
| TopicGPT-liteBackbone Model=Gemma 2 2B, k (number of topics)=1572025.05 | 33 | 32 | 35 | |
| TopicGPT-liteBackbone Model=GPT-4o mini, k (number of topics)=892025.05 | 28 | 30 | 42 | |
| LDABackbone Model=None, k (number of topics)=5382025.05 | 27 | 17 | 57 | |
| LDABackbone Model=None, k (number of topics)=892025.05 | 22 | 20 | 58 | |
| LDABackbone Model=None, k (number of topics)=1572025.05 | 22 | 15 | 63 | |
| LDABackbone Model=None, k (number of topics)=2062025.05 | 20 | 12 | 68 | |
| LDABackbone Model=None, k (number of topics)=502025.05 | 3 | 12 | 85 |