Passage-pair linking on Literary analysis corpus (Goodreads, Sparknotes, Litcharts)
17PrecisionRetell-summarize
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| Retell-summarizeModel=Phi-3.5-mini, k=5382025.05 | 17 | 2 | |
| Retell-describeModel=Phi-3.5-mini, k=5382025.05 | 15.9 | 2.1 | |
| Retell-summarizeModel=Llama 3.1 8B, k=2062025.05 | 15.5 | 4 | |
| Retell-describeModel=GPT-4o mini, k=892025.05 | 15.4 | 8.2 | |
| Retell-describeModel=Llama 3.1 8B, k=2062025.05 | 14.9 | 3.9 | |
| TopicGPT-liteModel=GPT-4o mini, k=892025.05 | 14.8 | 18.6 | |
| Retell-summarizeModel=Gemma 2 2B, k=1572025.05 | 14.7 | 4.5 | |
| Retell-describeModel=Gemma 2 2B, k=1572025.05 | 14.5 | 4.6 | |
| Retell-summarizeModel=GPT-4o mini, k=502025.05 | 14.1 | 11.6 | |
| Retell-summarizeModel=GPT-4o mini, k=892025.05 | 14.1 | 7.9 | |
| TopicGPT-liteModel=Phi-3.5-mini, k=5382025.05 | 13.9 | 4.8 | |
| TopicGPT-liteModel=Gemma 2 2B, k=1572025.05 | 13.9 | 20.1 | |
| Retell-paraphraseModel=Gemma 2 2B, k=1572025.05 | 13.8 | 4.4 | |
| Retell-describeModel=GPT-4o mini, k=502025.05 | 13.7 | 11.8 | |
| Retell-paraphraseModel=Llama 3.1 8B, k=2062025.05 | 13.5 | 3.6 | |
| Retell-describeModel=Gemma 2 2B, k=502025.05 | 13.3 | 11.1 | |
| Retell-summarizeModel=Phi-3.5-mini, k=502025.05 | 13.1 | 11.2 | |
| Retell-describeModel=Phi-3.5-mini, k=502025.05 | 13.1 | 11.6 | |
| Retell-summarizeModel=Gemma 2 2B, k=502025.05 | 13.1 | 11.1 | |
| Retell-summarizeModel=Llama 3.1 8B, k=502025.05 | 12.8 | 10.3 | |
| Retell-describeModel=Llama 3.1 8B, k=502025.05 | 12.8 | 10.7 | |
| Retell-paraphraseModel=Phi-3.5-mini, k=5382025.05 | 12.8 | 1.7 | |
| TopicGPT-liteModel=Llama 3.1 8B, k=2062025.05 | 12.7 | 21.1 | |
| TopicGPT-liteModel=GPT-4o mini, k=142025.05 | 12.7 | 24.8 | |
| Retell-paraphraseModel=Gemma 2 2B, k=502025.05 | 12.3 | 11 | |
| Retell-paraphraseModel=Llama 3.1 8B, k=502025.05 | 12.1 | 10.5 | |
| Retell-describeModel=GPT-4o mini, k=142025.05 | 11.8 | 24.3 | |
| default LDA (no LM)Model=Phi-3.5-mini, k=5382025.05 | 11.4 | 1.8 | |
| Retell-paraphraseModel=GPT-4o mini, k=892025.05 | 11.2 | 6.8 | |
| Retell-paraphraseModel=Phi-3.5-mini, k=502025.05 | 11 | 10.6 | |
| Retell-summarizeModel=GPT-4o mini, k=142025.05 | 11 | 24.4 | |
| default LDA (no LM)Model=Llama 3.1 8B, k=2062025.05 | 10.7 | 3.6 | |
| Retell-paraphraseModel=GPT-4o mini, k=502025.05 | 10.6 | 10.5 | |
| default LDA (no LM)Model=GPT-4o mini, k=892025.05 | 10.5 | 6.9 | |
| default LDA (no LM)Model=Gemma 2 2B, k=1572025.05 | 10.4 | 4.2 | |
| default LDA (no LM)Model=Llama 3.1 8B, k=502025.05 | 10 | 10.8 | |
| default LDA (no LM)Model=Phi-3.5-mini, k=502025.05 | 10 | 10.8 | |
| default LDA (no LM)Model=Gemma 2 2B, k=502025.05 | 10 | 10.8 | |
| default LDA (no LM)Model=GPT-4o mini, k=502025.05 | 10 | 10.8 | |
| Retell-paraphraseModel=GPT-4o mini, k=142025.05 | 10 | 22.6 | |
| default LDA (no LM)Model=GPT-4o mini, k=142025.05 | 9.2 | 22.2 |