Salience detection on Wikipedia narratives
0.76Average AUCGPT-4.1
Evaluation results
| Method | Links | |
|---|---|---|
| GPT-4.12026.01 | 0.76 | |
| GPT-52026.01 | 0.76 | |
| GPT-5 mini2026.01 | 0.72 | |
| GPT-4.1 mini2026.01 | 0.7 | |
| Narrative Twinstraining_negatives=Only in-story negatives, operation=Summariz.2026.01 | 0.66 | |
| Narrative Twinstraining_negatives=LLM-negatives + in-story negatives, operation=Summariz.2026.01 | 0.65 | |
| Dropout Twinstraining_negatives=Only in-story negatives, operation=Summariz.2026.01 | 0.65 | |
| Dropout Twinstraining_negatives=LLM-negatives + in-story negatives, operation=Summariz.2026.01 | 0.63 | |
| Masked LMtraining_negatives=LLM-negatives + in-story negatives, operation=Summariz.2026.01 | 0.6 | |
| Narrative Twinstraining_negatives=LLM-negatives + in-story negatives, operation=Deletion2026.01 | 0.6 | |
| Narrative Twinstraining_negatives=LLM-negatives + in-story negatives, operation=Shifting2026.01 | 0.6 | |
| Narrative Twinstraining_negatives=Only in-story negatives, operation=Deletion2026.01 | 0.6 | |
| Narrative Twinstraining_negatives=Only in-story negatives, operation=Shifting2026.01 | 0.6 | |
| Masked LMtraining_negatives=Only in-story negatives, operation=Summariz.2026.01 | 0.59 | |
| Dropout Twinstraining_negatives=LLM-negatives + in-story negatives, operation=Deletion2026.01 | 0.58 | |
| Dropout Twinstraining_negatives=LLM-negatives + in-story negatives, operation=Shifting2026.01 | 0.58 | |
| Dropout Twinstraining_negatives=Only in-story negatives, operation=Shifting2026.01 | 0.57 | |
| Increasing2026.01 | 0.56 | |
| Masked LMtraining_negatives=LLM-negatives + in-story negatives, operation=Deletion2026.01 | 0.56 | |
| Masked LMtraining_negatives=Only in-story negatives, operation=Deletion2026.01 | 0.56 | |
| Dropout Twinstraining_negatives=Only in-story negatives, operation=Deletion2026.01 | 0.56 | |
| Masked LMtraining_negatives=LLM-negatives + in-story negatives, operation=Shifting2026.01 | 0.54 | |
| Narrative Twinstraining_negatives=LLM-negatives + in-story negatives, operation=Disruption2026.01 | 0.54 | |
| Masked LMtraining_negatives=Only in-story negatives, operation=Shifting2026.01 | 0.54 | |
| Narrative Twinstraining_negatives=Only in-story negatives, operation=Disruption2026.01 | 0.54 | |
| Dropout Twinstraining_negatives=LLM-negatives + in-story negatives, operation=Disruption2026.01 | 0.53 | |
| Dropout Twinstraining_negatives=Only in-story negatives, operation=Disruption2026.01 | 0.53 | |
| Random2026.01 | 0.5 | |
| Masked LMtraining_negatives=LLM-negatives + in-story negatives, operation=Disruption2026.01 | 0.49 | |
| Masked LMtraining_negatives=Only in-story negatives, operation=Disruption2026.01 | 0.49 | |
| Decreasing2026.01 | 0.44 |