Summarization on Summarization (Edit Distance, Accuracy, Expense)
6,573Edit DistanceOracle Preference
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Oracle PreferenceDescription=Uses true user preference2024.04 | 6,573 | 100 | 1.67 | |
| CIPHER-5-MPNETk=5, Encoder model=MPNET2024.04 | 32,974 | 47.8 | 3 | |
| CIPHER-1-MPNETk=1, Encoder model=MPNET2024.04 | 33,926 | 52 | 2.74 | |
| CIPHER-5-BERTk=5, Encoder model=BERT2024.04 | 35,811 | 47.8 | 3.03 | |
| CIPHER-1-BERTk=1, Encoder model=BERT2024.04 | 37,637 | 56.5 | 2.81 | |
| ICL-edit-5-MPNETk=5, Encoder model=MPNET2024.04 | 38,560 | — | 8 | |
| ICL-edit-5-BERTk=5, Encoder model=BERT2024.04 | 39,734 | — | 7.96 | |
| CoT-edit-5-MPNETk=5, Encoder model=MPNET2024.04 | 40,747 | 23 | 6.82 | |
| CoT-edit-5-BERTk=5, Encoder model=BERT2024.04 | 41,088 | 23 | 6.92 | |
| No Learning2024.04 | 48,269 | — | 1.5 | |
| Continual LPIProtocol=Learn context-agnostic preferences2024.04 | 57,915 | 23.3 | 8.89 | |
| E-then-e LPIProtocol=Learn context-agnostic preferences2024.04 | 65,218 | 21.8 | 1.99 |