Retrieval-Augmented Generation on NaturalQuestion 10 documents
66.17Average ScoreLongLLMLingua-rk + Cal.
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| LongLLMLingua-rk + Cal.Model=Vicuna2024.06 | 66.17 | — | — | — | |
| Attention sortingModel=Vicuna2024.06 | 65.06 | 72.54 | 59.54 | 63.12 | |
| Prompt reorderingModel=Vicuna2024.06 | 64.63 | — | — | — | |
| Calibrated attentionModel=Tulu2024.06 | 64.06 | 71.52 | 57.13 | 63.54 | |
| LongLLMLingua-rkModel=Vicuna2024.06 | 63.95 | — | — | — | |
| Calibrated attentionModel=Vicuna2024.06 | 63.07 | 70.84 | 62.61 | 55.78 | |
| LongLLMLingua-rk + Cal.Model=Tulu2024.06 | 61.31 | — | — | — | |
| Attention sortingModel=Tulu2024.06 | 60.71 | 62.52 | 56.43 | 63.2 | |
| Vanilla attentionModel=Vicuna2024.06 | 60.39 | 74.35 | 54.83 | 52.01 | |
| Prompt reorderingModel=Tulu2024.06 | 58.77 | — | — | — | |
| LongLLMLingua-rkModel=Tulu2024.06 | 56.39 | — | — | — | |
| Vanilla attentionModel=Tulu2024.06 | 56.19 | 70.5 | 48.81 | 49.26 |