Retrieval-Augmented Generation on NaturalQuestion 20 documents
0.6289Average ScoreAttention sorting
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Attention sortingModel=Vicuna2024.06 | 0.6289 | 0.6937 | 0.5691 | 0.6241 | |
| LongLLMLingua-rk + Cal.Model=Vicuna2024.06 | 0.6222 | — | — | — | |
| LongLLMLingua-rkModel=Vicuna2024.06 | 0.5992 | — | — | — | |
| Prompt reorderingModel=Vicuna2024.06 | 0.5868 | — | — | — | |
| Calibrated attentionModel=Vicuna2024.06 | 0.5811 | 0.664 | 0.5619 | 0.5175 | |
| Vanilla attentionModel=Vicuna2024.06 | 0.5664 | 0.7193 | 0.4734 | 0.5065 | |
| Calibrated attentionModel=Tulu2024.06 | 0.5391 | 0.5717 | 0.4308 | 0.615 | |
| LongLLMLingua-rk + Cal.Model=Tulu2024.06 | 0.4734 | — | — | — | |
| Vanilla attentionModel=Tulu2024.06 | 0.4628 | 0.5694 | 0.3532 | 0.4659 | |
| Prompt reorderingModel=Tulu2024.06 | 0.4464 | — | — | — | |
| Attention sortingModel=Tulu2024.06 | 0.4457 | 0.4557 | 0.4312 | 0.4504 | |
| LongLLMLingua-rkModel=Tulu2024.06 | 0.439 | — | — | — |