Question Answering on General Domain QA (ASQA, HotpotQA, NQ, TriviaQA, POPQA)
78ASQA ScorePISCO - Solar
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| PISCO - SolarCompression rate=x16, Number of retrieved documents=5, Backbone=Solar-10.7B2025.01 | 78 | 57 | 70 | 94 | 71 | 74 | |
| Solar-10.7BCompression rate=1x, Number of retrieved documents=52025.01 | 75 | 55 | 71 | 93 | 71 | 73 | |
| Mistral-7BCompression rate=1x, Number of retrieved documents=52025.01 | 74 | 51 | 69 | 92 | 70 | 71 | |
| PISCO - MistralCompression rate=x16, Number of retrieved documents=5, Backbone=Mistral-7B2025.01 | 72 | 48 | 65 | 90 | 66 | 68 | |
| PISCO - LlamaCompression rate=x16, Number of retrieved documents=5, Backbone=Llama-3.1-8B2025.01 | 72 | 50 | 64 | 91 | 66 | 69 | |
| Llama-3.1-8BCompression rate=1x, Number of retrieved documents=52025.01 | 71 | 50 | 65 | 90 | 68 | 69 | |
| PISCO - Mistral (x128)Compression rate=x128, Number of retrieved documents=5, Backbone=Mistral-7B2025.01 | 68 | 46 | 61 | 89 | 55 | 64 | |
| COCOMCompression rate=x16, Number of retrieved documents=52025.01 | 63 | 46 | 58 | 89 | 48 | 61 | |
| AutoCompressorCompression rate=x4, Number of retrieved documents=5, Backbone=Llama-2-8b2025.01 | 57 | 31 | 35 | 70 | 24 | 43 | |
| DODOCompression rate=x10, Number of retrieved documents=52025.01 | 52 | 38 | 48 | 82 | 47 | 53 | |
| ICAECompression rate=x4, Number of retrieved documents=52025.01 | 47 | 29 | 42 | 78 | 43 | 48 | |
| xRAG-mistral-7BCompression rate=x128, Number of retrieved documents=5, Backbone=Mistral-7B2025.01 | 34 | 27 | 32 | 77 | 33 | 40 |