Reading Comprehension on NarrativeQA
6.804Overall ScoreSLM
Evaluation results
| Method | Links | ||
|---|---|---|---|
| SLMBackbone=Llama3-70B-Instruct, ratio=0.52025.02 | 6.804 | 2.069 | |
| SnapKVBackbone=Llama3-70B-Instruct, ratio=0.52025.02 | 6.795 | 2.474 | |
| H2OBackbone=Llama3-70B-Instruct, ratio=0.52025.02 | 6.788 | 2.298 | |
| Llama3 (Full)Backbone=Llama3-70B-Instruct2025.02 | 6.285 | — | |
| ReFreeKVBackbone=Llama3-70B-Instruct, k=12025.02 | 6.042 | 2.345 | |
| Llama3 (Full)Backbone=Llama3-8B-Instruct2025.02 | 3.441 | — | |
| SnapKVBackbone=Llama3-8B-Instruct, ratio=0.52025.02 | 3.307 | 0.265 | |
| H2OBackbone=Llama3-8B-Instruct, ratio=0.52025.02 | 3.243 | 0.264 | |
| SLMBackbone=Llama3-8B-Instruct, ratio=0.52025.02 | 3.23 | 0.257 | |
| ReFreeKVBackbone=Llama3-8B-Instruct, k=12025.02 | 3.159 | 0.255 |