Summarization on TREC (Acc, AMSR)
65.2AccuracyReasonCache
Evaluation results
| Method | Links | ||
|---|---|---|---|
| ReasonCacheModel=QwQ 32B2025.07 | 65.2 | 18.1 | |
| DenseModel=QwQ 32B2025.07 | 64.8 | 0 | |
| DenseModel=Phi 4 reasoning plus2025.07 | 63.5 | 0 | |
| DenseModel=DeepSeek R1 Distill Qwen 32B2025.07 | 62 | 0 | |
| ReasonCacheModel=DeepSeek R1 Distill Qwen 32B2025.07 | 61 | 15.9 | |
| SnapKVModel=QwQ 32B2025.07 | 61 | 15.5 | |
| ReasonCacheModel=Phi 4 reasoning plus2025.07 | 61 | 14.2 | |
| QuestModel=QwQ 32B2025.07 | 56 | 17.8 | |
| SnapKVModel=DeepSeek R1 Distill Qwen 32B2025.07 | 51 | 16 | |
| StreamingLLMModel=QwQ 32B2025.07 | 49 | 17.3 | |
| QuestModel=DeepSeek R1 Distill Qwen 32B2025.07 | 48 | 15.8 | |
| StreamingLLMModel=DeepSeek R1 Distill Qwen 32B2025.07 | 47 | 15.5 | |
| SnapKVModel=Phi 4 reasoning plus2025.07 | 47 | 13.7 | |
| QuestModel=Phi 4 reasoning plus2025.07 | 47 | 13.6 | |
| StreamingLLMModel=Phi 4 reasoning plus2025.07 | 45 | 13.4 |