Summarization on TREC (Acc, throughput)
65.2AccuracyReasonCache
Evaluation results
| Method | Links | ||
|---|---|---|---|
| ReasonCacheModel=QwQ 32B2025.07 | 65.2 | 287.59 | |
| DenseModel=QwQ 32B2025.07 | 64.8 | 178.37 | |
| DenseModel=Phi 4 reasoning plus2025.07 | 63.5 | 391.85 | |
| DenseModel=DeepSeek R1 Distill Qwen 32B2025.07 | 62 | 262.38 | |
| ReasonCacheModel=DeepSeek R1 Distill Qwen 32B2025.07 | 61 | 416.84 | |
| ReasonCacheModel=Phi 4 reasoning plus2025.07 | 61 | 556.84 | |
| RandomModel=QwQ 32B2025.07 | 33.3 | 286.98 | |
| RandomModel=Phi 4 reasoning plus2025.07 | 31.5 | 552.08 | |
| RandomModel=DeepSeek R1 Distill Qwen 32B2025.07 | 28 | 406.4 |