Accuracy and Latency on MT-Bench (Multi-turn conversation)
8.54AccuracyVanilla
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Vanilla2025.12 | 8.54 | 10.55 | |
| Rhea2025.12 | 8.49 | 11.95 | |
| Reply-Soft-Compress2025.12 | 8.31 | 11.61 | |
| Summary2025.12 | 8.12 | 13.66 | |
| MemochaBase Model=Vicuna-7B2025.12 | 7.52 | 6.21 | |
| LongAlpacaBase Model=Vicuna-7B2025.12 | 6.69 | 5.06 | |
| LlmLingua22025.12 | 6.55 | 10.07 |