Language Modeling on Cross-family Language Modeling
35.08ScoreLoRA
Evaluation results
| Method | Links | |
|---|---|---|
| LoRAModel=Qwen2.5-7B, KV retention=100%2026.04 | 35.08 | |
| MoE-LoRAModel=Qwen2.5-7B, KV retention=100%2026.04 | 33.72 | |
| Sub-TokenModel=Qwen2.5-7B, KV retention=75%2026.04 | 31.41 | |
| LoRAModel=Mistral-7B, KV retention=100%2026.04 | 28.71 | |
| MoE-LoRAModel=Mistral-7B, KV retention=100%2026.04 | 26.16 | |
| Sub-TokenModel=Mistral-7B, KV retention=75%2026.04 | 24.81 |