Long-context Retrieval on NIAH (Needle In A Haystack) Multi-task Qwen2.5-7B
96Accuracy (Single)fp16
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| fp16Backbone=Qwen2.5-7B-Instruct, Rate=fp162026.06 | 96 | 92.9 | 83.8 | 31.5 | 66.3 | 32 | 67.1 | |
| Block-GTQBackbone=Qwen2.5-7B-Instruct, Rate=K3V3, Calibration=2048-token WikiText-22026.06 | 96 | 91.4 | 78.8 | 30.6 | 62 | 31.8 | 65.1 | |
| Block-GTQBackbone=Qwen2.5-7B-Instruct, Rate=K3V2, Calibration=2048-token WikiText-22026.06 | 92.9 | 90.9 | 75.3 | 37.4 | 61.3 | 30.8 | 64.8 | |
| Block-GTQBackbone=Qwen2.5-7B-Instruct, Rate=K2V2, Calibration=2048-token WikiText-22026.06 | 85.9 | 77.8 | 73.2 | 40.1 | 54.7 | 29.1 | 60.1 | |
| KIVI-ScaleOnlyBackbone=Qwen2.5-7B-Instruct, Rate=K3V32026.06 | 59.1 | 44.4 | 37.9 | 20.7 | 32.9 | 16.3 | 35.2 | |
| KIVI-ScaleOnlyBackbone=Qwen2.5-7B-Instruct, Rate=K3V22026.06 | 59.1 | 42.4 | 24.8 | 23.2 | 24.1 | 19.5 | 32.2 | |
| KIVI-ScaleOnlyBackbone=Qwen2.5-7B-Instruct, Rate=K2V22026.06 | 20.7 | 14.1 | 10.6 | 12.1 | 6.6 | 7.9 | 12 | |
| TQ-MSEBackbone=Qwen2.5-7B-Instruct, Rate=K3V32026.06 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | |
| TQ-MSEBackbone=Qwen2.5-7B-Instruct, Rate=K3V22026.06 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | |
| TQ-MSEBackbone=Qwen2.5-7B-Instruct, Rate=K2V22026.06 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |