Long-context Language Modeling on LongBench v2
29.62AccuracyFoLoRA
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| FoLoRAFine-tuning dataset=MetaMathQA, Calibration data=NQ Open, Calibration model size=4B2026.05 | 29.62 | — | — | — | — | — | — | — | |
| Qwen3-1.7BType=Base Model2026.05 | 29.42 | — | — | — | — | — | — | — | |
| LoRA-NullFine-tuning dataset=MetaMathQA, Calibration data=model-generated pretraining-proxy, Calibration model size=4B2026.05 | 29.42 | — | — | — | — | — | — | — | |
| LoRAFine-tuning dataset=MetaMathQA2026.05 | 29.22 | — | — | — | — | — | — | — | |
| OPLoRAFine-tuning dataset=MetaMathQA2026.05 | 29.02 | — | — | — | — | — | — | — | |
| FoLoRAFine-tuning dataset=MetaMathQA, Calibration data=NQ Open2026.05 | 29.02 | — | — | — | — | — | — | — | |
| FoLoRAFine-tuning dataset=MetaMathQA, Calibration data=NQ Open, Calibration model size=1.7B2026.05 | 28.82 | — | — | — | — | — | — | — | |
| MiLoRAFine-tuning dataset=MetaMathQA2026.05 | 28.03 | — | — | — | — | — | — | — | |
| LoRA-NullFine-tuning dataset=MetaMathQA, Calibration data=model-generated pretraining-proxy2026.05 | 28.03 | — | — | — | — | — | — | — | |
| FoLoRAFine-tuning dataset=MetaMathQA, Calibration data=NQ Open, Calibration model size=0.6B2026.05 | 28.03 | — | — | — | — | — | — | — | |
| CorDAFine-tuning dataset=MetaMathQA, Calibration data=model-generated pretraining-proxy2026.05 | 27.83 | — | — | — | — | — | — | — | |
| CorDAFine-tuning dataset=MetaMathQA, Calibration data=model-generated pretraining-proxy, Calibration model size=4B2026.05 | 27.83 | — | — | — | — | — | — | — | |
| AdmTreeBackbone=LLama-2-7B2025.12 | — | 22.9 | 19.2 | 27.2 | 25.6 | 24 | 24.2 | 23.1 | |
| AdmTreeBackbone=Qwen-2-7B2025.12 | — | 34.9 | 25.6 | 33.3 | 38.5 | 22 | 33.3 | 31.9 | |
| AutoCompressorBackbone=LLama-2-7B2025.12 | — | 3.1 | 1.5 | 2.6 | 0.8 | 8.9 | 7.2 | 3.1 | |
| BeaconBackbone=LLama-2-7B2025.12 | — | 20 | 14.4 | 25.9 | 18 | 22 | 21.2 | 19.5 | |
| BeaconBackbone=Qwen-2-7B2025.12 | — | 34.3 | 18.4 | 29.6 | 30.8 | 16 | 33.3 | 28.3 | |
| ICAEBackbone=LLama-2-7B2025.12 | — | 4.2 | 2.7 | 2.1 | 3.5 | 9.6 | 8.5 | 4.1 | |
| LongLLMLinguaBackbone=LLama-2-7B2025.12 | — | 20 | 17.6 | 8.6 | 2.6 | 16 | 11.8 | 14.9 | |
| LongLLMLinguaBackbone=Qwen-2-7B2025.12 | — | 22.3 | 19.6 | 12.3 | 8.5 | 13.2 | 20.1 | 18.1 | |
| SnapKVBackbone=LLama-2-7B2025.12 | — | 18.1 | 15.2 | 23.1 | 16.2 | 18.3 | 19.2 | 18.1 | |
| SnapKVBackbone=Qwen-2-7B2025.12 | — | 29.5 | 27.2 | 28.6 | 29.1 | 15.3 | 27.9 | 27.9 |