General Language Understanding on GLUE
92.5AccuracyFull FT
Evaluation results
| Method | Links | |
|---|---|---|
| Full FTBackbone=Llama3.1, Parameters=405B2025.12 | 92.5 | |
| HybridBackbone=Llama3.1, Parameters=405B2025.12 | 92.3 | |
| LoRA-GABackbone=Llama3.1, Parameters=405B2025.12 | 91.9 | |
| BOFTBackbone=Llama3.1, Parameters=405B2025.12 | 91.7 | |
| LoRABackbone=Llama3.1, Parameters=405B2025.12 | 91.3 | |
| uRNNBackbone=Llama3.1, Parameters=405B2025.12 | 90.9 | |
| Full FTBackbone=Llama3.3, Parameters=70B2025.12 | 90.7 | |
| HybridBackbone=Llama3.3, Parameters=70B2025.12 | 90.2 | |
| LoRA-GABackbone=Llama3.3, Parameters=70B2025.12 | 89.6 | |
| BOFTBackbone=Llama3.3, Parameters=70B2025.12 | 89.4 | |
| LoRABackbone=Llama3.3, Parameters=70B2025.12 | 89.1 | |
| Full FTBackbone=Wizard-Vicuna, Parameters=30B2025.12 | 88.9 | |
| uRNNBackbone=Llama3.3, Parameters=70B2025.12 | 88.5 | |
| HybridBackbone=Wizard-Vicuna, Parameters=30B2025.12 | 88.4 | |
| LoRA-GABackbone=Wizard-Vicuna, Parameters=30B2025.12 | 87.9 | |
| BOFTBackbone=Wizard-Vicuna, Parameters=30B2025.12 | 87.8 | |
| LoRABackbone=Wizard-Vicuna, Parameters=30B2025.12 | 87.5 | |
| uRNNBackbone=Wizard-Vicuna, Parameters=30B2025.12 | 86.7 | |
| Full FTBackbone=BloomZ, Parameters=7B12025.12 | 85.8 | |
| NSP Context-Free Data Aug.Domain=Combined2026.01 | 85.74 | |
| NHP Loss Context-AwareDomain=News2026.01 | 85.49 | |
| HybridBackbone=BloomZ, Parameters=7B12025.12 | 85.1 | |
| LoRA-GABackbone=BloomZ, Parameters=7B12025.12 | 84.4 | |
| NSP Loss Context-AwareDomain=YouTube2026.01 | 84.33 | |
| NSP Context-Aware Data Aug.Domain=Combined2026.01 | 84.33 | |
| BOFTBackbone=BloomZ, Parameters=7B12025.12 | 84.3 | |
| NSP Loss Context-FreeDomain=News2026.01 | 84.27 | |
| NSP Loss Context-FreeDomain=ArXiv2026.01 | 84.23 | |
| LoRABackbone=BloomZ, Parameters=7B12025.12 | 84 | |
| NHP Loss Context-AwareDomain=ArXiv2026.01 | 83.63 | |
| NHP Loss Context-FreeDomain=ArXiv2026.01 | 83.63 | |
| NHP Context-Aware Data Aug.Domain=YouTube2026.01 | 83.63 | |
| uRNNBackbone=BloomZ, Parameters=7B12025.12 | 83.5 | |
| NTP Hypernym Baseline Fine-TunedDomain=YouTube2026.01 | 83.43 | |
| MSRSBackbone=Qwen2-7B-Instruct2025.08 | 83.2 | |
| NHP Context-Aware Data Aug.Domain=News2026.01 | 82.87 | |
| NHP Context-Free Data Aug.Domain=Combined2026.01 | 82.82 | |
| NTP Hypernym Baseline Fine-TunedDomain=News2026.01 | 82.62 | |
| VanillaBackbone=Qwen2-7B-Instruct2025.08 | 82.5 | |
| NHP Loss Context-FreeDomain=News2026.01 | 81.66 | |
| NHP Loss Context-FreeDomain=Combined2026.01 | 81.26 | |
| NHP Context-Aware Data Aug.Domain=Combined2026.01 | 81.11 | |
| NSP Context-Free Data Aug.Domain=YouTube2026.01 | 80.81 | |
| NHP Loss Context-AwareDomain=Combined2026.01 | 80.65 | |
| NSP Loss Context-AwareDomain=News2026.01 | 80.3 | |
| NSP Context-Free Data Aug.Domain=News2026.01 | 80.3 | |
| NSP Loss Context-FreeDomain=YouTube2026.01 | 78.44 | |
| MSRSBackbone=Llama-3-8B-Instruct2025.08 | 77.5 | |
| NSP Loss Context-AwareDomain=ArXiv2026.01 | 76.98 | |
| NHP Context-Free Data Aug.Domain=News2026.01 | 76.62 | |
| NSP Context-Aware Data Aug.Domain=YouTube2026.01 | 76.52 | |
| NSP Context-Aware Data Aug.Domain=ArXiv2026.01 | 76.37 | |
| NTP Synonym Baseline Fine-TunedDomain=ArXiv2026.01 | 75.26 | |
| NTP Synonym Baseline Fine-TunedDomain=Combined2026.01 | 73.95 | |
| VanillaBackbone=Llama-3-8B-Instruct2025.08 | 72.6 | |
| NTP Hypernym Baseline Fine-TunedDomain=ArXiv2026.01 | 71.49 | |
| MSRSBackbone=Mistral-7B-v0.32025.08 | 70.7 | |
| NSP Context-Aware Data Aug.Domain=News2026.01 | 69.22 | |
| VanillaBackbone=Mistral-7B-v0.32025.08 | 68.1 | |
| NHP Context-Free Data Aug.Domain=ArXiv2026.01 | 67.76 | |
| NTP Hypernym Baseline Fine-TunedDomain=Combined2026.01 | 62.47 | |
| NSP Loss Context-FreeDomain=Combined2026.01 | 58.79 | |
| NSP Context-Free Data Aug.Domain=ArXiv2026.01 | 58.29 | |
| NHP Context-Free Data Aug.Domain=YouTube2026.01 | 55.87 | |
| NTP Synonym Baseline Fine-TunedDomain=YouTube2026.01 | 49.87 | |
| NHP Context-Aware Data Aug.Domain=ArXiv2026.01 | 47.61 | |
| TransformerTraining=compute-matched, Training dataset=FineWeb-edu2026.05 | 40.3 | |
| NTP Synonym Baseline Fine-TunedDomain=News2026.01 | 34.96 | |
| NSP Loss Context-AwareDomain=Combined2026.01 | 33.65 | |
| Base Model Fine-TunedFine-Tuned=true2026.01 | 33.65 | |
| NHP Loss Context-AwareDomain=YouTube2026.01 | 32.54 | |
| NHP Loss Context-FreeDomain=YouTube2026.01 | 32.54 | |
| Base ModelZero-Shot=true2026.01 | 32.04 | |
| SRMTraining=compute-matched, Training dataset=FineWeb-edu2026.05 | 0.4577 | |
| MambaTraining=compute-matched, Training dataset=FineWeb-edu2026.05 | 0.4552 |