General Language Understanding on NLU Suite (Average Accuracy)
89.9Average AccuracyReLoRA
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| ReLoRAUpdate Source=OpenOrca, Evaluation Protocol=Task-adaptation Methods2026.05 | 89.9 | — | — | — | — | — | — | |
| ReLoRAUpdate Source=OpenPlatypus, Evaluation Protocol=Task-adaptation Methods2026.05 | 89.9 | — | — | — | — | — | — | |
| ReLoRAModel Backbone=LLaMA3.1-8B, Adaptation Protocol=Task-adaptation Methods2026.05 | 89.9 | 83.9 | 97 | 95.3 | 76.6 | 93.5 | 93.1 | |
| ReLoRAUpdate Source=AlpacaGPT4, Evaluation Protocol=Task-adaptation Methods2026.05 | 89.6 | — | — | — | — | — | — | |
| PortLLM+FTUpdate Source=OpenPlatypus, Evaluation Protocol=Task-adaptation Methods2026.05 | 89.3 | — | — | — | — | — | — | |
| PortLLM+FTUpdate Source=OpenOrca, Evaluation Protocol=Task-adaptation Methods2026.05 | 89.2 | — | — | — | — | — | — | |
| PortLLM+FTModel Backbone=LLaMA3.1-8B, Adaptation Protocol=Task-adaptation Methods2026.05 | 89.2 | 82.5 | 96.6 | 95.2 | 76.4 | 92.6 | 92.1 | |
| PortLLM+FTUpdate Source=AlpacaGPT4, Evaluation Protocol=Task-adaptation Methods2026.05 | 89.1 | — | — | — | — | — | — | |
| LoRA-ScratchUpdate Source=OpenOrca, Evaluation Protocol=Task-adaptation Methods2026.05 | 88.8 | — | — | — | — | — | — | |
| ORALUpdate Source=OpenOrca, Evaluation Protocol=Task-adaptation Methods2026.05 | 88.8 | — | — | — | — | — | — | |
| LoRA-ScratchModel Backbone=LLaMA3.1-8B, Adaptation Protocol=Task-adaptation Methods2026.05 | 88.8 | 81.4 | 96.3 | 95.2 | 76.2 | 91.6 | 91.8 | |
| ORALModel Backbone=LLaMA3.1-8B, Adaptation Protocol=Task-adaptation Methods2026.05 | 88.8 | 81.2 | 96.4 | 95 | 76.1 | 91.9 | 92 | |
| ORALUpdate Source=AlpacaGPT4, Evaluation Protocol=Task-adaptation Methods2026.05 | 88.4 | — | — | — | — | — | — | |
| LoRA-ScratchUpdate Source=AlpacaGPT4, Evaluation Protocol=Task-adaptation Methods2026.05 | 88.3 | — | — | — | — | — | — | |
| LoRA-ScratchUpdate Source=OpenPlatypus, Evaluation Protocol=Task-adaptation Methods2026.05 | 88.2 | — | — | — | — | — | — | |
| ORALUpdate Source=OpenPlatypus, Evaluation Protocol=Task-adaptation Methods2026.05 | 88.2 | — | — | — | — | — | — | |
| ReLoRAModel Backbone=Mistral-7B, Adaptation Protocol=Task-adaptation Methods2026.05 | 88.1 | 76.6 | 96.9 | 95.5 | 76.3 | 91.2 | 92.1 | |
| PortLLM+FTModel Backbone=Mistral-7B, Adaptation Protocol=Task-adaptation Methods2026.05 | 87.6 | 76 | 96.6 | 95.3 | 74.9 | 91 | 91.9 | |
| LoRA-ScratchModel Backbone=Mistral-7B, Adaptation Protocol=Task-adaptation Methods2026.05 | 87.4 | 75.3 | 96.8 | 95.3 | 73.4 | 91.5 | 92.1 | |
| ReLoRAModel Backbone=LLaMA2-7B, Adaptation Protocol=Task-adaptation Methods2026.05 | 87.1 | 70.9 | 97.5 | 95.4 | 75.8 | 91 | 91.7 | |
| ORALModel Backbone=Mistral-7B, Adaptation Protocol=Task-adaptation Methods2026.05 | 87.1 | 75.5 | 96.7 | 95.2 | 72.9 | 90.8 | 91.6 | |
| PortLLM+FTModel Backbone=LLaMA2-7B, Adaptation Protocol=Task-adaptation Methods2026.05 | 86.5 | 68.3 | 97.1 | 95.4 | 75.7 | 90.7 | 91.6 | |
| LoRA-ScratchModel Backbone=LLaMA2-7B, Adaptation Protocol=Task-adaptation Methods2026.05 | 86.3 | 67.9 | 96.6 | 95.3 | 75.5 | 90.7 | 91.8 | |
| ORALModel Backbone=LLaMA2-7B, Adaptation Protocol=Task-adaptation Methods2026.05 | 86.2 | 67.8 | 96.7 | 95.2 | 75.5 | 90.8 | 91.4 | |
| Evolved LLM Θ′Update Source=OpenOrca, Evaluation Protocol=Zero-shot2026.05 | 29.8 | — | — | — | — | — | — | |
| Evolved LLM Θ'Model Backbone=LLaMA3.1-8B, Adaptation Protocol=Zero-shot2026.05 | 29.8 | 24.5 | 56 | 29.2 | 3.7 | 32.3 | 33 | |
| Pretrained LLM ΘEvaluation Protocol=Zero-shot2026.05 | 29.6 | — | — | — | — | — | — | |
| Evolved LLM Θ'Model Backbone=LLaMA2-7B, Adaptation Protocol=Zero-shot2026.05 | 29.1 | 25 | 52 | 26.3 | 5.1 | 33.3 | 33.1 | |
| Evolved LLM Θ′Update Source=AlpacaGPT4, Evaluation Protocol=Zero-shot2026.05 | 29 | — | — | — | — | — | — | |
| Evolved LLM Θ'Model Backbone=Mistral-7B, Adaptation Protocol=Zero-shot2026.05 | 28.9 | 25.2 | 50.9 | 24.2 | 6.2 | 33.3 | 33.3 | |
| Evolved LLM Θ′Update Source=OpenPlatypus, Evaluation Protocol=Zero-shot2026.05 | 28.7 | — | — | — | — | — | — |