Commonsense Reasoning on ARC Challenge (test)
57.8AccuracyLoRA
Evaluation results
| Method | Links | |
|---|---|---|
| LoRABackbone=Llama-2-13b, Rank=32, Param × 10⁵=262.1, Protocol=Fine-tuning2025.06 | 57.8 | |
| LoRABackbone=Llama-2-13b, Rank=8, Param × 10⁵=65.5, Protocol=Fine-tuning2025.06 | 57.6 | |
| LoTRBackbone=Llama-2-13b, Rank=64, Param × 10⁵=9.83, Protocol=Fine-tuning2025.06 | 57 | |
| MetaTT-4DBackbone=Llama-2-13b, Rank=64, Param × 10⁵=8.3, Protocol=Fine-tuning2025.06 | 56.9 | |
| LoTRBackbone=Llama-2-13b, Rank=16, Param × 10⁵=1.84, Protocol=Fine-tuning2025.06 | 55.5 | |
| MetaTT-4DBackbone=Llama-2-13b, Rank=16, Param × 10⁵=1.75, Protocol=Fine-tuning2025.06 | 55.2 | |
| LoRABackbone=Llama-2-7b, Rank=8, Param × 10⁵=41.9, Protocol=Fine-tuning2025.06 | 53.3 | |
| VeRABackbone=Llama-2-13b, Rank=256, Param × 10⁵=4.3, Protocol=Fine-tuning2025.06 | 53.2 | |
| LoRABackbone=Llama-2-7b, Rank=16, Param × 10⁵=83.9, Protocol=Fine-tuning2025.06 | 52.6 | |
| LoTRBackbone=Llama-2-7b, Rank=64, Param × 10⁵=7.86, Protocol=Fine-tuning2025.06 | 52.1 | |
| MetaTT-4DBackbone=Llama-2-7b, Rank=64, Param × 10⁵=6.6, Protocol=Fine-tuning2025.06 | 51.8 | |
| LoTRBackbone=Llama-2-7b, Rank=16, Param × 10⁵=1.47, Protocol=Fine-tuning2025.06 | 51.2 | |
| MetaTT-4DBackbone=Llama-2-7b, Rank=16, Param × 10⁵=1.40, Protocol=Fine-tuning2025.06 | 50.2 | |
| Zero ShotBackbone=Llama-2-13b, Protocol=Zero-shot2025.06 | 48.9 | |
| VeRABackbone=Llama-2-7b, Rank=1024, Param × 10⁵=3.27, Protocol=Fine-tuning2025.06 | 47.8 | |
| Zero ShotBackbone=Llama-2-7b, Protocol=Zero-shot2025.06 | 46.5 | |
| BAM SSMaxModel Size=1B, Activation=SSMax, Scale=Large-Scale2025.05 | 41.32 | |
| RoPE SSMaxModel Size=1B, Activation=SSMax, Scale=Large-Scale2025.05 | 41.23 |