Commonsense Reasoning on SIQA (test)
57AccuracyLoRA
Evaluation results
| Method | Links | |
|---|---|---|
| LoRABackbone=Llama-2-13b, Rank=32, Param × 10⁵=262.1, Protocol=Fine-tuning2025.06 | 57 | |
| LoRABackbone=Llama-2-13b, Rank=8, Param × 10⁵=65.5, Protocol=Fine-tuning2025.06 | 56 | |
| LoRABackbone=Llama-2-7b, Rank=16, Param × 10⁵=83.9, Protocol=Fine-tuning2025.06 | 55.3 | |
| LoTRBackbone=Llama-2-13b, Rank=64, Param × 10⁵=9.83, Protocol=Fine-tuning2025.06 | 55.1 | |
| LoRABackbone=Llama-2-7b, Rank=8, Param × 10⁵=41.9, Protocol=Fine-tuning2025.06 | 54.5 | |
| MetaTT-4DBackbone=Llama-2-13b, Rank=64, Param × 10⁵=8.3, Protocol=Fine-tuning2025.06 | 54.11 | |
| LoTRBackbone=Llama-2-7b, Rank=64, Param × 10⁵=7.86, Protocol=Fine-tuning2025.06 | 54.1 | |
| MetaTT-4DBackbone=Llama-2-13b, Rank=16, Param × 10⁵=1.75, Protocol=Fine-tuning2025.06 | 53.9 | |
| MetaTT-4DBackbone=Llama-2-7b, Rank=64, Param × 10⁵=6.6, Protocol=Fine-tuning2025.06 | 53.8 | |
| LoTRBackbone=Llama-2-13b, Rank=16, Param × 10⁵=1.84, Protocol=Fine-tuning2025.06 | 52.7 | |
| LoTRBackbone=Llama-2-7b, Rank=16, Param × 10⁵=1.47, Protocol=Fine-tuning2025.06 | 52 | |
| MetaTT-4DBackbone=Llama-2-7b, Rank=16, Param × 10⁵=1.40, Protocol=Fine-tuning2025.06 | 51.1 | |
| VeRABackbone=Llama-2-13b, Rank=256, Param × 10⁵=4.3, Protocol=Fine-tuning2025.06 | 49.7 | |
| VeRABackbone=Llama-2-7b, Rank=1024, Param × 10⁵=3.27, Protocol=Fine-tuning2025.06 | 47.68 | |
| Zero ShotBackbone=Llama-2-13b, Protocol=Zero-shot2025.06 | 47.2 | |
| Zero ShotBackbone=Llama-2-7b, Protocol=Zero-shot2025.06 | 46.1 | |
| PGM 6 / 6 (1024)distillation=Before2025.05 | 40.28 | |
| PGM 8 / 8distillation=Before2025.05 | 39.97 | |
| PGM 8 / 8distillation=After Distillation (SDTT)2025.05 | 39.61 | |
| PGM 6 / 6 (1024)distillation=After Distillation (SDTT)2025.05 | 39.25 | |
| MDLMdistillation=Before2025.05 | 38.84 | |
| MDLMdistillation=After Distillation (SDTT)2025.05 | 38.79 |