Commonsense Reasoning on CommonsenseQA (val)
82.06AccuracyQwen-1.5 14B (Teacher)
Evaluation Results
| Method | Links | |
|---|---|---|
| Qwen-1.5 14B (Teacher)Model Scale=14B2024.07 | 82.06 | |
| TAIATraining Dataset=CoT-Collection, Model=Qwen1.5-7B, FT Method=LoRA, Infer Mode=TAIA2024.05 | 80.1 | |
| TAIATraining Dataset=CoT-Collection, Model=Qwen1.5-7B, FT Method=MOLORA, Infer Mode=TAIA2024.05 | 78.84 | |
| Qwen1.5-7BTraining Dataset=Base Model, Model=Qwen1.5-7B2024.05 | 78.3 | |
| TAIATraining Dataset=Alpaca-GPT4, Model=Qwen1.5-7B, FT Method=MOLORA, Infer Mode=TAIA2024.05 | 77.56 | |
| TAIATraining Dataset=Alpaca-GPT4, Model=LLaMA3-8B, FT Method=MOLORA, Infer Mode=TAIA2024.05 | 77.48 | |
| LoRATraining Dataset=Alpaca-GPT4, Model=Qwen1.5-7B, FT Method=LoRA, Infer Mode=Vanilla2024.05 | 77.31 | |
| TAIATraining Dataset=Alpaca-GPT4, Model=Qwen1.5-7B, FT Method=LoRA, Infer Mode=TAIA2024.05 | 77.31 | |
| MOLORATraining Dataset=Alpaca-GPT4, Model=LLaMA3-8B, FT Method=MOLORA, Infer Mode=Vanilla2024.05 | 77.15 | |
| MOLORATraining Dataset=Alpaca-GPT4, Model=Qwen1.5-7B, FT Method=MOLORA, Infer Mode=Vanilla2024.05 | 76.9 | |
| TAIATraining Dataset=Alpaca-GPT4, Model=LLaMA3-8B, FT Method=LoRA, Infer Mode=TAIA2024.05 | 75.92 | |
| LoRATraining Dataset=Alpaca-GPT4, Model=LLaMA3-8B, FT Method=LoRA, Infer Mode=Vanilla2024.05 | 75.68 | |
| MOLORATraining Dataset=CoT-Collection, Model=Qwen1.5-7B, FT Method=MOLORA, Infer Mode=Vanilla2024.05 | 75.59 | |
| LLaMA3-8BTraining Dataset=Base Model, Model=LLaMA3-8B2024.05 | 73.71 | |
| TAIATraining Dataset=CoT-Collection, Model=LLaMA3-8B, FT Method=MOLORA, Infer Mode=TAIA2024.05 | 73.22 | |
| LoRATraining Dataset=CoT-Collection, Model=Qwen1.5-7B, FT Method=LoRA, Infer Mode=Vanilla2024.05 | 72.65 | |
| MOLORATraining Dataset=CoT-Collection, Model=LLaMA3-8B, FT Method=MOLORA, Infer Mode=Vanilla2024.05 | 71.74 | |
| TAIATraining Dataset=CoT-Collection, Model=LLaMA3-8B, FT Method=LoRA, Infer Mode=TAIA2024.05 | 71.25 | |
| LoRATraining Dataset=CoT-Collection, Model=LLaMA3-8B, FT Method=LoRA, Infer Mode=Vanilla2024.05 | 67.98 | |
| TAIATraining Dataset=CoT-Collection, Model=Qwen1.5-1.8B, FT Method=MOLORA, Infer Mode=TAIA2024.05 | 67.73 | |
| TAIATraining Dataset=CoT-Collection, Model=Qwen1.5-1.8B, FT Method=LoRA, Infer Mode=TAIA2024.05 | 67.32 | |
| Qwen-1.5 1.8B + DDKDistillation=DDK2024.07 | 66.75 | |
| Qwen-1.5 1.8B + KDDistillation=KD2024.07 | 66.26 | |
| Qwen-1.5 1.8B + TEDDistillation=TED2024.07 | 65.27 | |
| Qwen-1.5 1.8B + CPT & DoReMiDistillation=CPT & DoReMi2024.07 | 65.11 | |
| MOLORATraining Dataset=Alpaca-GPT4, Model=LLaMA2-7B, FT Method=MOLORA, Infer Mode=Vanilla2024.05 | 64.95 | |
| Qwen-1.5 1.8B + CPTDistillation=CPT2024.07 | 64.78 | |
| Qwen-1.5 1.8B (Student)Model Scale=1.8B2024.07 | 64.7 | |
| Qwen-1.5 1.8B + MiniLLMDistillation=MiniLLM2024.07 | 64.46 | |
| TAIATraining Dataset=Alpaca-GPT4, Model=Qwen1.5-1.8B, FT Method=LoRA, Infer Mode=TAIA2024.05 | 64.29 | |
| LoRATraining Dataset=Alpaca-GPT4, Model=LLaMA2-7B, FT Method=LoRA, Infer Mode=Vanilla2024.05 | 63.96 | |
| TAIATraining Dataset=Alpaca-GPT4, Model=LLaMA2-7B, FT Method=MOLORA, Infer Mode=TAIA2024.05 | 63.8 | |
| TAIATraining Dataset=CoT-Collection, Model=LLaMA2-7B, FT Method=MOLORA, Infer Mode=TAIA2024.05 | 63.49 | |
| TAIATraining Dataset=Alpaca-GPT4, Model=Qwen1.5-1.8B, FT Method=MOLORA, Infer Mode=TAIA2024.05 | 63.47 | |
| TAIATraining Dataset=Alpaca-GPT4, Model=LLaMA2-7B, FT Method=LoRA, Infer Mode=TAIA2024.05 | 63.06 | |
| MOLORATraining Dataset=CoT-Collection, Model=LLaMA2-7B, FT Method=MOLORA, Infer Mode=Vanilla2024.05 | 61.55 | |
| TAIATraining Dataset=CoT-Collection, Model=LLaMA2-7B, FT Method=LoRA, Infer Mode=TAIA2024.05 | 60.77 | |
| MOLORATraining Dataset=Alpaca-GPT4, Model=Qwen1.5-1.8B, FT Method=MOLORA, Infer Mode=Vanilla2024.05 | 60.69 | |
| LoRATraining Dataset=Alpaca-GPT4, Model=Qwen1.5-1.8B, FT Method=LoRA, Infer Mode=Vanilla2024.05 | 60.44 | |
| MOLORATraining Dataset=CoT-Collection, Model=Qwen1.5-1.8B, FT Method=MOLORA, Infer Mode=Vanilla2024.05 | 58.8 | |
| Qwen1.5-1.8BTraining Dataset=Base Model, Model=Qwen1.5-1.8B2024.05 | 58.39 | |
| LoRATraining Dataset=CoT-Collection, Model=Qwen1.5-1.8B, FT Method=LoRA, Infer Mode=Vanilla2024.05 | 58.07 | |
| LoRATraining Dataset=CoT-Collection, Model=LLaMA2-7B, FT Method=LoRA, Infer Mode=Vanilla2024.05 | 56.43 | |
| LLaMA2-13BModel Size=13B, Shots=0-shot2024.07 | 52.17 | |
| LLaMA2-7BTraining Dataset=Base Model, Model=LLaMA2-7B2024.05 | 48.4 | |
| TinyLLaMA-1.1B + DDKModel Size=1.1B, Shots=0-shot2024.07 | 25.39 | |
| TinyLLaMA-1.1B + TEDModel Size=1.1B, Shots=0-shot2024.07 | 22.93 | |
| TinyLLaMA-1.1B + MiniLLMModel Size=1.1B, Shots=0-shot2024.07 | 22.93 | |
| TinyLLaMA-1.1B + KDModel Size=1.1B, Shots=0-shot2024.07 | 22.52 | |
| TinyLLaMA-1.1B + CPT & DoReMiModel Size=1.1B, Shots=0-shot2024.07 | 20.88 | |
| TinyLLaMA-1.1B + CPTModel Size=1.1B, Shots=0-shot2024.07 | 20.39 | |
| TinyLLaMA-1.1BModel Size=1.1B, Shots=0-shot2024.07 | 19.08 |