List your company
ResearchFormerly WizWand

Commonsense Reasoning on CommonsenseQA (val)

82.06AccuracyQwen-1.5 14B (Teacher)
16.560833.565450.5767.5746May 30, 2024Jun 8, 2024Jun 17, 2024Jun 26, 2024Jul 5, 2024Jul 14, 2024Jul 23, 2024
Updated 8mo ago

Evaluation results

MethodLinks
Qwen-1.5 14B (Teacher)Model Scale=14B2024.0782.06
TAIATraining Dataset=CoT-Collection, Model=Qwen1.5-7B, FT Method=LoRA, Infer Mode=TAIA2024.0580.1
TAIATraining Dataset=CoT-Collection, Model=Qwen1.5-7B, FT Method=MOLORA, Infer Mode=TAIA2024.0578.84
Qwen1.5-7BTraining Dataset=Base Model, Model=Qwen1.5-7B2024.0578.3
TAIATraining Dataset=Alpaca-GPT4, Model=Qwen1.5-7B, FT Method=MOLORA, Infer Mode=TAIA2024.0577.56
TAIATraining Dataset=Alpaca-GPT4, Model=LLaMA3-8B, FT Method=MOLORA, Infer Mode=TAIA2024.0577.48
LoRATraining Dataset=Alpaca-GPT4, Model=Qwen1.5-7B, FT Method=LoRA, Infer Mode=Vanilla2024.0577.31
TAIATraining Dataset=Alpaca-GPT4, Model=Qwen1.5-7B, FT Method=LoRA, Infer Mode=TAIA2024.0577.31
MOLORATraining Dataset=Alpaca-GPT4, Model=LLaMA3-8B, FT Method=MOLORA, Infer Mode=Vanilla2024.0577.15
MOLORATraining Dataset=Alpaca-GPT4, Model=Qwen1.5-7B, FT Method=MOLORA, Infer Mode=Vanilla2024.0576.9
TAIATraining Dataset=Alpaca-GPT4, Model=LLaMA3-8B, FT Method=LoRA, Infer Mode=TAIA2024.0575.92
LoRATraining Dataset=Alpaca-GPT4, Model=LLaMA3-8B, FT Method=LoRA, Infer Mode=Vanilla2024.0575.68
MOLORATraining Dataset=CoT-Collection, Model=Qwen1.5-7B, FT Method=MOLORA, Infer Mode=Vanilla2024.0575.59
LLaMA3-8BTraining Dataset=Base Model, Model=LLaMA3-8B2024.0573.71
TAIATraining Dataset=CoT-Collection, Model=LLaMA3-8B, FT Method=MOLORA, Infer Mode=TAIA2024.0573.22
LoRATraining Dataset=CoT-Collection, Model=Qwen1.5-7B, FT Method=LoRA, Infer Mode=Vanilla2024.0572.65
MOLORATraining Dataset=CoT-Collection, Model=LLaMA3-8B, FT Method=MOLORA, Infer Mode=Vanilla2024.0571.74
TAIATraining Dataset=CoT-Collection, Model=LLaMA3-8B, FT Method=LoRA, Infer Mode=TAIA2024.0571.25
LoRATraining Dataset=CoT-Collection, Model=LLaMA3-8B, FT Method=LoRA, Infer Mode=Vanilla2024.0567.98
TAIATraining Dataset=CoT-Collection, Model=Qwen1.5-1.8B, FT Method=MOLORA, Infer Mode=TAIA2024.0567.73
TAIATraining Dataset=CoT-Collection, Model=Qwen1.5-1.8B, FT Method=LoRA, Infer Mode=TAIA2024.0567.32
Qwen-1.5 1.8B + DDKDistillation=DDK2024.0766.75
Qwen-1.5 1.8B + KDDistillation=KD2024.0766.26
Qwen-1.5 1.8B + TEDDistillation=TED2024.0765.27
Qwen-1.5 1.8B + CPT & DoReMiDistillation=CPT & DoReMi2024.0765.11
MOLORATraining Dataset=Alpaca-GPT4, Model=LLaMA2-7B, FT Method=MOLORA, Infer Mode=Vanilla2024.0564.95
Qwen-1.5 1.8B + CPTDistillation=CPT2024.0764.78
Qwen-1.5 1.8B (Student)Model Scale=1.8B2024.0764.7
Qwen-1.5 1.8B + MiniLLMDistillation=MiniLLM2024.0764.46
TAIATraining Dataset=Alpaca-GPT4, Model=Qwen1.5-1.8B, FT Method=LoRA, Infer Mode=TAIA2024.0564.29
LoRATraining Dataset=Alpaca-GPT4, Model=LLaMA2-7B, FT Method=LoRA, Infer Mode=Vanilla2024.0563.96
TAIATraining Dataset=Alpaca-GPT4, Model=LLaMA2-7B, FT Method=MOLORA, Infer Mode=TAIA2024.0563.8
TAIATraining Dataset=CoT-Collection, Model=LLaMA2-7B, FT Method=MOLORA, Infer Mode=TAIA2024.0563.49
TAIATraining Dataset=Alpaca-GPT4, Model=Qwen1.5-1.8B, FT Method=MOLORA, Infer Mode=TAIA2024.0563.47
TAIATraining Dataset=Alpaca-GPT4, Model=LLaMA2-7B, FT Method=LoRA, Infer Mode=TAIA2024.0563.06
MOLORATraining Dataset=CoT-Collection, Model=LLaMA2-7B, FT Method=MOLORA, Infer Mode=Vanilla2024.0561.55
TAIATraining Dataset=CoT-Collection, Model=LLaMA2-7B, FT Method=LoRA, Infer Mode=TAIA2024.0560.77
MOLORATraining Dataset=Alpaca-GPT4, Model=Qwen1.5-1.8B, FT Method=MOLORA, Infer Mode=Vanilla2024.0560.69
LoRATraining Dataset=Alpaca-GPT4, Model=Qwen1.5-1.8B, FT Method=LoRA, Infer Mode=Vanilla2024.0560.44
MOLORATraining Dataset=CoT-Collection, Model=Qwen1.5-1.8B, FT Method=MOLORA, Infer Mode=Vanilla2024.0558.8
Qwen1.5-1.8BTraining Dataset=Base Model, Model=Qwen1.5-1.8B2024.0558.39
LoRATraining Dataset=CoT-Collection, Model=Qwen1.5-1.8B, FT Method=LoRA, Infer Mode=Vanilla2024.0558.07
LoRATraining Dataset=CoT-Collection, Model=LLaMA2-7B, FT Method=LoRA, Infer Mode=Vanilla2024.0556.43
LLaMA2-13BModel Size=13B, Shots=0-shot2024.0752.17
LLaMA2-7BTraining Dataset=Base Model, Model=LLaMA2-7B2024.0548.4
TinyLLaMA-1.1B + DDKModel Size=1.1B, Shots=0-shot2024.0725.39
TinyLLaMA-1.1B + TEDModel Size=1.1B, Shots=0-shot2024.0722.93
TinyLLaMA-1.1B + MiniLLMModel Size=1.1B, Shots=0-shot2024.0722.93
TinyLLaMA-1.1B + KDModel Size=1.1B, Shots=0-shot2024.0722.52
TinyLLaMA-1.1B + CPT & DoReMiModel Size=1.1B, Shots=0-shot2024.0720.88
TinyLLaMA-1.1B + CPTModel Size=1.1B, Shots=0-shot2024.0720.39
TinyLLaMA-1.1BModel Size=1.1B, Shots=0-shot2024.0719.08