List your company
ResearchFormerly WizWand

General Reasoning on BIG-Bench Hard (val)

43.46AccuracyTAIA
11.771219.998128.22536.4519May 30, 2024
Updated 8mo ago

Evaluation results

MethodLinks
TAIATraining Dataset=Alpaca-GPT4, Model=Qwen1.5-7B, FT Method=LoRA, Infer Mode=TAIA2024.0543.46
TAIATraining Dataset=Alpaca-GPT4, Model=Qwen1.5-7B, FT Method=MOLORA, Infer Mode=TAIA2024.0542.54
LoRATraining Dataset=Alpaca-GPT4, Model=LLaMA3-8B, FT Method=LoRA, Infer Mode=Vanilla2024.0537.35
LoRATraining Dataset=Alpaca-GPT4, Model=Qwen1.5-7B, FT Method=LoRA, Infer Mode=Vanilla2024.0536.09
MOLORATraining Dataset=Alpaca-GPT4, Model=LLaMA3-8B, FT Method=MOLORA, Infer Mode=Vanilla2024.0535.85
MOLORATraining Dataset=Alpaca-GPT4, Model=Qwen1.5-7B, FT Method=MOLORA, Infer Mode=Vanilla2024.0535.05
LoRATraining Dataset=Alpaca-GPT4, Model=LLaMA2-7B, FT Method=LoRA, Infer Mode=Vanilla2024.0533.19
MOLORATraining Dataset=Alpaca-GPT4, Model=LLaMA2-7B, FT Method=MOLORA, Infer Mode=Vanilla2024.0532.64
TAIATraining Dataset=CoT-Collection, Model=Qwen1.5-7B, FT Method=LoRA, Infer Mode=TAIA2024.0532.54
TAIATraining Dataset=Alpaca-GPT4, Model=LLaMA2-7B, FT Method=LoRA, Infer Mode=TAIA2024.0531.47
TAIATraining Dataset=Alpaca-GPT4, Model=LLaMA3-8B, FT Method=LoRA, Infer Mode=TAIA2024.0531.3
TAIATraining Dataset=CoT-Collection, Model=Qwen1.5-7B, FT Method=MOLORA, Infer Mode=TAIA2024.0530.96
TAIATraining Dataset=Alpaca-GPT4, Model=LLaMA2-7B, FT Method=MOLORA, Infer Mode=TAIA2024.0530.93
Qwen1.5-7BTraining Dataset=Base Model, Model=Qwen1.5-7B2024.0530.76
TAIATraining Dataset=CoT-Collection, Model=LLaMA2-7B, FT Method=MOLORA, Infer Mode=TAIA2024.0530.2
TAIATraining Dataset=Alpaca-GPT4, Model=Qwen1.5-1.8B, FT Method=LoRA, Infer Mode=TAIA2024.0530.03
LLaMA3-8BTraining Dataset=Base Model, Model=LLaMA3-8B2024.0529.58
TAIATraining Dataset=Alpaca-GPT4, Model=LLaMA3-8B, FT Method=MOLORA, Infer Mode=TAIA2024.0529.1
LoRATraining Dataset=Alpaca-GPT4, Model=Qwen1.5-1.8B, FT Method=LoRA, Infer Mode=Vanilla2024.0528.8
TAIATraining Dataset=CoT-Collection, Model=LLaMA3-8B, FT Method=MOLORA, Infer Mode=TAIA2024.0528.63
TAIATraining Dataset=Alpaca-GPT4, Model=Qwen1.5-1.8B, FT Method=MOLORA, Infer Mode=TAIA2024.0527.71
LLaMA2-7BTraining Dataset=Base Model, Model=LLaMA2-7B2024.0526.36
TAIATraining Dataset=CoT-Collection, Model=LLaMA2-7B, FT Method=LoRA, Infer Mode=TAIA2024.0526
LoRATraining Dataset=CoT-Collection, Model=Qwen1.5-7B, FT Method=LoRA, Infer Mode=Vanilla2024.0524.07
MOLORATraining Dataset=Alpaca-GPT4, Model=Qwen1.5-1.8B, FT Method=MOLORA, Infer Mode=Vanilla2024.0523.67
LoRATraining Dataset=CoT-Collection, Model=LLaMA3-8B, FT Method=LoRA, Infer Mode=Vanilla2024.0523.24
MOLORATraining Dataset=CoT-Collection, Model=Qwen1.5-7B, FT Method=MOLORA, Infer Mode=Vanilla2024.0522.5
TAIATraining Dataset=CoT-Collection, Model=Qwen1.5-1.8B, FT Method=MOLORA, Infer Mode=TAIA2024.0522.49
MOLORATraining Dataset=CoT-Collection, Model=LLaMA3-8B, FT Method=MOLORA, Infer Mode=Vanilla2024.0522.24
TAIATraining Dataset=CoT-Collection, Model=Qwen1.5-1.8B, FT Method=LoRA, Infer Mode=TAIA2024.0521.93
MOLORATraining Dataset=CoT-Collection, Model=LLaMA2-7B, FT Method=MOLORA, Infer Mode=Vanilla2024.0520.21
LoRATraining Dataset=CoT-Collection, Model=LLaMA2-7B, FT Method=LoRA, Infer Mode=Vanilla2024.0519.09
TAIATraining Dataset=CoT-Collection, Model=LLaMA3-8B, FT Method=LoRA, Infer Mode=TAIA2024.0518.86
Qwen1.5-1.8BTraining Dataset=Base Model, Model=Qwen1.5-1.8B2024.0516.8
LoRATraining Dataset=CoT-Collection, Model=Qwen1.5-1.8B, FT Method=LoRA, Infer Mode=Vanilla2024.0513.9
MOLORATraining Dataset=CoT-Collection, Model=Qwen1.5-1.8B, FT Method=MOLORA, Infer Mode=Vanilla2024.0512.99