List your company
ResearchFormerly WizWand

Medical Knowledge Evaluation on MMedbench English subset (val)

60.33AccuracyLLaMA3-8B
20.570830.892941.21551.5371May 30, 2024
Updated 8mo ago

Evaluation results

MethodLinks
LLaMA3-8BTraining Dataset=Base Model, Model=LLaMA3-8B2024.0560.33
TAIATraining Dataset=CoT-Collection, Model=LLaMA3-8B, FT Method=MOLORA, Infer Mode=TAIA2024.0560.02
TAIATraining Dataset=Alpaca-GPT4, Model=LLaMA3-8B, FT Method=MOLORA, Infer Mode=TAIA2024.0559.07
TAIATraining Dataset=Alpaca-GPT4, Model=LLaMA3-8B, FT Method=LoRA, Infer Mode=TAIA2024.0558.68
TAIATraining Dataset=CoT-Collection, Model=LLaMA3-8B, FT Method=LoRA, Infer Mode=TAIA2024.0558.68
MOLORATraining Dataset=CoT-Collection, Model=LLaMA3-8B, FT Method=MOLORA, Infer Mode=Vanilla2024.0558.44
MOLORATraining Dataset=Alpaca-GPT4, Model=LLaMA3-8B, FT Method=MOLORA, Infer Mode=Vanilla2024.0557.97
LoRATraining Dataset=Alpaca-GPT4, Model=LLaMA3-8B, FT Method=LoRA, Infer Mode=Vanilla2024.0557.5
LoRATraining Dataset=CoT-Collection, Model=LLaMA3-8B, FT Method=LoRA, Infer Mode=Vanilla2024.0555.3
TAIATraining Dataset=Alpaca-GPT4, Model=Qwen1.5-7B, FT Method=LoRA, Infer Mode=TAIA2024.0546.9
TAIATraining Dataset=CoT-Collection, Model=Qwen1.5-7B, FT Method=LoRA, Infer Mode=TAIA2024.0546.74
TAIATraining Dataset=Alpaca-GPT4, Model=Qwen1.5-7B, FT Method=MOLORA, Infer Mode=TAIA2024.0546.58
TAIATraining Dataset=CoT-Collection, Model=Qwen1.5-7B, FT Method=MOLORA, Infer Mode=TAIA2024.0546.03
MOLORATraining Dataset=Alpaca-GPT4, Model=Qwen1.5-7B, FT Method=MOLORA, Infer Mode=Vanilla2024.0545.56
Qwen1.5-7BTraining Dataset=Base Model, Model=Qwen1.5-7B2024.0545.09
LoRATraining Dataset=Alpaca-GPT4, Model=Qwen1.5-7B, FT Method=LoRA, Infer Mode=Vanilla2024.0544.85
LoRATraining Dataset=CoT-Collection, Model=LLaMA2-7B, FT Method=LoRA, Infer Mode=Vanilla2024.0538.73
TAIATraining Dataset=CoT-Collection, Model=LLaMA2-7B, FT Method=LoRA, Infer Mode=TAIA2024.0538.33
TAIATraining Dataset=Alpaca-GPT4, Model=LLaMA2-7B, FT Method=LoRA, Infer Mode=TAIA2024.0538.1
LoRATraining Dataset=Alpaca-GPT4, Model=LLaMA2-7B, FT Method=LoRA, Infer Mode=Vanilla2024.0538.02
MOLORATraining Dataset=CoT-Collection, Model=LLaMA2-7B, FT Method=MOLORA, Infer Mode=Vanilla2024.0537.86
TAIATraining Dataset=CoT-Collection, Model=LLaMA2-7B, FT Method=MOLORA, Infer Mode=TAIA2024.0537.55
MOLORATraining Dataset=Alpaca-GPT4, Model=LLaMA2-7B, FT Method=MOLORA, Infer Mode=Vanilla2024.0537.23
TAIATraining Dataset=CoT-Collection, Model=Qwen1.5-1.8B, FT Method=MOLORA, Infer Mode=TAIA2024.0536.68
TAIATraining Dataset=Alpaca-GPT4, Model=LLaMA2-7B, FT Method=MOLORA, Infer Mode=TAIA2024.0536.53
TAIATraining Dataset=CoT-Collection, Model=Qwen1.5-1.8B, FT Method=LoRA, Infer Mode=TAIA2024.0534.64
MOLORATraining Dataset=Alpaca-GPT4, Model=Qwen1.5-1.8B, FT Method=MOLORA, Infer Mode=Vanilla2024.0534.33
TAIATraining Dataset=Alpaca-GPT4, Model=Qwen1.5-1.8B, FT Method=MOLORA, Infer Mode=TAIA2024.0534.33
Qwen1.5-1.8BTraining Dataset=Base Model, Model=Qwen1.5-1.8B2024.0533.78
LLaMA2-7BTraining Dataset=Base Model, Model=LLaMA2-7B2024.0532.21
TAIATraining Dataset=Alpaca-GPT4, Model=Qwen1.5-1.8B, FT Method=LoRA, Infer Mode=TAIA2024.0529.9
LoRATraining Dataset=CoT-Collection, Model=Qwen1.5-1.8B, FT Method=LoRA, Infer Mode=Vanilla2024.0527.65
MOLORATraining Dataset=CoT-Collection, Model=Qwen1.5-7B, FT Method=MOLORA, Infer Mode=Vanilla2024.0527.57
LoRATraining Dataset=CoT-Collection, Model=Qwen1.5-7B, FT Method=LoRA, Infer Mode=Vanilla2024.0527.49
MOLORATraining Dataset=CoT-Collection, Model=Qwen1.5-1.8B, FT Method=MOLORA, Infer Mode=Vanilla2024.0527.42
LoRATraining Dataset=Alpaca-GPT4, Model=Qwen1.5-1.8B, FT Method=LoRA, Infer Mode=Vanilla2024.0522.1