List your company
ResearchFormerly WizWand

Chat Evaluation on MT-Bench 1.0 (test)

8MT-Bench ScoreLlama-3.1-Instruct
5.54566.18286.827.4572Aug 27, 2024
Updated 8mo ago

Evaluation results

MethodLinks
Llama-3.1-InstructSize=8B, Alignment=RLHF2024.088——
Llama-3-InstructSize=8B, Alignment=RLHF2024.088——
GPT-3.5-turboAlignment=RLHF2024.087.94——
Mamba-Llama3.1Attention %=50%, Size=8B2024.087.787.3
Mamba2-Llama3.1Attention %=50%, Size=8B2024.087.68.17
Mamba-Llama3Attention %=50%, Size=8B, Alignment=DPO2024.087.357.826.88
ZephyrSize=7B, Alignment=DPO2024.087.34——
Mamba2-Llama3Attention %=50%, Size=8B, Alignment=DPO2024.087.327.936.7
Mamba-ZephyrAttention %=50%, Size=7B, Alignment=DPO2024.087.31——
Mamba-ZephyrAttention %=25%, Size=7B, Alignment=DPO2024.087.03——
Mamba-Llama3.2Attention %=50%, Size=3B2024.086.97.66.1
Mamba-Llama3Attention %=25%, Size=8B, Alignment=DPO2024.086.867.566.15
Mamba2-Llama3Attention %=25%, Size=8B, Alignment=DPO2024.086.747.246.24
Mamba2-Llama3.2Attention %=50%, Size=3B2024.086.57.15.8
Mamba2-Llama3Attention %=12.5%, Size=8B, Alignment=DPO2024.086.486.836.13
Mamba-Llama3Attention %=12.5%, Size=8B, Alignment=DPO2024.086.466.916.01
Mamba-ZephyrAttention %=12.5%, Size=7B, Alignment=DPO2024.086.4——
Falcon Mamba InstructSize=7B, Alignment=SFT2024.086.47.255.55
Mamba2-Llama3Attention %=0%, Size=8B, Alignment=DPO2024.085.646.165.11