List your company
ResearchFormerly WizWand

Instruction Following on Vicuna Eval

66.3Win Rate (A)BPO-aligned claude-instant-1.2
40.347.0553.860.55Nov 7, 2023
Updated 8mo ago

Evaluation results

MethodLinks
BPO-aligned claude-instant-1.2Base LLM=claude-instant-1.2, Evaluator=gpt-42023.1166.3528.7
BPO-aligned text-bisonBase LLM=text-bison, Evaluator=gpt-42023.11651025
BPO-aligned vicuna-v1.3-7bBase LLM=vicuna-v1.3-7b, Reference Baseline=vicuna-v1.3-7b original, Evaluator=gpt-42023.11658.726.3
BPOBase LLM=vicuna-v1.3, Method A=7B + BPO, Method B=7B, Evaluator=Claude-v1.32023.1165—35
BPO-aligned llama-2-13b-chatBase LLM=llama-2-13b-chat, Reference Baseline=llama-2-13b-chat original, Evaluator=gpt-42023.1161.32.536.2
BPO-aligned llama-2-13b-chat (scaling test)Base LLM=llama-2-13b-chat, Reference Baseline=llama-2-70b-chat original, Evaluator=gpt-42023.1161.3038.7
BPO-aligned gpt-3.5-turboBase LLM=gpt-3.5-turbo, Evaluator=gpt-42023.11608.731.3
BPO-aligned llama-2-7b-chatBase LLM=llama-2-7b-chat, Reference Baseline=llama-2-7b-chat original, Evaluator=gpt-42023.11602.537.5
BPO-aligned llama-2-70b-chatBase LLM=llama-2-70b-chat, Reference Baseline=llama-2-70b-chat original, Evaluator=gpt-42023.1159.35.535.2
BPO-aligned claude-2Base LLM=claude-2, Evaluator=gpt-42023.1157.5537.5
BPOBase LLM=vicuna-v1.3, Method A=13B + BPO, Method B=13B, Evaluator=Claude-v1.32023.1157.5—42.5
BPOBase LLM=llama-2-chat, Method A=7B + BPO, Method B=7B, Evaluator=Claude-v1.32023.1155—45
BPO-aligned vicuna-v1.3-13bBase LLM=vicuna-v1.3-13b, Reference Baseline=vicuna-v1.3-13b original, Evaluator=gpt-42023.1152.53.743.8
BPOBase LLM=llama-2-chat, Method A=13B + BPO, Method B=13B, Evaluator=Claude-v1.32023.1152.5—47.5
BPOBase LLM=llama-2-chat, Method A=70B + BPO, Method B=70B, Evaluator=Claude-v1.32023.1152.5—47.5
BPO-aligned llama-2-7b-chat (scaling test)Base LLM=llama-2-7b-chat, Reference Baseline=llama-2-70b-chat original, Evaluator=gpt-42023.1148.83.747.5
BPOBase LLM=llama-2-chat, Method A=7B + BPO, Method B=70B, Evaluator=Claude-v1.32023.1148.8—51.2
BPOBase LLM=llama-2-chat, Method A=13B + BPO, Method B=70B, Evaluator=Claude-v1.32023.1146.3—53.7
BPO-aligned gpt-4Base LLM=gpt-4, Evaluator=gpt-42023.1141.323.735