List your company
ResearchFormerly WizWand

Instruction Following on BPO Eval (test)

59.5A WinsBPO-aligned vicuna-v1.3-13b
38.1843.71549.2554.785Nov 7, 2023
Updated 8mo ago

Evaluation results

MethodLinks
BPO-aligned vicuna-v1.3-13bBase LLM=vicuna-v1.3-13b, Reference Baseline=vicuna-v1.3-13b original, Evaluator=gpt-42023.1159.5634.5
BPOBase LLM=llama-2-chat, Method A=7B + BPO, Method B=7B, Evaluator=Claude-v1.32023.1158—42
BPOBase LLM=llama-2-chat, Method A=13B + BPO, Method B=13B, Evaluator=Claude-v1.32023.1157.5—42.5
BPOBase LLM=vicuna-v1.3, Method A=13B + BPO, Method B=13B, Evaluator=Claude-v1.32023.1157.5—42.5
BPO-aligned llama-2-70b-chatBase LLM=llama-2-70b-chat, Reference Baseline=llama-2-70b-chat original, Evaluator=gpt-42023.1153.51135.5
BPOBase LLM=llama-2-chat, Method A=13B + BPO, Method B=70B, Evaluator=Claude-v1.32023.1153.5—46.5
BPO-aligned llama-2-7b-chatBase LLM=llama-2-7b-chat, Reference Baseline=llama-2-7b-chat original, Evaluator=gpt-42023.115310.536.5
BPO-aligned llama-2-13b-chatBase LLM=llama-2-13b-chat, Reference Baseline=llama-2-13b-chat original, Evaluator=gpt-42023.115312.534.5
BPOBase LLM=vicuna-v1.3, Method A=7B + BPO, Method B=7B, Evaluator=Claude-v1.32023.1153—47
BPOBase LLM=llama-2-chat, Method A=70B + BPO, Method B=70B, Evaluator=Claude-v1.32023.1152.5—47.5
BPO-aligned gpt-3.5-turboBase LLM=gpt-3.5-turbo, Evaluator=gpt-42023.11511831
BPO-aligned llama-2-13b-chat (scaling test)Base LLM=llama-2-13b-chat, Reference Baseline=llama-2-70b-chat original, Evaluator=gpt-42023.1151742
BPOBase LLM=llama-2-chat, Method A=7B + BPO, Method B=70B, Evaluator=Claude-v1.32023.1151—49
BPO-aligned text-bisonBase LLM=text-bison, Evaluator=gpt-42023.1150.510.539
BPO-aligned vicuna-v1.3-7bBase LLM=vicuna-v1.3-7b, Reference Baseline=vicuna-v1.3-7b original, Evaluator=gpt-42023.11462232
BPO-aligned claude-instant-1.2Base LLM=claude-instant-1.2, Evaluator=gpt-42023.114510.544.5
BPO-aligned claude-2Base LLM=claude-2, Evaluator=gpt-42023.11451342
BPO-aligned llama-2-7b-chat (scaling test)Base LLM=llama-2-7b-chat, Reference Baseline=llama-2-70b-chat original, Evaluator=gpt-42023.1140555
BPO-aligned gpt-4Base LLM=gpt-4, Evaluator=gpt-42023.11392635