List your company
ResearchFormerly WizWand

Instruction Following Evaluation on Average (Vicuna, Self-instruct, Dolly, BPO) (test)

22Delta Win Rate (ΔWR)BPO-aligned gpt-3.5-turbo
-8.264-0.4077.4515.307Nov 7, 2023
Updated 8mo ago

Evaluation results

MethodLinks
BPO-aligned gpt-3.5-turboBase LLM=gpt-3.5-turbo, Evaluator=gpt-42023.1122
BPO-aligned text-bisonBase LLM=text-bison, Evaluator=gpt-42023.1120.5
BPOBase LLM=gpt-3.5-turbo, Method A=BPO, Method B=Original, Evaluator=Claude-v1.32023.1119.3
BPO-aligned vicuna-v1.3-7bBase LLM=vicuna-v1.3-7b, Reference Baseline=vicuna-v1.3-7b original, Evaluator=gpt-42023.1118.5
BPO-aligned llama-2-13b-chatBase LLM=llama-2-13b-chat, Reference Baseline=llama-2-13b-chat original, Evaluator=gpt-42023.1118.1
BPO-aligned llama-2-7b-chatBase LLM=llama-2-7b-chat, Reference Baseline=llama-2-7b-chat original, Evaluator=gpt-42023.1117.4
BPO-aligned llama-2-70b-chatBase LLM=llama-2-70b-chat, Reference Baseline=llama-2-70b-chat original, Evaluator=gpt-42023.1116.8
BPOBase LLM=vicuna-v1.3, Method A=7B + BPO, Method B=7B, Evaluator=Claude-v1.32023.1114.4
BPOBase LLM=text-bison, Method A=BPO, Method B=Original, Evaluator=Claude-v1.32023.1114.3
BPO-aligned vicuna-v1.3-13bBase LLM=vicuna-v1.3-13b, Reference Baseline=vicuna-v1.3-13b original, Evaluator=gpt-42023.1113.1
BPO-aligned claude-instant-1.2Base LLM=claude-instant-1.2, Evaluator=gpt-42023.1112.9
BPOBase LLM=vicuna-v1.3, Method A=13B + BPO, Method B=13B, Evaluator=Claude-v1.32023.1112.8
BPO-aligned llama-2-13b-chat (scaling test)Base LLM=llama-2-13b-chat, Reference Baseline=llama-2-70b-chat original, Evaluator=gpt-42023.1111.9
BPOBase LLM=llama-2-chat, Method A=13B + BPO, Method B=13B, Evaluator=Claude-v1.32023.1111.7
BPOBase LLM=llama-2-chat, Method A=7B + BPO, Method B=7B, Evaluator=Claude-v1.32023.1110.5
BPO-aligned gpt-4Base LLM=gpt-4, Evaluator=gpt-42023.1110.1
BPOBase LLM=gpt-4, Method A=BPO, Method B=Original, Evaluator=Claude-v1.32023.119.2
BPO-aligned claude-2Base LLM=claude-2, Evaluator=gpt-42023.118.8
BPOBase LLM=llama-2-chat, Method A=13B + BPO, Method B=70B, Evaluator=Claude-v1.32023.118.7
BPOBase LLM=claude-instant-1.2, Method A=BPO, Method B=Original, Evaluator=Claude-v1.32023.118.5
BPOBase LLM=claude-2, Method A=BPO, Method B=Original, Evaluator=Claude-v1.32023.117.1
BPOBase LLM=llama-2-chat, Method A=70B + BPO, Method B=70B, Evaluator=Claude-v1.32023.116.7
BPOBase LLM=llama-2-chat, Method A=7B + BPO, Method B=70B, Evaluator=Claude-v1.32023.11-0.6
BPO-aligned llama-2-7b-chat (scaling test)Base LLM=llama-2-7b-chat, Reference Baseline=llama-2-70b-chat original, Evaluator=gpt-42023.11-7.1