Instruction Following Evaluation on Average (Vicuna, Self-instruct, Dolly, BPO) (test)
22Delta Win Rate (ΔWR)BPO-aligned gpt-3.5-turbo
Evaluation Results
| Method | Links | |
|---|---|---|
| BPO-aligned gpt-3.5-turboBase LLM=gpt-3.5-turbo, Evaluator=gpt-42023.11 | 22 | |
| BPO-aligned text-bisonBase LLM=text-bison, Evaluator=gpt-42023.11 | 20.5 | |
| BPOBase LLM=gpt-3.5-turbo, Method A=BPO, Method B=Original, Evaluator=Claude-v1.32023.11 | 19.3 | |
| BPO-aligned vicuna-v1.3-7bBase LLM=vicuna-v1.3-7b, Reference Baseline=vicuna-v1.3-7b original, Evaluator=gpt-42023.11 | 18.5 | |
| BPO-aligned llama-2-13b-chatBase LLM=llama-2-13b-chat, Reference Baseline=llama-2-13b-chat original, Evaluator=gpt-42023.11 | 18.1 | |
| BPO-aligned llama-2-7b-chatBase LLM=llama-2-7b-chat, Reference Baseline=llama-2-7b-chat original, Evaluator=gpt-42023.11 | 17.4 | |
| BPO-aligned llama-2-70b-chatBase LLM=llama-2-70b-chat, Reference Baseline=llama-2-70b-chat original, Evaluator=gpt-42023.11 | 16.8 | |
| BPOBase LLM=vicuna-v1.3, Method A=7B + BPO, Method B=7B, Evaluator=Claude-v1.32023.11 | 14.4 | |
| BPOBase LLM=text-bison, Method A=BPO, Method B=Original, Evaluator=Claude-v1.32023.11 | 14.3 | |
| BPO-aligned vicuna-v1.3-13bBase LLM=vicuna-v1.3-13b, Reference Baseline=vicuna-v1.3-13b original, Evaluator=gpt-42023.11 | 13.1 | |
| BPO-aligned claude-instant-1.2Base LLM=claude-instant-1.2, Evaluator=gpt-42023.11 | 12.9 | |
| BPOBase LLM=vicuna-v1.3, Method A=13B + BPO, Method B=13B, Evaluator=Claude-v1.32023.11 | 12.8 | |
| BPO-aligned llama-2-13b-chat (scaling test)Base LLM=llama-2-13b-chat, Reference Baseline=llama-2-70b-chat original, Evaluator=gpt-42023.11 | 11.9 | |
| BPOBase LLM=llama-2-chat, Method A=13B + BPO, Method B=13B, Evaluator=Claude-v1.32023.11 | 11.7 | |
| BPOBase LLM=llama-2-chat, Method A=7B + BPO, Method B=7B, Evaluator=Claude-v1.32023.11 | 10.5 | |
| BPO-aligned gpt-4Base LLM=gpt-4, Evaluator=gpt-42023.11 | 10.1 | |
| BPOBase LLM=gpt-4, Method A=BPO, Method B=Original, Evaluator=Claude-v1.32023.11 | 9.2 | |
| BPO-aligned claude-2Base LLM=claude-2, Evaluator=gpt-42023.11 | 8.8 | |
| BPOBase LLM=llama-2-chat, Method A=13B + BPO, Method B=70B, Evaluator=Claude-v1.32023.11 | 8.7 | |
| BPOBase LLM=claude-instant-1.2, Method A=BPO, Method B=Original, Evaluator=Claude-v1.32023.11 | 8.5 | |
| BPOBase LLM=claude-2, Method A=BPO, Method B=Original, Evaluator=Claude-v1.32023.11 | 7.1 | |
| BPOBase LLM=llama-2-chat, Method A=70B + BPO, Method B=70B, Evaluator=Claude-v1.32023.11 | 6.7 | |
| BPOBase LLM=llama-2-chat, Method A=7B + BPO, Method B=70B, Evaluator=Claude-v1.32023.11 | -0.6 | |
| BPO-aligned llama-2-7b-chat (scaling test)Base LLM=llama-2-7b-chat, Reference Baseline=llama-2-70b-chat original, Evaluator=gpt-42023.11 | -7.1 |