Harmlessness evaluation on Beavertails
58.4Helpful ScoreAligner
Evaluation results
| Method | Links | ||
|---|---|---|---|
| AlignerAligner size=7B, Upstream LLM=Claude 2, Zero-shot=true2024.02 | 58.4 | 30.3 | |
| AlignerAligner size=7B, Upstream LLM=Vicuna-33B, Zero-shot=true2024.02 | 51 | 55.9 | |
| AlignerAligner size=13B, Upstream LLM=Claude 2, Zero-shot=true2024.02 | 50 | 30 | |
| AlignerAligner size=2B, Upstream LLM=Claude 2, Zero-shot=true2024.02 | 38.3 | 15.1 | |
| AlignerAligner size=7B, Upstream LLM=Vicuna-13B, Zero-shot=true2024.02 | 37.6 | 16.6 | |
| AlignerAligner size=7B, Upstream LLM=Alpaca-7B, Zero-shot=true2024.02 | 34.9 | 47 | |
| AlignerAligner size=13B, Upstream LLM=GPT-4, Zero-shot=true2024.02 | 33.9 | 25.1 | |
| AlignerAligner size=13B, Upstream LLM=Vicuna-33B, Zero-shot=true2024.02 | 33.3 | 63.3 | |
| AlignerAligner size=13B, Upstream LLM=Vicuna-13B, Zero-shot=true2024.02 | 31.8 | 26.7 | |
| AlignerAligner size=7B, Upstream LLM=Vicuna-7B, Zero-shot=true2024.02 | 26.4 | 15.9 | |
| AlignerAligner size=2B, Upstream LLM=Llama2-7B-Chat, Zero-shot=true2024.02 | 25.4 | 7.2 | |
| AlignerAligner size=2B, Upstream LLM=Vicuna-33B, Zero-shot=true2024.02 | 24.4 | 52.4 | |
| AlignerAligner size=7B, Upstream LLM=Beaver-7B, Zero-shot=true2024.02 | 21.9 | 12 | |
| AlignerAligner size=7B, Upstream LLM=Llama2-13B-Chat, Zero-shot=true2024.02 | 20.1 | 10.3 | |
| AlignerAligner size=7B, Upstream LLM=Llama2-7B-Chat, Zero-shot=true2024.02 | 19.9 | 7.4 | |
| AlignerAligner size=2B, Upstream LLM=Vicuna-13B, Zero-shot=true2024.02 | 19.4 | 14.9 | |
| AlignerAligner size=13B, Upstream LLM=Vicuna-7B, Zero-shot=true2024.02 | 19.1 | 24 | |
| AlignerAligner size=7B, Upstream LLM=GPT-4, Zero-shot=true2024.02 | 18.6 | 25.8 | |
| AlignerAligner size=2B, Upstream LLM=Llama2-13B-Chat, Zero-shot=true2024.02 | 18.4 | 12.3 | |
| AlignerAligner size=2B, Upstream LLM=Llama2-70B-Chat, Zero-shot=true2024.02 | 17.8 | 5.5 | |
| AlignerAligner size=2B, Upstream LLM=Vicuna-7B, Zero-shot=true2024.02 | 16.9 | 15.8 | |
| AlignerAligner size=13B, Upstream LLM=Llama2-13B-Chat, Zero-shot=true2024.02 | 16.7 | 10.6 | |
| AlignerAligner size=13B, Upstream LLM=GPT-3.5, Zero-shot=true2024.02 | 15.1 | 10.9 | |
| AlignerAligner size=13B, Upstream LLM=Beaver-7B, Zero-shot=true2024.02 | 14.2 | 19.1 | |
| AlignerAligner size=13B, Upstream LLM=Llama2-7B-Chat, Zero-shot=true2024.02 | 13.5 | 4.6 | |
| AlignerAligner size=13B, Upstream LLM=Llama2-70B-Chat, Zero-shot=true2024.02 | 10.6 | 1.9 | |
| AlignerAligner size=2B, Upstream LLM=Beaver-7B, Zero-shot=true2024.02 | 9.9 | 12.1 | |
| AlignerAligner size=7B, Upstream LLM=GPT-3.5, Zero-shot=true2024.02 | 9.3 | 9.3 | |
| AlignerAligner size=13B, Upstream LLM=Alpaca-7B, Zero-shot=true2024.02 | 8.5 | 53.4 | |
| AlignerAligner size=2B, Upstream LLM=GPT-4, Zero-shot=true2024.02 | 8 | 28.6 | |
| AlignerAligner size=2B, Upstream LLM=Alpaca-7B, Zero-shot=true2024.02 | 5.8 | 45 | |
| AlignerAligner size=7B, Upstream LLM=Llama2-70B-Chat, Zero-shot=true2024.02 | 5.2 | 2.4 | |
| AlignerAligner size=2B, Upstream LLM=GPT-3.5, Zero-shot=true2024.02 | 3.1 | 7.6 |