Instruction Generalization on RoleBench Instruction Generalization (CUS/RAW/SPE)
57.6CUS ScoreRoleGPT
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| RoleGPT2023.10 | 57.6 | 53.2 | 32.3 | 47.7 | |
| Character.AI2023.10 | 41.9 | 45.7 | 30.3 | 39.3 | |
| RoleLLaMA2parameters=13B, mode=Chat2023.10 | 37.5 | 47.9 | 48.8 | 44.7 | |
| RoleLLaMAparameters=7B2023.10 | 32.9 | 37.6 | 38.1 | 36.2 | |
| Alpaca2023.10 | 24.2 | 35.3 | 27 | 28.8 | |
| ChatPLUG2023.10 | 24 | 34.7 | 25.8 | 28.2 | |
| Vicuna2023.10 | 21 | 25.5 | 29.1 | 25.2 | |
| LLaMA-2-Chatparameters=7B2023.10 | 16.2 | 26.2 | 21.4 | 21.2 | |
| LLaMA2023.10 | 12.9 | 12.3 | 25.5 | 16.9 | |
| LLaMASetup=script2023.10 | 8.3 | 5.1 | 10.8 | 8.1 |