Rule Alignment Evaluation on RuLES
0.602P_manualPRIORITYRULES
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| PRIORITYRULESBackbone=Llama-2-13B, Data Censorship=Uncensored, Reference Signal (+Lref)=true2024.02 | 0.602 | 0.436 | |
| PRIORITYRULESBackbone=Llama-2-7B, Data Censorship=Uncensored, Reference Signal (+Lref)=true2024.02 | 0.6 | 0.383 | |
| PRIORITYRULESBackbone=Llama-2-13B, Training Data=ShareGPT2024.02 | 0.577 | 0.382 | |
| PRIORITYRULESBackbone=Llama-2-13B, Training Data=ShareGPT, Reference Signal (+Lref)=true2024.02 | 0.577 | 0.445 | |
| PRIORITYRULESBackbone=Llama-2-13B, Data Censorship=Uncensored, System Message (+sys)=true2024.02 | 0.576 | 0.39 | |
| PRIORITYRULESBackbone=Llama-2-7B, Training Data=ShareGPT2024.02 | 0.572 | 0.361 | |
| PRIORITYRULESBackbone=Llama-2-7B, Training Data=ShareGPT, Reference Signal (+Lref)=true2024.02 | 0.57 | 0.378 | |
| PRIORITYRULESBackbone=Llama-2-13B, Data Censorship=Uncensored2024.02 | 0.568 | 0.447 | |
| PRIORITYRULESBackbone=Llama-2-7B, Data Censorship=Uncensored2024.02 | 0.559 | 0.387 | |
| PRIORITYRULESBackbone=Llama-2-7B, System Message (+sys)=true, Training Data=ShareGPT2024.02 | 0.549 | 0.415 | |
| PRIORITYRULESBackbone=Llama-2-7B, Data Censorship=Uncensored, System Message (+sys)=true2024.02 | 0.548 | 0.372 | |
| PRIORITYRULESBackbone=Llama-2-13B, System Message (+sys)=true, Training Data=ShareGPT2024.02 | 0.493 | 0.39 | |
| ShareGPT (+sys)Backbone=Llama-2-7B, System Message (+sys)=true2024.02 | 0.453 | 0.267 | |
| ShareGPT (+sys)Backbone=Llama-2-13B, System Message (+sys)=true2024.02 | 0.414 | 0.241 | |
| Llama-2 ChatBackbone=Llama-2-13B2024.02 | 0.409 | 0.218 | |
| ShareGPTBackbone=Llama-2-13B2024.02 | 0.375 | 0.196 | |
| Unc. ShareGPT (+sys)Backbone=Llama-2-7B, Data Censorship=Uncensored, System Message (+sys)=true2024.02 | 0.362 | 0.212 | |
| Llama-2 ChatBackbone=Llama-2-7B2024.02 | 0.349 | 0.198 | |
| ShareGPTBackbone=Llama-2-7B2024.02 | 0.347 | 0.223 | |
| Unc. ShareGPTBackbone=Llama-2-7B, Data Censorship=Uncensored2024.02 | 0.339 | 0.186 | |
| Unc. ShareGPT (+sys)Backbone=Llama-2-13B, Data Censorship=Uncensored, System Message (+sys)=true2024.02 | 0.326 | 0.162 | |
| Unc. ShareGPTBackbone=Llama-2-13B, Data Censorship=Uncensored2024.02 | 0.3 | 0.151 |