Opinion Editing on FOE 1.0 (test)
1.91EfficacyROMEINST
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| ROMEINSTBase Model=Llama3.12026.06 | 1.91 | 1.89 | 1.91 | 1.43 | 84.8 | 77 | 1.15 | 0.18 | 0.98 | |
| FT-MEABase Model=Llama3.12026.06 | 1.9 | 1.58 | 1.55 | 1.28 | 85.86 | 70.84 | 0.67 | 0.36 | 0.27 | |
| ROMEINSTBase Model=Mistral32026.06 | 1.9 | 1.82 | 1.9 | 1.51 | 89.85 | 75.98 | 1.43 | 0.13 | 1.59 | |
| FT-MEABase Model=Mistral32026.06 | 1.9 | 1.81 | 1.63 | 1.31 | 60.14 | 57.71 | 0.65 | 0.21 | 0.31 | |
| ROMEEABase Model=Mistral32026.06 | 1.76 | 1.67 | 1.77 | 1.39 | 69.97 | 74.89 | 1.05 | 0.22 | 1.41 | |
| ROMEEABase Model=Llama3.12026.06 | 1.64 | 1.61 | 1.57 | 1.23 | 80.99 | 73.05 | 1.21 | 0.29 | 1.46 | |
| ROMEBase Model=Mistral32026.06 | 1.3 | 1.29 | 1.26 | 1.18 | 86 | 46.37 | 0.98 | 0.07 | 1.14 | |
| FT-MBase Model=Mistral32026.06 | 1.09 | 1.06 | 1.08 | 0.86 | 68.18 | 20.89 | 0.66 | 0.14 | 0.16 | |
| LoRABase Model=Mistral32026.06 | 1.09 | 1.09 | 1.07 | 1.02 | 45.19 | 23.72 | 0.68 | 0.12 | 0.02 | |
| AdaLoRABase Model=Mistral32026.06 | 1.03 | 1.07 | 1.02 | 1.05 | 47.76 | 20.19 | 0.73 | 0.05 | 0.01 | |
| FT-MBase Model=Llama3.12026.06 | 1 | 0.97 | 0.96 | 0.78 | 81.68 | 9.14 | 0.48 | 0.16 | 0.07 | |
| LoRABase Model=Llama3.12026.06 | 1 | 0.98 | 0.99 | 0.89 | 60.58 | 0.96 | 0.86 | 0.04 | 0.01 | |
| AdaLoRABase Model=Llama3.12026.06 | 1 | 0.99 | 1 | 0.82 | 57.37 | 3.53 | 0.89 | 0.03 | 0 | |
| ROMEBase Model=Llama3.12026.06 | 0.99 | 1.04 | 0.98 | 0.79 | 94.53 | 32.97 | 0.91 | 0.08 | 0.75 | |
| DPOBase Model=Llama3.12026.06 | 0.75 | 0.73 | 0.64 | 0.77 | 46.47 | 28.21 | 0.53 | 0.79 | 0.69 | |
| BiPOBase Model=Llama3.12026.06 | 0.54 | 0.51 | 0.44 | 0.33 | 7.37 | 13.14 | 0.13 | 1.26 | 1.09 | |
| DPOBase Model=Mistral32026.06 | 0.45 | 0.35 | 0.31 | 0.41 | 45.83 | 6.09 | 0.23 | 0.62 | 0.59 | |
| ActAddBase Model=Llama3.12026.06 | 0.26 | 0.39 | 0.17 | 0.34 | 16.99 | 10.58 | 0.07 | 0.45 | 0.29 | |
| CAABase Model=Llama3.12026.06 | 0.25 | 0.31 | 0.21 | 0.21 | 52.24 | 14.1 | 0.07 | 0.91 | 0.95 | |
| ActAddBase Model=Mistral32026.06 | 0.25 | 0.23 | 0.14 | 0.21 | 14.42 | 6.09 | 0.11 | 0.99 | 0.78 | |
| BiPOBase Model=Mistral32026.06 | 0.09 | 0.11 | 0.09 | 0.05 | 11.22 | 1.6 | 0.04 | 1.43 | 1.37 | |
| CAABase Model=Mistral32026.06 | 0.08 | 0.09 | 0.06 | 0.05 | 37.82 | 12.8 | 0.03 | 1.09 | 1.04 | |
| Llama3.1Base Model=Llama3.12026.06 | 0.04 | 0.05 | 0.09 | 0.03 | 0 | 1.7 | 0.02 | 1.73 | 1.74 | |
| Mistral3Base Model=Mistral32026.06 | 0.03 | 0.04 | 0.08 | 0.04 | 7.39 | 0 | 0.03 | 1.42 | 1.46 |