Policy-refinement on ToolEmu
33.1IORVanilla
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| VanillaModel=LLaMA-3.1-405B2026.04 | 33.1 | — | 6.22 | |
| VanillaModel=GPT-3.52026.04 | 31.69 | — | 6.24 | |
| VanillaModel=GPT-42026.04 | 22.22 | — | 7.57 | |
| VanillaModel=Qwen2.5-72B2026.04 | 21.53 | — | 6.69 | |
| Goal PriorityModel=Qwen2.5-72B2026.04 | 17.36 | 64 | 7.47 | |
| Self DefenseModel=Qwen2.5-72B2026.04 | 16.67 | 43 | 4.35 | |
| Goal PriorityModel=GPT-42026.04 | 16.23 | 67 | 7.58 | |
| Goal PriorityModel=LLaMA-3.1-405B2026.04 | 14.58 | 59 | 7.7 | |
| Self DefenseModel=GPT-3.52026.04 | 9.86 | 71 | 4.57 | |
| Self DefenseModel=LLaMA-3.1-405B2026.04 | 9.79 | 38 | 4.37 | |
| Goal PriorityModel=GPT-3.52026.04 | 6.99 | 68 | 8.41 | |
| LCOModel=GPT-3.52026.04 | 6.99 | 72 | 6.24 | |
| Self DefenseModel=GPT-42026.04 | 6.99 | 33 | 4.17 | |
| LCOModel=GPT-42026.04 | 6.99 | 70 | 7.28 | |
| LCOModel=LLaMA-3.1-405B2026.04 | 6.29 | 70 | 6.36 | |
| LCOModel=Qwen2.5-72B2026.04 | 4.17 | 82 | 7.06 |