# Jailbreaking Large Language Models with Morality Attacks

Source: [Jailbreaking Large Language Models with Morality Attacks](<https://www.sota2.com/research/paper/69e704c37d0172140ec7eef6>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-04-22T05:21:28.944Z

Authors: Ying Su, Mingen Zheng, Weili Diao, Haoran Li

Published: 2026-04-18T16:22:25.000Z

## Abstract

Pluralism alignment with AI has the sophisticated and necessary goal of creating AI that can coexist with and serve morally multifaceted humanity. Research towards pluralism alignment has many efforts in enhancing the learning of large language models (LLMs) to accomplish pluralism. Although this is essential, the robustness of LLMs to produce moral content over pluralistic values is still under exploration.Inspired by the astonishing persuasion abilities via jailbreak prompts, we propose to leverage jailbreak attacks to study LLMs' internal pluralistic values. In detail, we develop a morality dataset with 10.3K instances in two categories: Value Ambiguity and Value Conflict. We further formalize four adversarial attacks with the constructed dataset, to manipulate LLMs' judgment over the morality questions. We evaluate both the large language models and guardrail models which are typically used in generative systems with flexible user input. Our experiment results show that there is a critical vulnerability of LLMs and guardrail models to these subtle and sophisticated moral-aware attacks.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Morality Attack on Designed Morality Attacks 1.0 (test)](<https://www.sota2.com/research/sota/morality-attack-on-designed-morality-attacks-1-0-test.md>): Morality Attack; Designed Morality Attacks 1.0 (test); RN: not recorded.

- [Morality Attack (Defense on User Input) on Designed Morality Attacks 1.0 (test)](<https://www.sota2.com/research/sota/morality-attack-defense-on-user-input-on-designed-morality-attacks-1-0-test.md>): Morality Attack (Defense on User Input); Designed Morality Attacks 1.0 (test); RN: not recorded.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2604.17053>)

[PDF](<https://arxiv.org/pdf/2604.17053.pdf>)