Framing Sensitivity Evaluation on FRAGILE Overall
13.2Flip Rate (%)VALIGN
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| VALIGNLLM Backbone=LLaMA-3.1-8B-Instruct2026.05 | 13.2 | 9.8 | 4.9 | 81.9 | |
| VALIGNLLM Backbone=Mistral-7B-Instruct-v0.32026.05 | 18.2 | 17.6 | 6.8 | 75 | |
| VALIGNLLM Backbone=Qwen2.5-7B-Instruct2026.05 | 21.1 | 20.5 | 13.8 | 65.1 | |
| BaseLLM Backbone=Mistral-7B-Instruct-v0.32026.05 | 21.9 | 20.8 | 12.7 | 65.4 | |
| BaseLLM Backbone=LLaMA-3.1-8B-Instruct2026.05 | 28.6 | 11.2 | 4.3 | 67.1 | |
| BaseLLM Backbone=Qwen2.5-7B-Instruct2026.05 | 32.3 | 23.1 | 16.9 | 50.8 | |
| CAALLM Backbone=LLaMA-3.1-8B-Instruct2026.05 | 34.4 | 32.5 | 29.6 | 36 | |
| CAALLM Backbone=Qwen2.5-7B-Instruct2026.05 | 36.5 | 35.8 | 15.6 | 47.9 | |
| CAALLM Backbone=Mistral-7B-Instruct-v0.32026.05 | 38.8 | 38.4 | 13.2 | 48 | |
| Instruc.LLM Backbone=LLaMA-3.1-8B-Instruct2026.05 | 50.7 | 27.6 | 26.7 | 22.5 | |
| Instruc.LLM Backbone=Mistral-7B-Instruct-v0.32026.05 | 52.3 | 23.7 | 14.7 | 32.7 | |
| Instruc.LLM Backbone=Qwen2.5-7B-Instruct2026.05 | 53.1 | 21.9 | 19.7 | 26.7 | |
| CoTLLM Backbone=LLaMA-3.1-8B-Instruct2026.05 | 57.1 | 31.1 | 23.5 | 19.5 | |
| CoTLLM Backbone=Qwen2.5-7B-Instruct2026.05 | 58.3 | 25.4 | 16.9 | 24.7 | |
| CoTLLM Backbone=Mistral-7B-Instruct-v0.32026.05 | 61.3 | 25.8 | 12.8 | 25.9 |