# Framing Sensitivity Evaluation on FRAGILE Narrative vividness

Source: [Framing Sensitivity Evaluation on FRAGILE Narrative vividness](<https://www.sota2.com/research/sota/framing-sensitivity-evaluation-on-fragile-narrative-vividness>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-05-28T09:04:48.256Z

Evaluation of framing sensitivity using the FRAGILE Narrative vividness dataset, measured by Flip%, FH%, NH%, and NL% metrics.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Framing Sensitivity Evaluation |
| Dataset (including any stated split) | FRAGILE Narrative vividness |
| Metric | Flip% |
| Unit | PERCENT |
| Direction | Lower is better |
| Results loaded | 15 of 15 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [Flip%](<https://www.sota2.com/research/sota/framing-sensitivity-evaluation-on-fragile-narrative-vividness.md?metric=Flip%25>): Lower is better; unit: PERCENT.

- [FH%](<https://www.sota2.com/research/sota/framing-sensitivity-evaluation-on-fragile-narrative-vividness.md?metric=FH%25>): Lower is better; unit: PERCENT.

- [NH%](<https://www.sota2.com/research/sota/framing-sensitivity-evaluation-on-fragile-narrative-vividness.md?metric=NH%25>): Lower is better; unit: PERCENT.

- [NL%](<https://www.sota2.com/research/sota/framing-sensitivity-evaluation-on-fragile-narrative-vividness.md?metric=NL%25>): Lower is better; unit: PERCENT.

## Recorded results

| Method | Flip% | Result date | Report paper |
| --- | --- | --- | --- |
| VALIGN | 6.4 | Not recorded | 6a17cc05dbd7150506f81282 |
| Base | 8.6 | Not recorded | 6a17cc05dbd7150506f81282 |
| VALIGN | 11.3 | Not recorded | 6a17cc05dbd7150506f81282 |
| VALIGN | 13.4 | Not recorded | 6a17cc05dbd7150506f81282 |
| Base | 14 | Not recorded | 6a17cc05dbd7150506f81282 |
| Base | 17.4 | Not recorded | 6a17cc05dbd7150506f81282 |
| CAA | 24.8 | Not recorded | 6a17cc05dbd7150506f81282 |
| CAA | 28.9 | Not recorded | 6a17cc05dbd7150506f81282 |
| CAA | 29.2 | Not recorded | 6a17cc05dbd7150506f81282 |
| Instruc. | 46.2 | Not recorded | 6a17cc05dbd7150506f81282 |
| Instruc. | 48.7 | Not recorded | 6a17cc05dbd7150506f81282 |
| Instruc. | 53.5 | Not recorded | 6a17cc05dbd7150506f81282 |
| CoT | 53.6 | Not recorded | 6a17cc05dbd7150506f81282 |
| CoT | 56.2 | Not recorded | 6a17cc05dbd7150506f81282 |
| CoT | 58.8 | Not recorded | 6a17cc05dbd7150506f81282 |

### VALIGN

Flip%: 6.4. Result ID: 6a180515fbc373592c11f0aa.

[Reporting paper](<https://www.sota2.com/research/paper/6a17cc05dbd7150506f81282.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a17cc05dbd7150506f81282.md>)

Recorded evaluation configuration: {"LLM Backbone":"Mistral-7B-Instruct-v0.3"}. This record does not assert independent reproduction.

### Base

Flip%: 8.6. Result ID: 6a180515fbc373592c11f0a6.

[Reporting paper](<https://www.sota2.com/research/paper/6a17cc05dbd7150506f81282.md>)

Recorded evaluation configuration: {"LLM Backbone":"Mistral-7B-Instruct-v0.3"}. This record does not assert independent reproduction.

### VALIGN

Flip%: 11.3. Result ID: 6a180515fbc373592c11f0af.

[Reporting paper](<https://www.sota2.com/research/paper/6a17cc05dbd7150506f81282.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a17cc05dbd7150506f81282.md>)

Recorded evaluation configuration: {"LLM Backbone":"Qwen2.5-7B-Instruct"}. This record does not assert independent reproduction.

### VALIGN

Flip%: 13.4. Result ID: 6a180515fbc373592c11f0a5.

[Reporting paper](<https://www.sota2.com/research/paper/6a17cc05dbd7150506f81282.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a17cc05dbd7150506f81282.md>)

Recorded evaluation configuration: {"LLM Backbone":"LLaMA-3.1-8B-Instruct"}. This record does not assert independent reproduction.

### Base

Flip%: 14. Result ID: 6a180515fbc373592c11f0ab.

[Reporting paper](<https://www.sota2.com/research/paper/6a17cc05dbd7150506f81282.md>)

Recorded evaluation configuration: {"LLM Backbone":"Qwen2.5-7B-Instruct"}. This record does not assert independent reproduction.

### Base

Flip%: 17.4. Result ID: 6a180515fbc373592c11f0a1.

[Reporting paper](<https://www.sota2.com/research/paper/6a17cc05dbd7150506f81282.md>)

Recorded evaluation configuration: {"LLM Backbone":"LLaMA-3.1-8B-Instruct"}. This record does not assert independent reproduction.

### CAA

Flip%: 24.8. Result ID: 6a180515fbc373592c11f0a4.

[Reporting paper](<https://www.sota2.com/research/paper/6a17cc05dbd7150506f81282.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa78991e209131cea7939d.md>)

Recorded evaluation configuration: {"LLM Backbone":"LLaMA-3.1-8B-Instruct"}. This record does not assert independent reproduction.

### CAA

Flip%: 28.9. Result ID: 6a180515fbc373592c11f0a9.

[Reporting paper](<https://www.sota2.com/research/paper/6a17cc05dbd7150506f81282.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa78991e209131cea7939d.md>)

Recorded evaluation configuration: {"LLM Backbone":"Mistral-7B-Instruct-v0.3"}. This record does not assert independent reproduction.

### CAA

Flip%: 29.2. Result ID: 6a180515fbc373592c11f0ae.

[Reporting paper](<https://www.sota2.com/research/paper/6a17cc05dbd7150506f81282.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa78991e209131cea7939d.md>)

Recorded evaluation configuration: {"LLM Backbone":"Qwen2.5-7B-Instruct"}. This record does not assert independent reproduction.

### Instruc.

Flip%: 46.2. Result ID: 6a180515fbc373592c11f0a3.

[Reporting paper](<https://www.sota2.com/research/paper/6a17cc05dbd7150506f81282.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7bd31e209131cea8abad.md>)

Recorded evaluation configuration: {"LLM Backbone":"LLaMA-3.1-8B-Instruct"}. This record does not assert independent reproduction.

### Instruc.

Flip%: 48.7. Result ID: 6a180515fbc373592c11f0a8.

[Reporting paper](<https://www.sota2.com/research/paper/6a17cc05dbd7150506f81282.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7bd31e209131cea8abad.md>)

Recorded evaluation configuration: {"LLM Backbone":"Mistral-7B-Instruct-v0.3"}. This record does not assert independent reproduction.

### Instruc.

Flip%: 53.5. Result ID: 6a180515fbc373592c11f0ad.

[Reporting paper](<https://www.sota2.com/research/paper/6a17cc05dbd7150506f81282.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7bd31e209131cea8abad.md>)

Recorded evaluation configuration: {"LLM Backbone":"Qwen2.5-7B-Instruct"}. This record does not assert independent reproduction.

### CoT

Flip%: 53.6. Result ID: 6a180515fbc373592c11f0a2.

[Reporting paper](<https://www.sota2.com/research/paper/6a17cc05dbd7150506f81282.md>)

[Method origin paper](<https://www.sota2.com/research/paper/697c3b06a8b01e76cc8da48a.md>)

Recorded evaluation configuration: {"LLM Backbone":"LLaMA-3.1-8B-Instruct"}. This record does not assert independent reproduction.

### CoT

Flip%: 56.2. Result ID: 6a180515fbc373592c11f0ac.

[Reporting paper](<https://www.sota2.com/research/paper/6a17cc05dbd7150506f81282.md>)

[Method origin paper](<https://www.sota2.com/research/paper/697c3b06a8b01e76cc8da48a.md>)

Recorded evaluation configuration: {"LLM Backbone":"Qwen2.5-7B-Instruct"}. This record does not assert independent reproduction.

### CoT

Flip%: 58.8. Result ID: 6a180515fbc373592c11f0a7.

[Reporting paper](<https://www.sota2.com/research/paper/6a17cc05dbd7150506f81282.md>)

[Method origin paper](<https://www.sota2.com/research/paper/697c3b06a8b01e76cc8da48a.md>)

Recorded evaluation configuration: {"LLM Backbone":"Mistral-7B-Instruct-v0.3"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.
