# Dialogue Alignment on Relational Failure Mode Benchmark (test)

Source: [Dialogue Alignment on Relational Failure Mode Benchmark (test)](<https://www.sota2.com/research/sota/dialogue-alignment-on-relational-failure-mode-benchmark-test>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-04-03T07:44:14.978Z

Evaluation of model performance on dialogue alignment using the test split of the Relational Failure Mode Benchmark, focusing on utility and delta metrics.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Dialogue Alignment |
| Dataset (including any stated split) | Relational Failure Mode Benchmark (test) |
| Metric | Mean Utility |
| Unit | SCALAR |
| Direction | Higher is better |
| Results loaded | 4 of 4 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [Mean Utility](<https://www.sota2.com/research/sota/dialogue-alignment-on-relational-failure-mode-benchmark-test.md?metric=Mean%20Utility>): Higher is better; unit: SCALAR.

- [Delta vs SFT](<https://www.sota2.com/research/sota/dialogue-alignment-on-relational-failure-mode-benchmark-test.md?metric=Delta%20vs%20SFT>): Higher is better; unit: SCALAR.

## Recorded results

| Method | Mean Utility | Result date | Report paper |
| --- | --- | --- | --- |
| Reranked-best | 0.4116 | Not recorded | 69cf49770636e5bf4afcd984 |
| Manual-DPO | 0.3422 | Not recorded | 69cf49770636e5bf4afcd984 |
| SFT-baseline | 0.1618 | Not recorded | 69cf49770636e5bf4afcd984 |
| CCN-candidate | -0.1498 | Not recorded | 69cf49770636e5bf4afcd984 |

### Reranked-best

Mean Utility: 0.4116. Result ID: 69cf6f0447f34ce924b8fb12.

[Reporting paper](<https://www.sota2.com/research/paper/69cf49770636e5bf4afcd984.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69cf49770636e5bf4afcd984.md>)

Recorded evaluation configuration: {"strategy":"utility-based reranking"}. This record does not assert independent reproduction.

### Manual-DPO

Mean Utility: 0.3422. Result ID: 69cf6f0447f34ce924b8fb11.

[Reporting paper](<https://www.sota2.com/research/paper/69cf49770636e5bf4afcd984.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa74511e209131cea6105d.md>)

Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction.

### SFT-baseline

Mean Utility: 0.1618. Result ID: 69cf6f0447f34ce924b8fb0f.

[Reporting paper](<https://www.sota2.com/research/paper/69cf49770636e5bf4afcd984.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6bf71e209131cea314c0.md>)

[Implementation](<https://github.com/openai/following-instructions-human-feedback>)

Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction.

### CCN-candidate

Mean Utility: -0.1498. Result ID: 69cf6f0447f34ce924b8fb10.

[Reporting paper](<https://www.sota2.com/research/paper/69cf49770636e5bf4afcd984.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69cf49770636e5bf4afcd984.md>)

Recorded evaluation configuration: {"mode":"candidate generation"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.