# Formatting Compliance on RULER NIAH (1K-16K Context)

Source: [Formatting Compliance on RULER NIAH (1K-16K Context)](<https://www.sota2.com/research/sota/formatting-compliance-on-ruler-niah-1k-16k-context>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-05-08T11:03:58.479Z

Evaluation of model performance on the RULER NIAH dataset focusing on Formatting Compliance, measured across context lengths of 1K, 2K, 4K, 8K, and 16K tokens.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Formatting Compliance |
| Dataset (including any stated split) | RULER NIAH |
| Metric | Formatting Compliance (1K Context) |
| Unit | PERCENT |
| Direction | Higher is better |
| Results loaded | 3 of 3 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [Formatting Compliance (1K Context)](<https://www.sota2.com/research/sota/formatting-compliance-on-ruler-niah-1k-16k-context.md?metric=Formatting%20Compliance%20(1K%20Context)>): Higher is better; unit: PERCENT.

- [Formatting Compliance (2K Context)](<https://www.sota2.com/research/sota/formatting-compliance-on-ruler-niah-1k-16k-context.md?metric=Formatting%20Compliance%20(2K%20Context)>): Higher is better; unit: PERCENT.

- [Formatting Compliance (4K Context)](<https://www.sota2.com/research/sota/formatting-compliance-on-ruler-niah-1k-16k-context.md?metric=Formatting%20Compliance%20(4K%20Context)>): Higher is better; unit: PERCENT.

- [Formatting Compliance (8K Context)](<https://www.sota2.com/research/sota/formatting-compliance-on-ruler-niah-1k-16k-context.md?metric=Formatting%20Compliance%20(8K%20Context)>): Higher is better; unit: PERCENT.

- [Formatting Compliance (16K Context)](<https://www.sota2.com/research/sota/formatting-compliance-on-ruler-niah-1k-16k-context.md?metric=Formatting%20Compliance%20(16K%20Context)>): Higher is better; unit: PERCENT.

## Recorded results

| Method | Formatting Compliance (1K Context) | Result date | Report paper |
| --- | --- | --- | --- |
| Comm Steer | 89.6 | Not recorded | 69fd6e565497b3b670c896a4 |
| DISCO-Q | 88.5 | Not recorded | 69fd6e565497b3b670c896a4 |
| SKOP | 84.6 | Not recorded | 69fd6e565497b3b670c896a4 |

### Comm Steer

Formatting Compliance (1K Context): 89.6. Result ID: 69fdc31247f34ce924e19098.

[Reporting paper](<https://www.sota2.com/research/paper/69fd6e565497b3b670c896a4.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7b6f1e209131cea886d6.md>)

Recorded evaluation configuration: {"Model":"LLaMA-3.1-8B-Instruct","Steering Task":"Formatting steering","Steering Type":"Unconditional"}. This record does not assert independent reproduction.

### DISCO-Q

Formatting Compliance (1K Context): 88.5. Result ID: 69fdc31247f34ce924e19097.

[Reporting paper](<https://www.sota2.com/research/paper/69fd6e565497b3b670c896a4.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7b6f1e209131cea886d6.md>)

Recorded evaluation configuration: {"Model":"LLaMA-3.1-8B-Instruct","Steering Task":"Formatting steering","Steering Type":"Unconditional"}. This record does not assert independent reproduction.

### SKOP

Formatting Compliance (1K Context): 84.6. Result ID: 69fdc31247f34ce924e19099.

[Reporting paper](<https://www.sota2.com/research/paper/69fd6e565497b3b670c896a4.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69fd6e565497b3b670c896a4.md>)

Recorded evaluation configuration: {"Model":"LLaMA-3.1-8B-Instruct","Steering Task":"Formatting steering","Steering Type":"Unconditional"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.