# Knowledge & Instruction Following on ARC Challenge

Source: [Knowledge & Instruction Following on ARC Challenge](<https://www.sota2.com/research/sota/knowledge-and-instruction-following-on-arc-challenge>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-06-02T12:07:20.394Z

Evaluation of model performance on knowledge recall and instruction following capabilities using the Abstract Reasoning Corpus (ARC Challenge) dataset, measured by Accuracy.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Knowledge & Instruction Following |
| Dataset (including any stated split) | ARC Challenge |
| Metric | Accuracy (ARC) |
| Unit | PERCENT |
| Direction | Higher is better |
| Results loaded | 15 of 15 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [Accuracy (ARC)](<https://www.sota2.com/research/sota/knowledge-and-instruction-following-on-arc-challenge.md?metric=Accuracy%20(ARC)>): Higher is better; unit: PERCENT.

## Recorded results

| Method | Accuracy (ARC) | Result date | Report paper |
| --- | --- | --- | --- |
| Qwen3.5-9B | 97.7 | Not recorded | 6a1e6435dbd7150506f85838 |
| FLARE-9B | 96.33 | Not recorded | 6a1e6435dbd7150506f85838 |
| Qwen3.5 | 96.33 | Not recorded | 6a1e6435dbd7150506f85838 |
| LLaDA-2.0-flash | 95.93 | Not recorded | 6a1e6435dbd7150506f85838 |
| FLARE-9B | 95.65 | Not recorded | 6a1e6435dbd7150506f85838 |
| FLARE | 94.62 | Not recorded | 6a1e6435dbd7150506f85838 |
| LLaDA-2.0-mini | 93.56 | Not recorded | 6a1e6435dbd7150506f85838 |
| FLARE | 93.52 | Not recorded | 6a1e6435dbd7150506f85838 |
| SDAR 30B-A3B | 93.2 | Not recorded | 6a1e6435dbd7150506f85838 |
| Qwen3.5 | 92.15 | Not recorded | 6a1e6435dbd7150506f85838 |
| SDAR | 91.9 | Not recorded | 6a1e6435dbd7150506f85838 |
| SDAR | 90.5 | Not recorded | 6a1e6435dbd7150506f85838 |
| FLARE | 85.84 | Not recorded | 6a1e6435dbd7150506f85838 |
| SDAR | 85.4 | Not recorded | 6a1e6435dbd7150506f85838 |
| FLARE | 85.07 | Not recorded | 6a1e6435dbd7150506f85838 |

### Qwen3.5-9B

Accuracy (ARC): 97.7. Result ID: 6a1ec715fbc373592c1a48fd.

[Reporting paper](<https://www.sota2.com/research/paper/6a1e6435dbd7150506f85838.md>)

Recorded evaluation configuration: {"Params":"9B","Sampling mode":"AR"}. This record does not assert independent reproduction.

### FLARE-9B

Accuracy (ARC): 96.33. Result ID: 6a1ec715fbc373592c1a48fb.

[Reporting paper](<https://www.sota2.com/research/paper/6a1e6435dbd7150506f85838.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a1e6435dbd7150506f85838.md>)

Recorded evaluation configuration: {"Params":"9B","Sampling mode":"AR-Trust"}. This record does not assert independent reproduction.

### Qwen3.5

Accuracy (ARC): 96.33. Result ID: 6a1ec715fbc373592c1a494d.

[Reporting paper](<https://www.sota2.com/research/paper/6a1e6435dbd7150506f85838.md>)

Recorded evaluation configuration: {"Params":"4B","Sampling mode":"AR"}. This record does not assert independent reproduction.

### LLaDA-2.0-flash

Accuracy (ARC): 95.93. Result ID: 6a1ec715fbc373592c1a48f9.

[Reporting paper](<https://www.sota2.com/research/paper/6a1e6435dbd7150506f85838.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6944dc05423ea42265729a7b.md>)

[Implementation](<https://github.com/inclusionAI/LLaDA2.0>)

Recorded evaluation configuration: {"Params":"100B-A5B","Sampling mode":"Diffusion"}. This record does not assert independent reproduction.

### FLARE-9B

Accuracy (ARC): 95.65. Result ID: 6a1ec715fbc373592c1a48fc.

[Reporting paper](<https://www.sota2.com/research/paper/6a1e6435dbd7150506f85838.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a1e6435dbd7150506f85838.md>)

Recorded evaluation configuration: {"Params":"9B","Sampling mode":"Diffusion-Trust"}. This record does not assert independent reproduction.

### FLARE

Accuracy (ARC): 94.62. Result ID: 6a1ec715fbc373592c1a494c.

[Reporting paper](<https://www.sota2.com/research/paper/6a1e6435dbd7150506f85838.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a1e6435dbd7150506f85838.md>)

Recorded evaluation configuration: {"Params":"4B","Sampling mode":"Diffusion-Trust"}. This record does not assert independent reproduction.

### LLaDA-2.0-mini

Accuracy (ARC): 93.56. Result ID: 6a1ec715fbc373592c1a48f8.

[Reporting paper](<https://www.sota2.com/research/paper/6a1e6435dbd7150506f85838.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6944dc05423ea42265729a7b.md>)

[Implementation](<https://github.com/inclusionAI/LLaDA2.0>)

Recorded evaluation configuration: {"Params":"16B-A1B","Sampling mode":"Diffusion"}. This record does not assert independent reproduction.

### FLARE

Accuracy (ARC): 93.52. Result ID: 6a1ec715fbc373592c1a494b.

[Reporting paper](<https://www.sota2.com/research/paper/6a1e6435dbd7150506f85838.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a1e6435dbd7150506f85838.md>)

Recorded evaluation configuration: {"Params":"4B","Sampling mode":"AR-Trust"}. This record does not assert independent reproduction.

### SDAR 30B-A3B

Accuracy (ARC): 93.2. Result ID: 6a1ec715fbc373592c1a48fa.

[Reporting paper](<https://www.sota2.com/research/paper/6a1e6435dbd7150506f85838.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7c431e209131cea8d3a7.md>)

Recorded evaluation configuration: {"Params":"30B-A3B","Sampling mode":"Diffusion"}. This record does not assert independent reproduction.

### Qwen3.5

Accuracy (ARC): 92.15. Result ID: 6a1ec715fbc373592c1a494a.

[Reporting paper](<https://www.sota2.com/research/paper/6a1e6435dbd7150506f85838.md>)

Recorded evaluation configuration: {"Params":"2B","Sampling mode":"AR"}. This record does not assert independent reproduction.

### SDAR

Accuracy (ARC): 91.9. Result ID: 6a1ec715fbc373592c1a4947.

[Reporting paper](<https://www.sota2.com/research/paper/6a1e6435dbd7150506f85838.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7c431e209131cea8d3a7.md>)

Recorded evaluation configuration: {"Params":"8B","Sampling mode":"Diffusion"}. This record does not assert independent reproduction.

### SDAR

Accuracy (ARC): 90.5. Result ID: 6a1ec715fbc373592c1a4946.

[Reporting paper](<https://www.sota2.com/research/paper/6a1e6435dbd7150506f85838.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7c431e209131cea8d3a7.md>)

Recorded evaluation configuration: {"Params":"4B","Sampling mode":"Diffusion"}. This record does not assert independent reproduction.

### FLARE

Accuracy (ARC): 85.84. Result ID: 6a1ec715fbc373592c1a4949.

[Reporting paper](<https://www.sota2.com/research/paper/6a1e6435dbd7150506f85838.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a1e6435dbd7150506f85838.md>)

Recorded evaluation configuration: {"Params":"2B","Sampling mode":"Diffusion-Trust"}. This record does not assert independent reproduction.

### SDAR

Accuracy (ARC): 85.4. Result ID: 6a1ec715fbc373592c1a4945.

[Reporting paper](<https://www.sota2.com/research/paper/6a1e6435dbd7150506f85838.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7c431e209131cea8d3a7.md>)

Recorded evaluation configuration: {"Params":"1.7B","Sampling mode":"Diffusion"}. This record does not assert independent reproduction.

### FLARE

Accuracy (ARC): 85.07. Result ID: 6a1ec715fbc373592c1a4948.

[Reporting paper](<https://www.sota2.com/research/paper/6a1e6435dbd7150506f85838.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a1e6435dbd7150506f85838.md>)

Recorded evaluation configuration: {"Params":"2B","Sampling mode":"AR-Trust"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.
