# Qwen Circuit Recovery Suite 0.5B -\> 1.5B 2.5

Source: [Qwen Circuit Recovery Suite 0.5B -\> 1.5B 2.5](<https://www.sota2.com/research/sota/qwen-circuit-recovery-suite-0-5b-1-5b-2-5>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-04-28T10:11:43.706Z

Evaluation of model performance on the Circuit Recovery task using the Qwen Circuit Recovery Suite, covering sub-datasets including IOI, MCQA, Arithmetic, and ARC splits for models in the 0.5B to 1.5B parameter range.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Circuit Recovery |
| Dataset (including any stated split) | Qwen Circuit Recovery Suite 0.5B -\> 1.5B 2.5 |
| Metric | Arithmetic (-) Score |
| Unit | PERCENT |
| Direction | Higher is better |
| Results loaded | 13 of 13 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [Arithmetic (-) Score](<https://www.sota2.com/research/sota/qwen-circuit-recovery-suite-0-5b-1-5b-2-5.md?metric=Arithmetic%20(-)%20Score>): Higher is better; unit: PERCENT.

- [IOI Score](<https://www.sota2.com/research/sota/qwen-circuit-recovery-suite-0-5b-1-5b-2-5.md?metric=IOI%20Score>): Higher is better; unit: PERCENT.

- [MCQA Score](<https://www.sota2.com/research/sota/qwen-circuit-recovery-suite-0-5b-1-5b-2-5.md?metric=MCQA%20Score>): Higher is better; unit: PERCENT.

- [Arithmetic (+) Score](<https://www.sota2.com/research/sota/qwen-circuit-recovery-suite-0-5b-1-5b-2-5.md?metric=Arithmetic%20(%2B)%20Score>): Higher is better; unit: SCALAR.

- [ARC-E Score](<https://www.sota2.com/research/sota/qwen-circuit-recovery-suite-0-5b-1-5b-2-5.md?metric=ARC-E%20Score>): Higher is better; unit: PERCENT.

- [ARC-C Score](<https://www.sota2.com/research/sota/qwen-circuit-recovery-suite-0-5b-1-5b-2-5.md?metric=ARC-C%20Score>): Higher is better; unit: PERCENT.

- [Average Score](<https://www.sota2.com/research/sota/qwen-circuit-recovery-suite-0-5b-1-5b-2-5.md?metric=Average%20Score>): Higher is better; unit: PERCENT.

## Recorded results

| Method | Arithmetic (-) Score | Result date | Report paper |
| --- | --- | --- | --- |
| DFA (In-Distribution) | 40 | Not recorded | 69f03f7555a3c32af89fa4df |
| DFA (In-Distribution) | 40 | Not recorded | 69f03f7555a3c32af89fa4df |
| DFA (Far/Zero-Shot) | 40 | Not recorded | 69f03f7555a3c32af89fa4df |
| DFA (Far/Zero-Shot) | 39 | Not recorded | 69f03f7555a3c32af89fa4df |
| DFA (In-Distribution) | 27 | Not recorded | 69f03f7555a3c32af89fa4df |
| DFA (Near-Distribution) | 27 | Not recorded | 69f03f7555a3c32af89fa4df |
| Random Alignment | 25 | Not recorded | 69f03f7555a3c32af89fa4df |
| Gold (Upper Bound) | 25 | Not recorded | 69f03f7555a3c32af89fa4df |
| DFA (Near-Distribution) | 25 | Not recorded | 69f03f7555a3c32af89fa4df |
| DFA (Far/Zero-Shot) | 25 | Not recorded | 69f03f7555a3c32af89fa4df |
| DFA (Near-Distribution) | 25 | Not recorded | 69f03f7555a3c32af89fa4df |
| Gold (Upper Bound) | 24 | Not recorded | 69f03f7555a3c32af89fa4df |
| Gold (Upper Bound) | 24 | Not recorded | 69f03f7555a3c32af89fa4df |

### DFA (In-Distribution)

Arithmetic (-) Score: 40. Result ID: 69f087cc47f34ce924d70ef6.

[Reporting paper](<https://www.sota2.com/research/paper/69f03f7555a3c32af89fa4df.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69f03f7555a3c32af89fa4df.md>)

Recorded evaluation configuration: {"Method Category":"NAP","Transfer Setting":"DFA (In-Distribution)"}. This record does not assert independent reproduction.

### DFA (In-Distribution)

Arithmetic (-) Score: 40. Result ID: 69f087cc47f34ce924d70efa.

[Reporting paper](<https://www.sota2.com/research/paper/69f03f7555a3c32af89fa4df.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69f03f7555a3c32af89fa4df.md>)

Recorded evaluation configuration: {"Method Category":"NAP-IG-Activations","Transfer Setting":"DFA (In-Distribution)"}. This record does not assert independent reproduction.

### DFA (Far/Zero-Shot)

Arithmetic (-) Score: 40. Result ID: 69f087cc47f34ce924d70efc.

[Reporting paper](<https://www.sota2.com/research/paper/69f03f7555a3c32af89fa4df.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69f03f7555a3c32af89fa4df.md>)

Recorded evaluation configuration: {"Method Category":"NAP-IG-Activations","Transfer Setting":"DFA (Far/Zero-Shot)"}. This record does not assert independent reproduction.

### DFA (Far/Zero-Shot)

Arithmetic (-) Score: 39. Result ID: 69f087cc47f34ce924d70ef4.

[Reporting paper](<https://www.sota2.com/research/paper/69f03f7555a3c32af89fa4df.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69f03f7555a3c32af89fa4df.md>)

Recorded evaluation configuration: {"Method Category":"NAP-IG-Inputs","Transfer Setting":"DFA (Far/Zero-Shot)"}. This record does not assert independent reproduction.

### DFA (In-Distribution)

Arithmetic (-) Score: 27. Result ID: 69f087cc47f34ce924d70ef2.

[Reporting paper](<https://www.sota2.com/research/paper/69f03f7555a3c32af89fa4df.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69f03f7555a3c32af89fa4df.md>)

Recorded evaluation configuration: {"Method Category":"NAP-IG-Inputs","Transfer Setting":"DFA (In-Distribution)"}. This record does not assert independent reproduction.

### DFA (Near-Distribution)

Arithmetic (-) Score: 27. Result ID: 69f087cc47f34ce924d70ef3.

[Reporting paper](<https://www.sota2.com/research/paper/69f03f7555a3c32af89fa4df.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69f03f7555a3c32af89fa4df.md>)

Recorded evaluation configuration: {"Method Category":"NAP-IG-Inputs","Transfer Setting":"DFA (Near-Distribution)"}. This record does not assert independent reproduction.

### Random Alignment

Arithmetic (-) Score: 25. Result ID: 69f087cc47f34ce924d70ef0.

[Reporting paper](<https://www.sota2.com/research/paper/69f03f7555a3c32af89fa4df.md>)

Recorded evaluation configuration: {"Method Category":"Baseline","Transfer Setting":"Random Alignment"}. This record does not assert independent reproduction.

### Gold (Upper Bound)

Arithmetic (-) Score: 25. Result ID: 69f087cc47f34ce924d70ef1.

[Reporting paper](<https://www.sota2.com/research/paper/69f03f7555a3c32af89fa4df.md>)

Recorded evaluation configuration: {"Method Category":"NAP-IG-Inputs","Transfer Setting":"Gold (Upper Bound)"}. This record does not assert independent reproduction.

### DFA (Near-Distribution)

Arithmetic (-) Score: 25. Result ID: 69f087cc47f34ce924d70ef7.

[Reporting paper](<https://www.sota2.com/research/paper/69f03f7555a3c32af89fa4df.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69f03f7555a3c32af89fa4df.md>)

Recorded evaluation configuration: {"Method Category":"NAP","Transfer Setting":"DFA (Near-Distribution)"}. This record does not assert independent reproduction.

### DFA (Far/Zero-Shot)

Arithmetic (-) Score: 25. Result ID: 69f087cc47f34ce924d70ef8.

[Reporting paper](<https://www.sota2.com/research/paper/69f03f7555a3c32af89fa4df.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69f03f7555a3c32af89fa4df.md>)

Recorded evaluation configuration: {"Method Category":"NAP","Transfer Setting":"DFA (Far/Zero-Shot)"}. This record does not assert independent reproduction.

### DFA (Near-Distribution)

Arithmetic (-) Score: 25. Result ID: 69f087cc47f34ce924d70efb.

[Reporting paper](<https://www.sota2.com/research/paper/69f03f7555a3c32af89fa4df.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69f03f7555a3c32af89fa4df.md>)

Recorded evaluation configuration: {"Method Category":"NAP-IG-Activations","Transfer Setting":"DFA (Near-Distribution)"}. This record does not assert independent reproduction.

### Gold (Upper Bound)

Arithmetic (-) Score: 24. Result ID: 69f087cc47f34ce924d70ef5.

[Reporting paper](<https://www.sota2.com/research/paper/69f03f7555a3c32af89fa4df.md>)

Recorded evaluation configuration: {"Method Category":"NAP","Transfer Setting":"Gold (Upper Bound)"}. This record does not assert independent reproduction.

### Gold (Upper Bound)

Arithmetic (-) Score: 24. Result ID: 69f087cc47f34ce924d70ef9.

[Reporting paper](<https://www.sota2.com/research/paper/69f03f7555a3c32af89fa4df.md>)

Recorded evaluation configuration: {"Method Category":"NAP-IG-Activations","Transfer Setting":"Gold (Upper Bound)"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.
