# Multiple-choice science question answering on ARC-C Smaller Dist. Shift 18

Source: [Multiple-choice science question answering on ARC-C Smaller Dist. Shift 18](<https://www.sota2.com/research/sota/multiple-choice-science-question-answering-on-arc-c-smaller-dist-shift-18>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-02-26T05:30:16.849Z

Evaluation of model performance on the specific 'Smaller Dist. Shift 18' subset of the ARC-C dataset for multiple-choice science question answering.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Multiple-choice science question answering |
| Dataset (including any stated split) | ARC-C Smaller Dist. Shift 18 |
| Metric | Accuracy |
| Unit | PERCENT |
| Direction | Higher is better |
| Results loaded | 9 of 9 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [Accuracy](<https://www.sota2.com/research/sota/multiple-choice-science-question-answering-on-arc-c-smaller-dist-shift-18.md?metric=Accuracy>): Higher is better; unit: PERCENT.

- [Expected Calibration Error](<https://www.sota2.com/research/sota/multiple-choice-science-question-answering-on-arc-c-smaller-dist-shift-18.md?metric=Expected%20Calibration%20Error>): Lower is better; unit: PERCENT.

- [Negative Log Likelihood](<https://www.sota2.com/research/sota/multiple-choice-science-question-answering-on-arc-c-smaller-dist-shift-18.md?metric=Negative%20Log%20Likelihood>): Lower is better; unit: SCALAR.

## Recorded results

| Method | Accuracy | Result date | Report paper |
| --- | --- | --- | --- |
| BLoB | 69.93 | Not recorded | 68fa76b91e209131cea6ebb3 |
| MAP | 69.59 | Not recorded | 68fa76b91e209131cea6ebb3 |
| LAP | 69.14 | Not recorded | 68fa76b91e209131cea6ebb3 |
| MCD | 69.03 | Not recorded | 68fa76b91e209131cea6ebb3 |
| BLoB | 68.36 | Not recorded | 68fa76b91e209131cea6ebb3 |
| BLoB | 67.71 | Not recorded | 68fa76b91e209131cea6ebb3 |
| ENS | 67.34 | Not recorded | 68fa76b91e209131cea6ebb3 |
| BBB | 67.25 | Not recorded | 68fa76b91e209131cea6ebb3 |
| MLE | 66.2 | Not recorded | 68fa76b91e209131cea6ebb3 |

### BLoB

Accuracy: 69.93. Result ID: 69944c093a6c3148491c550b.

[Reporting paper](<https://www.sota2.com/research/paper/68fa76b91e209131cea6ebb3.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa76b91e209131cea6ebb3.md>)

Recorded evaluation configuration: {"Backbone":"Llama2-7B","PEFT Method":"LoRA","Number of samples (N)":0}. This record does not assert independent reproduction.

### MAP

Accuracy: 69.59. Result ID: 69944c093a6c3148491c5506.

[Reporting paper](<https://www.sota2.com/research/paper/68fa76b91e209131cea6ebb3.md>)

Recorded evaluation configuration: {"Backbone":"Llama2-7B","PEFT Method":"LoRA"}. This record does not assert independent reproduction.

### LAP

Accuracy: 69.14. Result ID: 69944c093a6c3148491c550a.

[Reporting paper](<https://www.sota2.com/research/paper/68fa76b91e209131cea6ebb3.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa72111e209131cea54493.md>)

Recorded evaluation configuration: {"Backbone":"Llama2-7B","PEFT Method":"LoRA"}. This record does not assert independent reproduction.

### MCD

Accuracy: 69.03. Result ID: 69944c093a6c3148491c5507.

[Reporting paper](<https://www.sota2.com/research/paper/68fa76b91e209131cea6ebb3.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa62d31e209131ce9fb5ff.md>)

Recorded evaluation configuration: {"Backbone":"Llama2-7B","PEFT Method":"LoRA"}. This record does not assert independent reproduction.

### BLoB

Accuracy: 68.36. Result ID: 69944c093a6c3148491c550c.

[Reporting paper](<https://www.sota2.com/research/paper/68fa76b91e209131cea6ebb3.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa76b91e209131cea6ebb3.md>)

Recorded evaluation configuration: {"Backbone":"Llama2-7B","PEFT Method":"LoRA","Number of samples (N)":5}. This record does not assert independent reproduction.

### BLoB

Accuracy: 67.71. Result ID: 69944c093a6c3148491c550d.

[Reporting paper](<https://www.sota2.com/research/paper/68fa76b91e209131cea6ebb3.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa76b91e209131cea6ebb3.md>)

Recorded evaluation configuration: {"Backbone":"Llama2-7B","PEFT Method":"LoRA","Number of samples (N)":10}. This record does not assert independent reproduction.

### ENS

Accuracy: 67.34. Result ID: 69944c093a6c3148491c5508.

[Reporting paper](<https://www.sota2.com/research/paper/68fa76b91e209131cea6ebb3.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa63bc1e209131cea0084f.md>)

Recorded evaluation configuration: {"Backbone":"Llama2-7B","PEFT Method":"LoRA"}. This record does not assert independent reproduction.

### BBB

Accuracy: 67.25. Result ID: 69944c093a6c3148491c5509.

[Reporting paper](<https://www.sota2.com/research/paper/68fa76b91e209131cea6ebb3.md>)

Recorded evaluation configuration: {"Backbone":"Llama2-7B","PEFT Method":"LoRA"}. This record does not assert independent reproduction.

### MLE

Accuracy: 66.2. Result ID: 69944c093a6c3148491c5505.

[Reporting paper](<https://www.sota2.com/research/paper/68fa76b91e209131cea6ebb3.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6b121e209131cea2bf59.md>)

[Implementation](<https://github.com/microsoft/LoRA>)

Recorded evaluation configuration: {"Backbone":"Llama2-7B","PEFT Method":"LoRA"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.