# Common Sense Reasoning on WG

Source: [Common Sense Reasoning on WG](<https://www.sota2.com/research/sota/common-sense-reasoning-on-wg>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-06-26T08:06:02.317Z

Evaluation of model performance on the WG dataset for common sense reasoning tasks, measured by Accuracy.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Common Sense Reasoning |
| Dataset (including any stated split) | WG |
| Metric | Accuracy |
| Unit | PERCENT |
| Direction | Higher is better |
| Results loaded | 61 of 61 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [Accuracy](<https://www.sota2.com/research/sota/common-sense-reasoning-on-wg.md?metric=Accuracy>): Higher is better; unit: PERCENT.

## Recorded results

| Method | Accuracy | Result date | Report paper |
| --- | --- | --- | --- |
| Human | 94.1 | Not recorded | 69aa5f8d2d603e6de3e418cb |
| DeBERTa-v3-L (Supervised) | 84.1 | Not recorded | 69aa5f8d2d603e6de3e418cb |
| IMAGINE-DeBERTa-v3-L | 79.3 | Not recorded | 69aa5f8d2d603e6de3e418cb |
| RoBERTa-L (Supervised) | 79.3 | Not recorded | 69aa5f8d2d603e6de3e418cb |
| IMAGINE-DeBERTa-v3-L (Retrieval) | 79.2 | Not recorded | 69aa5f8d2d603e6de3e418cb |
| Qwen3-235B-A22B | 79.01 | Not recorded | 6a3e07bb206542251d7cb62a |
| CANDLE-DeBERTa-v3-L | 78.3 | Not recorded | 69aa5f8d2d603e6de3e418cb |
| CAR-DeBERTa-v3-L | 78.2 | Not recorded | 69aa5f8d2d603e6de3e418cb |
| Llama2-70B | 77.58 | Not recorded | 6a3e07bb206542251d7cb62a |
| GPT-4 (gpt-4) | 77 | Not recorded | 69aa5f8d2d603e6de3e418cb |
| IMAGINE-DeBERTa-v3-L | 76.7 | Not recorded | 69aa5f8d2d603e6de3e418cb |
| DeBERTa-v3-L (MR) | 76 | Not recorded | 69aa5f8d2d603e6de3e418cb |
| CAT-Q | 75.34 | Not recorded | 6a3e07bb206542251d7cb62a |
| Mistral-v0.1-7B | 75.3 | Not recorded | 69aa5f8d2d603e6de3e418cb |
| Qwen3-32B | 73.56 | Not recorded | 6a3e07bb206542251d7cb62a |
| Qwen3-14B | 73.16 | Not recorded | 6a3e07bb206542251d7cb62a |
| LLAMA2-13B | 72.8 | Not recorded | 69aa5f8d2d603e6de3e418cb |
| CAT-Q | 71.74 | Not recorded | 6a3e07bb206542251d7cb62a |
| CANDLE-VERA-T5-xxl | 71.3 | Not recorded | 69aa5f8d2d603e6de3e418cb |
| RF2-100B-A6.1B | 70.32 | Not recorded | 6a3e07bb206542251d7cb62a |
| Qwen3-30B-A3B | 69.93 | Not recorded | 6a3e07bb206542251d7cb62a |
| CAT-Q | 69.85 | Not recorded | 6a3e07bb206542251d7cb62a |
| Llama2-7B | 69.46 | Not recorded | 6a3e07bb206542251d7cb62a |
| VERA-T5-xxl | 68.1 | Not recorded | 69aa5f8d2d603e6de3e418cb |
| Qwen3-8B | 68.03 | Not recorded | 6a3e07bb206542251d7cb62a |
| CAT-Q | 68.03 | Not recorded | 6a3e07bb206542251d7cb62a |
| VERA-T5-xxl | 67.2 | Not recorded | 69aa5f8d2d603e6de3e418cb |
| Qwen3-4B | 65.51 | Not recorded | 6a3e07bb206542251d7cb62a |
| CAT-Q | 65.43 | Not recorded | 6a3e07bb206542251d7cb62a |
| ChatGPT (gpt-3.5-turbo) | 64.1 | Not recorded | 69aa5f8d2d603e6de3e418cb |
| CAT-Q | 63.93 | Not recorded | 6a3e07bb206542251d7cb62a |
| CAT-Q | 63.85 | Not recorded | 6a3e07bb206542251d7cb62a |
| CAT-Q | 63.51 | Not recorded | 6a3e07bb206542251d7cb62a |
| CAT-Q | 62.67 | Not recorded | 6a3e07bb206542251d7cb62a |
| CAT-Q | 62.19 | Not recorded | 6a3e07bb206542251d7cb62a |
| CAR-RoBERTa-L | 62 | Not recorded | 69aa5f8d2d603e6de3e418cb |
| Qwen3-1.7B | 61.88 | Not recorded | 6a3e07bb206542251d7cb62a |
| GenKnowSub | 61.24 | Not recorded | 68fa7a661e209131cea82edb |
| IMAGINE-RoBERTa-L | 61.2 | Not recorded | 69aa5f8d2d603e6de3e418cb |
| Multi-hop Knowledge Injection | 61 | Not recorded | 69aa5f8d2d603e6de3e418cb |
| CAT-Q | 60.93 | Not recorded | 6a3e07bb206542251d7cb62a |
| Arrow | 60.85 | Not recorded | 68fa7a661e209131cea82edb |
| Zero-shot Fusion | 60.8 | Not recorded | 69aa5f8d2d603e6de3e418cb |
| GPT-3.5 (text-davinci-003) | 60.7 | Not recorded | 69aa5f8d2d603e6de3e418cb |
| GenKnowSub | 60.69 | Not recorded | 68fa7a661e209131cea82edb |
| H128-MQA-CLA2 | 60.69 | Not recorded | 68fa73401e209131cea5b426 |
| H128-MQA | 60.46 | Not recorded | 68fa73401e209131cea5b426 |
| RoBERTa-L (MR) | 59.6 | Not recorded | 69aa5f8d2d603e6de3e418cb |
| InstructBLIP-Vicuna-7B | 59.6 | Not recorded | 69aa5f8d2d603e6de3e418cb |
| H64-MQA | 57.85 | Not recorded | 68fa73401e209131cea5b426 |
| RoBERTa-L | 57.5 | Not recorded | 69aa5f8d2d603e6de3e418cb |
| Phi-2 | 56.51 | Not recorded | 68fa7a661e209131cea82edb |
| CAR-GPT-2-L | 55.2 | Not recorded | 69aa5f8d2d603e6de3e418cb |
| IMAGINE-GPT-2-L | 55.2 | Not recorded | 69aa5f8d2d603e6de3e418cb |
| CAT-Q | 54.83 | Not recorded | 6a3e07bb206542251d7cb62a |
| Self-talk | 54.7 | Not recorded | 69aa5f8d2d603e6de3e418cb |
| GPT-2-L (MR) | 54.7 | Not recorded | 69aa5f8d2d603e6de3e418cb |
| CAT-Q | 54.62 | Not recorded | 6a3e07bb206542251d7cb62a |
| LLaVA-1.5-7B | 54.5 | Not recorded | 69aa5f8d2d603e6de3e418cb |
| GPT-2-L | 53.2 | Not recorded | 69aa5f8d2d603e6de3e418cb |
| DeBERTa-v3-L | 50.3 | Not recorded | 69aa5f8d2d603e6de3e418cb |

### Human

Accuracy: 94.1. Result ID: 69aa98b36dbb7075b9548625.

[Reporting paper](<https://www.sota2.com/research/paper/69aa5f8d2d603e6de3e418cb.md>)

Recorded evaluation configuration: {"KB":"-"}. This record does not assert independent reproduction.

### DeBERTa-v3-L (Supervised)

Accuracy: 84.1. Result ID: 69aa98b36dbb7075b9548624.

[Reporting paper](<https://www.sota2.com/research/paper/69aa5f8d2d603e6de3e418cb.md>)

Recorded evaluation configuration: {"KB":"-","Evaluation Protocol":"Supervised"}. This record does not assert independent reproduction.

### IMAGINE-DeBERTa-v3-L

Accuracy: 79.3. Result ID: 69aa98b36dbb7075b9548621.

[Reporting paper](<https://www.sota2.com/research/paper/69aa5f8d2d603e6de3e418cb.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69aa5f8d2d603e6de3e418cb.md>)

Recorded evaluation configuration: {"KB":"Synthetic VQA+","Evaluation Protocol":"Zero-shot"}. This record does not assert independent reproduction.

### RoBERTa-L (Supervised)

Accuracy: 79.3. Result ID: 69aa98b36dbb7075b9548623.

[Reporting paper](<https://www.sota2.com/research/paper/69aa5f8d2d603e6de3e418cb.md>)

Recorded evaluation configuration: {"KB":"-","Evaluation Protocol":"Supervised"}. This record does not assert independent reproduction.

### IMAGINE-DeBERTa-v3-L (Retrieval)

Accuracy: 79.2. Result ID: 69aa98b36dbb7075b9548622.

[Reporting paper](<https://www.sota2.com/research/paper/69aa5f8d2d603e6de3e418cb.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69aa5f8d2d603e6de3e418cb.md>)

Recorded evaluation configuration: {"KB":"Synthetic VQA+","Inference Strategy":"Retrieval","Evaluation Protocol":"Zero-shot"}. This record does not assert independent reproduction.

### Qwen3-235B-A22B

Accuracy: 79.01. Result ID: 6a3e32e9fbc373592c5fd2e1.

[Reporting paper](<https://www.sota2.com/research/paper/6a3e07bb206542251d7cb62a.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa788e1e209131cea79116.md>)

Recorded evaluation configuration: {"#Bits":"W16A16","Regime":"PTQ","Zero-shot":"true","Architecture":"MoE"}. This record does not assert independent reproduction.

### CANDLE-DeBERTa-v3-L

Accuracy: 78.3. Result ID: 69aa98b36dbb7075b9548613.

[Reporting paper](<https://www.sota2.com/research/paper/69aa5f8d2d603e6de3e418cb.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73411e209131cea5b4d0.md>)

[Implementation](<https://github.com/HKUST-KnowComp/CANDLE>)

Recorded evaluation configuration: {"KB":"CANDLE","Evaluation Protocol":"Zero-shot"}. This record does not assert independent reproduction.

### CAR-DeBERTa-v3-L

Accuracy: 78.2. Result ID: 69aa98b36dbb7075b9548612.

[Reporting paper](<https://www.sota2.com/research/paper/69aa5f8d2d603e6de3e418cb.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa70e81e209131cea4de50.md>)

Recorded evaluation configuration: {"KB":"AbsAT","Evaluation Protocol":"Zero-shot"}. This record does not assert independent reproduction.

### Llama2-70B

Accuracy: 77.58. Result ID: 6a3e32e9fbc373592c5fd2db.

[Reporting paper](<https://www.sota2.com/research/paper/6a3e07bb206542251d7cb62a.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa701f1e209131cea492fe.md>)

[Implementation](<https://github.com/facebookresearch/llama>)

Recorded evaluation configuration: {"#Bits":"W16A16","Regime":"PTQ","Zero-shot":"true","Architecture":"Dense"}. This record does not assert independent reproduction.

### GPT-4 (gpt-4)

Accuracy: 77. Result ID: 69aa98b36dbb7075b9548616.

[Reporting paper](<https://www.sota2.com/research/paper/69aa5f8d2d603e6de3e418cb.md>)

Recorded evaluation configuration: {"KB":"-","Evaluation Protocol":"Zero-shot"}. This record does not assert independent reproduction.

### IMAGINE-DeBERTa-v3-L

Accuracy: 76.7. Result ID: 69aa98b36dbb7075b9548620.

[Reporting paper](<https://www.sota2.com/research/paper/69aa5f8d2d603e6de3e418cb.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69aa5f8d2d603e6de3e418cb.md>)

Recorded evaluation configuration: {"KB":"Synthetic VQA","Evaluation Protocol":"Zero-shot"}. This record does not assert independent reproduction.

### DeBERTa-v3-L (MR)

Accuracy: 76. Result ID: 69aa98b36dbb7075b954860d.

[Reporting paper](<https://www.sota2.com/research/paper/69aa5f8d2d603e6de3e418cb.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa68f61e209131cea1f563.md>)

Recorded evaluation configuration: {"KB":"AT","Evaluation Protocol":"Zero-shot"}. This record does not assert independent reproduction.

### CAT-Q

Accuracy: 75.34. Result ID: 6a3e32e9fbc373592c5fd2dc.

[Reporting paper](<https://www.sota2.com/research/paper/6a3e07bb206542251d7cb62a.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a3e07bb206542251d7cb62a.md>)

Recorded evaluation configuration: {"Base Model":"Llama2-70B","#Bits":"W1.58A16","Regime":"PTQ","Zero-shot":"true","Architecture":"Dense"}. This record does not assert independent reproduction.

### Mistral-v0.1-7B

Accuracy: 75.3. Result ID: 69aa98b36dbb7075b9548618.

[Reporting paper](<https://www.sota2.com/research/paper/69aa5f8d2d603e6de3e418cb.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa70d61e209131cea4d909.md>)

[Implementation](<https://github.com/mistralai/mistral-src>)

Recorded evaluation configuration: {"KB":"-","Evaluation Protocol":"Zero-shot"}. This record does not assert independent reproduction.

### Qwen3-32B

Accuracy: 73.56. Result ID: 6a3e32e9fbc373592c5fd2d7.

[Reporting paper](<https://www.sota2.com/research/paper/6a3e07bb206542251d7cb62a.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa788e1e209131cea79116.md>)

Recorded evaluation configuration: {"#Bits":"W16A16","Regime":"PTQ","Zero-shot":"true","Architecture":"Dense"}. This record does not assert independent reproduction.

### Qwen3-14B

Accuracy: 73.16. Result ID: 6a3e32e9fbc373592c5fd2d5.

[Reporting paper](<https://www.sota2.com/research/paper/6a3e07bb206542251d7cb62a.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa788e1e209131cea79116.md>)

Recorded evaluation configuration: {"#Bits":"W16A16","Regime":"PTQ","Zero-shot":"true","Architecture":"Dense"}. This record does not assert independent reproduction.

### LLAMA2-13B

Accuracy: 72.8. Result ID: 69aa98b36dbb7075b9548617.

[Reporting paper](<https://www.sota2.com/research/paper/69aa5f8d2d603e6de3e418cb.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa701f1e209131cea492fe.md>)

[Implementation](<https://github.com/facebookresearch/llama>)

Recorded evaluation configuration: {"KB":"-","Evaluation Protocol":"Zero-shot"}. This record does not assert independent reproduction.

### CAT-Q

Accuracy: 71.74. Result ID: 6a3e32e9fbc373592c5fd2d8.

[Reporting paper](<https://www.sota2.com/research/paper/6a3e07bb206542251d7cb62a.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a3e07bb206542251d7cb62a.md>)

Recorded evaluation configuration: {"Base Model":"Qwen3-32B","#Bits":"W1.58A16","Regime":"PTQ","Zero-shot":"true","Architecture":"Dense"}. This record does not assert independent reproduction.

### CANDLE-VERA-T5-xxl

Accuracy: 71.3. Result ID: 69aa98b36dbb7075b954861b.

[Reporting paper](<https://www.sota2.com/research/paper/69aa5f8d2d603e6de3e418cb.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73411e209131cea5b4d0.md>)

[Implementation](<https://github.com/HKUST-KnowComp/CANDLE>)

Recorded evaluation configuration: {"KB":"CANDLE","Evaluation Protocol":"Zero-shot"}. This record does not assert independent reproduction.

### RF2-100B-A6.1B

Accuracy: 70.32. Result ID: 6a3e32e9fbc373592c5fd2df.

[Reporting paper](<https://www.sota2.com/research/paper/6a3e07bb206542251d7cb62a.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7c6d1e209131cea8e049.md>)

Recorded evaluation configuration: {"#Bits":"W16A16","Regime":"PTQ","Zero-shot":"true","Architecture":"MoE"}. This record does not assert independent reproduction.

### Qwen3-30B-A3B

Accuracy: 69.93. Result ID: 6a3e32e9fbc373592c5fd2dd.

[Reporting paper](<https://www.sota2.com/research/paper/6a3e07bb206542251d7cb62a.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa788e1e209131cea79116.md>)

Recorded evaluation configuration: {"#Bits":"W16A16","Regime":"PTQ","Zero-shot":"true","Architecture":"MoE"}. This record does not assert independent reproduction.

### CAT-Q

Accuracy: 69.85. Result ID: 6a3e32e9fbc373592c5fd2e2.

[Reporting paper](<https://www.sota2.com/research/paper/6a3e07bb206542251d7cb62a.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a3e07bb206542251d7cb62a.md>)

Recorded evaluation configuration: {"Base Model":"Qwen3-235B-A22B","#Bits":"W1.58A16","Regime":"PTQ","Zero-shot":"true","Architecture":"MoE"}. This record does not assert independent reproduction.

### Llama2-7B

Accuracy: 69.46. Result ID: 6a3e32e9fbc373592c5fd2d9.

[Reporting paper](<https://www.sota2.com/research/paper/6a3e07bb206542251d7cb62a.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa701f1e209131cea492fe.md>)

[Implementation](<https://github.com/facebookresearch/llama>)

Recorded evaluation configuration: {"#Bits":"W16A16","Regime":"PTQ","Zero-shot":"true","Architecture":"Dense"}. This record does not assert independent reproduction.

### VERA-T5-xxl

Accuracy: 68.1. Result ID: 69aa98b36dbb7075b954861a.

[Reporting paper](<https://www.sota2.com/research/paper/69aa5f8d2d603e6de3e418cb.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa70e51e209131cea4dbc7.md>)

Recorded evaluation configuration: {"KB":"AbsAT","Evaluation Protocol":"Zero-shot"}. This record does not assert independent reproduction.

### Qwen3-8B

Accuracy: 68.03. Result ID: 6a3e32e9fbc373592c5fd2d2.

[Reporting paper](<https://www.sota2.com/research/paper/6a3e07bb206542251d7cb62a.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa788e1e209131cea79116.md>)

Recorded evaluation configuration: {"#Bits":"W16A16","Regime":"PTQ","Zero-shot":"true","Architecture":"Dense"}. This record does not assert independent reproduction.

### CAT-Q

Accuracy: 68.03. Result ID: 6a3e32e9fbc373592c5fd2d6.

[Reporting paper](<https://www.sota2.com/research/paper/6a3e07bb206542251d7cb62a.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a3e07bb206542251d7cb62a.md>)

Recorded evaluation configuration: {"Base Model":"Qwen3-14B","#Bits":"W1.58A16","Regime":"PTQ","Zero-shot":"true","Architecture":"Dense"}. This record does not assert independent reproduction.

### VERA-T5-xxl

Accuracy: 67.2. Result ID: 69aa98b36dbb7075b9548619.

[Reporting paper](<https://www.sota2.com/research/paper/69aa5f8d2d603e6de3e418cb.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa70e51e209131cea4dbc7.md>)

Recorded evaluation configuration: {"KB":"AT","Evaluation Protocol":"Zero-shot"}. This record does not assert independent reproduction.

### Qwen3-4B

Accuracy: 65.51. Result ID: 6a3e32e9fbc373592c5fd2cf.

[Reporting paper](<https://www.sota2.com/research/paper/6a3e07bb206542251d7cb62a.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa788e1e209131cea79116.md>)

Recorded evaluation configuration: {"#Bits":"W16A16","Regime":"PTQ","Zero-shot":"true","Architecture":"Dense"}. This record does not assert independent reproduction.

### CAT-Q

Accuracy: 65.43. Result ID: 6a3e32e9fbc373592c5fd2d4.

[Reporting paper](<https://www.sota2.com/research/paper/6a3e07bb206542251d7cb62a.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a3e07bb206542251d7cb62a.md>)

Recorded evaluation configuration: {"Base Model":"Qwen3-8B","#Bits":"W1.58A16","Regime":"PTQ","Zero-shot":"true","Architecture":"Dense"}. This record does not assert independent reproduction.

### ChatGPT (gpt-3.5-turbo)

Accuracy: 64.1. Result ID: 69aa98b36dbb7075b9548615.

[Reporting paper](<https://www.sota2.com/research/paper/69aa5f8d2d603e6de3e418cb.md>)

Recorded evaluation configuration: {"KB":"-","Evaluation Protocol":"Zero-shot"}. This record does not assert independent reproduction.

### CAT-Q

Accuracy: 63.93. Result ID: 6a3e32e9fbc373592c5fd2e0.

[Reporting paper](<https://www.sota2.com/research/paper/6a3e07bb206542251d7cb62a.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a3e07bb206542251d7cb62a.md>)

Recorded evaluation configuration: {"Base Model":"RF2-100B-A6.1B","#Bits":"W1.58A16","Regime":"PTQ","Zero-shot":"true","Architecture":"MoE"}. This record does not assert independent reproduction.

### CAT-Q

Accuracy: 63.85. Result ID: 6a3e32e9fbc373592c5fd2de.

[Reporting paper](<https://www.sota2.com/research/paper/6a3e07bb206542251d7cb62a.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a3e07bb206542251d7cb62a.md>)

Recorded evaluation configuration: {"Base Model":"Qwen3-30B-A3B","#Bits":"W1.58A16","Regime":"PTQ","Zero-shot":"true","Architecture":"MoE"}. This record does not assert independent reproduction.

### CAT-Q

Accuracy: 63.51. Result ID: 6a3e32e9fbc373592c5fd2d0.

[Reporting paper](<https://www.sota2.com/research/paper/6a3e07bb206542251d7cb62a.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a3e07bb206542251d7cb62a.md>)

Recorded evaluation configuration: {"Base Model":"Qwen3-4B","#Bits":"W1.58A8","Regime":"PTQ","Zero-shot":"true","Architecture":"Dense"}. This record does not assert independent reproduction.

### CAT-Q

Accuracy: 62.67. Result ID: 6a3e32e9fbc373592c5fd2d3.

[Reporting paper](<https://www.sota2.com/research/paper/6a3e07bb206542251d7cb62a.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a3e07bb206542251d7cb62a.md>)

Recorded evaluation configuration: {"Base Model":"Qwen3-8B","#Bits":"W1.58A8","Regime":"PTQ","Zero-shot":"true","Architecture":"Dense"}. This record does not assert independent reproduction.

### CAT-Q

Accuracy: 62.19. Result ID: 6a3e32e9fbc373592c5fd2d1.

[Reporting paper](<https://www.sota2.com/research/paper/6a3e07bb206542251d7cb62a.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a3e07bb206542251d7cb62a.md>)

Recorded evaluation configuration: {"Base Model":"Qwen3-4B","#Bits":"W1.58A16","Regime":"PTQ","Zero-shot":"true","Architecture":"Dense"}. This record does not assert independent reproduction.

### CAR-RoBERTa-L

Accuracy: 62. Result ID: 69aa98b36dbb7075b9548611.

[Reporting paper](<https://www.sota2.com/research/paper/69aa5f8d2d603e6de3e418cb.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa70e81e209131cea4de50.md>)

Recorded evaluation configuration: {"KB":"AbsAT","Evaluation Protocol":"Zero-shot"}. This record does not assert independent reproduction.

### Qwen3-1.7B

Accuracy: 61.88. Result ID: 6a3e32e9fbc373592c5fd2cc.

[Reporting paper](<https://www.sota2.com/research/paper/6a3e07bb206542251d7cb62a.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa788e1e209131cea79116.md>)

Recorded evaluation configuration: {"#Bits":"W16A16","Regime":"PTQ","Zero-shot":"true","Architecture":"Dense"}. This record does not assert independent reproduction.

### GenKnowSub

Accuracy: 61.24. Result ID: 698b76e27a967eddb76eccfe.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7a661e209131cea82edb.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7a661e209131cea82edb.md>)

[Implementation](<https://github.com/saharsamr/Modular-LLM>)

Recorded evaluation configuration: {"Base Model":"Phi-2","Setting":"En","Evaluation Protocol":"zero-shot"}. This record does not assert independent reproduction.

### IMAGINE-RoBERTa-L

Accuracy: 61.2. Result ID: 69aa98b36dbb7075b954861f.

[Reporting paper](<https://www.sota2.com/research/paper/69aa5f8d2d603e6de3e418cb.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69aa5f8d2d603e6de3e418cb.md>)

Recorded evaluation configuration: {"KB":"Synthetic VQA","Evaluation Protocol":"Zero-shot"}. This record does not assert independent reproduction.

### Multi-hop Knowledge Injection

Accuracy: 61. Result ID: 69aa98b36dbb7075b954860f.

[Reporting paper](<https://www.sota2.com/research/paper/69aa5f8d2d603e6de3e418cb.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6f451e209131cea44a5c.md>)

Recorded evaluation configuration: {"KB":"AT, CN, WD, WN","Evaluation Protocol":"Zero-shot"}. This record does not assert independent reproduction.

### CAT-Q

Accuracy: 60.93. Result ID: 6a3e32e9fbc373592c5fd2da.

[Reporting paper](<https://www.sota2.com/research/paper/6a3e07bb206542251d7cb62a.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a3e07bb206542251d7cb62a.md>)

Recorded evaluation configuration: {"Base Model":"Llama2-7B","#Bits":"W1.58A16","Regime":"PTQ","Zero-shot":"true","Architecture":"Dense"}. This record does not assert independent reproduction.

### Arrow

Accuracy: 60.85. Result ID: 698b76e27a967eddb76eccfd.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7a661e209131cea82edb.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73471e209131cea5b839.md>)

Recorded evaluation configuration: {"Base Model":"Phi-2","Evaluation Protocol":"zero-shot"}. This record does not assert independent reproduction.

### Zero-shot Fusion

Accuracy: 60.8. Result ID: 69aa98b36dbb7075b954860e.

[Reporting paper](<https://www.sota2.com/research/paper/69aa5f8d2d603e6de3e418cb.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6cd31e209131cea36050.md>)

Recorded evaluation configuration: {"KB":"AT, CN, WD, WN","Evaluation Protocol":"Zero-shot"}. This record does not assert independent reproduction.

### GPT-3.5 (text-davinci-003)

Accuracy: 60.7. Result ID: 69aa98b36dbb7075b9548614.

[Reporting paper](<https://www.sota2.com/research/paper/69aa5f8d2d603e6de3e418cb.md>)

Recorded evaluation configuration: {"KB":"-","Evaluation Protocol":"Zero-shot"}. This record does not assert independent reproduction.

### GenKnowSub

Accuracy: 60.69. Result ID: 698b76e27a967eddb76eccff.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7a661e209131cea82edb.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7a661e209131cea82edb.md>)

[Implementation](<https://github.com/saharsamr/Modular-LLM>)

Recorded evaluation configuration: {"Base Model":"Phi-2","Setting":"Avg","Evaluation Protocol":"zero-shot"}. This record does not assert independent reproduction.

### H128-MQA-CLA2

Accuracy: 60.69. Result ID: 6992ccbb3a6c3148491195fa.

[Reporting paper](<https://www.sota2.com/research/paper/68fa73401e209131cea5b426.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73401e209131cea5b426.md>)

Recorded evaluation configuration: {"Model Scale":"3B","Head Dimension":"128","Sharing Factor":"2"}. This record does not assert independent reproduction.

### H128-MQA

Accuracy: 60.46. Result ID: 6992ccbb3a6c3148491195f9.

[Reporting paper](<https://www.sota2.com/research/paper/68fa73401e209131cea5b426.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa66971e209131cea112e4.md>)

Recorded evaluation configuration: {"Model Scale":"3B","Head Dimension":"128"}. This record does not assert independent reproduction.

### RoBERTa-L (MR)

Accuracy: 59.6. Result ID: 69aa98b36dbb7075b954860c.

[Reporting paper](<https://www.sota2.com/research/paper/69aa5f8d2d603e6de3e418cb.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa68f61e209131cea1f563.md>)

Recorded evaluation configuration: {"KB":"AT","Evaluation Protocol":"Zero-shot"}. This record does not assert independent reproduction.

### InstructBLIP-Vicuna-7B

Accuracy: 59.6. Result ID: 69aa98b36dbb7075b954861d.

[Reporting paper](<https://www.sota2.com/research/paper/69aa5f8d2d603e6de3e418cb.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6fb71e209131cea47021.md>)

[Implementation](<https://github.com/salesforce/LAVIS/tree/main/projects/instructblip>)

Recorded evaluation configuration: {"KB":"-","Evaluation Protocol":"Zero-shot"}. This record does not assert independent reproduction.

### H64-MQA

Accuracy: 57.85. Result ID: 6992ccbb3a6c3148491195fb.

[Reporting paper](<https://www.sota2.com/research/paper/68fa73401e209131cea5b426.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa66971e209131cea112e4.md>)

Recorded evaluation configuration: {"Model Scale":"3B","Head Dimension":"64"}. This record does not assert independent reproduction.

### RoBERTa-L

Accuracy: 57.5. Result ID: 69aa98b36dbb7075b9548608.

[Reporting paper](<https://www.sota2.com/research/paper/69aa5f8d2d603e6de3e418cb.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa66131e209131cea0e18b.md>)

[Implementation](<https://github.com/pytorch/fairseq>)

Recorded evaluation configuration: {"KB":"-","Evaluation Protocol":"Zero-shot"}. This record does not assert independent reproduction.

### Phi-2

Accuracy: 56.51. Result ID: 698b76e27a967eddb76eccfc.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7a661e209131cea82edb.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73471e209131cea5b839.md>)

Recorded evaluation configuration: {"Base Model":"Phi-2","Evaluation Protocol":"zero-shot"}. This record does not assert independent reproduction.

### CAR-GPT-2-L

Accuracy: 55.2. Result ID: 69aa98b36dbb7075b9548610.

[Reporting paper](<https://www.sota2.com/research/paper/69aa5f8d2d603e6de3e418cb.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa70e81e209131cea4de50.md>)

Recorded evaluation configuration: {"KB":"AbsAT","Evaluation Protocol":"Zero-shot"}. This record does not assert independent reproduction.

### IMAGINE-GPT-2-L

Accuracy: 55.2. Result ID: 69aa98b36dbb7075b954861e.

[Reporting paper](<https://www.sota2.com/research/paper/69aa5f8d2d603e6de3e418cb.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69aa5f8d2d603e6de3e418cb.md>)

Recorded evaluation configuration: {"KB":"Synthetic VQA","Evaluation Protocol":"Zero-shot"}. This record does not assert independent reproduction.

### CAT-Q

Accuracy: 54.83. Result ID: 6a3e32e9fbc373592c5fd2ce.

[Reporting paper](<https://www.sota2.com/research/paper/6a3e07bb206542251d7cb62a.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a3e07bb206542251d7cb62a.md>)

Recorded evaluation configuration: {"Base Model":"Qwen3-1.7B","#Bits":"W1.58A16","Regime":"PTQ","Zero-shot":"true","Architecture":"Dense"}. This record does not assert independent reproduction.

### Self-talk

Accuracy: 54.7. Result ID: 69aa98b36dbb7075b954860a.

[Reporting paper](<https://www.sota2.com/research/paper/69aa5f8d2d603e6de3e418cb.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa68551e209131cea1b93d.md>)

Recorded evaluation configuration: {"KB":"-","Evaluation Protocol":"Zero-shot"}. This record does not assert independent reproduction.

### GPT-2-L (MR)

Accuracy: 54.7. Result ID: 69aa98b36dbb7075b954860b.

[Reporting paper](<https://www.sota2.com/research/paper/69aa5f8d2d603e6de3e418cb.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa68f61e209131cea1f563.md>)

Recorded evaluation configuration: {"KB":"AT","Evaluation Protocol":"Zero-shot"}. This record does not assert independent reproduction.

### CAT-Q

Accuracy: 54.62. Result ID: 6a3e32e9fbc373592c5fd2cd.

[Reporting paper](<https://www.sota2.com/research/paper/6a3e07bb206542251d7cb62a.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a3e07bb206542251d7cb62a.md>)

Recorded evaluation configuration: {"Base Model":"Qwen3-1.7B","#Bits":"W1.58A8","Regime":"PTQ","Zero-shot":"true","Architecture":"Dense"}. This record does not assert independent reproduction.

### LLaVA-1.5-7B

Accuracy: 54.5. Result ID: 69aa98b36dbb7075b954861c.

[Reporting paper](<https://www.sota2.com/research/paper/69aa5f8d2d603e6de3e418cb.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa717d1e209131cea51970.md>)

[Implementation](<https://llava-vl.github.io/>)

Recorded evaluation configuration: {"KB":"-","Evaluation Protocol":"Zero-shot"}. This record does not assert independent reproduction.

### GPT-2-L

Accuracy: 53.2. Result ID: 69aa98b36dbb7075b9548607.

[Reporting paper](<https://www.sota2.com/research/paper/69aa5f8d2d603e6de3e418cb.md>)

Recorded evaluation configuration: {"KB":"-","Evaluation Protocol":"Zero-shot"}. This record does not assert independent reproduction.

### DeBERTa-v3-L

Accuracy: 50.3. Result ID: 69aa98b36dbb7075b9548609.

[Reporting paper](<https://www.sota2.com/research/paper/69aa5f8d2d603e6de3e418cb.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6ee11e209131cea4297b.md>)

[Implementation](<https://github.com/microsoft/DeBERTa>)

Recorded evaluation configuration: {"KB":"-","Evaluation Protocol":"Zero-shot"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.
