# Social Interaction Question Answering on SIQA

Source: [Social Interaction Question Answering on SIQA](<https://www.sota2.com/research/sota/social-interaction-question-answering-on-siqa>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-06-01T09:42:42.317Z

Evaluation of model performance on the SIQA dataset for Social Interaction Question Answering.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Social Interaction Question Answering |
| Dataset (including any stated split) | SIQA |
| Metric | Accuracy |
| Unit | PERCENT |
| Direction | Higher is better |
| Results loaded | 157 of 157 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [Accuracy](<https://www.sota2.com/research/sota/social-interaction-question-answering-on-siqa.md?metric=Accuracy>): Higher is better; unit: PERCENT.

## Recorded results

| Method | Accuracy | Result date | Report paper |
| --- | --- | --- | --- |
| Human Performance | 86.9 | Not recorded | 68fa73411e209131cea5b4d0 |
| T5 | 81.4 | Not recorded | 68fa68811e209131cea1c854 |
| UNIFIEDQA | 81.4 | Not recorded | 68fa68811e209131cea1c854 |
| LoRA-RITE | 80.25 | Not recorded | 69aa5f922d603e6de3e4196f |
| Stable-LoRA | 80.25 | Not recorded | 69aa5f922d603e6de3e4196f |
| DeBERTa-v3-L (Supervised) | 80.1 | Not recorded | 68fa73411e209131cea5b4d0 |
| VERA-T5 (Multitask Supervised) | 80.1 | Not recorded | 68fa73411e209131cea5b4d0 |
| LoRA+ | 79.94 | Not recorded | 69aa5f922d603e6de3e4196f |
| AdamW | 79.89 | Not recorded | 69aa5f922d603e6de3e4196f |
| Riemann | 79.89 | Not recorded | 69aa5f922d603e6de3e4196f |
| RoBERTa | 78 | Not recorded | 68fa68811e209131cea1c854 |
| Stable-LoRA | 77.64 | Not recorded | 69aa5f922d603e6de3e4196f |
| LoRA+ | 77.38 | Not recorded | 69aa5f922d603e6de3e4196f |
| AdamW | 77.33 | Not recorded | 69aa5f922d603e6de3e4196f |
| LoRA-RITE | 77.23 | Not recorded | 69aa5f922d603e6de3e4196f |
| Riemann | 76.87 | Not recorded | 69aa5f922d603e6de3e4196f |
| ROBERTa-L (Supervised) | 76.6 | Not recorded | 68fa73411e209131cea5b4d0 |
| BART-large | 74 | Not recorded | 68fa68811e209131cea1c854 |
| UnifiedQA-BART | 73.2 | Not recorded | 68fa68811e209131cea1c854 |
| Stable-LoRA | 72.26 | Not recorded | 69aa5f922d603e6de3e4196f |
| AdamW | 71.7 | Not recorded | 69aa5f922d603e6de3e4196f |
| LoRA-RITE | 71.6 | Not recorded | 69aa5f922d603e6de3e4196f |
| LoRA+ | 71.19 | Not recorded | 69aa5f922d603e6de3e4196f |
| ChatGPT + Chain-of-thought | 70.7 | Not recorded | 68fa73411e209131cea5b4d0 |
| Riemann | 70.62 | Not recorded | 69aa5f922d603e6de3e4196f |
| ChatGPT + Self-consistent chain-of-thought | 69.7 | Not recorded | 68fa73411e209131cea5b4d0 |
| ChatGPT | 69.5 | Not recorded | 68fa73411e209131cea5b4d0 |
| Stable-LoRA | 68.27 | Not recorded | 69aa5f922d603e6de3e4196f |
| GPT-3.5 | 68 | Not recorded | 68fa73411e209131cea5b4d0 |
| LoRA+ | 67.91 | Not recorded | 69aa5f922d603e6de3e4196f |
| AdamW | 67.04 | Not recorded | 69aa5f922d603e6de3e4196f |
| Riemann | 66.94 | Not recorded | 69aa5f922d603e6de3e4196f |
| LoRA-RITE | 66.89 | Not recorded | 69aa5f922d603e6de3e4196f |
| ZS-Fusion | 66.7 | Not recorded | 68fa73411e209131cea5b4d0 |
| DeBERTa-v3-L (CANDLE Distilled) | 65.9 | Not recorded | 68fa73411e209131cea5b4d0 |
| CAR-ROBERTa-L | 64.8 | Not recorded | 68fa73411e209131cea5b4d0 |
| Baseline | 64.8 | Not recorded | 68fa7b951e209131cea89474 |
| STL-Adapter | 64.7 | Not recorded | 68fa73411e209131cea5b4d0 |
| STL-Adapter | 64.4 | Not recorded | 68fa73411e209131cea5b4d0 |
| CAR-ROBERTa-L | 64 | Not recorded | 68fa73411e209131cea5b4d0 |
| CAR-DEBERTa-v3-L | 64 | Not recorded | 68fa73411e209131cea5b4d0 |
| CAR-DEBERTa-v3-L | 63.8 | Not recorded | 68fa73411e209131cea5b4d0 |
| STL-PLM | 63.2 | Not recorded | 68fa73411e209131cea5b4d0 |
| ROBERTa-L (MR) | 63.2 | Not recorded | 68fa73411e209131cea5b4d0 |
| ROBERTa-L (MR) | 63.1 | Not recorded | 68fa73411e209131cea5b4d0 |
| DeBERTa-v3-L (MR) | 62.1 | Not recorded | 68fa73411e209131cea5b4d0 |
| MTL | 61.9 | Not recorded | 68fa73411e209131cea5b4d0 |
| SEED | 61.8 | Not recorded | 6a0a9cdedbc836b87d82b23f |
| ROBERTa-L (MR) | 61 | Not recorded | 68fa73411e209131cea5b4d0 |
| DeBERTa-v3-L (MR) | 59.7 | Not recorded | 68fa73411e209131cea5b4d0 |
| VERA-T5-xxl (CANDLE Distilled) | 59.4 | Not recorded | 68fa73411e209131cea5b4d0 |
| FULL | 59.4 | Not recorded | 6a0a9cdedbc836b87d82b23f |
| Tensor RMS + C | 58.6 | Not recorded | 68fa7b951e209131cea89474 |
| Task-specific FT | 58.39 | Not recorded | 698575ddc97a0b3e3b5d5ecc |
| VERA-T5-xxl | 58.2 | Not recorded | 68fa73411e209131cea5b4d0 |
| VERA-T5-xxl | 58.1 | Not recorded | 68fa73411e209131cea5b4d0 |
| TSV-M | 58.03 | Not recorded | 698575ddc97a0b3e3b5d5ecc |
| VERA-T5-xxl | 57.7 | Not recorded | 68fa73411e209131cea5b4d0 |
| GPT-4 | 57 | Not recorded | 68fa73411e209131cea5b4d0 |
| TIES | 56.96 | Not recorded | 698575ddc97a0b3e3b5d5ecc |
| Task-specific FT | 56.81 | Not recorded | 698575ddc97a0b3e3b5d5ecc |
| MICO | 56 | Not recorded | 68fa73411e209131cea5b4d0 |
| RANDOM | 55.6 | Not recorded | 6a0a9cdedbc836b87d82b23f |
| OrthoMerge | 55.17 | Not recorded | 698575ddc97a0b3e3b5d5ecc |
| TSV-M | 55.02 | Not recorded | 698575ddc97a0b3e3b5d5ecc |
| ROBERTa-L (MR) | 54.8 | Not recorded | 68fa73411e209131cea5b4d0 |
| ProQA | 54.5 | Not recorded | 68fa6f471e209131cea44b64 |
| TA | 54.15 | Not recorded | 698575ddc97a0b3e3b5d5ecc |
| DARE | 53.84 | Not recorded | 698575ddc97a0b3e3b5d5ecc |
| TIES | 53.74 | Not recorded | 698575ddc97a0b3e3b5d5ecc |
| ZS-Fusion | 53.7 | Not recorded | 68fa73411e209131cea5b4d0 |
| Tensor RMS + Sp | 53.6 | Not recorded | 68fa7b951e209131cea89474 |
| KoCo | 53.4 | Not recorded | 69df1b777d0172140ec7b810 |
| URL Prefix (MeCo) | 52.9 | Not recorded | 69df1b777d0172140ec7b810 |
| Standard CPT | 52.7 | Not recorded | 69df1b777d0172140ec7b810 |
| Muppet | 52.63 | Not recorded | 68fa6f471e209131cea44b64 |
| Data Selection | 52.6 | Not recorded | 69df1b777d0172140ec7b810 |
| OLTQA | 52.51 | Not recorded | 68fa6f471e209131cea44b64 |
| Hyperformer++ | 52.46 | Not recorded | 68fa6f471e209131cea44b64 |
| TA | 52.2 | Not recorded | 698575ddc97a0b3e3b5d5ecc |

### Human Performance

Accuracy: 86.9. Result ID: 6992b2823a6c31484910d514.

[Reporting paper](<https://www.sota2.com/research/paper/68fa73411e209131cea5b4d0.md>)

Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction.

### T5

Accuracy: 81.4. Result ID: 698d736b8b517eb433219bd2.

[Reporting paper](<https://www.sota2.com/research/paper/68fa68811e209131cea1c854.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa709e1e209131cea4c470.md>)

[Implementation](<https://github.com/google-research/text-to-text-transfer-transformer>)

Recorded evaluation configuration: {"Model Architecture":"T5"}. This record does not assert independent reproduction.

### UNIFIEDQA

Accuracy: 81.4. Result ID: 698d736b8b517eb433219bd3.

[Reporting paper](<https://www.sota2.com/research/paper/68fa68811e209131cea1c854.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa68811e209131cea1c854.md>)

[Implementation](<https://github.com/allenai/unifiedqa>)

Recorded evaluation configuration: {"Model Architecture":"T5"}. This record does not assert independent reproduction.

### LoRA-RITE

Accuracy: 80.25. Result ID: 69aa9d126dbb7075b954a3d8.

[Reporting paper](<https://www.sota2.com/research/paper/69aa5f922d603e6de3e4196f.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7a091e209131cea80b04.md>)

Recorded evaluation configuration: {"Model":"3B"}. This record does not assert independent reproduction.

### Stable-LoRA

Accuracy: 80.25. Result ID: 69aa9d126dbb7075b954a3d9.

[Reporting paper](<https://www.sota2.com/research/paper/69aa5f922d603e6de3e4196f.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69aa5f922d603e6de3e4196f.md>)

Recorded evaluation configuration: {"Model":"3B"}. This record does not assert independent reproduction.

### DeBERTa-v3-L (Supervised)

Accuracy: 80.1. Result ID: 6992b2823a6c31484910d512.

[Reporting paper](<https://www.sota2.com/research/paper/68fa73411e209131cea5b4d0.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6ee11e209131cea4297b.md>)

[Implementation](<https://github.com/microsoft/DeBERTa>)

Recorded evaluation configuration: {"Evaluation Protocol":"Supervised"}. This record does not assert independent reproduction.

### VERA-T5 (Multitask Supervised)

Accuracy: 80.1. Result ID: 6992b2823a6c31484910d513.

[Reporting paper](<https://www.sota2.com/research/paper/68fa73411e209131cea5b4d0.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa70e51e209131cea4dbc7.md>)

Recorded evaluation configuration: {"Evaluation Protocol":"Supervised"}. This record does not assert independent reproduction.

### LoRA+

Accuracy: 79.94. Result ID: 69aa9d126dbb7075b954a3d6.

[Reporting paper](<https://www.sota2.com/research/paper/69aa5f922d603e6de3e4196f.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa74061e209131cea5f4d8.md>)

Recorded evaluation configuration: {"Model":"3B"}. This record does not assert independent reproduction.

### AdamW

Accuracy: 79.89. Result ID: 69aa9d126dbb7075b954a3d5.

[Reporting paper](<https://www.sota2.com/research/paper/69aa5f922d603e6de3e4196f.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa63171e209131ce9fca0b.md>)

Recorded evaluation configuration: {"Model":"3B"}. This record does not assert independent reproduction.

### Riemann

Accuracy: 79.89. Result ID: 69aa9d126dbb7075b954a3d7.

[Reporting paper](<https://www.sota2.com/research/paper/69aa5f922d603e6de3e4196f.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa5be71e209131ce9cc789.md>)

Recorded evaluation configuration: {"Model":"3B"}. This record does not assert independent reproduction.

### RoBERTa

Accuracy: 78. Result ID: 698d736b8b517eb433219bcf.

[Reporting paper](<https://www.sota2.com/research/paper/68fa68811e209131cea1c854.md>)

Recorded evaluation configuration: {"Model Architecture":"RoBERTa"}. This record does not assert independent reproduction.

### Stable-LoRA

Accuracy: 77.64. Result ID: 69aa9d126dbb7075b954a3d4.

[Reporting paper](<https://www.sota2.com/research/paper/69aa5f922d603e6de3e4196f.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69aa5f922d603e6de3e4196f.md>)

Recorded evaluation configuration: {"Model":"1.5B"}. This record does not assert independent reproduction.

### LoRA+

Accuracy: 77.38. Result ID: 69aa9d126dbb7075b954a3d1.

[Reporting paper](<https://www.sota2.com/research/paper/69aa5f922d603e6de3e4196f.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa74061e209131cea5f4d8.md>)

Recorded evaluation configuration: {"Model":"1.5B"}. This record does not assert independent reproduction.

### AdamW

Accuracy: 77.33. Result ID: 69aa9d126dbb7075b954a3d0.

[Reporting paper](<https://www.sota2.com/research/paper/69aa5f922d603e6de3e4196f.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa63171e209131ce9fca0b.md>)

Recorded evaluation configuration: {"Model":"1.5B"}. This record does not assert independent reproduction.

### LoRA-RITE

Accuracy: 77.23. Result ID: 69aa9d126dbb7075b954a3d3.

[Reporting paper](<https://www.sota2.com/research/paper/69aa5f922d603e6de3e4196f.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7a091e209131cea80b04.md>)

Recorded evaluation configuration: {"Model":"1.5B"}. This record does not assert independent reproduction.

### Riemann

Accuracy: 76.87. Result ID: 69aa9d126dbb7075b954a3d2.

[Reporting paper](<https://www.sota2.com/research/paper/69aa5f922d603e6de3e4196f.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa5be71e209131ce9cc789.md>)

Recorded evaluation configuration: {"Model":"1.5B"}. This record does not assert independent reproduction.

### ROBERTa-L (Supervised)

Accuracy: 76.6. Result ID: 6992b2823a6c31484910d511.

[Reporting paper](<https://www.sota2.com/research/paper/68fa73411e209131cea5b4d0.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa66131e209131cea0e18b.md>)

[Implementation](<https://github.com/pytorch/fairseq>)

Recorded evaluation configuration: {"Evaluation Protocol":"Supervised"}. This record does not assert independent reproduction.

### BART-large

Accuracy: 74. Result ID: 698d736b8b517eb433219bd0.

[Reporting paper](<https://www.sota2.com/research/paper/68fa68811e209131cea1c854.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa66921e209131cea10fff.md>)

Recorded evaluation configuration: {"Model Architecture":"BART-large"}. This record does not assert independent reproduction.

### UnifiedQA-BART

Accuracy: 73.2. Result ID: 698d736b8b517eb433219bd1.

[Reporting paper](<https://www.sota2.com/research/paper/68fa68811e209131cea1c854.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa68811e209131cea1c854.md>)

[Implementation](<https://github.com/allenai/unifiedqa>)

Recorded evaluation configuration: {"Model Architecture":"BART"}. This record does not assert independent reproduction.

### Stable-LoRA

Accuracy: 72.26. Result ID: 69aa9d126dbb7075b954a3cf.

[Reporting paper](<https://www.sota2.com/research/paper/69aa5f922d603e6de3e4196f.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69aa5f922d603e6de3e4196f.md>)

Recorded evaluation configuration: {"Model":"1B"}. This record does not assert independent reproduction.

### AdamW

Accuracy: 71.7. Result ID: 69aa9d126dbb7075b954a3cb.

[Reporting paper](<https://www.sota2.com/research/paper/69aa5f922d603e6de3e4196f.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa63171e209131ce9fca0b.md>)

Recorded evaluation configuration: {"Model":"1B"}. This record does not assert independent reproduction.

### LoRA-RITE

Accuracy: 71.6. Result ID: 69aa9d126dbb7075b954a3ce.

[Reporting paper](<https://www.sota2.com/research/paper/69aa5f922d603e6de3e4196f.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7a091e209131cea80b04.md>)

Recorded evaluation configuration: {"Model":"1B"}. This record does not assert independent reproduction.

### LoRA+

Accuracy: 71.19. Result ID: 69aa9d126dbb7075b954a3cc.

[Reporting paper](<https://www.sota2.com/research/paper/69aa5f922d603e6de3e4196f.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa74061e209131cea5f4d8.md>)

Recorded evaluation configuration: {"Model":"1B"}. This record does not assert independent reproduction.

### ChatGPT + Chain-of-thought

Accuracy: 70.7. Result ID: 6992b2823a6c31484910d507.

[Reporting paper](<https://www.sota2.com/research/paper/68fa73411e209131cea5b4d0.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6e571e209131cea3f27b.md>)

Recorded evaluation configuration: {"Version":"gpt-3.5-turbo","Evaluation Protocol":"Zero-shot"}. This record does not assert independent reproduction.

### Riemann

Accuracy: 70.62. Result ID: 69aa9d126dbb7075b954a3cd.

[Reporting paper](<https://www.sota2.com/research/paper/69aa5f922d603e6de3e4196f.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa5be71e209131ce9cc789.md>)

Recorded evaluation configuration: {"Model":"1B"}. This record does not assert independent reproduction.

### ChatGPT + Self-consistent chain-of-thought

Accuracy: 69.7. Result ID: 6992b2823a6c31484910d508.

[Reporting paper](<https://www.sota2.com/research/paper/68fa73411e209131cea5b4d0.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6eb61e209131cea41b33.md>)

Recorded evaluation configuration: {"Version":"gpt-3.5-turbo","Evaluation Protocol":"Zero-shot"}. This record does not assert independent reproduction.

### ChatGPT

Accuracy: 69.5. Result ID: 6992b2823a6c31484910d506.

[Reporting paper](<https://www.sota2.com/research/paper/68fa73411e209131cea5b4d0.md>)

Recorded evaluation configuration: {"Version":"gpt-3.5-turbo","Evaluation Protocol":"Zero-shot"}. This record does not assert independent reproduction.

### Stable-LoRA

Accuracy: 68.27. Result ID: 69aa9d126dbb7075b954a3ca.

[Reporting paper](<https://www.sota2.com/research/paper/69aa5f922d603e6de3e4196f.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69aa5f922d603e6de3e4196f.md>)

Recorded evaluation configuration: {"Model":"0.5B"}. This record does not assert independent reproduction.

### GPT-3.5

Accuracy: 68. Result ID: 6992b2823a6c31484910d505.

[Reporting paper](<https://www.sota2.com/research/paper/68fa73411e209131cea5b4d0.md>)

Recorded evaluation configuration: {"Version":"text-davinci-003","Evaluation Protocol":"Zero-shot"}. This record does not assert independent reproduction.

### LoRA+

Accuracy: 67.91. Result ID: 69aa9d126dbb7075b954a3c7.

[Reporting paper](<https://www.sota2.com/research/paper/69aa5f922d603e6de3e4196f.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa74061e209131cea5f4d8.md>)

Recorded evaluation configuration: {"Model":"0.5B"}. This record does not assert independent reproduction.

### AdamW

Accuracy: 67.04. Result ID: 69aa9d126dbb7075b954a3c6.

[Reporting paper](<https://www.sota2.com/research/paper/69aa5f922d603e6de3e4196f.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa63171e209131ce9fca0b.md>)

Recorded evaluation configuration: {"Model":"0.5B"}. This record does not assert independent reproduction.

### Riemann

Accuracy: 66.94. Result ID: 69aa9d126dbb7075b954a3c8.

[Reporting paper](<https://www.sota2.com/research/paper/69aa5f922d603e6de3e4196f.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa5be71e209131ce9cc789.md>)

Recorded evaluation configuration: {"Model":"0.5B"}. This record does not assert independent reproduction.

### LoRA-RITE

Accuracy: 66.89. Result ID: 69aa9d126dbb7075b954a3c9.

[Reporting paper](<https://www.sota2.com/research/paper/69aa5f922d603e6de3e4196f.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7a091e209131cea80b04.md>)

Recorded evaluation configuration: {"Model":"0.5B"}. This record does not assert independent reproduction.

### ZS-Fusion

Accuracy: 66.7. Result ID: 6992b2823a6c31484910d4ff.

[Reporting paper](<https://www.sota2.com/research/paper/68fa73411e209131cea5b4d0.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6cd31e209131cea36050.md>)

Recorded evaluation configuration: {"CSKB":"CSKG","Evaluation Protocol":"Zero-shot"}. This record does not assert independent reproduction.

### DeBERTa-v3-L (CANDLE Distilled)

Accuracy: 65.9. Result ID: 6992b2823a6c31484910d504.

[Reporting paper](<https://www.sota2.com/research/paper/68fa73411e209131cea5b4d0.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73411e209131cea5b4d0.md>)

[Implementation](<https://github.com/HKUST-KnowComp/CANDLE>)

Recorded evaluation configuration: {"CSKB":"CANDLE","Evaluation Protocol":"Zero-shot"}. This record does not assert independent reproduction.

### CAR-ROBERTa-L

Accuracy: 64.8. Result ID: 6992b2823a6c31484910d500.

[Reporting paper](<https://www.sota2.com/research/paper/68fa73411e209131cea5b4d0.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa70e81e209131cea4de50.md>)

Recorded evaluation configuration: {"CSKB":"ATOMIC","Evaluation Protocol":"Zero-shot"}. This record does not assert independent reproduction.

### Baseline

Accuracy: 64.8. Result ID: 699a47a75698665ec4cef800.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7b951e209131cea89474.md>)

Recorded evaluation configuration: {"Bit width (b)":"16.00"}. This record does not assert independent reproduction.

### STL-Adapter

Accuracy: 64.7. Result ID: 6992b2823a6c31484910d4f7.

[Reporting paper](<https://www.sota2.com/research/paper/68fa73411e209131cea5b4d0.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6cd31e209131cea36050.md>)

Recorded evaluation configuration: {"CSKB":"CSKG","Evaluation Protocol":"Zero-shot"}. This record does not assert independent reproduction.

### STL-Adapter

Accuracy: 64.4. Result ID: 6992b2823a6c31484910d4f6.

[Reporting paper](<https://www.sota2.com/research/paper/68fa73411e209131cea5b4d0.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6cd31e209131cea36050.md>)

Recorded evaluation configuration: {"CSKB":"ATOMIC","Evaluation Protocol":"Zero-shot"}. This record does not assert independent reproduction.

### CAR-ROBERTa-L

Accuracy: 64. Result ID: 6992b2823a6c31484910d501.

[Reporting paper](<https://www.sota2.com/research/paper/68fa73411e209131cea5b4d0.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa70e81e209131cea4de50.md>)

Recorded evaluation configuration: {"CSKB":"AbsATM","Evaluation Protocol":"Zero-shot"}. This record does not assert independent reproduction.

### CAR-DEBERTa-v3-L

Accuracy: 64. Result ID: 6992b2823a6c31484910d503.

[Reporting paper](<https://www.sota2.com/research/paper/68fa73411e209131cea5b4d0.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa70e81e209131cea4de50.md>)

Recorded evaluation configuration: {"CSKB":"AbsATM","Evaluation Protocol":"Zero-shot"}. This record does not assert independent reproduction.

### CAR-DEBERTa-v3-L

Accuracy: 63.8. Result ID: 6992b2823a6c31484910d502.

[Reporting paper](<https://www.sota2.com/research/paper/68fa73411e209131cea5b4d0.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa70e81e209131cea4de50.md>)

Recorded evaluation configuration: {"CSKB":"ATOMIC","Evaluation Protocol":"Zero-shot"}. This record does not assert independent reproduction.

### STL-PLM

Accuracy: 63.2. Result ID: 6992b2823a6c31484910d4f3.

[Reporting paper](<https://www.sota2.com/research/paper/68fa73411e209131cea5b4d0.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6cd31e209131cea36050.md>)

Recorded evaluation configuration: {"CSKB":"ATOMIC","Evaluation Protocol":"Zero-shot"}. This record does not assert independent reproduction.

### ROBERTa-L (MR)

Accuracy: 63.2. Result ID: 6992b2823a6c31484910d4fb.

[Reporting paper](<https://www.sota2.com/research/paper/68fa73411e209131cea5b4d0.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa68f61e209131cea1f563.md>)

Recorded evaluation configuration: {"CSKB":"CSKG","Evaluation Protocol":"Zero-shot"}. This record does not assert independent reproduction.

### ROBERTa-L (MR)

Accuracy: 63.1. Result ID: 6992b2823a6c31484910d4f9.

[Reporting paper](<https://www.sota2.com/research/paper/68fa73411e209131cea5b4d0.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa68f61e209131cea1f563.md>)

Recorded evaluation configuration: {"CSKB":"ATOMIC","Evaluation Protocol":"Zero-shot"}. This record does not assert independent reproduction.

### DeBERTa-v3-L (MR)

Accuracy: 62.1. Result ID: 6992b2823a6c31484910d4fd.

[Reporting paper](<https://www.sota2.com/research/paper/68fa73411e209131cea5b4d0.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa68f61e209131cea1f563.md>)

Recorded evaluation configuration: {"CSKB":"ATOMIC","Evaluation Protocol":"Zero-shot"}. This record does not assert independent reproduction.

### MTL

Accuracy: 61.9. Result ID: 6992b2823a6c31484910d4f5.

[Reporting paper](<https://www.sota2.com/research/paper/68fa73411e209131cea5b4d0.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6cd31e209131cea36050.md>)

Recorded evaluation configuration: {"CSKB":"CSKG","Evaluation Protocol":"Zero-shot"}. This record does not assert independent reproduction.

### SEED

Accuracy: 61.8. Result ID: 6a0ac75247f34ce924f0b72f.

[Reporting paper](<https://www.sota2.com/research/paper/6a0a9cdedbc836b87d82b23f.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a0a9cdedbc836b87d82b23f.md>)

Recorded evaluation configuration: {"Backbone":"LLaMA2-7B","Selection Ratio":"5%"}. This record does not assert independent reproduction.

### ROBERTa-L (MR)

Accuracy: 61. Result ID: 6992b2823a6c31484910d4f8.

[Reporting paper](<https://www.sota2.com/research/paper/68fa73411e209131cea5b4d0.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa68f61e209131cea1f563.md>)

Recorded evaluation configuration: {"CSKB":"ATM10X","Evaluation Protocol":"Zero-shot"}. This record does not assert independent reproduction.

### DeBERTa-v3-L (MR)

Accuracy: 59.7. Result ID: 6992b2823a6c31484910d4fc.

[Reporting paper](<https://www.sota2.com/research/paper/68fa73411e209131cea5b4d0.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa68f61e209131cea1f563.md>)

Recorded evaluation configuration: {"CSKB":"ATM10X","Evaluation Protocol":"Zero-shot"}. This record does not assert independent reproduction.

### VERA-T5-xxl (CANDLE Distilled)

Accuracy: 59.4. Result ID: 6992b2823a6c31484910d510.

[Reporting paper](<https://www.sota2.com/research/paper/68fa73411e209131cea5b4d0.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73411e209131cea5b4d0.md>)

[Implementation](<https://github.com/HKUST-KnowComp/CANDLE>)

Recorded evaluation configuration: {"CSKB":"CANDLE","Evaluation Protocol":"Zero-shot"}. This record does not assert independent reproduction.

### FULL

Accuracy: 59.4. Result ID: 6a0ac75247f34ce924f0b72d.

[Reporting paper](<https://www.sota2.com/research/paper/6a0a9cdedbc836b87d82b23f.md>)

Recorded evaluation configuration: {"Backbone":"LLaMA2-7B","Selection Ratio":"100%"}. This record does not assert independent reproduction.

### Tensor RMS + C

Accuracy: 58.6. Result ID: 699a47a75698665ec4cef801.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7b951e209131cea89474.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7b951e209131cea89474.md>)

Recorded evaluation configuration: {"Bit width (b)":"3.00"}. This record does not assert independent reproduction.

### Task-specific FT

Accuracy: 58.39. Result ID: 6998ed165698665ec451a005.

[Reporting paper](<https://www.sota2.com/research/paper/698575ddc97a0b3e3b5d5ecc.md>)

Recorded evaluation configuration: {"Base Model":"Llama-3.1-8B","Fine-tuning Technique":"LoRA","Merging Strategy":"None"}. This record does not assert independent reproduction.

### VERA-T5-xxl

Accuracy: 58.2. Result ID: 6992b2823a6c31484910d50e.

[Reporting paper](<https://www.sota2.com/research/paper/68fa73411e209131cea5b4d0.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa70e51e209131cea4dbc7.md>)

Recorded evaluation configuration: {"CSKB":"ATM10X","Evaluation Protocol":"Zero-shot"}. This record does not assert independent reproduction.

### VERA-T5-xxl

Accuracy: 58.1. Result ID: 6992b2823a6c31484910d50f.

[Reporting paper](<https://www.sota2.com/research/paper/68fa73411e209131cea5b4d0.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa70e51e209131cea4dbc7.md>)

Recorded evaluation configuration: {"CSKB":"AbsATM","Evaluation Protocol":"Zero-shot"}. This record does not assert independent reproduction.

### TSV-M

Accuracy: 58.03. Result ID: 6998ed165698665ec451a009.

[Reporting paper](<https://www.sota2.com/research/paper/698575ddc97a0b3e3b5d5ecc.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa77c11e209131cea751d7.md>)

Recorded evaluation configuration: {"Base Model":"Llama-3.1-8B","Fine-tuning Technique":"LoRA","Merging Strategy":"TSV-M"}. This record does not assert independent reproduction.

### VERA-T5-xxl

Accuracy: 57.7. Result ID: 6992b2823a6c31484910d50d.

[Reporting paper](<https://www.sota2.com/research/paper/68fa73411e209131cea5b4d0.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa70e51e209131cea4dbc7.md>)

Recorded evaluation configuration: {"CSKB":"ATOMIC","Evaluation Protocol":"Zero-shot"}. This record does not assert independent reproduction.

### GPT-4

Accuracy: 57. Result ID: 6992b2823a6c31484910d509.

[Reporting paper](<https://www.sota2.com/research/paper/68fa73411e209131cea5b4d0.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa727a1e209131cea56a1e.md>)

[Implementation](<https://github.com/openai/evals>)

Recorded evaluation configuration: {"Evaluation Protocol":"Zero-shot"}. This record does not assert independent reproduction.

### TIES

Accuracy: 56.96. Result ID: 6998ed165698665ec451a007.

[Reporting paper](<https://www.sota2.com/research/paper/698575ddc97a0b3e3b5d5ecc.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa71091e209131cea4ecd8.md>)

Recorded evaluation configuration: {"Base Model":"Llama-3.1-8B","Fine-tuning Technique":"LoRA","Merging Strategy":"TIES"}. This record does not assert independent reproduction.

### Task-specific FT

Accuracy: 56.81. Result ID: 6998ed165698665ec451a00a.

[Reporting paper](<https://www.sota2.com/research/paper/698575ddc97a0b3e3b5d5ecc.md>)

Recorded evaluation configuration: {"Base Model":"Llama-3.1-8B","Fine-tuning Technique":"OFT","Merging Strategy":"None"}. This record does not assert independent reproduction.

### MICO

Accuracy: 56. Result ID: 6992b2823a6c31484910d4f2.

[Reporting paper](<https://www.sota2.com/research/paper/68fa73411e209131cea5b4d0.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6daa1e209131cea3b4ca.md>)

Recorded evaluation configuration: {"CSKB":"ATOMIC","Evaluation Protocol":"Zero-shot"}. This record does not assert independent reproduction.

### RANDOM

Accuracy: 55.6. Result ID: 6a0ac75247f34ce924f0b72e.

[Reporting paper](<https://www.sota2.com/research/paper/6a0a9cdedbc836b87d82b23f.md>)

Recorded evaluation configuration: {"Backbone":"LLaMA2-7B","Selection Ratio":"5%"}. This record does not assert independent reproduction.

### OrthoMerge

Accuracy: 55.17. Result ID: 6998ed165698665ec451a00f.

[Reporting paper](<https://www.sota2.com/research/paper/698575ddc97a0b3e3b5d5ecc.md>)

[Method origin paper](<https://www.sota2.com/research/paper/698575ddc97a0b3e3b5d5ecc.md>)

Recorded evaluation configuration: {"Base Model":"Llama-3.1-8B","Fine-tuning Technique":"OFT","Merging Strategy":"OrthoMerge"}. This record does not assert independent reproduction.

### TSV-M

Accuracy: 55.02. Result ID: 6998ed165698665ec451a00e.

[Reporting paper](<https://www.sota2.com/research/paper/698575ddc97a0b3e3b5d5ecc.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa77c11e209131cea751d7.md>)

Recorded evaluation configuration: {"Base Model":"Llama-3.1-8B","Fine-tuning Technique":"OFT","Merging Strategy":"TSV-M"}. This record does not assert independent reproduction.

### ROBERTa-L (MR)

Accuracy: 54.8. Result ID: 6992b2823a6c31484910d4fa.

[Reporting paper](<https://www.sota2.com/research/paper/68fa73411e209131cea5b4d0.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa68f61e209131cea1f563.md>)

Recorded evaluation configuration: {"CSKB":"CWWV","Evaluation Protocol":"Zero-shot"}. This record does not assert independent reproduction.

### ProQA

Accuracy: 54.5. Result ID: 6990b35f3a6c314849028d6e.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6f471e209131cea44b64.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6e2c1e209131cea3e54d.md>)

Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction.

### TA

Accuracy: 54.15. Result ID: 6998ed165698665ec451a006.

[Reporting paper](<https://www.sota2.com/research/paper/698575ddc97a0b3e3b5d5ecc.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6ef01e209131cea43012.md>)

Recorded evaluation configuration: {"Base Model":"Llama-3.1-8B","Fine-tuning Technique":"LoRA","Merging Strategy":"TA"}. This record does not assert independent reproduction.

### DARE

Accuracy: 53.84. Result ID: 6998ed165698665ec451a008.

[Reporting paper](<https://www.sota2.com/research/paper/698575ddc97a0b3e3b5d5ecc.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73921e209131cea5d4aa.md>)

Recorded evaluation configuration: {"Base Model":"Llama-3.1-8B","Fine-tuning Technique":"LoRA","Merging Strategy":"DARE"}. This record does not assert independent reproduction.

### TIES

Accuracy: 53.74. Result ID: 6998ed165698665ec451a00c.

[Reporting paper](<https://www.sota2.com/research/paper/698575ddc97a0b3e3b5d5ecc.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa71091e209131cea4ecd8.md>)

Recorded evaluation configuration: {"Base Model":"Llama-3.1-8B","Fine-tuning Technique":"OFT","Merging Strategy":"TIES"}. This record does not assert independent reproduction.

### ZS-Fusion

Accuracy: 53.7. Result ID: 6992b2823a6c31484910d4fe.

[Reporting paper](<https://www.sota2.com/research/paper/68fa73411e209131cea5b4d0.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6cd31e209131cea36050.md>)

Recorded evaluation configuration: {"CSKB":"CWWV","Evaluation Protocol":"Zero-shot"}. This record does not assert independent reproduction.

### Tensor RMS + Sp

Accuracy: 53.6. Result ID: 699a47a75698665ec4cef802.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7b951e209131cea89474.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7b951e209131cea89474.md>)

Recorded evaluation configuration: {"Bit width (b)":"3.05"}. This record does not assert independent reproduction.

### KoCo

Accuracy: 53.4. Result ID: 69df4f4947f34ce924c820e3.

[Reporting paper](<https://www.sota2.com/research/paper/69df1b777d0172140ec7b810.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69df1b777d0172140ec7b810.md>)

Recorded evaluation configuration: {"Model Scale":"1.6B Parameters"}. This record does not assert independent reproduction.

### URL Prefix (MeCo)

Accuracy: 52.9. Result ID: 69df4f4947f34ce924c820e0.

[Reporting paper](<https://www.sota2.com/research/paper/69df1b777d0172140ec7b810.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa79b31e209131cea7efb3.md>)

Recorded evaluation configuration: {"Model Scale":"1.6B Parameters"}. This record does not assert independent reproduction.

### Standard CPT

Accuracy: 52.7. Result ID: 69df4f4947f34ce924c820e1.

[Reporting paper](<https://www.sota2.com/research/paper/69df1b777d0172140ec7b810.md>)

Recorded evaluation configuration: {"Model Scale":"1.6B Parameters"}. This record does not assert independent reproduction.

### Muppet

Accuracy: 52.63. Result ID: 6990b35f3a6c314849028d6f.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6f471e209131cea44b64.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa69321e209131cea20c16.md>)

[Implementation](<https://github.com/google-research/electra#expected-results>)

Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction.

### Data Selection

Accuracy: 52.6. Result ID: 69df4f4947f34ce924c820e2.

[Reporting paper](<https://www.sota2.com/research/paper/69df1b777d0172140ec7b810.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa78351e209131cea77359.md>)

Recorded evaluation configuration: {"Model Scale":"1.6B Parameters"}. This record does not assert independent reproduction.

### OLTQA

Accuracy: 52.51. Result ID: 6990b35f3a6c314849028d7a.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6f471e209131cea44b64.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6f471e209131cea44b64.md>)

[Implementation](<https://github.com/AlibabaResearch/DAMO-ConvAI/tree/main/oltqa>)

Recorded evaluation configuration: {"Configuration":"Full"}. This record does not assert independent reproduction.

### Hyperformer++

Accuracy: 52.46. Result ID: 6990b35f3a6c314849028d70.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6f471e209131cea44b64.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6a1d1e209131cea25f67.md>)

[Implementation](<https://github.com/rabeehk/hyperformer>)

Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction.

### TA

Accuracy: 52.2. Result ID: 6998ed165698665ec451a00b.

[Reporting paper](<https://www.sota2.com/research/paper/698575ddc97a0b3e3b5d5ecc.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6ef01e209131cea43012.md>)

Recorded evaluation configuration: {"Base Model":"Llama-3.1-8B","Fine-tuning Technique":"OFT","Merging Strategy":"TA"}. This record does not assert independent reproduction.

Page 1 of 2. This is an index page, not the full collection.

- [Next index page](<https://www.sota2.com/research/sota/social-interaction-question-answering-on-siqa.md?metric=Accuracy&page=2>): Follow this link to discover more pages.
