Commonsense Question Answering on CSQA (Accuracy)
88.9AccuracyHuman
Evaluation results
| Method | Links | |
|---|---|---|
| HumanKB=-2026.03 | 88.9 | |
| BEAMSparsity Level=Mid Sparsity, beta=0.012026.05 | 86.4 | |
| Qwen3-30B-A3BSparsity Level=Base, K=82026.05 | 86.24 | |
| Top-K ReducedSparsity Level=Mid Sparsity, K=42026.05 | 85.18 | |
| MoE-DynamicSparsity Level=Mid Sparsity, phi=0.32026.05 | 85.18 | |
| IoTModel=GPT-4o mini2026.03 | 84.54 | |
| BEAMSparsity Level=High Sparsity, beta=0.12026.05 | 84.28 | |
| CoTModel=GPT-4o mini2026.03 | 83.21 | |
| DeBERTa-v3-L (Supervised)KB=-, Evaluation Protocol=Supervised2026.03 | 82.1 | |
| SCModel=Olmo-2-13B2026.03 | 80.18 | |
| BEAMSparsity Level=Extreme Sparsity, beta=1.02026.05 | 80.1 | |
| Top-K ReducedSparsity Level=High Sparsity, K=22026.05 | 80.02 | |
| IoTModel=Olmo-2-13B2026.03 | 80.01 | |
| CoTModel=Olmo-2-13B2026.03 | 79.69 | |
| MoE-DynamicSparsity Level=High Sparsity, phi=0.12026.05 | 78.87 | |
| RoBERTa-L (Supervised)KB=-, Evaluation Protocol=Supervised2026.03 | 78.5 | |
| AdaMoESparsity Level=Mid Sparsity, Null=1282026.05 | 77.15 | |
| IoTModel=Olmo-2-7B2026.03 | 76.58 | |
| SCModel=Olmo-2-7B2026.03 | 76.42 | |
| IMAGINE-DeBERTa-v3-LKB=Synthetic VQA+, Evaluation Protocol=Zero-shot2026.03 | 76.3 | |
| IMAGINE-DeBERTa-v3-L (Retrieval)KB=Synthetic VQA+, Inference Strategy=Retrieval, Evaluation Protocol=Zero-shot2026.03 | 76.2 | |
| IoTModel=Llama-3.3-8B2026.03 | 75.84 | |
| ChatGPT (gpt-3.5-turbo)KB=-, Evaluation Protocol=Zero-shot2026.03 | 75.7 | |
| Top-K PruningSparsity Level=Mid Sparsity, K=42026.05 | 75.02 | |
| SCModel=Llama-3.3-8B2026.03 | 74.77 | |
| CoTModel=Olmo-2-7B2026.03 | 74.53 | |
| EoTModel=Llama-3.3-8B2026.03 | 74.44 | |
| IMAGINE-DeBERTa-v3-LKB=Synthetic VQA, Evaluation Protocol=Zero-shot2026.03 | 74 | |
| EoTModel=Olmo-2-7B2026.03 | 73.79 | |
| EoTModel=Olmo-2-13B2026.03 | 73.38 | |
| CoTModel=Llama-3.3-8B2026.03 | 72.89 | |
| Multi-hop Knowledge InjectionKB=AT, CN, WD, WN, Evaluation Protocol=Zero-shot2026.03 | 71 | |
| CANDLE-DeBERTa-v3-LKB=CANDLE, Evaluation Protocol=Zero-shot2026.03 | 69.9 | |
| CAR-DeBERTa-v3-LKB=AbsAT, Evaluation Protocol=Zero-shot2026.03 | 69.3 | |
| GPT-3.5 (text-davinci-003)KB=-, Evaluation Protocol=Zero-shot2026.03 | 68.9 | |
| Top-K ReducedSparsity Level=Extreme Sparsity, K=12026.05 | 68.88 | |
| Zero-shot FusionKB=AT, CN, WD, WN, Evaluation Protocol=Zero-shot2026.03 | 68.2 | |
| IMAGINE-RoBERTa-LKB=Synthetic VQA, Evaluation Protocol=Zero-shot2026.03 | 67.5 | |
| LLAMA2-13BKB=-, Evaluation Protocol=Zero-shot2026.03 | 67.3 | |
| DeBERTa-v3-L (MR)KB=AT, Evaluation Protocol=Zero-shot2026.03 | 67 | |
| CAR-RoBERTa-LKB=AbsAT, Evaluation Protocol=Zero-shot2026.03 | 66.3 | |
| CANDLE-VERA-T5-xxlKB=CANDLE, Evaluation Protocol=Zero-shot2026.03 | 64.7 | |
| RoBERTa-L (MR)KB=AT, Evaluation Protocol=Zero-shot2026.03 | 64.2 | |
| URL Prefix (MeCo)Model Scale=1.6B Parameters2026.04 | 63.96 | |
| IMAGINE-GPT-2-LKB=Synthetic VQA, Evaluation Protocol=Zero-shot2026.03 | 63.9 | |
| Data SelectionModel Scale=1.6B Parameters2026.04 | 63.31 | |
| VERA-T5-xxlKB=AbsAT, Evaluation Protocol=Zero-shot2026.03 | 63 | |
| KoCoModel Scale=1.6B Parameters2026.04 | 61.83 | |
| VERA-T5-xxlKB=AT, Evaluation Protocol=Zero-shot2026.03 | 61.7 | |
| PPFT w/o noiseBackbone=Llama-3.1-8B2026.04 | 60.86 | |
| AdaMoESparsity Level=High Sparsity, Null=2562026.05 | 60.44 | |
| Mistral-v0.1-7BKB=-, Evaluation Protocol=Zero-shot2026.03 | 59.6 | |
| Standard CPTModel Scale=1.6B Parameters2026.04 | 59.54 | |
| PPFT w/o noiseBackbone=Llama-3.2-1B2026.04 | 54.3 | |
| PPFTBackbone=Llama-3.1-8B2026.04 | 52.78 | |
| PPFTBackbone=Llama-3.2-1B2026.04 | 51.25 | |
| CAR-GPT-2-LKB=AbsAT, Evaluation Protocol=Zero-shot2026.03 | 50 | |
| GPT-2-L (MR)KB=AT, Evaluation Protocol=Zero-shot2026.03 | 48 | |
| RoBERTa-LKB=-, Evaluation Protocol=Zero-shot2026.03 | 45 | |
| MICOKB=AT, Evaluation Protocol=Zero-shot2026.03 | 44.2 | |
| GPT-4 (gpt-4)KB=-, Evaluation Protocol=Zero-shot2026.03 | 43 | |
| GPT-2-LKB=-, Evaluation Protocol=Zero-shot2026.03 | 41.4 | |
| InstructBLIP-Vicuna-7BKB=-, Evaluation Protocol=Zero-shot2026.03 | 40.5 | |
| Self-talkKB=-, Evaluation Protocol=Zero-shot2026.03 | 32.4 | |
| LLaVA-1.5-7BKB=-, Evaluation Protocol=Zero-shot2026.03 | 29.4 | |
| DeBERTa-v3-LKB=-, Evaluation Protocol=Zero-shot2026.03 | 25.4 | |
| dχ-privacyBackbone=Llama-3.1-8B2026.04 | 18.19 | |
| Top-K PruningSparsity Level=High Sparsity, K=22026.05 | 16.87 | |
| dχ-privacyBackbone=Llama-3.2-1B2026.04 | 12.1 | |
| ParaphraseBackbone=Llama-3.1-8B2026.04 | 6.49 | |
| ParaphraseBackbone=Llama-3.2-1B2026.04 | 4.7 |