Commonsense Reasoning on PIQA
94.9AccuracyHUMAN
Evaluation results
| Method | Links | ||
|---|---|---|---|
| HUMAN2021.03 | 94.9 | — | |
| Cont-Squeezerank=128 to 1, additional_steps=7002026.02 | 93.22 | — | |
| Direct Fine-tuningrank=1, additional_steps=2002026.02 | 93.11 | — | |
| Direct Fine-tuningrank=1, additional_steps=02026.02 | 93.05 | — | |
| Direct Fine-tuningrank=1, additional_steps=7002026.02 | 93.02 | — | |
| Cont-Squeezerank=128 to 1, additional_steps=2002026.02 | 92.97 | — | |
| In-Squeezerank=128 to 1, schedule=Standard2026.02 | 92.89 | — | |
| In-Squeezerank=128 to 1, schedule=Min steps2026.02 | 92.8 | — | |
| Cont-Squeezerank=128 to 1, additional_steps=02026.02 | 92.49 | — | |
| UNICORN2021.03 | 90.1 | — | |
| Supervised SOTAsupervised=true2022.03 | 90.1 | — | |
| FlexoraBackbone=Gemma-7B2024.08 | 87.54 | — | |
| FlexoraBackbone=Mistral-7B-v0.12024.08 | 86.89 | — | |
| FlexoraBackbone=Qwen2.5-32B2024.08 | 86.72 | — | |
| FlexoraLoRA Rank (r)=322024.08 | 86.02 | — | |
| FlexoraBackbone=Llama3-8B2024.08 | 85.91 | — | |
| FlexoraLoRA Rank (r)=82024.08 | 85.91 | — | |
| Llama 3 405BParameters=405B2024.07 | 85.6 | — | |
| FlexoraBackbone=Zephyr-7B-beta2024.08 | 85.58 | — | |
| Mixtral 8×22BParameters=8×22B2024.07 | 85.5 | — | |
| UNIFIEDQA-3BParameters=3B2021.03 | 85.3 | — | |
| FlexoraLoRA Rank (r)=162024.08 | 85.26 | — | |
| PaLM 2-Lprompting=1-shot2023.05 | 85 | — | |
| FalconModel Size=180B2023.11 | 84.9 | — | |
| FlexoraBackbone=Yi-6B2024.08 | 84.82 | — | |
| ShapLoRATunable Params=22.8M, Backbone=LLaMA-3 8B2026.01 | 84.5 | — | |
| FlexoraBackbone=Qwen1.5-7B2024.08 | 84.33 | — | |
| Inflection-12023.11 | 84.2 | — | |
| PaLMprompting=1-shot2023.05 | 83.9 | — | |
| Llama 3 70BParameters=70B2024.07 | 83.8 | — | |
| Mixtral 8x7BActive Params=13B2024.01 | 83.6 | — | |
| PaLM 2-Mprompting=1-shot2023.05 | 83.2 | — | |
| LoRABackbone=Gemma-7B2024.08 | 83.19 | — | |
| FalconModel Size=40B2023.11 | 83 | — | |
| Mistral 7BParameters=7B2024.07 | 83 | — | |
| RFID-MoEBackbone=Qwen2-57B-A14B, Ratio=40%, Evaluation Protocol=Zero-shot2026.02 | 83 | — | |
| GPT-3Setting=Few-Shot2020.05 | 82.8 | — | |
| LLaMAParameters=65B, Zero-shot=true, Likelihood Normalization=Per-character2023.02 | 82.8 | — | |
| LLaMA-2Model Size=70B2023.11 | 82.8 | — | |
| DoRATunable Params=22.6M, Backbone=LLaMA-3 8B2026.01 | 82.7 | — | |
| LLaMA 2 70BActive Params=70B2024.01 | 82.6 | — | |
| Llama-3-70B#Bit=16.00, Base Model=Llama-3-70B, Evaluation Protocol=Zero-shot2025.03 | 82.5 | — | |
| AutoLoRATunable Params=23.1M, Backbone=LLaMA-3 8B2026.01 | 82.5 | — | |
| MOELoRATunable Params=29.9M, Backbone=LLaMA-3 8B2026.01 | 82.4 | — | |
| PaLMParameters=540B, Zero-shot=true, Likelihood Normalization=Per-character2023.02 | 82.3 | — | |
| LLaMAParameters=33B, Zero-shot=true, Likelihood Normalization=Per-character2023.02 | 82.3 | — | |
| PaLM2023.11 | 82.3 | — | |
| PaLM-540BModel Category=LLM, Evaluation Mode=Zero-shot2023.09 | 82.3 | — | |
| PaLM 2-Sprompting=1-shot2023.05 | 82.2 | — | |
| LLaMA 1 33BActive Params=33B2024.01 | 82.2 | — | |
| Mistral 7BActive Params=7B2024.01 | 82.2 | — | |
| MistralSize=7B, Context=16K2024.04 | 82.2 | — | |
| LoRABackbone=Mistral-7B-v0.12024.08 | 82.15 | — | |
| AdaLoRATunable Params=23.2M, Backbone=LLaMA-3 8B2026.01 | 82.1 | — | |
| MT-NLG 530Bzero-shot=true2022.03 | 82 | — | |
| MT-NLG2023.11 | 82 | — | |
| RFID-MoEBackbone=Qwen3-30B-A3B-2507, Ratio=20%, Evaluation Protocol=Zero-shot2026.02 | 82 | — | |
| LLaMA-2Model Size=34B2023.11 | 81.9 | — | |
| Chinchillazero-shot=true2022.03 | 81.8 | — | |
| Gopherzero-shot=true2022.03 | 81.8 | — | |
| GopherParameters=280B, Zero-shot=true, Likelihood Normalization=Per-character2023.02 | 81.8 | — | |
| ChinchillaParameters=70B, Zero-shot=true, Likelihood Normalization=Per-character2023.02 | 81.8 | — | |
| Gopher2023.11 | 81.8 | — | |
| Chinchilla2023.11 | 81.8 | — | |
| FlexoraBackbone=Llama2-7B2024.08 | 81.72 | — | |
| Random (Greedy)Selection=randomly selected layers, LoRA Rank (r)=82024.08 | 81.54 | — | |
| Gemma 7BParameters=7B2024.07 | 81.5 | — | |
| PaLM-contParameters=62B, Zero-shot=true, Likelihood Normalization=Per-character2023.02 | 81.4 | — | |
| LoRATunable Params=22.5M, Backbone=LLaMA-3 8B2026.01 | 81.4 | — | |
| GemmaSize=8B, Tokens=6T, Context=8K2024.04 | 81.2 | — | |
| Full ModelBackbone=Llama-3.1-8B-Instruct, Pruning Ratio=0%2026.02 | 81.12 | — | |
| DRAGON2022.10 | 81.1 | — | |
| LoRA-Drop (25%)Backbone=Qwen2.5-14B, Evaluation Protocol=Zero-shot, Drop Ratio=25%, Speedup (x)=1.352026.01 | 81.1 | — | |
| GPT-3zero-shot=true2022.03 | 81 | — | |
| GPT-3Parameters=175B, Zero-shot=true, Likelihood Normalization=Per-character2023.02 | 81 | — | |
| GPT-32023.11 | 81 | — | |
| GPT-3Model Category=LLM, Evaluation Mode=Zero-shot2023.09 | 81 | — | |
| Llama 3 8BParameters=8B2024.07 | 81 | — | |
| FP16 BaselineModel=LLaMA-2-70B, #Bits=W16A162024.02 | 80.85 | — | |
| FP-16Backbone=LLaMA-1-65B, Bit-width=W16A16, Evaluation Protocol=zero-shot2024.02 | 80.85 | — | |
| LLaMA 2 13BActive Params=13B2024.01 | 80.8 | — | |
| WandaBackbone=Llama-3.1-8B-Instruct, Pruning Ratio=30%2026.02 | 80.74 | — | |
| DeepSeekMoE 145B# Shot=0-shot2024.01 | 80.7 | — | |
| OLMoE-1B-7B-0924Model Type=MoE2026.04 | 80.7 | — | |
| MPTSize=7B, Tokens=1T, Context=4K2024.04 | 80.6 | — | |
| OLMoE-1B-7B#Params=1.3B/7B, #Tokens=5T, Zero-shot=true2026.02 | 80.6 | — | |
| DenseModel=Llama2-13B, Pruning Ratio (%)=0, Zero-shot=true2024.06 | 80.52 | — | |
| GPT-3Setting=Zero-Shot2020.05 | 80.5 | — | |
| GPT-3Setting=One-Shot2020.05 | 80.5 | — | |
| PaLMParameters=62B, Zero-shot=true, Likelihood Normalization=Per-character2023.02 | 80.5 | — | |
| LLaMA-2Model Size=13B2023.11 | 80.5 | — | |
| LLAMA2Size=13B, Tokens=2T, Context=4K2024.04 | 80.5 | — | |
| Llama-2-13B#Bit=16, Base Model=Llama-2-13B, Zero-shot=true2025.03 | 80.5 | — | |
| 27B w/ mHC# Shots=0-shot2025.12 | 80.5 | — | |
| WandaBackbone=Qwen3-VL-8B-Instruct, Pruning Ratio=30%2026.02 | 80.47 | — | |
| POPBackbone=Llama-3.1-8B-Instruct, Pruning Ratio=31.25%2026.02 | 80.36 | — | |
| FalconModel Size=7B2023.11 | 80.3 | — | |
| Llama-3.1Size=8B, Type=AR2026.01 | 80.3 | — | |
| AdaLoRAro=4, LoRA Rank (r)=82024.08 | 80.2 | — | |
| DeepSeekMoE 142B (Half Activated)# Shot=0-shot2024.01 | 80.2 | — |