Commonsense reasoning on PIQA 1.0 (test)
82.21AccuracyMistral-7B + Uniform
Evaluation results
| Method | Links | |
|---|---|---|
| Mistral-7B + UniformModel=Mistral-7B, Selection Method=Uniform, Protocol=6-shot2024.02 | 82.21 | |
| Mistral-7B + AutoDSModel=Mistral-7B, Selection Method=AutoDS, Protocol=6-shot2024.02 | 82.21 | |
| Mistral-7B BaseModel=Mistral-7B, Selection Method=Base, Protocol=6-shot2024.02 | 82.1 | |
| Mistral-7BTraining Tokens (B)=8000*, Evaluation Protocol=Zero-shot, Model Architecture Category=Transformer2025.07 | 82.1 | |
| LLaMA-3-8B-LizardTraining Tokens (B)=0.04, Evaluation Protocol=Zero-shot, Model Architecture Category=Linearized, Linearized Model Constraint=Unbounded2025.07 | 82 | |
| Mistral-7B + QuRatingModel=Mistral-7B, Selection Method=QuRating, Protocol=6-shot2024.02 | 81.99 | |
| Mistral-7B + DSIRModel=Mistral-7B, Selection Method=DSIR, Protocol=6-shot2024.02 | 81.94 | |
| Gemma-7BTraining Tokens (B)=6000, Evaluation Protocol=Zero-shot, Model Architecture Category=Transformer2025.07 | 81.9 | |
| Mistral-7B-LizardTraining Tokens (B)=0.04, Evaluation Protocol=Zero-shot, Model Architecture Category=Linearized, Linearized Model Constraint=Unbounded2025.07 | 81.8 | |
| Mistral-7B-LoLCATsTraining Tokens (B)=0.04, Evaluation Protocol=Zero-shot, Model Architecture Category=Linearized, Linearized Model Constraint=Bounded2025.07 | 81.5 | |
| Mamba-7BTraining Tokens (B)=1200, Evaluation Protocol=Zero-shot, Model Architecture Category=Subquadratic2025.07 | 81 | |
| Griffin-7BTraining Tokens (B)=300, Evaluation Protocol=Zero-shot, Model Architecture Category=Subquadratic2025.07 | 81 | |
| LLaMA-3-8B-LoLCATsTraining Tokens (B)=0.04, Evaluation Protocol=Zero-shot, Model Architecture Category=Linearized, Linearized Model Constraint=Bounded2025.07 | 80.9 | |
| Mistral-7B-SUPRATraining Tokens (B)=100, Evaluation Protocol=Zero-shot, Model Architecture Category=Linearized, Linearized Model Constraint=Bounded2025.07 | 80.4 | |
| Liger-GLA-Llama-3-8BTraining Tokens (B)=0.02, Evaluation Protocol=Zero-shot, Model Architecture Category=Linearized, Linearized Model Constraint=Bounded2025.07 | 80.3 | |
| TransNormerLLM-7BTraining Tokens (B)=1400, Evaluation Protocol=Zero-shot, Model Architecture Category=Subquadratic2025.07 | 80.1 | |
| Liger-GLA-Mistral-7BTraining Tokens (B)=0.02, Evaluation Protocol=Zero-shot, Model Architecture Category=Linearized, Linearized Model Constraint=Bounded2025.07 | 80.1 | |
| Hawk-7BTraining Tokens (B)=300, Evaluation Protocol=Zero-shot, Model Architecture Category=Subquadratic2025.07 | 80 | |
| LLaMA-3-8BTraining Tokens (B)=15000, Evaluation Protocol=Zero-shot, Model Architecture Category=Transformer2025.07 | 79.9 | |
| LLaMA2-7B + UniformModel=LLaMA2-7B, Selection Method=Uniform, Protocol=6-shot2024.02 | 79.54 | |
| LLaMA2-7B BaseModel=LLaMA2-7B, Selection Method=Base, Protocol=6-shot2024.02 | 79.43 | |
| LLaMA2-7B + QuRatingModel=LLaMA2-7B, Selection Method=QuRating, Protocol=6-shot2024.02 | 79 | |
| LLaMA2-7B + DSIRModel=LLaMA2-7B, Selection Method=DSIR, Protocol=6-shot2024.02 | 78.84 | |
| RWKV-6-v2.1-7BTraining Tokens (B)=1420, Evaluation Protocol=Zero-shot, Model Architecture Category=Subquadratic2025.07 | 78.7 | |
| LLaMA2-7B + AutoDSModel=LLaMA2-7B, Selection Method=AutoDS, Protocol=6-shot2024.02 | 78.18 | |
| Gemma-2B + QuRatingModel=Gemma-2B, Selection Method=QuRating, Protocol=6-shot2024.02 | 77.53 | |
| Gemma-2B + AutoDSModel=Gemma-2B, Selection Method=AutoDS, Protocol=6-shot2024.02 | 77.42 | |
| Gemma-2B + DSIRModel=Gemma-2B, Selection Method=DSIR, Protocol=6-shot2024.02 | 77.15 | |
| Mamba2-LLaMA-3-8BTraining Tokens (B)=20, Evaluation Protocol=Zero-shot, Model Architecture Category=Linearized, Linearized Model Constraint=Unbounded2025.07 | 76.8 | |
| Gemma-2B + UniformModel=Gemma-2B, Selection Method=Uniform, Protocol=6-shot2024.02 | 76.71 | |
| Gemma-2B BaseModel=Gemma-2B, Selection Method=Base, Protocol=6-shot2024.02 | 70.67 | |
| TituLLM-3b-v2.0Shots=52025.02 | 60 | |
| BLOOM-3bShots=02025.02 | 58 | |
| BLOOM-3bShots=52025.02 | 58 | |
| TituLLM-1b-v2.0Shots=02025.02 | 58 | |
| TituLLM-3b-v2.0Shots=02025.02 | 58 | |
| Llama-3.2-3bShots=02025.02 | 57 | |
| Llama-3.2-3bShots=52025.02 | 57 | |
| TituLLM-1b-v2.0Shots=52025.02 | 57 | |
| Gemma-2-2bShots=02025.02 | 56 | |
| Gemma-2-2bShots=52025.02 | 56 | |
| BLOOM-1b7Shots=52025.02 | 56 | |
| BLOOM-1b1Shots=52025.02 | 55 | |
| BLOOM-1b7Shots=02025.02 | 55 | |
| Llama-3.2-1bShots=52025.02 | 54 | |
| Qwen-2.5-0.5bShots=02025.02 | 54 | |
| SmolLM2-360mShots=02025.02 | 54 | |
| BLOOM-560mShots=02025.02 | 54 | |
| BLOOM-560mShots=52025.02 | 54 | |
| BLOOM-1b1Shots=02025.02 | 54 | |
| Llama-3.2-1bShots=02025.02 | 53 | |
| Qwen-2.5-0.5bShots=52025.02 | 53 | |
| Qwen-2.5-1.5bShots=02025.02 | 53 | |
| SmolLM2-360mShots=52025.02 | 53 | |
| SmolLM2-1.7bShots=02025.02 | 53 | |
| SmolLM2-1.7bShots=52025.02 | 53 | |
| BongLLaMA-3.2-1bShots=52025.02 | 53 | |
| davinciShots=02025.02 | 52 | |
| Qwen-2.5-1.5bShots=52025.02 | 52 | |
| SmolLM2-135mShots=02025.02 | 52 | |
| SmolLM2-135mShots=52025.02 | 52 | |
| BongLLaMA-3.2-1bShots=02025.02 | 52 | |
| BongLLaMA-3.2-3bShots=02025.02 | 51 | |
| BongLLaMA-3.2-3bShots=52025.02 | 50 |