Commonsense Reasoning on PIQA (Normalized Accuracy)
68.39Normalized AccuracyMHF (H4B16K)
Evaluation results
| Method | Links | |
|---|---|---|
| MHF (H4B16K)Model Scale=3B, Embedding Parameters (Emb)=138M, Decoder Parameters (Dec)=2.8B, Evaluation Protocol=Zero-shot2026.06 | 68.39 | |
| MHF (H3B10K)Model Scale=1B, Embedding Parameters (Emb)=65M, Decoder Parameters (Dec)=0.9B, Evaluation Protocol=Zero-shot2026.06 | 68.17 | |
| MHF (H4B16K)Model Scale=1B, Embedding Parameters (Emb)=138M, Decoder Parameters (Dec)=0.9B, Evaluation Protocol=Zero-shot2026.06 | 66.49 | |
| StandardModel Scale=3B, Embedding Parameters (Emb)=65M, Decoder Parameters (Dec)=2.8B, Evaluation Protocol=Zero-shot2026.06 | 65.83 | |
| StandardModel Scale=1B, Embedding Parameters (Emb)=65M, Decoder Parameters (Dec)=0.9B, Evaluation Protocol=Zero-shot2026.06 | 65.78 | |
| Standard+2LModel Scale=1B, Embedding Parameters (Emb)=65M, Decoder Parameters (Dec)=1.0B, Evaluation Protocol=Zero-shot2026.06 | 65.02 | |
| MHF (H3B10K)Model Scale=100M, Embedding Parameters (Emb)=25M, Decoder Parameters (Dec)=87M, Evaluation Protocol=Zero-shot2026.06 | 60.23 | |
| Standard+4LModel Scale=100M, Embedding Parameters (Emb)=25M, Decoder Parameters (Dec)=116M, Evaluation Protocol=Zero-shot2026.06 | 60.17 | |
| MHF (H4B16K)Model Scale=100M, Embedding Parameters (Emb)=52M, Decoder Parameters (Dec)=87M, Evaluation Protocol=Zero-shot2026.06 | 59.03 | |
| StandardModel Scale=100M, Embedding Parameters (Emb)=25M, Decoder Parameters (Dec)=87M, Evaluation Protocol=Zero-shot2026.06 | 58.87 | |
| Rnd. GuessModel Scale=N/A, Embedding Parameters (Emb)=-, Decoder Parameters (Dec)=-, Evaluation Protocol=Random Guessing2026.06 | 50 |