Commonsense Reasoning on PIQA, WinoGrande, HellaSwag, BoolQ, SocialIQA, and OpenBookQA
90.2PIQA AccuracyQwen2.5-14B
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| Qwen2.5-14BParams=14B, Latency (s/token)=0.0982025.12 | 90.2 | 85.8 | 92.9 | 71.2 | 78.9 | 91.4 | 85.1 | |
| LLMBOOST (7B × 2)Params=14B, Latency (s/token)=0.0752025.12 | 89.5 | 86.7 | 95.1 | 71.6 | 79.1 | 91.6 | 85.6 | |
| Qwen2.5-7BParams=7B, Latency (s/token)=0.0562025.12 | 86.7 | 83.9 | 89.4 | 68.7 | 77.6 | 87.4 | 82.3 | |
| LLMBOOST (3B × 2)Params=6B (-1B), Latency (s/token)=0.0432025.12 | 85.9 | 81.8 | 91.7 | 70.4 | 80 | 86.2 | 82.3 | |
| LLMBOOST (8B + 3B)Params=11B, Latency (s/token)=0.049, Ensemble Type=Weakly-Heterogeneous2025.12 | 85.9 | 85 | 93.8 | 71.3 | 80.4 | 84 | 83.7 | |
| Llama 3.1-8BParams=8B, Latency (s/token)=0.038, Ensemble Type=Weakly-Heterogeneous2025.12 | 83.3 | 81.8 | 90.6 | 69.5 | 75.8 | 79 | 80 | |
| LLama-3.2-3BParams=3B, Latency (s/token)=0.027, Ensemble Type=Weakly-Heterogeneous2025.12 | 78.5 | 77.7 | 83.9 | 62.4 | 74.3 | 76.8 | 75.6 |