Hard Reasoning Tasks on BBH (0-shot/3-shot)
52.1BBH Accuracy (0-shot)Yi-Chat-8bits(GPTQ)
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| Yi-Chat-8bits(GPTQ)Size=34B2024.03 | 52.1 | 71 | — | |
| Yi-ChatSize=34B2024.03 | 51.4 | 71.7 | — | |
| Qwen-ChatSize=14B2024.03 | 49.7 | 55 | — | |
| Yi-Chat-4bits(AWQ)Size=34B2024.03 | 48.3 | 69.4 | — | |
| LLaMA2-ChatSize=70B2024.03 | 42.4 | 58.5 | — | |
| InternLM-ChatSize=20B2024.03 | 42.4 | 36.7 | — | |
| Yi-Chat-8bits(GPTQ)Size=6B2024.03 | 40.4 | 47.3 | — | |
| Yi-ChatSize=6B2024.03 | 39.7 | 47.2 | — | |
| Baichuan2-ChatSize=13B2024.03 | 38.8 | 47.2 | — | |
| Yi-Chat-4bits(AWQ)Size=6B2024.03 | 37.7 | 43.6 | — | |
| LLaMA2-ChatSize=13B2024.03 | 32.9 | 58.2 | — | |
| AquilaChat2Size=34B2024.03 | 20.1 | 34.3 | — | |
| CONCEPTINFBase model=Llama-2-7B2026.05 | — | — | 37.87 | |
| CONCEPTINFBase model=Llama-2-13B2026.05 | — | — | 49.04 | |
| LESSBase model=Llama-2-7B2026.05 | — | — | 41.7 | |
| LESSBase model=Llama-2-13B2026.05 | — | — | 50.09 | |
| PRISMBase model=Llama-2-7B2026.05 | — | — | 38.12 | |
| PRISMBase model=Llama-2-13B2026.05 | — | — | 49.75 | |
| RandomBase model=Llama-2-7B2026.05 | — | — | 40.06 | |
| RandomBase model=Llama-2-13B2026.05 | — | — | 50.09 | |
| VECFILTERBase model=Llama-2-7B2026.05 | — | — | 37.35 | |
| VECFILTERBase model=Llama-2-13B2026.05 | — | — | 49.04 |