Multiple Choice Question Answering on BIG-bench HHH Eval
87Overall ScoreChatGPT
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| ChatGPT2023.05 | 87 | 95 | 85 | 80 | 0.91 | |
| Anthropic-LMEvaluation Protocol=Preference Model (PM)2023.05 | 86 | — | — | — | — | |
| Dromedary-65Bnumber of parameters=65B, verbosity=non-verbose2023.05 | 85 | 91 | 86 | 74 | 0.88 | |
| Dromedary-65Bnumber of parameters=65B, verbosity=final2023.05 | 83 | 91 | 85 | 74 | 0.81 | |
| Alpaca-65Bnumber of parameters=65B, reproduction=true2023.05 | 79 | 76 | 85 | 72 | 0.86 | |
| Anthropic-LMEvaluation Protocol=Context Distillation (CD)2023.05 | 77 | — | — | — | — | |
| LLaMA-65Bnumber of parameters=65B2023.05 | 77 | 71 | 83 | 72 | 0.84 | |
| DModel=LLaMA-2-13B2025.10 | 75.1 | — | — | — | — | |
| DModel=LLaMA-2-7B2025.10 | 72.9 | — | — | — | — | |
| SGModel=LLaMA-2-7B, Decision Layer=smooth (sigmoid), smoothness (κ)=302025.10 | 72.9 | — | — | — | — | |
| ER-GModel=LLaMA-2-13B2025.10 | 71.5 | — | — | — | — | |
| ER-DModel=LLaMA-2-13B2025.10 | 71.5 | — | — | — | — | |
| SCModel=GPT-OSS-20B2025.10 | 71.5 | — | — | — | — | |
| SGModel=GPT-OSS-20B, Decision Layer=smooth (sigmoid), smoothness (κ)=302025.10 | 71.5 | — | — | — | — | |
| ER-GModel=LLaMA-2-7B2025.10 | 69.7 | — | — | — | — | |
| ER-DModel=LLaMA-2-7B2025.10 | 69.7 | — | — | — | — | |
| SGModel=LLaMA-2-13B, Decision Layer=smooth (sigmoid), smoothness (κ)=302025.10 | 69.2 | — | — | — | — | |
| SCModel=LLaMA-2-13B2025.10 | 68.3 | — | — | — | — | |
| SGModel=Llama-3.1-8B, Decision Layer=smooth (sigmoid), smoothness (κ)=302025.10 | 67.9 | — | — | — | — | |
| MIModel=LLaMA-2-7B2025.10 | 66.5 | — | — | — | — | |
| GModel=LLaMA-2-7B2025.10 | 66.5 | — | — | — | — | |
| ER-GModel=GPT-OSS-20B2025.10 | 66.1 | — | — | — | — | |
| ER-DModel=GPT-OSS-20B2025.10 | 66.1 | — | — | — | — | |
| MIModel=Llama-3.1-8B2025.10 | 65.6 | — | — | — | — | |
| SCModel=LLaMA-2-7B2025.10 | 65.2 | — | — | — | — | |
| MIModel=LLaMA-2-13B2025.10 | 65.2 | — | — | — | — | |
| GModel=LLaMA-2-13B2025.10 | 65.2 | — | — | — | — | |
| GModel=Llama-3.1-8B2025.10 | 65.2 | — | — | — | — | |
| MIModel=Llama-3.2-1B2025.10 | 65.2 | — | — | — | — | |
| GModel=Llama-3.2-1B2025.10 | 65.2 | — | — | — | — | |
| DModel=GPT-OSS-20B2025.10 | 63.3 | — | — | — | — | |
| ER-GModel=Llama-3.1-8B2025.10 | 62 | — | — | — | — | |
| ER-DModel=Llama-3.1-8B2025.10 | 62 | — | — | — | — | |
| DModel=Llama-3.2-1B2025.10 | 61.5 | — | — | — | — | |
| DModel=Llama-3.1-8B2025.10 | 59.7 | — | — | — | — | |
| SCModel=Llama-3.1-8B2025.10 | 59.7 | — | — | — | — | |
| SGModel=Llama-3.2-1B, Decision Layer=smooth (sigmoid), smoothness (κ)=302025.10 | 54.3 | — | — | — | — | |
| SCModel=Llama-3.2-1B2025.10 | 49.3 | — | — | — | — | |
| ER-GModel=Llama-3.2-1B2025.10 | 47.5 | — | — | — | — | |
| ER-DModel=Llama-3.2-1B2025.10 | 47.5 | — | — | — | — | |
| MIModel=GPT-OSS-20B2025.10 | 43.9 | — | — | — | — | |
| GModel=GPT-OSS-20B2025.10 | 43.9 | — | — | — | — |