Same-task discrimination accuracy on BBEH
72.6Same-task AccuracyOE
Evaluation results
| Method | Links | |
|---|---|---|
| OELLM=Llama 70B2026.05 | 72.6 | |
| OELLM=Llama 8B2026.05 | 68.8 | |
| OELLM=DS-R1 32B2026.05 | 63.5 | |
| OELLM=Sky-OR1 32B2026.05 | 63.4 | |
| OELLM=GPT-OSS 20B2026.05 | 62.4 | |
| STLLM=DS-R1 32B2026.05 | 54.8 | |
| STLLM=Llama 8B2026.05 | 54.7 | |
| LTLLM=Llama 8B2026.05 | 54.7 | |
| IELLM=Llama 8B2026.05 | 54.5 | |
| STLLM=Sky-OR1 32B2026.05 | 54.2 | |
| IELLM=Sky-OR1 32B2026.05 | 54 | |
| LITLLM=Llama 8B2026.05 | 53.9 | |
| STNLLM=Llama 8B2026.05 | 53.5 | |
| IELLM=DS-R1 32B2026.05 | 53.5 | |
| LITLLM=Sky-OR1 32B2026.05 | 53.3 | |
| STLLM=Llama 70B2026.05 | 52.9 | |
| STNLLM=Llama 70B2026.05 | 52.8 | |
| LITLLM=DS-R1 32B2026.05 | 52.6 | |
| IELLM=Llama 70B2026.05 | 52.1 | |
| STNLLM=DS-R1 32B2026.05 | 51.8 | |
| STNLLM=Sky-OR1 32B2026.05 | 51.8 | |
| STNLLM=GPT-OSS 20B2026.05 | 51.8 | |
| LITLLM=Llama 70B2026.05 | 51.6 | |
| LTLLM=Llama 70B2026.05 | 51.4 | |
| LTLLM=Sky-OR1 32B2026.05 | 51.2 | |
| LTLLM=GPT-OSS 20B2026.05 | 51.2 | |
| RVLLM=GPT-OSS 20B2026.05 | 51 | |
| STLLM=GPT-OSS 20B2026.05 | 50.7 | |
| LITLLM=GPT-OSS 20B2026.05 | 50.4 | |
| LTLLM=DS-R1 32B2026.05 | 50.3 | |
| RVLLM=DS-R1 32B2026.05 | 50.3 | |
| RVLLM=Sky-OR1 32B2026.05 | 49.9 | |
| RVLLM=Llama 70B2026.05 | 49.7 | |
| IELLM=GPT-OSS 20B2026.05 | 49.5 | |
| RVLLM=Llama 8B2026.05 | 48.9 |