Legal Question Answering on MMLU-Pro Law n=1,101 (test)
75.7AccuracyEP-HUBO (Qwen-35B)
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| EP-HUBO (Qwen-35B)Adj.=Opus4.6, Trace generator=Qwen3.5-35B, Traces per question=202026.06 | 75.7 | — | — | — | — | |
| ZS (no traces/HUBO)Adj.=Opus4.6, Traces=None2026.06 | 74.2 | — | — | — | — | |
| EP-HUBO (OSS-20B)Adj.=Opus4.6, Trace generator=OSS-20B, Traces per question=202026.06 | 74.1 | — | — | — | — | |
| EP-HUBO (Qwen-35B)Adj.=Sonnet4.6, Trace generator=Qwen3.5-35B, Traces per question=202026.06 | 67.1 | — | — | — | — | |
| EP-HUBO (OSS-20B)Adj.=Sonnet4.6, Trace generator=OSS-20B, Traces per question=202026.06 | 63.9 | — | — | — | — | |
| ZS (no traces/HUBO)Adj.=Sonnet4.6, Traces=None2026.06 | 63.1 | — | — | — | — | |
| MV (Qwen-35B)Trace generator=Qwen3.5-35B, Traces per question=202026.06 | 63.1 | — | — | — | — | |
| MV (OSS-20B)Trace generator=OSS-20B, Traces per question=202026.06 | 46.2 | — | — | — | — | |
| HUBO prec (Opus, OSS-20B)Adj.=Opus, Trace generator=OSS-20B2026.06 | — | — | 55 | 56 | 49.5 | |
| HUBO prec (Opus, Qwen-35B)Adj.=Opus, Trace generator=Qwen-35B2026.06 | — | — | 72 | 55 | 56.7 | |
| HUBO prec (Sonnet, OSS-20B)Adj.=Sonnet, Trace generator=OSS-20B2026.06 | — | — | 91 | 83 | 52.3 | |
| HUBO prec (Sonnet, Qwen-35B)Adj.=Sonnet, Trace generator=Qwen-35B2026.06 | — | — | 129 | 85 | 60.3 | |
| Net Opus vs MV (OSS-20B)Trace generator=OSS-20B2026.06 | — | 27.9 | — | — | — | |
| Net Opus vs MV (Qwen-35B)Trace generator=Qwen3.5-35B2026.06 | — | 12.6 | — | — | — | |
| Net Opus vs ZS Opus (OSS-20B)Trace generator=OSS-20B2026.06 | — | 0.1 | — | — | — | |
| Net Opus vs ZS Opus (Qwen-35B)Trace generator=Qwen3.5-35B2026.06 | — | 1.5 | — | — | — | |
| Net Sonnet vs MV (OSS-20B)Trace generator=OSS-20B2026.06 | — | 17.6 | — | — | — | |
| Net Sonnet vs MV (Qwen-35B)Trace generator=Qwen3.5-35B2026.06 | — | 4 | — | — | — | |
| Net Sonnet vs ZS Son (OSS-20B)Trace generator=OSS-20B2026.06 | — | 0.7 | — | — | — | |
| Net Sonnet vs ZS Son (Qwen-35B)Trace generator=Qwen3.5-35B2026.06 | — | 4 | — | — | — |