Multitask Language Understanding on MMLU Redux
84.13AccuracyT2T
Evaluation results
| Method | Links | ||
|---|---|---|---|
| T2TPair=8B → 14B, Regime=context-aware2026.06 | 84.13 | 67.08 | |
| T2TPair=14B → 8B, Regime=context-aware2026.06 | 83.43 | 55.5 | |
| T2TPair=4B → 14B, Regime=context-aware2026.06 | 82.87 | 56.24 | |
| T2TPair=4B → 8B, Regime=context-aware2026.06 | 80.75 | 37.73 | |
| T2TPair=14B → 4B, Regime=context-aware2026.06 | 80.61 | 43.48 | |
| T2TPair=8B → 4B, Regime=context-aware2026.06 | 79.39 | 33.91 | |
| dense alignmentPair=4B → 14B, Regime=context-aware2026.06 | 78.57 | 21.54 | |
| dense alignmentPair=4B → 8B, Regime=context-aware2026.06 | 78.52 | 12.56 | |
| dense alignmentPair=14B → 8B, Regime=context-aware2026.06 | 78.04 | 15.64 | |
| dense alignmentPair=8B → 4B, Regime=context-aware2026.06 | 77.86 | 7.95 | |
| dense alignmentPair=14B → 4B, Regime=context-aware2026.06 | 77.66 | 10.18 | |
| dense alignmentPair=8B → 14B, Regime=context-aware2026.06 | 77.38 | 21.79 | |
| Ours-context-unawarePair=8B → 4B, Context Setting=context-unaware2026.06 | 77.06 | 6.56 | |
| C2CPair=14B → 8B, Regime=context-aware2026.06 | 75.99 | 7.83 | |
| C2CPair=4B → 8B, Regime=context-aware2026.06 | 75.87 | 4.5 | |
| Ours-context-unawarePair=4B → 8B, Context Setting=context-unaware2026.06 | 74.59 | 9.42 | |
| C2CPair=8B → 14B, Regime=context-aware2026.06 | 73.81 | 7.42 | |
| C2CPair=4B → 14B, Regime=context-aware2026.06 | 72.76 | 6.64 | |
| Receiver-onlyPair=4B → 8B, Regime=context-aware2026.06 | 72.1 | 19.85 | |
| Receiver-onlyPair=14B → 8B, Regime=context-aware2026.06 | 72.1 | 19.85 | |
| Receiver-onlyPair=4B → 14B, Regime=context-aware2026.06 | 71.8 | 31.16 | |
| Receiver-onlyPair=8B → 14B, Regime=context-aware2026.06 | 71.8 | 31.16 | |
| C2CPair=8B → 4B, Regime=context-aware2026.06 | 70.19 | 3.43 | |
| C2CPair=14B → 4B, Regime=context-aware2026.06 | 69.73 | 5.01 | |
| Ours-context-unawarePair=14B → 8B, Context Setting=context-unaware2026.06 | 67.84 | 13.28 | |
| Ours-context-unawarePair=14B → 4B, Context Setting=context-unaware2026.06 | 67.15 | 8.9 | |
| Receiver-onlyPair=8B → 4B, Regime=context-aware2026.06 | 65 | 9.18 | |
| Receiver-onlyPair=14B → 4B, Regime=context-aware2026.06 | 65 | 9.18 | |
| Ours-context-unawarePair=4B → 14B, Context Setting=context-unaware2026.06 | 62.86 | 17.15 | |
| Ours-context-unawarePair=8B → 14B, Context Setting=context-unaware2026.06 | 54.92 | 15.85 | |
| T2T-context-unawarePair=8B → 4B, Context Setting=context-unaware2026.06 | 25.18 | 32.75 | |
| T2T-context-unawarePair=14B → 4B, Context Setting=context-unaware2026.06 | 24.52 | 41.95 | |
| T2T-context-unawarePair=4B → 14B, Context Setting=context-unaware2026.06 | 23.06 | 60.83 | |
| T2T-context-unawarePair=8B → 14B, Context Setting=context-unaware2026.06 | 22.8 | 72.06 | |
| C2C-context-unawarePair=4B → 8B, Context Setting=context-unaware2026.06 | 21.91 | 14.94 | |
| T2T-context-unawarePair=14B → 8B, Context Setting=context-unaware2026.06 | 21.41 | 53.8 | |
| T2T-context-unawarePair=4B → 8B, Context Setting=context-unaware2026.06 | 21 | 37.86 | |
| C2C-context-unawarePair=4B → 14B, Context Setting=context-unaware2026.06 | 10.26 | 20.05 | |
| C2C-context-unawarePair=8B → 4B, Context Setting=context-unaware2026.06 | 8.2 | 14.76 | |
| C2C-context-unawarePair=8B → 14B, Context Setting=context-unaware2026.06 | 7.14 | 9.84 | |
| C2C-context-unawarePair=14B → 4B, Context Setting=context-unaware2026.06 | 6.11 | 17.93 | |
| C2C-context-unawarePair=14B → 8B, Context Setting=context-unaware2026.06 | 5.34 | 24.77 |